It wasn't until 1970 that the Census Bureau began asking households to fill out a form at home and send it in; before then, temporary census workers (and before them, federal marshals) collected the data in person.
Read what George Washington said after he was presented with the returns from the first-ever U.S. Census, conducted by federal marshals on horseback in 1790.
Returns of the Census have already been made from several of the States and a tolerably just estimate has been formed now in others, by which it appears that we shall hardly reach four millions; but one thing is certain: our real numbers will exceed, greatly, the official returns of them; because the religious scruples of some would not allow them to give in their lists; the fear of others that it was intended as the foundation of a tax induced them to conceal or diminish theirs; and thro' the indolence of the people and the negligence of many of the Officers, numbers are omitted.
-The Writings of George Washington, John C. Fitzpatrick, ed., vol. 31 Washington, DC: Government Printing Office, 1939: 329.
True then, true today.
Source: "American People: Politics and Science in Census Taking," by Kenneth Prewitt, Population Reference Bureau.