I was included in the last American Community Census. I filled out the online questionnaire after a little technical trouble which I never expect a government agency to be able to deal with. Bureaucracies and computers are not compatible. A bureaucracy is made up of countless warring factions and the programmers never do know who to listen to.
That is not the issue here. The issue is that I have been designing high quality surveys for the past 40 years as an expert in the field. I have conducted a vast number of surveys including several where I was in the field myself and several where I supervised the field interviewers. I have used every medium from face-to-face interviews, to focus groups to postcard questionnaires.
There are several serious problems with this annual sample for the American Community Census. Far too many of the questions are ambiguous. I could not fit my income into any of their categories because I have at least six different sources of income that overlap the three categories for which the census requires separate numbers. Meaning my total income and those in my household were wrong. (Rental income, Social Security income, consulting income, royalty income, dividend income and capital gains can not meaningfully be fit in three categories.)
This particular problem arises from the fact that the Census people have no conception of self employment. None whatsoever. They do not have categories for the dozens of types of self-employment. They give no priority to the self employment category and its overlap with other earning categories. Yet this is the category with the greatest amount of growth and jobs and vitality in American society.
Lastly, while they are using a computer for answers, they make no effort to include alternatives such as "other" with a pulldown for explanation or "none of the above" with a pulldown for explanations.
The clear reason for their very rigid and inappropriate questionnaire is their sample of several million is too gigantic for them to process the answers. A sample this size with poor questions is consequently irrelevant.
Sampling 2 million people with the wrong question is not in the same category of reliability as sampling 200 people with the right question. The latter is always preferred by those of us who know statistics and survey research.
My job in life is not to criticize an incompetent bureaucracy.
A proper study could be done with a small in-person proper random sample. Sample size would not need to be over 5000 completed interviews. The accuracy would be greater than anything currently done anywhere in the world.
The survey interviewers would have to be educated and experienced survey researchers who are required to make written comments about each response. Top flight survey firms such as NORC do this.
I know it will take time but someday we will have a reliable national census data base that will underlie the tens of thousands of commercial survey research studies that make use of standardized national data.
Comments