Are political surveys reliable?
The answer from a technical point of view is ‘absolutely not’.
In the early 1960’s I was able to do reliable survey research on any subject. We knocked on people’s doors, found the right person to interview, and did the interview. The person to be interviewed was selected from a random number table. Survey theory supported this methodology, as long as we were getting interviews with close to 80% of the selected interviewees. The main error was in the sample size. Even a sample of 25 was often very accurate.
Today survey’s are done with a mixture of random calls to landlines, cellphones and sometimes personal interviews at shopping malls. The error is entirely in self-selection by respondents. Who are the people willing to answer the survey questionnaire? I often hang up in the middle of a phone survey when the questionnaire was clearly designed by ignorant amateurs.
In trying to correct the result to represent the general voting population the survey instrument includes questions about the demographics of the respondent and the location of the respondent (with zip code). People lie and deliberately distort their responses but in general the lying and distortion remains the same over time.
The problem is that the self-selection bias is horrific. Past election data is used to estimate current survey results. The two biggest problems with this is that the responses to landlines and cellphones has been changing rapidly. On the otherhand people don’t move very much after age 25 so zip codes are fairly useful.
One of the two worst problems is ethnicity. High turnout in one year by blacks can be a surprise, the same for Latinos. The other is voter turnout for the leading candidates is variable and no question or measurement is a reliable predictor.
Nate Silver has his own website and funding where he uses everyone else’s survey data and tries to make his own adjustments. He is doing the right thing. So I am giving a great deal of weight to his material.