Why the Polls Are Apart

By HUMPHREY TAYLOR*

EVERY time the polls disagree, their methods are exposed to a critical examination. The sampling techniques, the control and administra- tion of interviews, and the wording of questions are all discussed. But the most persistent of all the pollsters' worries is seldom mentioned. This is 'differential turn-out' or 'differential abstention' and it probably explains more polling disasters than all the other factors combined.

To make a perfect voting forecast, a poll must be based on a good sample, competently ques- tioned by interviewers of scrupulous integrity. The electors interviewed must also be able and willing to say how they will vote. Fortunately, these necessary conditions can be satisfied most of the time—otherwise political polling would not have survived.

With care and attention to detail, one can reduce to a minimum, but never entirely eliminate, the possibility of getting a bad sample. There is always an element of chance, even when the best sampling methods are used. On the other hand, a badly drawn sample might, with luck, give a good result and be called a good sample.

Contrary to popular belief, a sample cannot be judged on its size alone. The biggest poll ever mounted, the Literary Digest presidential poll in 1936, • which used a sample of over two million, produced the worst forecast on record-

* The author is a director of National Opinion Polls Ltd. a 19 per cent error on President Roosevelt's share of the vote (or nearly 40 per cent error on the 'gap,' on which British polls are often judged). It was 'a thoroughly bad sample, drawn from telephone directories. Its huge size did nothing to make it better.

The local elections in May this year are almost certainly another example of a big but bad sample of the whole country. They indicated a 7 per cent swing to the Conservatives since last October, the equivalent of a 12 per cent lead in the polls. It is not credible that all the opinion polls could be this far out. Because a much higher proportion of Tory supporters than of Labour supporters turned out to vote, the results did not reflect the position in the country.

Dishonest and incompetent interviewing can be reduced to a minimum by supervision and control. Only 3 per cent or 4 per cent of electors in Britain adamantly refuse to be interviewed. A few more may not truthfully identify the party they support. However, these two groups usually seem to form an approximate microcosm of the whole electorate, and do not seriously upset a poll's accuracy.

In every poll before every election, the pro- portion of electors who say they are going to vote is higher than the proportion who actually do so. Some people say they arc gOing to vote but, on the day, abstain. There is a reluctance to admit that one is unlikely to vote as this appears to be failure in one's duty as a citizen. There

are other people who genuinely expect to vote, but for one reason or another do not do so. If the absention rate varies from one party to another a poll with a perfect sample and a perfect measure of party support will still produce an inaccurate forecast.

Fortunately (for the pollsters) differential turn-out on any scale is the exception rather than the rule. It is not normally necessary to incor- porate any correction for turn-out in the voting forecast. But it is always a threat, and was almost certainly a major cause of the polls' two biggest upsets in this country in recent years, at South Dorset in 1962, and at Leyton in January, 1965.

The lower the overall turn-out, the greater the possibility of differential turn-out affecting the result. But there are many examples of polls producing accurate forecasts although turn-out was very low. The NOP surveys of three boroughs at the LCC elections in 1961, and of

four boroughs at the any elections in 1964, were as accurate as any poll could hope to be, in spite of the very low turn-out.

This, then, is the problem. One solution is to ask a number of questions to measure the likeli- hood of each elector actually voting. But although the replies to these questions may indicate that one party's supporters are more likely to abstain than are the supporters of another party, it is difficult to know how great a 'correction' to make to the poll's findings on this account. Because of this difficulty, both Gallup and NOP made their final forecasts of the 1964 election without apply- ing any correction. In 1955, one of the opinion polls (Gallup) did apply a turn-out 'correction,' which made an almost perfect forecast less accurate.

Both NOP and Gallup put a further question to all those who are undecided or who refuse to reply to the straight voting question. They are asked which party they would be 'mast inclined' to support. One might expect that these 'in- cliners' were more likely to abstain than the committed voters, and conclude that they should be excluded from the voting tables. But by- elections during the last few years have taught us that by counting these 'incliners' as voters, we can improve our forecasts substantially in four out of five surveys.

One diniculty with 'turn-out' questions is that they appear only to operate effectively in a real. as distinct from a hypothetical, voting situation. And even when the poll data clearly shows that many supporters of one party are likely to abstain, any correction that the pollster makes will expose him to the charge of manipulation. For the size of the correction will be, in part, a matter of individual judgment and not cold science.

Methods of estimating the differential turn-out will continue to improve. But there are some factors which no poll, even one taken on the cve of the election, can measure. It is not the pollster's job to predict the weather or the efficiency of party organisation on polling day, h,,th of which can affect the result.

The pollsters measure voting intention, and they believe that their measure is accurate. But behaviour sometimes differs from intention. The replies to some of the questions in the latest NOP survey suggest that at the present time, a correction for turn-out might make the poll a more accurate forecast or voting behaviour. Many electors are dissatisfied with the Government and think that it has made a bad job of handling vital issues such as rising prices and home loans, but still believe that if there were a general election tomorrow, they would vote Labour. Experience suggests that many of them might abstain.