MARI HARRIS: The inner workings of opinion polls

The most important role of opinion polling in a free society is to provide information to the public on how other people think and feel about important issues. In this process leaders of society and political parties also learn how the public perceive them and their leadership, and issues concerning their political parties or their businesses.

The freedom to publish opinion polls is a fundamental right in most democracies, and serious pollsters abide by the codes of conduct of the SA Marketing Research Association, the European Society for Opinion & Marketing Research and the World Association of Public Opinion Research.

This obviously places a huge responsibility on the polling company, but also on the journalist reporting on the poll. Too often it happens that the technical detail of a poll is simply not included in a media report.

In my view the following should at least be mentioned when discussing the results of a poll:

• Who conducted the poll? A company like Ipsos does not only conduct polls close to an election but polls opinions throughout the year, most often for paying clients but also twice a year in a study called “Pulse of the People”, using the Ipsos Omnibus study as a vehicle.
• How were the survey questions phrased and how were they asked? In reporting it is important not to misconstrue how the questions were asked or how it was administered. It adds to clarity if the exact formulation of a question and the procedure that was followed during fieldwork are specified. For instance, in our face-to-face interviews an “imitation ballot paper” is handed to a respondent to draw his/her own cross next to their party of choice, simulating a “secret vote” process. In addition, several consistency checks are done, some computerised and others “by hand” by the checking division. In addition, backchecks are conducted on 20% of each interviewer’s work.
• Sample size. The sample size of the Ipsos Omnibus study is 3,600, which is adequate for the type of analysis we need to do. The size of the universe does not determine the size of the sample, but rather issues such as which types of analysis need to be conducted, which subgroups need to be analysed separately, and — to be practical — the availability of budget. As we draw a sample we are aware that survey research is not an exact science and therefore the margin of error needs to be specified in the technical detail. The results of every survey need to be evaluated within the margin of error of the survey. This is determined by sample size and response rate sampling methodology.
• When was the survey conducted? Keep in mind that surveys are only true for the time when the fieldwork is done. Conducting a survey is like taking a photograph of society. By conducting the Ipsos Khayabus survey regularly we can build up something akin to a movie … or at least a sequence of photographs. In election times it is important to conduct the last study before the election as close as possible to the election date, to measure accurately final changes in opinion. Closer to an election there is also time to take some possible wildcards into account: voter turnout, late swings, successful or unsuccessful election campaigns. An election is the most stringent test our work is subjected to, but it is actually quite amazing to see how close survey results can be to the actual election results.
• Interviewing methodology. Ipsos’ Omnibus interviewing is done in homes and in the preferred home languages of respondents. However, we also conduct large-scale computer-assisted telephone interviewing studies and find them to be generally accurate. Web-based interviewing is not ideal for opinion surveys in SA as the  use of the internet is too low to be representative.
• Who was surveyed and how representative was the survey? It is important to take the universe of a study into account when looking at opinion polls: is the study based on all adults in the country, on those who are registered to vote, or perhaps only on those living in metropolitan or urban areas?
• Sampling methodology. The science of survey research is coming through strongly in the way in which respondents are chosen. It is important that neither the interviewer nor the respondent should have any influence on who is chosen to be interviewed. Therefore the Ipsos marketing sciences department specifies this process of systematic random selection carefully, from the provincial level through to household level, and lastly the choice of the individual in the household.

Ipsos has developed an election turnout algorithm that we have used successfully, but these calculations are of specific value closer to an election. A few months before an election reporting on the proportion of registered voters who say that they “don’t know” who to vote for is actually useful. Strategically this information is important to political parties.

Why do people say this? Who did they vote for previously? How do they feel about the different political parties? And about political leaders? Where do they live? What is important to them? Just excluding those who say they “don’t know” would be an irresponsible way of dealing with those who do not express an opinion as by doing so you take the view that people who do not express an opinion will vote in the same way as those who did express an opinion. After working for many years with survey results I can assure you that this is not the case. The opinions of these voters deserve to be heard.

• Harris is Ipsos SSA knowledge director for public affairs.