This post is a miniature literature review conducted for an upcoming research project on E-Democracy. The project will involve sending out an email based questionnaire to various politicians and the purpose of this literature review is, prior to the undertaking of the primary research for this project, to research and highlight techniques for questionnaire design and attitude measurement. In doing so the author hopes ensure that the primary research conducted for this topic is both useful and carried out according to the most suitable and best possible methods.

Conducting research using questionnaires is nothing new and the fact that this particular questionnaire will be conducted by email does not mean that traditional techniques should not be consulted. The nature of the area into which research will be conducted, social media, is such that an understanding of the topic on the part of the respondent cannot be guaranteed. Writing well before the rise of the internet, Henerson, Lyons Morris, & Fitz-Gibbo (1978, p.71) correctly point out that simply by receiving a questionnaire covering a certain topic, the respondent may feel that they should know something about the topic concerned - otherwise why would you be asking them. Therefore it is useful to offer the respondent the ability to choose an answer, phrased in a non-demeaning way, which identifies their lack of knowledge in the area concerned. Henerson et al. (1978 p.71) also draw attention to the possibility that the researcher can actually influence responses by drawing respondents attention to a particular issue. If a respondent does not possess strong views on the subject concerned then they can be open to being influenced by the questioning process itself. To combat this, respondents should be given the opportunity to say that they have no substantial opinions on the matter.

 How to Measure Attitudes by Henerson et al. (1978) offers a number of useful insights into critiquing questions prior to sending out a questionnaire, choosing the sequence of questions and managing a questionnaire. It is interesting that Henerson et al. (1978) put much greater emphasis on the researcher reviewing their own questions than on conducting pilot surveys, or ‘try-outs’ as they refer to them (Henerson et al 1978, p. 81). Instead researchers are advised to look revise their own questions and search for potential problems within them, for example:

- Are there any phrases likely to influence a person’s response to the question?
- Is a question asked negatively? This sometimes confuses respondents.
- Does the question encourage one particular response over another?
- Does the question permit a response that indicates a lack of knowledge/opinion without demeaning the user?
(Henerson et al 1978, p. 78)

Analysing one’s own questionnaire is no doubt important, but conducting a round of pilot questionnaires should not be overlooked - especially as potential problems can lie not just in the questions themselves but also in the method of delivery. Henerson et al. (1978 p.81) go on to mention techniques for collecting mail-based questionnaires and do raise issues that could be relevant for this project (for example the inclusion of a deadline for returning the questionnaire); however for material relevant to an email questionnaire it will be necessary to look at more recent publications.

For a project like this that intends to conduct primary research using an email questionnaire, there are obvious benefits to using the Likert scale for data collection. A technique very popular with online surveys, a quick internet search will reveal many articles detailing the benefits of this method. The main motivation for the selection of this method is described by Oppenheim in his book Questionnaire design, interviewing and attitude measurement. Oppenheim called the Likert method “the most popular scaling procedure in use today” (Oppenheim, A. 1992, p. 195) and identifies it as comparatively less laborious than other questionnaire techniques. This is a quality that makes the technique well suited to a project such as this, being as it is carried out by a single researcher. But the Likert scale is not without criticism and Oppenheim (1992) goes on to say that the method of creating a total score of numerical answers ignores the fact that the same score could be obtained in many different ways. In addition, equal answer score intervals do not necessarily allow the researcher to make assumptions about the equality of underlying attitude differences (Oppenheim, A. 1992, p. 200). When proceeding with this method there are therefore a number of concerns that this project will have to take into account.

One answer to the problem described by Henerson et al. (1978) regarding the provision of a ‘don’t know/no opinion’ response is described by Sudman & Bradburn (1983) in their book Asking Questions. The authors strongly suggest that those designing questionnaires using numerical rating scales give an odd number of categories (‘on a scale of one to five…’, ‘on a scale of one to seven…’ etc.) in order to allow for a middle point (Sudman, & Bradburn, 1983, p. 157). Taking the example of a question along the lines of ‘on a scale from one to five…’, respondents who are undecided or indifferent would be able to choose ‘three’. However Oppenheim (1992, p. 200) reminds us that responses in the middle region could be due to lack of knowledge; furthermore, the neutral point on the scale does not necessarily represent the mid-point between the two extremes of the opposing potential answers to the question. All these issues are useful additions to the thought process that will be behind the creation of the research questionnaire for this project.

A book published by the Police Foundation offers a case-study style insight into different questionnaire techniques utilised by the police force. The Likert technique is described as attractive because it is relatively simple and the nature of results based on a numerical scale lends itself well to the measurement of attitudes (Hibberd & Bennett, 1990, p. 83). The example given by Hibberd & Bennett (1990, p. 87) to demonstrate potential uses of the Likert scale by the police shows a questionnaire consisting of five questions, each with five potential answers ranging from ‘strongly disagree’ to ‘strongly agree’. This appears to be a reoccurring and widely used answer format that would also seem appropriate for use with this project. It is interesting to note that Hibberd & Bennett (1990, p. 86) consider the Liker scale to be “…clearly an involved and time consuming business…”; however this comment must be taken in the context of the book itself and what may be seen to be ‘involved and time consuming’ for a regular police officer is quite probably routine from the point of view of a sociologist.

Another strong point of the Likert scale is that closed questions are more suitable for large scale surveys. Questionnaires produced using this method have shorter response time and are easier to analyse, enabling cross-group comparisons to be made (Lewin, 2005 p. 219). In the chapter written by Lewin of the book Research Methods in Social Sciences (Lewin, 2005 p. 220) insights are offered into how to maximise the number of responses to a questionnaire. It is argued that the questions should be preceded by a brief note stating how long they will take, as well as a note at the end to ask respondents to check they’ve answered all the questions correctly and thank them for their time. Lewin (2005, p. 221) also suggests the use of online questionnaires to facilitate easier data collection and analysis. This is a possibility that could be explored for this project as it would certainly save time on data entry and the creation of graphs. However using an online survey tool could mean it would not be possible to manipulate the data to anything like the extent required in order to make a detailed analysis of the results. For this reason it seems likely that Lewin’s suggestion will not be taken on board.

When it comes to piloting the questionnaire, Iarossi (2006, p. 87) states three different methods in his book The Power of Survey Design. The two most relevant to this project are what he calls the ‘conventional’ and ‘cognitive’; his third method, the ‘behavioural’ pre-test, is simply not possible for this project as the researcher is required to monitor participants while they complete the questionnaire. The cognitive method is similar to the techniques described by Henerson et al. (1978) and involves putting oneself in the position of the respondent and undertaking the questionnaire. Iarossi (2006, pp. 91-92) suggests looking for problems such as: questions that are difficult to read, complicated instructions, technical terminology, inappropriate assumptions, sensitive content, items that respondents may consider comical, long or ambiguous questions and any issues which in the eyes of the respondent may have been overlooked. Where Iarossi (2006, p. 87) goes further than Henerson et al. (1978) is in his description of the ‘conventional’ method of what he calls ‘pre-testing’. This involves carrying out small number of pilot questionnaires and requesting feedback from the respondents regarding the questionnaire itself. A miniature discussion with a small cross-section of the respondent population could be useful in identifying problems with the questionnaire early on. For this project, both the cognitive and conventional methods will be employed.

Conclusion

Given its nature an understanding of the topic on the part of the respondent cannot be guaranteed. As such, respondents should be given the opportunity to choose an answer which identifies their lack of knowledge or opinion in the area concerned. To reflect this, the questionnaire will make use of a numerical scale with an odd number of response options; thereby allowing for a median option for those with no opinion or knowledge of the topic covered.
The Likert scale will be utilised as this will reduce response time, maximise the number of responses and simplify response data entry. However the criticisms of the method noted above should not be ignored and need to be brought into the analysis of any results obtained. When using the Likert scale, particular issues that could affect the results analysis for this project include the problem that the same the total score for each group could be obtained in many different ways. Additionally, equal answer score intervals do not necessarily allow the researcher to make assumptions about the equality of underlying attitude differences (Oppenheim, A. 1992, p. 200).

When it comes to piloting the questionnaire, both the conventional and the cognitive methods described above will be used. The researcher will ‘revise’ the questions as described by Henerson et al. (1978 p.78) and make any corrections necessary. Following this, a cross-section of the participant population should be identified and the questionnaire piloted. Using the ‘conventional method’ described by Iarossi (2006, p. 87), a dialogue with this cross-section of participants will be attempted - with the aim of identifying problems with the questionnaire before deploying it formally.

Bibliography

Henerson, E, Lyons Morris, L, & Fitz-Gibbon, C. (1978). How to Measure attitudes. Beverly Hills, USA: Sage Publications.

Hibberd, M, & Bennett, M. (1990). Questionnaire and interview surveys: a manual for police officers. London: Police Foundation.

Iarossi, G. (2006). The Power of survey design. Washignton D.C: World Bank Publications.

Lewin, C. (2005). Elementary quantitative methods. In Somekh, B & Lewin, C (Eds.), Research methods in the social sciences (pp. 215-225).London: Sage.

Oppenheim, A. (1992). Questionnaire design, interviewing and attitude measurement. Guildford, UK: Printer.

Sudman, S, & Bradburn, N. (1983). Asking questions. San Francisco: Jossey-Bass.