Every now and then, the telephone will ring in the evening; on the other end of the line will be a pollster who wants to know our opinion on a variety of issues. For example, she might ask "On a scale of 1 to 10 where 1 means that you strongly disagree and 10 means that you strongly agree, what is your level of agreement with the following statement: "the sentence passed upon former Prime Minister Ehud Olmert was just". Normally this is said very fast, so I find it difficult to catch what is being asked. Also a scale of 1 to 10 gives a very large range: what's the difference between 6 and 7, or between 7 and 8?
In written questionnaires this appears as
Please indicate your level of agreement with the following statement: "The sentence passed upon former Prime Minister Ehud Olmert was just".
In written questionnaires this appears as
Please indicate your level of agreement with the following statement: "The sentence passed upon former Prime Minister Ehud Olmert was just".
- strongly disagree
- disagree
- neither disagree nor agree
- agree
- strongly agree
Note that I've halved the range and also given each value a name as opposed to a number, making it easier to choose a value.
Such a question is called a Likert item; a whole range of such questions is called a Likert scale and allows statisticians to assign a numerical value to a set of attitudes. This is what I have done in my research questionnaire with regards to concepts such as user ownership, user satisfaction and cognitive style.
There are arguments as to whether there should be an odd or even number of options. An odd number means that there is a central value which respondents can choose almost as a default, whereas an even number means that there is an uneven gap between the values (if the five values were mapped on a line, then there is an equal difference between values 1 to 2, 2 to 3, 3 to 4 and 4 to 5, but removing the central value means that the difference between 1 to 2 (strongly disagree to disagree) is not the same as the difference between 2 to 3 (disagree to agree)).
I thought that I was being clever when I decided to use four value Likert items but my mentor convinced me (or rather, told me) to use five value items.
in a computerized questionnaire, there could be a sequence of eight such questions; a lazy person could leave a finger on the '1' key and answer every question with '1' – strongly disagreeing with everything. One useful piece of advice which I obtained from an article which I read recently on building questionnaires is to randomize the options: thus for one question, the options may appear in the same order as the above example, but for another question, the options may appear in reverse order. This causes the respondent to think more deeply about her answer (it also means that the amount of time needed to answer the question will increase by about 50%). In total, there are 5! (that's 5 factorial) different ways of organizing five items: 5 X 4 X 3 X 2 X 1 = 120.
Naturally (at least for me), my first thoughts after reading about randomizing were "how am I going to implement this in my questionnaire?" .. The first thing that came to mind was that I don't actually need to store the textual representations for every question (there are at least 25 such questions in my questionnaire): if I mark a question as being Likert, then the program that presents the exam knows that it has to present the five options. Technically, this saves room in the database and makes the exam program smaller, but I doubt if this optimization is worth doing. I should explain at this stage that there is a program which 'manages' the questionnaire: questions are defined, given options and then ordered. This program exports a resource file containing all the questions and options (as well as a few other details); this resource file is then read into the questionnaire program. Thus the questionnaire program is written only once, but can be rebuilt many times, each time with a new version of the questions.
In the exam, the five numbered statements are presented; the respondent presses on a number key to signify her answer. If the statements are presented in reverse order and the respondent presses '1', then a naïve implementation might store this value ("strongly disagree'), whereas in fact the value to be stored should be '5' (strongly agree). It is a trivial matter for the program to store the real value of each statement in the (Delphi implementation details above) label's tag then save the tag's value in the data file. The real value could also be stored in an array.
The exam progam has to know that the specific question will require a random display of statements, which brings me back to the idea that I don't need to store the textual representations in the resource file.
Such a question is called a Likert item; a whole range of such questions is called a Likert scale and allows statisticians to assign a numerical value to a set of attitudes. This is what I have done in my research questionnaire with regards to concepts such as user ownership, user satisfaction and cognitive style.
There are arguments as to whether there should be an odd or even number of options. An odd number means that there is a central value which respondents can choose almost as a default, whereas an even number means that there is an uneven gap between the values (if the five values were mapped on a line, then there is an equal difference between values 1 to 2, 2 to 3, 3 to 4 and 4 to 5, but removing the central value means that the difference between 1 to 2 (strongly disagree to disagree) is not the same as the difference between 2 to 3 (disagree to agree)).
I thought that I was being clever when I decided to use four value Likert items but my mentor convinced me (or rather, told me) to use five value items.
in a computerized questionnaire, there could be a sequence of eight such questions; a lazy person could leave a finger on the '1' key and answer every question with '1' – strongly disagreeing with everything. One useful piece of advice which I obtained from an article which I read recently on building questionnaires is to randomize the options: thus for one question, the options may appear in the same order as the above example, but for another question, the options may appear in reverse order. This causes the respondent to think more deeply about her answer (it also means that the amount of time needed to answer the question will increase by about 50%). In total, there are 5! (that's 5 factorial) different ways of organizing five items: 5 X 4 X 3 X 2 X 1 = 120.
Naturally (at least for me), my first thoughts after reading about randomizing were "how am I going to implement this in my questionnaire?" .. The first thing that came to mind was that I don't actually need to store the textual representations for every question (there are at least 25 such questions in my questionnaire): if I mark a question as being Likert, then the program that presents the exam knows that it has to present the five options. Technically, this saves room in the database and makes the exam program smaller, but I doubt if this optimization is worth doing. I should explain at this stage that there is a program which 'manages' the questionnaire: questions are defined, given options and then ordered. This program exports a resource file containing all the questions and options (as well as a few other details); this resource file is then read into the questionnaire program. Thus the questionnaire program is written only once, but can be rebuilt many times, each time with a new version of the questions.
In the exam, the five numbered statements are presented; the respondent presses on a number key to signify her answer. If the statements are presented in reverse order and the respondent presses '1', then a naïve implementation might store this value ("strongly disagree'), whereas in fact the value to be stored should be '5' (strongly agree). It is a trivial matter for the program to store the real value of each statement in the (Delphi implementation details above) label's tag then save the tag's value in the data file. The real value could also be stored in an array.
The exam progam has to know that the specific question will require a random display of statements, which brings me back to the idea that I don't need to store the textual representations in the resource file.
(written on the train between Tel Aviv and Haifa)
No comments:
Post a Comment