Once the decision has been made to use a particular technique, the following questions should be considered before designing the questionnaire.
(a) What exactly do we want to measure according to the objectives formulated and variables identified?
(b) Of whom will we ask questions and what techniques will we follow?
(c) Are the respondents mainly illiterate?
(d) How large is the sample that will be interviewed?
The above questions are raised to ensure that the contents of the questionnaire are relevant to the
(i) goals of the study and (ii) individual respondents.
In a sample survey, it is customary to employ structured interview rather than unstructured ones, since the former lend themselves better to quantitative analysis and the latter create serious data processing difficulties, particularly if the sample is large. A large interview is one that employs a standard questionnaire (or interview schedule) to ensure that all respondents are asked exactly the same set of questions in the same sequence. The exact wording of each question to the respondent. This is also true for survey when information are sought by mail questionnaire.
The questioning of persons is an imposition and invasion of privacy, so it should not be surprising that some persons do not respond as we expect. In dealing with this problem, there are several problems to remember with respect to the designing and wording of the questions. We discuss below a few of these points.
(a) Use simple language
(b) Start with an interesting and easy question
(c) Use short language
(d) Avoid double-barrel questions
(e) Avoid ambiguous wording of questions
(f) Avoid leading questions
(g) Avoid questions with vague words
(h) Avoid presuming questions
(i) Avoid hypothetical questions
(j) V questions that involve memory
(k) Avoid sensitive or embarrassing questions
(l) Maintain sequencing of the questions
Thursday, December 17, 2009
Questionnaire and its Construction
Interview schedule or the self-administered questionnaires are probably the most important and commonly used research instruments for data collection. Construction of these tools thus occupies a central position in any scientific investigation. Before we discuss this issue, we distinguish between a questionnaire and a schedule.
Questionnaire: A questionnaire is an instrument that is generally mailed or handed over to the respondent and filled in by term with no help from the interviewer or any other person.
Schedule: A schedule also known as an interview schedule, is an instrument that is not given to the respondents but is filled in by interviewer himself who reads the questions to the respondents and records the answers as provided by the respondents.
Before elaborating the steps involved in designing a questionnaire, we need to know the types of questions used in questionnaire. Depending on how questions are asked and recorded, we can distinguish two types of questions: Open-ended questions and closed questions. A question that is formulated without pre-determined response is an open-ended question. An open-ended question permits free response that should be recorded in the respondents own words. Here the respondents are not provided with any possible answer to choose from. Such questions are useful to obtain information on:
• Facts with the researcher is not very familiar or difficult to recollect;
• Opinion, attitude and suggestions of informants etc;
• Sensitive issues
A closed question on the other hand, offers a list of possible options or answers with alternatives, from which the respondents must choose. Closed questions are useful if the range of possible responses is known. In practice, a questionnaire usually has a combination of open-ended and closed, arranged in such a way that the discussion follows as naturally as possible. Data processing is much easier in terms of time and resources when the interview schedule is structured and closed.
Open-ended question may provided valuable new insight into the problem relating to the issues not previously thought of at the planning stage. Closed questions, on the other hand, have the advantages of providing quick answer. The analysis is also easier with the closed question.
The most important disadvantages of open-ended question is that it may lead to distorted information when the interviewers are unskilled. Analysis is also time consuming with such question. Closed question are unsuitable for face-to-face interview. Options provided in the questionnaire may lead to bias and some important information may be missed if it is not asked.
In practice, a questionnaire usually has a combination of open-ended and closed questions, arranged in such way that the discussion follows as naturally as possible. For open-ended questions, multiple responses is usually allowed. The interviewers in such cases will not be in a hurry to skip to the next question. He should be trained to wait for additional answer that the respondent may provide. For closed question too, the interviewer must choose to tick the most appropriate answer(s).
A question may again be either pre-coded or post-coded. A pre-coded scheme may be followed for closed questions such s for sex. Thus, you may designates the male by a numerical code “1” and the female by “2”. You may do the other way round ‘1” for female and “2” for male, but the former is the most common practice.
When possible answers cannot be exactly comprehended in pre-coding scheme, the question is kept open and after getting the responses from the field, the answer are arranged in logical order and then numerical codes are assigned to each selected response.
Questionnaire: A questionnaire is an instrument that is generally mailed or handed over to the respondent and filled in by term with no help from the interviewer or any other person.
Schedule: A schedule also known as an interview schedule, is an instrument that is not given to the respondents but is filled in by interviewer himself who reads the questions to the respondents and records the answers as provided by the respondents.
Before elaborating the steps involved in designing a questionnaire, we need to know the types of questions used in questionnaire. Depending on how questions are asked and recorded, we can distinguish two types of questions: Open-ended questions and closed questions. A question that is formulated without pre-determined response is an open-ended question. An open-ended question permits free response that should be recorded in the respondents own words. Here the respondents are not provided with any possible answer to choose from. Such questions are useful to obtain information on:
• Facts with the researcher is not very familiar or difficult to recollect;
• Opinion, attitude and suggestions of informants etc;
• Sensitive issues
A closed question on the other hand, offers a list of possible options or answers with alternatives, from which the respondents must choose. Closed questions are useful if the range of possible responses is known. In practice, a questionnaire usually has a combination of open-ended and closed, arranged in such a way that the discussion follows as naturally as possible. Data processing is much easier in terms of time and resources when the interview schedule is structured and closed.
Open-ended question may provided valuable new insight into the problem relating to the issues not previously thought of at the planning stage. Closed questions, on the other hand, have the advantages of providing quick answer. The analysis is also easier with the closed question.
The most important disadvantages of open-ended question is that it may lead to distorted information when the interviewers are unskilled. Analysis is also time consuming with such question. Closed question are unsuitable for face-to-face interview. Options provided in the questionnaire may lead to bias and some important information may be missed if it is not asked.
In practice, a questionnaire usually has a combination of open-ended and closed questions, arranged in such way that the discussion follows as naturally as possible. For open-ended questions, multiple responses is usually allowed. The interviewers in such cases will not be in a hurry to skip to the next question. He should be trained to wait for additional answer that the respondent may provide. For closed question too, the interviewer must choose to tick the most appropriate answer(s).
A question may again be either pre-coded or post-coded. A pre-coded scheme may be followed for closed questions such s for sex. Thus, you may designates the male by a numerical code “1” and the female by “2”. You may do the other way round ‘1” for female and “2” for male, but the former is the most common practice.
When possible answers cannot be exactly comprehended in pre-coding scheme, the question is kept open and after getting the responses from the field, the answer are arranged in logical order and then numerical codes are assigned to each selected response.
Errors in Sample Survey
There is no denying of the fact that the whole sampling procedure is liable to varying degree of errors at all stages of its operation. The total errors involved in such operation can broadly be classified as Sampling Error and Non-Sampling Error.
The sampling error is always assessed with reference to the value of the population parameter. Whatever may be degree of cautiousness in selecting a sample from a population; there will always be a difference between the population value and its corresponding estimates. This difference is attributable to sampling and is termed sampling error.
A sampling error is usually measured in terms of the standard and no other reasons can be attributed to cause such error, is called sampling error.
A sampling error is usually measured in terms of the standard error for a particular statistic (e.g. mean, proportion, ratio etc.). If a sample of respondents had been selected as a simple random sample, it would have been possible to use straightforward formula for computing standard error as square root of the sample variance. In many occasions, we use more complex designs than SRS and consequently, measurement of standard error also warrants more complex formula. N this text, we have elaborated the discussion on estimating standard error in this case of some commonly used designs.
In practice, every operation of survey is a potential source of non-sampling errors. These are the results of mistakes made in implementing data collection and data processing, such as failure to locate and interview the correct household, misunderstanding of the questions on the part of either the respondent or the interviewer and the data entry error. While this suggests a multiplicity of sources of non-sampling errors, we can group them in to four broad categories as follows:
1. Non-Response Error
2. Measurement Error
3. Randomized Response
4. Processing Error
01. Non-Response Error: Element of non-response refers to the situation when no data are possible to be collected for one or more of the elements selected for the survey. An element may be an individual (respondent) or any other unit, such as a household. The reasons for such non-response may be either due to the fact that
(i) The respondent could not be contacted
(ii) They could be contacted but they refused to be interviewed or that
(iii) They were contacted and provided data but the elicited data were dubious in quality and thus were excluded from data processing.
Broadly speaking, non-response rates are relatively more pronounced in
(i) Mail surveys
(ii) Surveys dealing with sensitive issues and
(iii) Interview surveys with in adequate trained interviewers.
Since non-response can hardly be avoided, our attempt will be to achieve as low non-response rate as possible. Here are some measures that are likely to contribute towards achieving low non-response rate
• Making the audience survey oriented
• Imparting training to the survey statisticians
• Imparting training to the survey interviewers
• Call backs and reminders
• Sub-sampling the non-respondents.
If the people have a positive attitude and appreciation of the use of statistics, they are likely to co-operate to a large extent, thereby contributing to the response rate.
02. Measurement Error: By measurement, we understand determining the ‘true’ value of the variable or category of an attribute of interest. If we fall to do so, we encounter measurement error.
The potential sources of measurement error, among others are,
(a) Failure to understand the questions by the respondents
(b) The respondents are unaware of the true answer to the question
(c) The question are biased.
03. Randomized Response: In many cultures, one reason why people do not provide true response or altogether refuse to respond is their sensitivity to the question asked. Imagine a survey designed to estimate the proportion of persons who view X-rated video, addicted to marijuana, indulged in immoral activities, committed crime or ever have evaded taxes. A person, who does not view X-rated video, say will probably respond with a ‘No’. The response of a viewer, however, could be ‘Yes’ or ‘No’ or an outright refusal to the question. This is true for the other cases as well. Thus, direct questioning on these may introduce bias in the results.
A reasonable precaution is to treat the response of the individuals confidently and assure them that the response cannot be tracked back to the respondents. Such assurance can be given when the data are collected by personal interview or by means of a small questionnaire but not by any means where the person interviewed may feel alarmed, embarrassed or afraid of revealing the truth to the interviewer. Randomized response method have been developed to cope with this problem of ‘evasive’ bias. The method was introduced by Walner (1965) that aims at encouraging truthful response by disassociating the question from the response. Walner showed that it is possible to estimate the proportion of P of individuals who belong to some sensitive category by means of a survey using personal interview without the respondent revealing his or her personal status with respect to the question asked. The objective is to encourage truthful answers while fully preserving confidently.
04. Processing Error: After careful consideration of the objectives of the study, one must plan for processing of the data. Such a plan assures the researcher that all the information has indeed been collected in a standardized way. The processing of the data involves several steps, a few of which are: sorting, categorizing, coding and compiling. At any stages of these operations, errors may creep in.
Coding is a special kind of measurement operation. Its purpose is to classify the response to a question into meaningful and mutually exclusive categories, so as to bring out their essential pattern. These responses are then labeled by some numerical (1, 2 etc.) or letter codes (M,S etc.) for ease of computer entry. The coding operation is typically carried manually by special coding clerk. Experience has shown that this operation is susceptible to error. The verification of this coding error may be done in several ways: A lucid description is given by Dalenius (1985). He also presents a automatic coding scheme by computer, which is helpful to reduce the coding error rate.
Sorting data refers to grouping and categorizing data according to some common characteristics. Sorting operation may be effected either through computer or manually. Manual sorting is used when the size of the sample is small. Sorting error is more severe for manual sorting. Tallying process is a part of manual sorting, which is liable to larger margin of error.
When using computer, you may run the risk of compilation error if you fail to:
• Choose an appropriate computer program
• Ensure correct entry of the data
• Choose appropriate verification or validation program
• Use right programming
Sampling Error
The sampling error is always assessed with reference to the value of the population parameter. Whatever may be degree of cautiousness in selecting a sample from a population; there will always be a difference between the population value and its corresponding estimates. This difference is attributable to sampling and is termed sampling error.
A sampling error is usually measured in terms of the standard and no other reasons can be attributed to cause such error, is called sampling error.
A sampling error is usually measured in terms of the standard error for a particular statistic (e.g. mean, proportion, ratio etc.). If a sample of respondents had been selected as a simple random sample, it would have been possible to use straightforward formula for computing standard error as square root of the sample variance. In many occasions, we use more complex designs than SRS and consequently, measurement of standard error also warrants more complex formula. N this text, we have elaborated the discussion on estimating standard error in this case of some commonly used designs.
Non-Sampling Error
In practice, every operation of survey is a potential source of non-sampling errors. These are the results of mistakes made in implementing data collection and data processing, such as failure to locate and interview the correct household, misunderstanding of the questions on the part of either the respondent or the interviewer and the data entry error. While this suggests a multiplicity of sources of non-sampling errors, we can group them in to four broad categories as follows:
1. Non-Response Error
2. Measurement Error
3. Randomized Response
4. Processing Error
01. Non-Response Error: Element of non-response refers to the situation when no data are possible to be collected for one or more of the elements selected for the survey. An element may be an individual (respondent) or any other unit, such as a household. The reasons for such non-response may be either due to the fact that
(i) The respondent could not be contacted
(ii) They could be contacted but they refused to be interviewed or that
(iii) They were contacted and provided data but the elicited data were dubious in quality and thus were excluded from data processing.
Broadly speaking, non-response rates are relatively more pronounced in
(i) Mail surveys
(ii) Surveys dealing with sensitive issues and
(iii) Interview surveys with in adequate trained interviewers.
Since non-response can hardly be avoided, our attempt will be to achieve as low non-response rate as possible. Here are some measures that are likely to contribute towards achieving low non-response rate
• Making the audience survey oriented
• Imparting training to the survey statisticians
• Imparting training to the survey interviewers
• Call backs and reminders
• Sub-sampling the non-respondents.
If the people have a positive attitude and appreciation of the use of statistics, they are likely to co-operate to a large extent, thereby contributing to the response rate.
02. Measurement Error: By measurement, we understand determining the ‘true’ value of the variable or category of an attribute of interest. If we fall to do so, we encounter measurement error.
The potential sources of measurement error, among others are,
(a) Failure to understand the questions by the respondents
(b) The respondents are unaware of the true answer to the question
(c) The question are biased.
03. Randomized Response: In many cultures, one reason why people do not provide true response or altogether refuse to respond is their sensitivity to the question asked. Imagine a survey designed to estimate the proportion of persons who view X-rated video, addicted to marijuana, indulged in immoral activities, committed crime or ever have evaded taxes. A person, who does not view X-rated video, say will probably respond with a ‘No’. The response of a viewer, however, could be ‘Yes’ or ‘No’ or an outright refusal to the question. This is true for the other cases as well. Thus, direct questioning on these may introduce bias in the results.
A reasonable precaution is to treat the response of the individuals confidently and assure them that the response cannot be tracked back to the respondents. Such assurance can be given when the data are collected by personal interview or by means of a small questionnaire but not by any means where the person interviewed may feel alarmed, embarrassed or afraid of revealing the truth to the interviewer. Randomized response method have been developed to cope with this problem of ‘evasive’ bias. The method was introduced by Walner (1965) that aims at encouraging truthful response by disassociating the question from the response. Walner showed that it is possible to estimate the proportion of P of individuals who belong to some sensitive category by means of a survey using personal interview without the respondent revealing his or her personal status with respect to the question asked. The objective is to encourage truthful answers while fully preserving confidently.
04. Processing Error: After careful consideration of the objectives of the study, one must plan for processing of the data. Such a plan assures the researcher that all the information has indeed been collected in a standardized way. The processing of the data involves several steps, a few of which are: sorting, categorizing, coding and compiling. At any stages of these operations, errors may creep in.
Coding is a special kind of measurement operation. Its purpose is to classify the response to a question into meaningful and mutually exclusive categories, so as to bring out their essential pattern. These responses are then labeled by some numerical (1, 2 etc.) or letter codes (M,S etc.) for ease of computer entry. The coding operation is typically carried manually by special coding clerk. Experience has shown that this operation is susceptible to error. The verification of this coding error may be done in several ways: A lucid description is given by Dalenius (1985). He also presents a automatic coding scheme by computer, which is helpful to reduce the coding error rate.
Sorting data refers to grouping and categorizing data according to some common characteristics. Sorting operation may be effected either through computer or manually. Manual sorting is used when the size of the sample is small. Sorting error is more severe for manual sorting. Tallying process is a part of manual sorting, which is liable to larger margin of error.
When using computer, you may run the risk of compilation error if you fail to:
• Choose an appropriate computer program
• Ensure correct entry of the data
• Choose appropriate verification or validation program
• Use right programming
Pilot Survey
Pilot survey is a small-scale replica of the main survey, which goes beyond pre-tests by linking documents and procedure, which have already been individually pre-tested. It is a systematic and integrated inquiry in the form of a miniature or preliminary survey. A pilot survey is often compared with a theatrical dress rehearsal, which, before a final theater is staged. The pilot survey is designed to help the planners to clarify many of the problems left unsolved by pre-test operations. A pilot survey nearly always results in considerable improvement to the survey documents leading to a general increase in the efficiency of the survey. A well-planned pilot survey also offers an opportunity to the researchers on the basis of its results, as to whether the main survey still worth to carry out. Many standard designs require some prior or supplementary knowledge of the population elements to allow estimation based on sample observations. Pilot survey is a neat solution to the problem in such instances. In post-stratification, prior knowledge of the stratum, weight is needed for the estimation purpose. In sample size determination with SRS, we need some quessed values of p and s.
Pilot study might provide us with such values to determine. Since the pilot survey is the researchers last safe guard against the possibility that the main inquiry may be ineffective, its size and designed should be also so planned that it fulfills the above functions and the sample should ideally be of a comparable structure to that of the main survey.
Pilot study might provide us with such values to determine. Since the pilot survey is the researchers last safe guard against the possibility that the main inquiry may be ineffective, its size and designed should be also so planned that it fulfills the above functions and the sample should ideally be of a comparable structure to that of the main survey.
Acceptance Sampling
Acceptance sampling is an important field of statistical quality control that was popularized by Dodge and Roming (1959) and originally applied by the U.S. military to the testing of bullets during World War II. If every bullet were tested in advance, no bullet would be left to ship. If, on the other hand, none were tested, malfunctions might occur in the field of battle, with potentially disastrous results.
Dodge reasoned that a sample should be picked at random from the lot and on the basis of information that was yielded by the sample, a decision should be made regarding the disposition of the lot. The process is called lot acceptance sampling or just acceptance sampling.
Acceptance sampling is the task of taking samples from the lot and decides whether the lot is to be accepted or rejected, on the basis of evidence provided by inspection of samples drawn at random. If the average quality level is indicated by the sample, the lot is accepted, if not the lot is rejected.
The main purpose of acceptance sampling is to decide whether or not the lot likely to be acceptable, not to estimate the quality of the lot. The plan merely accepts or rejects the lots.
Acceptance sampling is employed when one or several of the following hold:
• Testing is destructive.
• The cost of 100% inspection is very high.
• 100% inspection takes too long.
Dodge reasoned that a sample should be picked at random from the lot and on the basis of information that was yielded by the sample, a decision should be made regarding the disposition of the lot. The process is called lot acceptance sampling or just acceptance sampling.
Acceptance sampling is the task of taking samples from the lot and decides whether the lot is to be accepted or rejected, on the basis of evidence provided by inspection of samples drawn at random. If the average quality level is indicated by the sample, the lot is accepted, if not the lot is rejected.
The main purpose of acceptance sampling is to decide whether or not the lot likely to be acceptable, not to estimate the quality of the lot. The plan merely accepts or rejects the lots.
Acceptance sampling is employed when one or several of the following hold:
• Testing is destructive.
• The cost of 100% inspection is very high.
• 100% inspection takes too long.
Subscribe to:
Posts (Atom)