Midterm Grading Guide
Survey Research (POS 4931)
Fall, 2009
A perfect essay would have
incorporated all these points:
1.
Question-wording and format is a potential
source of survey error (TSE) and needs to be considered carefully in
survey
design.
2.
Zaller & Feldman have taught us that many
respondents do not have preformed attitudes on questions asked in
surveys and
thus construct an answer based on the “considerations” in their mind at
the
time of the survey. So question form and wording may “cue” respondents
to
select answers by putting some considerations in mind and hiding others.
3.
Question A is a Likert-style question which
presents a statement and gives respondents a choice of four levels of
agreement. This may have several consequences:
a.
This question presents only one side of the
argument—the isolationist side—and thus may dispose respondents to
accept the
question
b.
There may be a general tendency for respondents
to acquiesce to the interviewer by agreeing due to the sense that this
is a
test or that the interviewer is asking him/her
to agree with a
statement of
opinion.
c.
This tendency to agree may be magnified in
cultures that prize harmony and encourage consensus as Javeline found
in
Kazakhastan (give details).
d.
All these raise the possibility of measurement
error.
4.
Question B is a balanced, forced choice question
with a category scale for response options. This design has several
potential
consequences for measurement error:
a.
A “balanced” design cues respondents to two
sides of an issue, raising an additional consideration, and thus is
more likely
to produce a valid answer.
b.
A “forced choice" design makes it less likely the
respondent will acquiesce or exhibit social desirability because it
forces
respondents to choose between two competing alternatives, something
that
Javeline showed reduced social desirability and acquiescence.
c.
This question gives respondent a greater array
of choices and probably encourages a wide range of responses.
d.
While all these should reduce error, it’s
possible that Question B’s length and complexity will have undesirable
effects:
i.
Some respondents may “hide” by picking
the
middle option.
ii.
Some respondents may suffer fatigue and either
fail to answer or answer carelessly
Some of you argued that the absence of a
“don’t know” or
middle option in Question A was an advantage or problem and that its
presence
in Question B made the question better. The jury is still out on that
issue
because adding a middle option or don’t know can have both good and bad
effects
in terms of measurement error.
In terms of assessing your essay grade, I
used the following scale:
Points
|
Letter Grade
|
27-30
|
A
|
24-26
|
B
|
21-23
|
C
|
18-20
|
D
|
0-17
|
E
|