Validity
Modes of Measurement
Modes of Measurement (II)
Item Analysis & Bias
Wildcard
100
Validity entails showing that a measure reflects the desired construct without contamination from what?
Other systematically varying constructs
100
What is one of the advantages of face-to-face interviews?
* High response rates (upward of 80%) * Can correct misunderstandings * Control over question order * Rich, detailed responses * Useful for special populations * Can maintain attention and motivation of respondents longer than with paper-and-pencil questionnaires
100
What is one of the two aspects of attitude that we seek to measure?
(1) Direction (of attitude) (2) Strength intensity (of attitude)
100
What is item discrimination?
An index of how well a test differentiates people who know the material and those who do not
100
What is the Hawthorne Effect?
People performed better simply because they knew that they were being watched/observed.
200
What is a domain in reference to content validity?
The hypothetical population of all items relevant to the construct we wish to measure
200
What is one of the disadvantages associated with surveying over the Internet?
* Technology * Response rate calculation * Are you getting responses from the actual person and only the person? * Sampling…
200
What is the one characteristic that is generally positive for all types of item content (facts, attitudes & beliefs, behavior)?
Items should be specific
200
What is the range of item difficulty, and what do the values mean?
0: A very difficulty item 1: A very easy item
200
What is halo bias?
Tendency for overall positive or negative evaluations to influence ratings on specific dimensions
300
What is a predictive criterion-related validity study?
A study where the criterion (behavior of interest, performance) is measured sometime after we collect predictor data
300
What is one reason you might want to use experience sampling?
* High data depth/quality for an individual is desired * Hypothesis of interest involves time and/or patterns of behavior
300
What is the funnel principle?
In regards to item sequencing, put general items first and place more specific questions later
300
What is a ceiling effect?
When an item is too easy to discriminate between those who are high on the construct and those who are low.
300
What are the two things that we must show when demonstrating the validity of a measure?
(1) Must show that the measure (effect) is sensitive to variation in the construct (cause) (2) Must show that the measure is not sensitive to other causes (alternative explanations)
400
What is internal structure validity (that is, how do we demonstrate it)?
Showing that the structure of our measure conforms to hypotheses that we might have about the structure of that measure
400
What are two (2) disadvantages of physiological measurement?
* Questionable construct validity * Hard to separate signal from noise * Some can be potentially faked (e.g., lie detectors) * Expensive
400
What is the most common type of summated rating scale?
The Likert scale
400
What are the two (2) requirements for a test to be unbiased?
(1) Test scores are affected by the same sources of variance in the various subgroups (2) None of the irrelevant sources of variance in the test provides an unfair advantage to some of the subgroups
400
What does it mean if an item has a negative item discrimination value?
The item is a poor item; those who are lower in the construct (the lower scoring group) do better on the item than those who are higher in the construct (the higher scoring group)
500
What type of correlations should be the smallest in a multitrait-multimethod matrix?
Heterotrait-Heteromethod Correlation (different trait, different method)
500
What are two of the three (3) reasons why direct questioning is often not feasible/ideal?
* Many subjects impacted by social desirability effects * Many constructs not easy to reflect upon or verbalize * Researcher unable to phrase questions so that others can understand
500
What are two (2) of the three advantages associated with multiple-item measures?
* Reduction in complexity in conducting research * Can test whether construct is unidimensional v. multidimensional * Improved reliability and validity
500
Mike wants to compute the discrimination for an item from a Final Exam he designed for an PSY395 class he is teaching. The difficulty for the high-scoring group was .90, while the difficulty for the low-scoring group was .55. What is the discrimination for the item?
.35
500
What are the two types of associations that provide evidence of construct validity?
(1) Association between alternate measures intended to tap the same construct (2) Association between theoretically-related constructs
M
e
n
u