Reliability
Reliability is primarily concerned with which of the following?
A. Whether a test measures what it claims to measure
B. How consistently a test measures
C. Whether a test predicts a diagnosis
D. Whether a test is clinically useful
B. How consistently a test measures
Validity asks primarily:
A. How consistent are scores?
B. How difficult is the test?
C. Whether interpretations of scores are supported by evidence
D. Whether scores follow a normal distribution
C. Whether interpretations of scores are supported by evidence
What is the mean and standard deviation of a typical T score?
Mean = 50; SD = 10
A test produces very similar scores every time you administer it—but consistently measures the wrong construct. Is it reliable, valid, both, or neither?
Reliable but not necessarily valid
A depression questionnaire contains 20 items designed to measure depressive symptoms. The items show strong consistency with one another. What type of reliability is being evaluated?
Internal Consistency
A reading comprehension test includes questions about vocabulary, understanding main ideas, making inferences, and interpreting passages. What type of validity evidence is most directly relevant to whether the test adequately samples the content of reading comprehension?
Content validity
What is the mean and standard deviation of a typical scaled score?
Mean = 10; SD = 3
A test has excellent sensitivity but poor specificity. What does this mean?
It is good at identifying people who have the condition but may incorrectly identify some people who do not have the condition as having it.
A client completes the same personality inventory twice, three weeks apart. The scores are highly similar. What type of reliability does this provide evidence for?
Test–retest reliability
A new anxiety questionnaire correlates strongly with established measures of anxiety. This provides evidence of:
Convergent validity
What is the mean and standard deviation of a typical standard score?
Answer: Mean = 100; SD = 15
A client scores at the 90th percentile on attention. Can you conclude that the client has 90% of the maximum possible attention ability?
No!
Two psychologists independently score the same behavioral observation. Their ratings are highly similar. What type of reliability is demonstrated?
Inter-rater reliability
A new anxiety questionnaire has only a weak relationship with a measure of mathematical reasoning. This provides evidence of:
Discriminant validity
A client receives a T score of 70. How many standard deviations above the mean is this score?
2 SD above the mean
A client's score increases from the 50th to the 60th percentile. Another client's score increases from the 80th to the 90th percentile. Can we assume these represent equivalent amounts of improvement?
No!
A test has excellent internal consistency but poor test–retest reliability. What might this tell us?
The items are working together consistently at a given administration, but scores may not be stable across time.
A test is designed to predict job performance six months after hiring. Researchers find that higher test scores are associated with better subsequent job performance. What type of validity evidence is this?
Criterion-related validity (Predictive Validity)
A client receives a standard score of 130. Approximately how many standard deviations above the mean is this?
2 SD above the mean
A client scores in the Very Low range on a computerized attention measure. The clinician concludes, “This client has ADHD.” What is the biggest psychometric/clinical reasoning problem?
The clinician has moved from a test finding to a diagnostic conclusion without sufficient evidence. The test may provide evidence of attention difficulties, but the score does not establish the cause of those difficulties or independently establish ADHD.