Reliability and Interpretation
More validity
Even more validity
Validity 4
Item Selection and Analysis
100
Reliability of a test can be increased by a) decreasing inter-item correlations b) decreasing number of test items c) increasing number of test items d) both a and c
c) increasing number of test items
100
Validity established through examination of the test itself is ________ validity. a) predictive b) concurrent c) construct d) content
d) content
100
When we ask, “Are the inferences being made from a set of test scores appropriate?” we are referring to A) internal consistency. B) face validity. C) reliability. D) validity.
D) validity.
100
_____ is concerned with how test takers perceive the attractiveness and appropriateness of a test. A) Content validity B) Construct validity C) Face validity D) Concurrent validity
C) Face validity
100
Item difficulty analysis shows a) relatedness of responses on one test to other test items on the test b) how many people chose each response for an item c) how many people chose the correct response to an item d) the proportion of people who chose each response for each item
c) how many people chose the correct response to an item
200
Changes over time in the attribute being measured are considered sources of measurement error in a) split-half reliability estimates b) alternate forms estimates c) internal consistency reliability estimates d) test-retest reliability estimates
d) test-retest reliability estimates
200
The _______ the match between expected correlations and actual correlations between test scores and behavior measures, the ______ the evidence of construct validity. a) weaker; stronger b) stronger; weaker c) stronger; stronger d) stronger; lower
c) stronger; stronger
200
If a test includes questions that are a representative sample of the material covered in a training course, the test demonstrates evidence of _____ A) reliability. B) validity based on a relations with other variables C) face validity. D) validity based on content
D) validity based on content
200
The predictive and concurrent methods provide two types of evidence for A) validity based on relations with a construct B) face validity C) validity based on relations with a criteria D) validity based on content
C) validity based on relations with a criteria
200
Suppose that for a question, 15% of people selected response "a", 20% selected response "b", 30% selected response "c" (which is the correct response), and 35% selected response "d”. Which item analysis data do you have for that item? a) item difficulty analysis data b) item discrimination analysis data c) item distractor analysis data d) both a and c
d) both a and c
300
If a test has a reliability coefficient of 0.00 and a standard error of 0.00, the test is a) reliable and accurate b) unreliable but accurate c) unreliable and inaccurate d) reliable but inaccurate
b) unreliable but accurate
300
When measures of unrelated constructs do not correlate with each other, _________ is demonstrated. a) divergent validity b) convergent validity c) discriminant validity d) concurrent validity
c) discriminant validity
300
When test scores used to select employees are systematically related to a scores on a job performance appraisal instrument, the test is demonstrating evidence of validity based on its A) content B) relations with another construct C) reliability D) relations with a criteria
D) relations with a criteria
300
The latest Standards for Educational and Psychological Testing recognize five sources of evidence of validity. Which of the following is NOT one of those sources? A) Evidence based on test content B) Evidence based on face validity C) Evidence based on response processes D) Evidence based on relations to other variables
B) Evidence based on face validity
300
A distractor that is chosen significantly fewer times than would be expected by chance a) lowers the difficulty of the item b) increases the difficulty of the item c) does not affect item difficulty d) reflects an item not in the content domain
a) lowers the difficulty of the item
400
If a person's true score is 110 on a test with a standard error of measurement of 3.7 and a mean of 100, we would expect 95% of the person's test scores to fall within a) 102.75 - 117.25 b) 92.75 - 107.25 c) 90.75 - 120.25 d) 100-110
a) 102.75 - 117.25
400
Content validity and construct validity are a) mutually exclusive b) very different c) highly related d) both a and b
c) highly related
400
Which of the following common job interview questions is most likely to be able to demonstrate content based evidence for validity? A) What are your greatest strengths? B) What is your favorite sport to watch on TV? C) Tell me about a time when you had to work on a team. D) Tell me about a time when you really enjoyed a movie.
C) Tell me about a time when you had to work on a team.
400
Correlating test scores with some measure of success is the same thing as correlating test scores with some a) concurrent validation b) sample referenced norm c) other test scores d) criterion
d) criterion
400
Item difficulty refers to a) item complexity b) item obscurity c) the number of people who answer an item correctly d) all of the above
c) the number of people who answer an item correctly
500
In criterion-referenced testing, our principle concern is with the reliability of a) test scores b) individual test items c) decisions d) reliability is not a concern in criterion-referenced testing
c) decisions
500
A validity study is conducted in the workplace, where many workers have had many years of on-the-job experience. This study is probably a _______ validity study. a) predictive b) concurrent c) construct d) content
b) concurrent
500
Two general strategies for assessing criterion-related validity are a) content and construct validity b) predictive and construct validity c) concurrent and predictive validity d) construct and concurrent validity
c) concurrent and predictive validity
500
A ________ represents some action which you take on the basis of your _______. a) prediction; decision b) decision; prediction c) criterion; predictor d) directive; assessment
b) decision; prediction
500
When everyone answers an item incorrectly the percent or proportion passing (p) value will be a) 0.0 b) 1.0 c) between 0.0 and 1.0 d) -1.0
a) 0.0
M
e
n
u