Test Construction, Midterm Review 4

Reliability and Interpretation

100

Reliability of a test can be increased by a) decreasing inter-item correlations b) decreasing number of test items c) increasing number of test items d) both a and c

c) increasing number of test items

200

Changes over time in the attribute being measured are considered sources of measurement error in a) split-half reliability estimates b) alternate forms estimates c) internal consistency reliability estimates d) test-retest reliability estimates

d) test-retest reliability estimates

300

If a test has a reliability coefficient of 0.00 and a standard error of 0.00, the test is a) reliable and accurate b) unreliable but accurate c) unreliable and inaccurate d) reliable but inaccurate

b) unreliable but accurate

400

If a person's true score is 110 on a test with a standard error of measurement of 3.7 and a mean of 100, we would expect 95% of the person's test scores to fall within a) 102.75 - 117.25 b) 92.75 - 107.25 c) 90.75 - 120.25 d) 100-110

a) 102.75 - 117.25

500

In criterion-referenced testing, our principle concern is with the reliability of a) test scores b) individual test items c) decisions d) reliability is not a concern in criterion-referenced testing

c) decisions

More validity

100

Validity established through examination of the test itself is ________ validity. a) predictive b) concurrent c) construct d) content

d) content

200

The _______ the match between expected correlations and actual correlations between test scores and behavior measures, the ______ the evidence of construct validity. a) weaker; stronger b) stronger; weaker c) stronger; stronger d) stronger; lower

c) stronger; stronger

300

When measures of unrelated constructs do not correlate with each other, _________ is demonstrated. a) divergent validity b) convergent validity c) discriminant validity d) concurrent validity

c) discriminant validity

400

Content validity and construct validity are a) mutually exclusive b) very different c) highly related d) both a and b

c) highly related

500

A validity study is conducted in the workplace, where many workers have had many years of on-the-job experience. This study is probably a _______ validity study. a) predictive b) concurrent c) construct d) content

b) concurrent

Even more validity

100

When we ask, “Are the inferences being made from a set of test scores appropriate?” we are referring to A) internal consistency. B) face validity. C) reliability. D) validity.

D) validity.

200

If a test includes questions that are a representative sample of the material covered in a training course, the test demonstrates evidence of _____ A) reliability. B) validity based on a relations with other variables C) face validity. D) validity based on content

D) validity based on content

300

When test scores used to select employees are systematically related to a scores on a job performance appraisal instrument, the test is demonstrating evidence of validity based on its A) content B) relations with another construct C) reliability D) relations with a criteria

D) relations with a criteria

400

Which of the following common job interview questions is most likely to be able to demonstrate content based evidence for validity? A) What are your greatest strengths? B) What is your favorite sport to watch on TV? C) Tell me about a time when you had to work on a team. D) Tell me about a time when you really enjoyed a movie.

C) Tell me about a time when you had to work on a team.

500

Two general strategies for assessing criterion-related validity are a) content and construct validity b) predictive and construct validity c) concurrent and predictive validity d) construct and concurrent validity

c) concurrent and predictive validity

Validity 4

100

_____ is concerned with how test takers perceive the attractiveness and appropriateness of a test. A) Content validity B) Construct validity C) Face validity D) Concurrent validity

C) Face validity

200

The predictive and concurrent methods provide two types of evidence for A) validity based on relations with a construct B) face validity C) validity based on relations with a criteria D) validity based on content

C) validity based on relations with a criteria

300

The latest Standards for Educational and Psychological Testing recognize five sources of evidence of validity. Which of the following is NOT one of those sources? A) Evidence based on test content B) Evidence based on face validity C) Evidence based on response processes D) Evidence based on relations to other variables

B) Evidence based on face validity

400

Correlating test scores with some measure of success is the same thing as correlating test scores with some a) concurrent validation b) sample referenced norm c) other test scores d) criterion

d) criterion

500

A ________ represents some action which you take on the basis of your _______. a) prediction; decision b) decision; prediction c) criterion; predictor d) directive; assessment

b) decision; prediction

Item Selection and Analysis

100

Item difficulty analysis shows a) relatedness of responses on one test to other test items on the test b) how many people chose each response for an item c) how many people chose the correct response to an item d) the proportion of people who chose each response for each item

c) how many people chose the correct response to an item

200

Suppose that for a question, 15% of people selected response "a", 20% selected response "b", 30% selected response "c" (which is the correct response), and 35% selected response "d”. Which item analysis data do you have for that item? a) item difficulty analysis data b) item discrimination analysis data c) item distractor analysis data d) both a and c

d) both a and c

300

A distractor that is chosen significantly fewer times than would be expected by chance a) lowers the difficulty of the item b) increases the difficulty of the item c) does not affect item difficulty d) reflects an item not in the content domain

a) lowers the difficulty of the item

400

Item difficulty refers to a) item complexity b) item obscurity c) the number of people who answer an item correctly d) all of the above

c) the number of people who answer an item correctly

500

When everyone answers an item incorrectly the percent or proportion passing (p) value will be a) 0.0 b) 1.0 c) between 0.0 and 1.0 d) -1.0

a) 0.0