Test Construction, Midterm Review 3

Potpourri

100

True scores and errors are assumed to be a) highly correlated b) not highly correlated c) uncorrelated d) negatively correlated

c) uncorrelated

200

Which one of the following processes can we use to answer the question: "If a student scores 85 on the academic abilities test, what course grade would we expect the student to receive?" A) Linear regression B) Multiple regression C) Correlation D) Test of significance

A) Linear regression

300

Which one of the following criteria used in educational settings is subjective? A) Grade point average (GPA) B) Instructors' letters of recommendation C) Number of dismissals or withdrawals D) Number of courses completed

B) Instructors' letters of recommendation

400

What is a factor? A) The concept measured by a subscale B) The underlying commonalities of tests or test questions that measure a construct C) An environmental force that changes interpretation of test scores D) Evidence that a test has construct validity

B) The underlying commonalities of tests or test questions that measure a construct

500

If an item analysis suggests that an item in a test is poor, and if that item is removed from the test, the reliability of the shorter test is likely to be a) lower than the original reliability coefficient b) reliability would not change, but validity would increase c) higher than the original reliability coefficient d) none of the above

c) higher than the original reliability coefficient

More reliability

100

2. Evidence of reliability indicates that A) test scores will likely be consistent across repeated measurements. B) the test is being used properly. C) the test is measuring what it is designed to measure. D) a test possesses an important, but not essential, property.

A) test scores will likely be consistent across repeated measurements.

200

3. When a test developer gives the same test to the same group of test takers on two different occasions, the developer is gathering evidence of A) internal consistency. B) internal reliability. C) test-retest reliability. D) scorer reliability.

C) test-retest reliability.

300

Administering a test to a group of individuals, dividing the test into two sections, and correlating the scores on each of the sections demonstrates the ________ method of estimating reliability. a) test-retest b) alternate forms c) split-half d) internal consistency

c) split-half

400

12. Which one of the following is required to yield an accurate estimate of reliability using the split halves method? A) The two halves must be equivalent in length and content. B) The scores on each half must be equal. C) The first half must contain the beginning questions and the second half must contain the ending questions. D) The tests must be administered twice.

A) The two halves must be equivalent in length and content.

500

Which one of the following is most helpful to test developers who wish to increase the reliability of a test’s scores? A) Spearman Brown formula B) Coefficient alpha C) Correlation D) Cohen’s kappa

A) Spearman Brown formula

Even more reliability

100

If Dr. Jonah gives students a math achievement test today and a different version of the same test in two months, we say Dr. Jonah is using A) supplemental forms. B) complementary forms . C) alternate forms D) non-parallel forms.

C) alternate forms

200

Which one of the following pairs of math questions would be most likely to produce an internally consistent result? A) 8-10 = ? and 500 + 224 = ? B) 22 X 48 = ? and 48 + 22 =? C) (-8) – (+10) = ? and 8 X 10 = ? D) 8 X 10 = ? and 10 X 8 = ?

D) 8 X 10 = ? and 10 X 8 = ?

300

16. The formula that Cronbach proposed that calculates internal consistency for questions that have more than two possible responses is called A) coefficient alpha. B) KR-20. C) product moment correlation. D) Spearman-Brown.

A) coefficient alpha.

400

When tests are heterogeneous, estimates of internal consistency are likely to be A) low. B) high. C) comparable. D) homogeneous.

A) low.

500

Judy wants to estimate the internal consistency of a survey on which the respondent marks (1) Not at all, (2) Sometimes, (3) Most of the time, or (4) Always. Which one of the following is most appropriate for her to use? A) Spearman Brown B) Coefficient Alpha C) Cohen’s kappa D) KR-20

B) Coefficient Alpha

Reliability again

100

What is the amount of consistency among scorers’ judgments called? A) Test-retest agreement. B) Interscorer agreeement. C) Intrascorer agreement. D) Internal agreemeent.

B) Interscorer agreeement.

200

An employment test for the job of sales manager that measures knowledge of sales theory, interpersonal skills, and ability to use text messaging is A) homogeneous. B) heterogeneous. C) reliable. D) generalizable.

B) heterogeneous.

300

Which one of the following statements is true about error? A) Systematic error increases the reliability of a test. B) Systematic error lowers the reliability of a test. C) Random error lowers the reliability of a test. D) Random and error increases the reliability of a test.

C) Random error lowers the reliability of a test.

400

32. The Spearman Brown formula is used when calculating A) only internal reliability. B) only split halves reliability. C) only scorer reliability. D) all forms of reliability.

B) only split halves reliability.

500

Cohen’s kappa is a statistical method for A) estimating test-retest reliability. B) estimating interrater agreement. C) estimating internal consistency. D) correlating ratings by two judges.

B) estimating interrater agreement.

Reliability and Standard Error of Measurement

100

The statistic that reflects the amount of inconsistency or error expected in an individual’s test score is called the A) reliability coefficient. B) standard deviation. C) standard error of measurement (SEM). D) correlation coefficient.

C) standard error of measurement (SEM).

200

The standard error of measurement is a function of which two factors? a) reliability of the test and range of test scores b) variability of test scores and range of test scores c) reliability of the test and variability of test scores d) variability of test scores and sample size

c) reliability of the test and variability of test scores

300

As the reliability of a test decreases, the _____ of the test increases. A) standard deviation B) standard error of measurement C) internal consistency D) difficulty

B) standard error of measurement

400

Rita scored 96 on an employment test, and Naomi scored 98 on the same test. Naomi believes that she has the highest score, but Rita disagrees. What information do you need to determine whether Naomi’s score is statistically higher than Rita’s score? A) standard deviation of the test scores B) standard error of measurement of the test scores C) average test score and the standard error of measurement for the test scores D) the standard deviation, standard error of measurement, and the average test score

B) standard error of measurement of the test scores

500

In order to increase internal consistency of a test, the developer could A) accurately measure the test-retest reliability of the test. B) add well written questions to each test form. C) add questions that measure the same factor. D) adjust the reliability coefficient using the KR-20 formula.

C) add questions that measure the same factor.