Reliability
When tests are heterogeneous, estimates of internal consistency are likely to be _______________.
A. low
B. high
C. comparable
D. homogeneous
A. low
Defining the testing universe is an important step for ensuring a test has evidence of validity based on its _______________.
A. content
B. reliability
C. relations with variables
D. constructs
A. content
How much variance do a test and criterion share if the coefficient of determination is .09?
A. 0.09%
B. 3%
C. 9%
D. 81%
C. 9%
When a group of test items all share something in common that accounts for the high correlations among them, we can say they all share a common _______________.
A. cause
B. theory
C. nomological network
D. factor
D. factor
What do we call a single source of error that always increases or decreases the true score by the same amount?
A. the true score
B. the average score
C. random error
D. systematic error
D. systematic error
Which one of the following statements best describes a practical test?
A. the driving portion of a driving test
B. the math portion of the SAT
C. tests such as temperature or weight measurement
D. a job interview
A. the driving portion of a driving test
In the equation Y’ = a + bX, what does b represent?
A. intercept
B. slope
C. individual’s score
D. predicted criterion score
B. slope
Evidence of construct validity is provided when convergent validity correlations are _______________.
A. higher than discriminant validity correlations
B. lower than discriminant validity correlations
C. the same as are discriminant validity correlations
D. similar as are discriminant validity correlations
A. higher than discriminant validity correlations
Cohen’s kappa is a statistical method for _______________.
A. estimating test–retest reliability
B. estimating interrater agreement
C. estimating internal consistency
D. correlating ratings by two judges
B. estimating interrater agreement
When you ask experts to rate the extent to which individual questions are essential to measure the testing universe, you are gathering evidence of validity based on _______________.
A. relations with other variables
B. internal structure
C. test content
D. response processes
C. test content
A statistical analysis that has more than one set of test scores used for predicting a criterion is called _______________.
A. multiple regression
B. linear regression
C. criterion validity
D. predictive validity
A. multiple regression
A multitrait–multimethod correlation matrix is shown below. What is the correlation of Test of Trait 1/Multiple Choice and Test of Trait 2/Multiple Choice and what is its type?
A. .75; monotrait–monomethod
B. .75; heterotrait–monomethod
C. .89; heterotrait–heteromethod
D. .89; montrait–heteromethod
B. .75; heterotrait–monomethod
The range of scores that we feel comfortable and includes the true score is called a _______________.
A. standard deviation
B. standard error of measurement
C. confidence interval
D. normal curve
C. confidence interval
Test specifications are very much like a(n) _______________.
A. blueprint
B. matrix
C. table of contents
D. index
A. blueprint
Which one of the following criteria used in educational settings is subjective?
A. grade point average
B. instructors’ letters of recommendation
C. number of dismissals or withdrawals
D. number of courses completed
B. instructors’ letters of recommendation
When an underlying theory explains the relation between a predictor and a criterion, there is evidence of _______________.
A. construct validity
B. criterion-related validity
C. predictive validity
D. concurrent validity
A. construct validity
According to classical test theory, what would the reliability coefficient be when the variance of observed scores is equal to the variance of true scores?
A. 0
B. 0.5
C. 0.75
D. 1.0
D. 1.0
Job analysis differs from competency modeling in that it _______________.
A. is a less technically rigorous process than competency modeling
B. usually focuses on a general family of jobs rather than one specific job
C. cannot be used to develop evidence of validity based on content
D. usually identifies the specific tasks an individual performs on the job
D. usually identifies the specific tasks an individual performs on the job
_______________ is a statistic used for interpreting the results of a multiple regression.
A. Coefficient of individual determination
B. Coefficient of multiple determination
C. Coefficient of validity
D. Coefficient of reliability
B. Coefficient of multiple determination
What is a confirmatory factor analysis?
A. a process that provides information on test bias and discrimination
B. a process that researchers use to determine what factors underlie a construct
C. a process of conducting a goodness-of-fit test
D. a process that researchers use to demonstrate that the constructs that theory suggests their test should contain actually exist
D. a process that researchers use to demonstrate that the constructs that theory suggests their test should contain actually exist