Show:
Threats to Validity
Applied Research
Observationa/Archival Research
Qualitative Research
Wildcard
100
Which types of threats to validity influence the generalizability of experimental findings to other populations, other settings, or other times?
Threats to external validity
100
What is the primary distinction between applied and basic research?
The end use of the results
100
What is nonreactive measurement?
Measurement procedures whereby participants’ responses should not be influenced by extraneous factors (e.g., desire to impress researcher, belief that certain behaviors will support hypothesis)
100
In regards to hypothesis generatoin and hypothesis test, which does qualitative research focus moreso on? Which does quantitative research focus moreso on?
Quant - hypothesis testing (deductive) Qual - hypothesis generation (inductive)
100
What is the name of the threat to validity where attrition rates of participants from a study are different for the various experimental groups? Which type of validity does this factor threaten?
Differential mortality -- internal validity
200
What is one of two factors that can lead to selection threats?
(1) Self-selection into groups (2) Failures of randomization
200
Which type of evaluation research answers the question, "Does it work?"
Summative evaluation
200
What does the dimension of natural behavior mean, in the context of naturalness in research? What does it impact upon (the setting, the IV, or the DV)?
Behaviors not established/maintained for the primary purpose of conduct research (dependent variable)
200
In regards to validity, what is the major limitation of the various qualitative research methods we discussed in class (which type of validity)
External validity
200
Which of the various types of qualitative research is confronted most with ethical dilemmas?
Participant observation
300
What threat includes events that occur during your study in addition to the experimental variable that is unrelated to the IV, but that may influence the DV
History
300
What are the three requirements for sound research by the Directorate for Education and Human Resources of the National Science Foundation?
Evaluation plan, evaluation instruments, evaluation report
300
What is one disadvantage to an emphasis on naturalness in social science research?
(1) Reduced ability to control for extraneous factors that introduce error/noise in the observations (reduced reliability) (2) Many constructs cannot be studied by observing an outward behavior
300
Which qualitative research technique ultimately codes the data and submits it to quantitative analysis?
Narrative analysis
300
What is one of the disadvantages of archival research in regards to interpretation?
(1) Spurious Associations (2) Lack of control over data collection - potential for sources of bias or unreliability (3) Potential for gaps/incompleteness in the data
400
What is differential mortality?
Attrition rates of participants from a study are different for the various experimental groups
400
What are two arguments against randomized experimental designs in evaluation?
(1) Not feasible (2) Narrow in scope (3) Doesn't say how to improve the program (4) Can be unethical
400
What is one of the two types of physical trace measures? What is its definition?
(1) Erosion measures: amount of wear on some material (2) Accretion measures: amount of depositing of some material
400
Which qualitative research technique includes structured group interviews where a small group of people is brought together to discuss a topic of interest to the researcher, under the guidance of a moderator
Focus groups
400
What threat to validity becomes possible when more than one treatment is applied to the same group of participants? What does this pose a threat to (validity)?
Multiple treatment interference; external validity
500
What is Reactive Effects of the Experimental Arrangements?
Something about the experimental setting itself could impact results to some extent, independent of what you’re actually doing in the experiment
500
Which of Kirkpatrick's four criteria for evaluation focuses on whether participants find the program to be useful or valuable?
Reactions Criteria
500
Which is structured approaches to Systematic Observation more appropriate for: hypothesis testing or hypothesis generation?
Hypothesis testing
500
How do experimenters and participant observers differ in regards to the importance of standardization and procedure?
Experimenters: reliant upon standardized questionnaires, administration, manipulations Participant observers: questions and actions exhibited by the participant observer varies from one respondent to the next
500
What are two challenges that are faced in conducting applied research?
(1) Need for quick fix (2) Multiple constituents or stakeholders (3) Expensive (4) Vested interests in outcomes and politics