Research Design & Levels of Evidence
Which level of evidence is strongest and includes systematic review, or meta-analysis, of multiple well-designed, randomized control trials?
A. Level V
B. Level III
C. Level I
✅ Correct Answer: C. Level I
Rationale
Level I evidence is considered the strongest level of research evidence because it includes:
These studies synthesize findings from several high-quality studies, providing the most reliable evidence for clinical decision-making.
An OTR develops a new screening tool to identify symptoms of depression in adolescents. During testing, the assessment consistently measures the construct it was intended to measure and accurately identifies individuals experiencing depression.
Which characteristic of the assessment is BEST demonstrated?
A. Reliability
B. Validity
C. Precision
D. Standardization
✅ Correct Answer: B. Validity
Rationale:
Validity refers to whether an assessment actually measures what it is intended to measure.
In this example:
Therefore, the assessment demonstrates validity.
What is the PRIMARY purpose for developing a clinical question using the Patient Intervention Comparison Outcome (PICO) method as part of the evidence-based practice process?
A. To critically appraise the quality of evidence associated with a client's problem
B. To ensure that the search of a scholarly database locates only randomized control trials
C. To guide an effective systematic search of published research and evidence
✅ C. To guide an effective systematic search of published research and evidence
Rationale
The PICO method helps create a focused clinical question that guides an efficient literature search during evidence-based practice (EBP).
Why the Other Answers Are Incorrect
❌ To critically appraise the quality of evidence
Critical appraisal occurs later during the Appraise step of EBP.
❌ To ensure the search locates only randomized controlled trials
PICO helps guide searches for relevant evidence, not only RCT
An OTR reviews a research study examining the effectiveness of a new upper extremity intervention for clients recovering from stroke. The study reports that clients who received the intervention demonstrated improved hand function with a p-value of .03. Which interpretation of this finding is MOST accurate?
A. The intervention guaranteed meaningful improvement in daily functioning for all clients
B. There is a high probability that the study results occurred by chance alone
C. The improvement in hand function was statistically significant and unlikely due to chance
D. The intervention was clinically significant because the p-value was less than .05
Correct Answer: C. The improvement in hand function was statistically significant and unlikely due to chance
Rationale
A p-value of .03 is below the common threshold of .05, indicating the results are statistically significant and unlikely to have occurred by chance alone.
A is incorrect: Research findings cannot guarantee outcomes for all clients.
B is incorrect: Smaller p-values indicate lower likelihood results occurred by chance.
D is incorrect: Statistical significance doesn't automatically mean clinical significance.
An OTR is interpreting the scores of a developmental test administered to a 4-year-old child. The test has a mean of 100and a standard deviation of 15. The child's index score is 68, and performance in three skill areas is rated as 2 standard deviations below the mean.
What can the OTR conclude based on these results?
A. Developmental services are indicated for this child.
B. Scores are within acceptable ranges for the child's age.
C. Approximately 30–34% of same-age children perform better on this test.
D. The child demonstrates average performance compared with same-age peers.
✅ Correct Answer: A. Developmental services are indicated for this child.
Rationale
The key clues are:
The child's score is more than 2 standard deviations below the mean.
Calculation:
100 − (2 × 15) = 70
A score of 68 falls below 70, indicating significant developmental delay compared to same-age peers.
Additionally, the child scored 2 standard deviations below the mean in three skill areas, suggesting deficits across multiple developmental domains.
These findings support the need for further evaluation and developmental intervention services.
What type of quantitative research design would provide the strongest level of evidence about the effectiveness of any intervention?
A. Quasi-experimental research
B. Randomized controlled trial
C. Phenomenological study
Correct Answer: B. Randomized controlled trial
Rationale
A Randomized Controlled Trial is considered the gold standard in quantitative research because participants are randomly assigned to either:
an intervention/treatment group, or
a control/comparison group.
An OTR reviews a newly developed assessment and determines that, at first glance, the assessment appears to measure the skill it was designed to evaluate. No statistical testing has been completed, but clinicians agree that the assessment "looks appropriate" for its intended purpose.
Which type of validity is BEST represented?
A. Construct Validity
B. Face Validity
C. Criterion Validity
D. Predictive Validity
✅ Correct Answer: B. Face Validity
Rationale:
Face validity refers to whether an assessment appears to measure what it is intended to measure based on a superficial review. It is based on judgment and appearance rather than statistical evidence.
As part of using the evidence-based practice process to determine the best intervention for a client, the OTR has posed a clinical question and has conducted a search of a scholarly electronic database. What is the NEXT step in the evidence-based practice process?
A. This step comes later in the EBP process, after you’ve appraised the literature and decided on the intervention.
B. Review evidence with the client to make a collaborative decision about the intervention plan
C. Appraise the literature to select quality evidence that answers the posed clinical question.
✅ Best Answer: C. Appraise the literature to select quality evidence that answers the posed clinical question.
Brief Rationale:
In the evidence-based practice (EBP) process, after:
Posing a clinical question and Searching for evidence
An OTR is interpreting scores from a developmental assessment administered to a 7-year-old child. The child scored at the 75th percentile for age and gender. What can the OTR conclude based on this score?
A. The child has minor developmental deficits compared to the normative sample
B. 25% of children in the normative sample scored higher than this child
C. The child demonstrates above-average developmental skills compared to similar children
D. The assessment is sensitive to measuring small developmental changes
Correct Answer: B. 25% of children in the normative sample scored higher than this child
Percentile rank indicates the percentage of individuals scoring at or below the child’s score. A score at the 75th percentile means 75% scored at or below the child, while 25% scored higher.
An OTR is planning to use constraint-induced movement therapy (CIMT) to improve upper extremity motor performance with a client who has hemiplegia secondary to a CVA.
When using an evidenced-based practice framework, which set of factors should the OTR use to guide clinical decision-making regarding the usefulness of integrating CIMT into the intervention plan?
A. Client preferences
B. Best research data
C. Standardized Assessments
D. Employers policies
E. Practitioners clinical experience
Documentation requirements
✅ Client preferences
✅ Best research data
✅ Practitioner’s clinical experience
Rationale: These are the three core components of evidence-based practice (EBP)
Which of the following clinically focused research project topics is an example of a qualitative research methodology?
A. Assessing the outcomes of a life skills training group to identify the change in job retention rates for clients who have a substance use disorder
B. Conducting focus groups with individuals whom have sustained a spinal cord injury in order to develop a theory of adjustment
C. Engaging kindergarten-age students who have learning disabilities in a sensory integrative program to improve academic performance
D. Using gardening activities to improve grip strength outcomes for clients who have mild hemiplegia
✅ Conducting focus groups with individuals who have sustained a spinal cord injury in order to develop a theory of adjustment
Rationale:
This is an example of qualitative research because it focuses on:
Participant experiences, perspectives,meanings,and development of theory through discussion and observation.
Key NBCOT Clue:
“Focus groups” → qualitative research
“Develop a theory” → qualitative methodology
An OTR develops a work-readiness assessment for clients participating in a work-hardening program. Six months after discharge, the OTR compares participants' assessment scores with their actual employment outcomes and finds that higher scores are associated with successful return to work.
Which type of validity is BEST demonstrated?
A. Face Validity
B. Construct Validity
C. Predictive Validity
D. Interrater Reliability
✅ Correct Answer: C. Predictive Validity
Rationale:
Predictive validity refers to an assessment's ability to predict future performance or outcomes.
In this example:
This demonstrates predictive validity.
An OTR wants to search for literature to inform clinical decision-making for a client who has spastic hemiplegia and upper extremity contractures. The OTR is developing a clinical question using the Patient Intervention Comparison Outcome (PICO) model. Intervention priorities for the client include preventing the development of contractures and increasing passive ROM. The OTR wants to compare the use of a custom-made orthosis to a daily passive ROM program. Using the PICO method, which of the following is the MOST EFFECTIVE clinical question to guide the search of the database?
A. In adults who have had a cerebral vascular accident, how does an upper extremity positioning program compared to traditional therapy affect functional abilities?
C. In adults who have spastic hemiplegia of the upper extremity, how does an orthosis compared to a passive ROM program affect upper extremity range of motion?
D. In adults who have decreased upper extremity range of motion, how does a passive ROM program compared to the application of an orthosis influence an effective outcome?
✅ Best Answer: C. In adults who have spastic hemiplegia of the upper extremity, how does an orthosis compared to a passive ROM program affect upper extremity range of motion?
Brief Rationale:
This question is the most effective because it clearly includes all parts of the PICO format:
P (Patient/Problem): adults with spastic hemiplegia of the upper extremity
I (Intervention): orthosis
C (Comparison): passive ROM program
O (Outcome): upper extremity range of motion
It is specific, measurable, and directly related to the client’s intervention priorities.
An OTR administers a standardized cognitive assessment to an older adult following a stroke. The client scores at the 15th percentile compared to their age group. Which interpretation is MOST accurate?
A. 85% of individuals in the normative sample scored lower than this client
B. The assessment has poor reliability for measuring cognition
C. The client performed better than 15% of the normative sample
D. The client answered 15% of the assessment items correctly
Correct Answer: C. The client performed better than 15% of the normative sample
A percentile rank indicates the percentage of individuals in the normative sample who scored at or below the client’s score. A score at the 15th percentile means the client performed better than 15% of similar individuals.
An OTR reviews a research article examining the effectiveness of a new home exercise program for adults with osteoarthritis. The researchers found that participants who completed the program reported slightly lower pain ratings at discharge.
Which conclusion is MOST appropriate for the OTR to make?
A. The home exercise program completely eliminated pain in adults with osteoarthritis.
B. The findings suggest the program may be helpful, but additional evidence is needed before making strong conclusions.
C. The program should immediately replace all other osteoarthritis interventions.
D. The lower pain ratings prove the intervention directly caused the improvement.
✅ Correct Answer: B. The findings suggest the program may be helpful, but additional evidence is needed before making strong conclusions.
Rationale
Evidence-based practice requires clinicians to interpret research findings cautiously.
The study found:
However, from the information provided, the OTR cannot determine:
Therefore, the most appropriate conclusion is that the intervention may be beneficial, but additional evidence is needed before making broad clinical recommendations.
An OTR and the rehabilitation manager in a hospital setting are conducting a program evaluation to determine if modifications to inpatient clinical protocols are needed. Currently, the OTR is required to provide a list of recommended DME to each patient being discharged from the hospital to home. The OTR plans to investigate if patients who acquire the recommended equipment for use in the home after discharge have a lower rate of readmission to the hospital than patients who did not acquire the recommended equipment. Which research design would be MOST EFFECTIVE for the study
A. Survey
B. Randomized controlled trial
C. Correlational
Correct Answer: Correlational
Rationale: The study is examining the relationship between two variables:
acquiring recommended DME, and
hospital readmission rates.
The researchers are not manipulating variables or randomly assigning groups, so a correlational design is the most appropriate.
An OTR working in a school-based setting creates a handwriting screening tool for first-grade students. To evaluate the tool, the OTR scores the handwriting samples of 20 students on Monday and then scores the same samples again one week later without looking at the original scores. Later, a second OTR independently scores the same handwriting samples to determine whether the scoring system is objective and consistent across therapists.
Which type of reliability is MOST directly assessed when the original OTR rescored the handwriting samples one week later?
A. Face validity
B. Intrarater reliability
C. Test- Retest reliability
D. Internal Consistency
Correct Answer: B. intrarater reliability
Rationale:
Intrarater reliability examines whether the same examiner produces consistent scores when scoring the same material at different times. The original OTR rescoring the same handwriting samples one week later is assessing intrarater reliability.
As part of a process improvement plan, the OTR supervisor in a rehabilitation facility reviewed the OT staff’s use of evidence in daily clinical practice. A key issue noted during the review was that the clinicians were aware of the benefits of evidence but were inconsistent in researching and integrating new evidence into client intervention. Which action would be MOST BENEFICIAL for reversing this trend?
A. Include evidence-based practice competencies into new staff orientation and the annual review process
B. Develop a policy outlining specific methods for implementing evidenced-based standards into daily practice.
C. Conduct inservice training to promote the use of resources for collecting and evaluating current evidence.
✅ Best Answer: C. Conduct inservice training to promote the use of resources for collecting and evaluating current evidence.
Rationale:
The main problem is that staff are inconsistent in researching and integrating new evidence despite understanding its benefits. Providing inservice training directly addresses the clinicians’ need for practical skills and strategies to locate, evaluate, and apply evidence in daily practice.
An OTR reviews two research studies examining the effectiveness of a fatigue management program for clients with multiple sclerosis. Both studies report statistically significant findings. Study A reports a narrow confidence interval, while Study B reports a wide confidence interval.
What should the OTR conclude about Study A?
A. The results are less precise and show greater variability.
B. The results provide a more precise estimate of the true effect.
C. The intervention was more clinically significant.
D. The study demonstrated a stronger negative correlation.
✅ Correct Answer: B. The results provide a more precise estimate of the true effect.
Rationale
A confidence interval (CI) provides a range within which the true population value is likely to fall.
A narrow confidence interval indicates:
Because Study A has a narrower confidence interval, its findings provide a more precise estimate of the true treatment effect than Study B.
An OTR administers a standardized handwriting assessment to a 7-year-old student. Results indicate the child scored 2 standard deviations below the mean when compared to same-age peers. The assessment manual reports a statistically significant difference (p < .05).
Which statement BEST describes how the OTR is interpreting the assessment results?
A. The assessment is criterion-referenced because the child's performance is compared to a fixed level of handwriting independence.
B. The assessment is norm-referenced because the child's performance is compared to scores from same-age peers.
C. The assessment is criterion-referenced because statistical significance and standard deviation are used in the scoring process.
D. The assessment is norm-referenced because the child demonstrated mastery of specific handwriting tasks.
✅ Correct Answer: B. The assessment is norm-referenced because the child's performance is compared to scores from same-age peers.
Rationale
The key clues are:
These characteristics indicate a norm-referenced assessment, which compares an individual's performance to that of a representative comparison group.
Norm-referenced assessments commonly report:
Because the child's score is being compared to peers, the assessment is norm-referenced.
An OTR is reviewing the results of a randomized control trial that studied the effectiveness of wearing a volar wrist orthosis to prevent contracture formation. The participants (n=6) in the study all had spastic hemiplegia and were greater than 6 months post-stroke. The intervention group followed an orthosis-wearing schedule of 10 hours per day, but the control group did not wear a wrist orthosis. Study results indicate there were no significant differences in muscle contracture formation between the two groups. Based on this review, what is the NEXT action the OTR should take?
A. Discontinue recommending orthoses for clients who have upper extremity spasticity.
B. Search a scholarly database for a systematic review or a meta-analysis on the topic.
C. Provide a passive ROM and therapeutic stretching program when a wrist orthosis is recommended.
Correct Answer: B. Search a scholarly database for a systematic review or a meta-analysis on the topic.
Rationale
This question is testing evidence-based practice (EBP) and the hierarchy of evidence.
The OTR reviewed:
Although RCTs are strong evidence, one small study is not enough to change practice.
The next step in the EBP process is to seek the highest level of evidence, which would be:
These synthesize findings from multiple studies and provide stronger evidence for clinical decision-making.
An OTR develops a new pediatric feeding assessment for children with sensory processing challenges. The assessment includes tolerance of food textures, utensil use, oral motor coordination, sitting posture during meals, and response to food smells.Before using the assessment clinically, a panel of experienced pediatric feeding therapists reviews the tool to determine whether it adequately covers all important areas involved in feeding performance.
Which type of validity is MOST directly being examined?
A. Concurrent validity
B. Predictive validity
C. Face validity
D. Content validity
✅ Correct Answer: D. Content validity
Rationale
The key clue is:
"A panel of experienced pediatric feeding therapists reviews the tool to determine whether it adequately covers all important areas involved in feeding performance."
This is the definition of content validity.
Content validity examines whether an assessment:
In this example, experts are evaluating whether the assessment covers all important aspects of feeding, including:
Therefore, they are evaluating content validity.
An OTR working in a children's hospital routinely reads pediatric-related research but is not always certain if the evidence presented in the research can be applied to OT interventions used with children in the hospital. After locating the evidence, what step should the OTR take NEXT to critically appraise the literature?
A. Create a list of questions to use for evaluating the applicability of the research to practice.
B. Select a systematic method to assign a meaningful value to each research article reviewed.
C. Organize the studies by age of children who participated in the intervention and control groups.
✅ A. Create a list of questions to use for evaluating the applicability of the research to practice.
Rationale
The OTR should evaluate whether the research is trustworthy, relevant, and useful for practice. This is part of the Appraise step of evidence-based practice.
Why the Others Are Incorrect
Assigning values to articles is not the next appraisal step.
Organizing studies by age does not evaluate study quality or usefulness.
An OTR working in an outpatient orthopedic clinic reviews a research article examining the relationship between chronic shoulder pain and participation in IADLs among older adults. The study found a strong negative correlation between pain ratings and IADL participation scores. Which conclusion is MOST appropriate?
A. Pain ratings and IADL participation scores are unrelated variables
B. Reducing shoulder pain will guarantee improved IADL participation
C. Higher pain levels are associated with lower participation in IADLs
D. Chronic shoulder pain directly causes decreased IADL participation
Correct Answer: C. Higher pain levels are associated with lower participation in IADLs
A negative correlation indicates that as one variable increases, the other variable tends to decrease. In this study, higher pain ratings were associated with lower levels of participation in IADLs.
An OTR reviews two research studies examining the effectiveness of a fatigue management program for clients with multiple sclerosis. Both studies report statistically significant findings. Study A reports a narrow confidence interval, while Study B reports a wide confidence interval.
What should the OTR conclude about Study A?
A. The results are less precise and show greater variability.
B. The results provide a more precise estimate of the true effect.
C. The intervention was more clinically significant.
D. The study demonstrated a stronger negative correlation.
✅ Correct Answer: B. The results provide a more precise estimate of the true effect.
Rationale
A confidence interval (CI) reflects the range within which the true population value is likely to fall.
A narrow confidence interval indicates:
Because Study A has a narrower confidence interval than Study B, its findings provide a more precise estimate of the true effect.