Research Control- Reliability, Validity
The types of data or levels of measurement. 2 categorical types and two continuous types.
Categorical- nominal and ordinal
Continuous- interval and ratio
Nominal (categorical)- think of names; categories that describe traits or characteristics participants can check; no ranking of data
Ordinal (categorical)- data ranked from smallest to largest; intervals may NOT be equal
Interval (continuous)- equal intervals between data levels or categories
Temperature
Ratio (continuous)- interval scale with an absolute zero
Grip strength testing
Length of time
A self-reported instrument that is generally mailed or handed to the respondent to complete with no help from the researcher.
Questionnaires
The two ways to calculate descriptive statistics.
Measures of central tendency (continuous variables)
Measures of variability/dispersion
This type of qualitative analysis is often used in OT research to understand the experience of the person, uses constant comparative approach and is often guided by a theoretical framework. This may or may not involve interpretation
Thematic analysis
Phenomenological approach does not interpret but distills to the essence the phenomena under investigation
Heuristic approach includes theoretical context and interpretation, uses open coding of interview data as soon as first interview is completed
In quantitative data you can tell if a sample is big/small enough by using this.
In qualitative data you can tell by this.
Power analysis
Data saturation
Reliability of a measure is the consistency or repeatability of that measure in quantitative research. It includes these three types.
Stability: test-retest
Internal consistency: testing homogeneity (split in half and compare results)
Equivalence: use of alternate forms
Reliability is degree to which random error exists in measurement instrument. Usually expressed as correlation coefficient (r).
Which type of questions on a questionnaire are structured and which are unstructured?
Open-ended questions- unstructured
Closed questions (structured)
This error is created by accepting the null hypothesis when it is false.
This error is created by rejecting the null hypothesis when it is true.
Type II error
Type I error
From interview data, these address "what" the content is within the data.
From interview data, these address the "why" the content is important.
Index codes/conceptual labels: words or phrases of chunks of content, often become codes, decisions and definitions recorded in audit trail, often revised with ongoing analysis.
Analytic memos: investigators thoughts about the content, statement about the content, developed as a reflection of understanding, return to questions posed and place the context of the phenomenon.
This type of design has less scientific rigor than randomized trials but more than analytical studies.
Non-randomized or quasi-experimental design.
The four ways to ensure rigor or trustworthiness in qualitative inquiry.
Credibility: (true value)- has the researcher represented multiple realities revealed by informants as adequately as possible?
Transferability: (applicability)- degree to which the findings fit into contexts outside the study situation that are determined by the degree of similarity or goodness of fit between the two contexts
Dependability: (consistency)- Can the variability in the study findings be ascribed to identify sources? The findings would be consistent if the inquiry were replicated with the same subjects or in a similar context
Confirmability: (neutrality, objectivity)- neutrality of data, participants voices, objective researcher is seen as scientifically distant
A downside to descriptive research.
A downside to questionnaires or interviews.
No follow-up; Cause and effect more difficult to establish.
Too exhausting/time-consuming to read all questions (can combat by making very clear concise statements)
The three measures of central tendency.
The three measures of variability/dispersion.
Mean, median and mode
Range, variance, standard deviation (most common)
This type of analysis uses artifacts, social and cultural groups to record the experience, the critical concern is what to include or exclude and it looks for patterns of behavior.
This type of analysis addresses how the individual "makes sense" of the situation and is derived from the construction of the "story".
Ethnographic analysis
Narrative analysis- does not have specific approach (considered a weakness), inherently dependent upon subjective interpretation and the interpretation occurs through collaboration with the individual under investigation
___ validity is the correspondence of conceptual and operational definitions (measuring what we want to measure/manipulate)
___ validity is for generalization and we can strengthen this type of validity by REPLICATION.
List the types of internal validity.
Internal
External
Types of internal validity:
Construct: definition and conceptual model of the attribute being measured (think of operational definitions)
Content: evidence that assessment adequately covers the relevant domains of the construct, and all content is relevant to the construct (evidence matches those operational definitions)
Criterion: evidence gathered by examining relations between the assessment and other measures administered to same person at the same point in time (concurrent validity, predictive validity)
The four ways to ensure rigor in QUANTITATIVE research
Internal validity, external validity, reliability, objectivity
When survey research is most likely the best method available to the social researcher.
A disadvantage to survey research.
To collect original data for describing a population too large to observe directly.
Careful probability sampling provides a group of respondents whose characteristics may taken to reflect those of the larger population
Carefully constructed standardized questionnaires provide data in the same form from all respondents
Disadvantage- heavily relies on response rate
A chi-square test is a ____ statistic used to analyze frequencies or proportions.
This statistic can only use a ___tailed test because ____.
nonparametric
one-tailed because no negative values are possible
The six types of quality assurance and what they are.
Interview training (skills), engagement in field, reflexivity, triangulation, stakeholder checks, audit trail.
Reflexivity- Researchers often keep a journal and jot down their possible biases and experiences while collecting the data
Triangulation- ensures quality of data; comparing data to other sources (taking your data and comparing it to other existing data)
Will show if it’s novel or if researcher is confirming data
Stakeholder checks- checking the quality and accuracy of data; ex. Taking audio interviews, transcribing interviews verbatim, checking back with interviewees to check the information is correct
The three things each question should have in a questionnaire.
Focus, brevity and simplicity
Some methods to enhance trustworthiness in qualitative research.
Name at least 3
Prolonged (breadth) and persistent engagement (depth)
Time sampling- systematize informant contacts, sample all possible situations, times, groups
Reflexivity- diary, notes, bracketing, self-interrogation, peer interview, group participation
Triangulation- convergence of multiple perspective to ensure that all aspects of a phenomenon have been investigated (triangulated sources are cross-checked: multiple methods, data sources, theory, investigators)
Member checking- ask participants to review and react to study data and emerging themes and conceptualizations
Interview techniques
Peer examination
Structural coherence- interpretation and analysis explains apparent contradictions
Authority of the researchers
Representatives of the participants
Audit trails- review raw data, process notes, data reduction, etc.
The types of scales in surveys and/or questionnaires. Define/explain them each.
Likert scale: level of agreement (favorable-unfavorable) indicated; usually represented by 5-7 responses
Usually uses ordinal data
Guttman scale: cumulative scaling of increasing intensity to establish a one-dimensional continuum for a concept; respondents check items with which they agree
Semantic differential scale: allows rating along a bipolar continuum: very black and white (one side or the other)
____ statistics show the strength of the relationship between two variables and include ____ coefficients.
List three types of tests of relationships.
Correlational statistics
correlation coefficients
Tests: Pearson r (parametric data), Spearman rank-order correlation coefficient (nonparametric data), Kappa, Kendall's tau, interclass correlation coefficient
NO CAUSE-EFFECT CAN BE INFERRED
List the types of data that would be collected if an ethnographic approach was used, if a phenomenological approach was used, if an heuristic approach was used, and if a grounded theory approach was used.
Ethnographic: artifacts, observer's notes, investigator immersion into the setting
Phenomenological: interviews, bracketing biases, limited interpretation with focus on essence of phenomenon, does not require multiple participants but often requires several interviews
Heuristic: interviews, can use constant comparative approach across interviews provided by multiple participants
Grounded theory: published data used in comparison to collected data for theory development
If a study is quantitative and there was not an intervention it is ___.
If there was an intervention but no control group it is ___.
If there was an intervention, a control group, but not a random sample it is ___.
If there was an intervention, a control group, and the sample was random, it is ____.
Descriptive (non-experimental)
Quasi-experimental
Quasi-experimental
RCT