measurement
levels of measurement
What is nominal, ordinal, interval, ratio
■ Nominal (discrete): classification without ranking, mutually exclusive and exhaustive
■ Ordinal: ranking, but no interval information ( Likert format)
■ Interval: ranking, equal intervals throughout scale, but no true zero so comparisons are difficult (e.g., IQ, )
■ Ratio: ranking, equal intervals, meaningful zero (theoretical as opposed to observable)
attributes of probability sampling
Probability sampling,
all elements (e.g., persons, households) in the population have a known, non-zero, independent opportunity of being included in the sample; the mathematical probability that any one of them will be selected can be calculated.
•‘Equal probability of selection' (EPS):When every element [and set of elements] in the population has the same probability of selection
•Also referred to as 'self-weighting' because all sampled units are given the same weight.
•May be an unequal probability of selection across groups (as opposed to individuals)
attributes of strong theory
Attributes of Strong Theory
Reasonable assumptions
Logically consistent
Unambiguous (e.g., concepts)
Suggest testable claims? (connections between abstract and concrete)
Generates new research
Accuracy: Becomes stronger as more supporting
evidence is gathered
Does it work better than rival theories or explanations?
Breadth: Does it apply across time, place, context?
Fallible: Can it be disproved? Karl Popper
key elements of informed consent
INFORMED CONSENT (UC DAVIS):
• Study involves research; purposes of research; length of research; what will happen to you; what is experimental; Risks or discomforts to you; Benefits to you or others; Other choices you might have; Who will see your information; You volunteer to be in a research study; You can agree to take part now and later change your mind; Whatever you decide it will not be held against you; If you have questions, concerns, or complaints, or think the research has hurt you, you can talk to the research team; Whether you will get treated or paid if injury occurs; possibility of unknown risks; when you may be taken off the research without your agreement; costs from taking part; what will happen if you stop taking part; steps to safely stop taking part; when and how new information will be told to you; number of people expected to take part in the research
points in C. Wright Mills sociological imagination
C. WRIGHT MILLS: SOCIOLOGICAL IMAGINATION
1. Do not split work from life. Both are part of a seriously accepted unity.
2. Keep files. Compendia of personal, professional, and intellectual experiences
3. Engage in continual review of thoughts and experiences.
4. Be as open to a truly bad book as a good one.
5. Adopt an attitude of playfulness toward phrases, words, and ideas.
6. Have a fierce drive to make sense out of the world.
7. Embrace seeing the world from other theories, perspective of others.
8. In preliminary stages of speculation, be open to imaginative extremes.
9. Express ideas in simple, direct language.
10. Look for ways to connect individual biography with macrosocial forces & historical conditions
ways of assessing the validity of measures
Validity:
External (generalizable): Sampling
Internal: Measurement
Theory based
Face: Judgement about the degree to which measure matches theoretical concept (exposure to law violating definitions=# of deviant peers) Content: Judgement about how well does the measure (scale/index/observation) capture all of the meanings/facets of a concept? Has anything been left out?
Criterion validity
1) Concurrent Validity: Is the new measure strongly associated with other, validated measures of the variable?
2) Predictive validity: Does the measure predict a future criterion? SAT and First year grades in college
Construct validity
1) Convergent: Correlation between different ways of measuring concept: Self-report condom use and counts of used condoms; Self-report classroom engagement and observer reports
2) Discriminate: Evidence that one concept is different from other closely related concepts: depression and anxiety; wealth and SES (factor analysis)
3) Hypothesis-testing: Evidence that the measured concept has the predicted relationship
attributes of non-probability sampling
• Any sampling method where:
1) some elements of population have no (zero) chance of selection (sometimes referred to as 'out of coverage'/'undercovered'),
or 2) where the probability of selection cannot be accurately determined.
• Select elements based on assumptions regarding the population of interest, which forms the criteria for selection. • Cannot estimate sampling error • Statistics less reliable • Less external validity • Population lists not available • Population is small and widely dispersed • In-depth, holistic understanding of a group, phenomenon
strong attributes of questions that guide research
STRONG QUESTIONS
Maximize variation: Zero-sum, binary, discrete vs continuous,variable
Disciplinary importance: appeal to other sociologists (Sociologicalquestions versus other: attributes of each?)
Substantive importance: potential to make a contribution, appealto other audiencesFeasibility: resources, data availability
Scientific: replicable, opposite outcome discoverable
Karl Popper: “Discovery contains ‘an irrational element, a creativeintuition’”
types of relationships
Direct: direct relationship between independent and dependent variable, net of other variables (background/antecedent/exogenous or intervening);
intervening/mediation: relationship is indirect through a third variable (3rd variable intervenes, mediates);
moderation/interaction: relationship depends upon (varies across the levels of ) a 3rd variable
exogenous antecedent relationship: changes among other variables don't affect it
direct: x----y
intervening: x1----x2----> Y
ways to assess reliability of measures
Reliability: dependability, stability, consistency
■ Does the measuring instrument/approach give the same results in different settings, regardless of it’s validity.
1) Stability:
a) Test-retest correlation over time. How high is the correlation between the measure when asked at two points in time of the same person? (same people in a panel study).
b) Alternate form reliability: similar, but different measure for same concept used
2) Equivalence: How high is the correlation between the measure when asked at one point in time of the same data by two researchers? (inter-coder reliability) OR How high when measured with different items or different types of data collection instruments AKA multiple-forms reliability. (e.g., self-report, observer data on classroom behavior)
3) Internal consistency (homogeneity), scales/indices
a) Average inter-item correlation. Most common (Cronbach’s alpha; factor analysis). How high are the correlations among the items that make up the scale? If scale is homogeneous, each question should be highly correlated with the other questions.
b) Average item-total correlation. How high does each question correlate with the scale of the remaining items?
b) Split-half correlation. Divide items into two groups (subscales). How high is the correlation between the two subscales?
3 major types of error in sampling
Sampling error (probability versus nonprobability)
Coverage error
Sampling frame: list from which the potential respondents are drawn • Best: identifies every single element of a population (exhaustive), by definition, representative • Preexisting population lists, constructed population lists • BUT, most lists have error • Reduce/assess Coverage error (update sampling frame) • Update list: missing • Clean list: duplicates, foreign elements
Participation (response)
Study nonresponse (distinct from item nonresponse) can turn any probability design into a nonprobability design
1) If it is systematic within a group(s)
2) If it systematic across groups
3) If it is excessive
attributes of the scientific method
DOING GOOD SCIENCE
• MAKE A PREDICTION BASED ON THE HYPOTHESIS.
• GATHER DATA TO INVESTIGATE QUESTION OR TEST THE PREDICTION.
• IDENTIFY OR IMAGINE OTHER ALTERNATIVE EXPLANATIONS
• TEST ALTERNATIVE EXPLANATION
• ITERATE: USE THE RESULTS TO FORM NEW HYPOTHESES OR predictions
In general, difference between qualitative and quantitative research
qualitative: internal validity (often uses non probability sampling)
explores unexplained settings by creating 'thick description' of relationships between variables;
quantitative: external validity (often uses probability sampling)
Is there a relationship between variables? How strong is it? Is it significant? How much does it explain of observed variation?