Types of Research Designs
Nate Silver's "Burrito Bracket"
What is a 'Sequential Mixed Design'?
Characterized by the use of (almost only) numerical data and view that scientists can be (mostly) objective as long as we control for the potential influence of values and use rigorous methods to make sure that the numerical data we're gathering is valid
What is 'Postpositivism'?
Reasoning from the general to the particular. (As opposed to reasoning from the particular to the general.)
What is 'Deductive logic' (as opposed to 'Inductive logic')?
Age, Race, Ethnicity, Average household income, Marital status, etc.
What is 'QUAN' or numeric data?
Tests of differences: T tests, ANOVA (analysis of variance), Mann-Whitney U test
What is 'QUAN' or statistical data analysis?
Researchers attend a climate justice protest to answer a research question about the motivations of attendees and organizers as well as their prior experiences at protests. Some of the researchers carry clipboards and ask attendee-protestors to participate in a 5-minute survey that includes closed-ended questions about whether or not they've participated in a protest before and asks respondents to select which reasons for attending the protest most resonate with them from a list of possible motivations; these researchers also ask basic demographic questions (race, gender, age). Other researchers carry tape recorders and ask other attendees as well as speakers/organizers a set of open-ended questions about why they're motivated to attend and organize protests like this and capture responses on the tape recorder.
What is a 'Parallel Mixed Methods' Design?
The two paradigms that are most fully Mixed Methods
What is 'Transformativism' and 'Pragmatism'?
Research methodology that is confirmatory, exploratory, and explanatory in purpose, uses the research question as the primary guide of all inquiry, and integrates the gathering and analysis of both numeric and narrative data. (As opposed to research that utilizes two or more mostly-independent strands of research where data is collected, analyzed, and inferences are made, but the inferences gleaned from each strand are relatively independent and methods and results are not integrated or synthesized.)
What is 'Mixed Methods Research' (as opposed to 'Multimethod Research')?
Responses to the following survey question: "I consider myself an environmentalist," with response options, "Agree," "Neither Agree nor Disagree," and "Disagree."
What is 'QUAN' data?
The use of Ostrom's 8 Design Principles to organize data collected from 5 Community Water Projects in Dell'Angelo et al. (2016).
What is 'QUAL' data analysis?
Researchers are interested in knowing more about a group of teens who participate in a weekly environmental education Saturday program. There are a dozen such programs throughout a school district. Researchers attend several Saturday sessions of one particular program in order to speak with the teens and learn more about who they are and why they're participating in the program. Based on these conversations, researchers then design and administer a survey to the participants at all dozen sites across the city to get a more representative sense of motivations for participation in an environmental education program.
What is a 'Sequential Mixed Methods Design'? (Bonus: What kind of sampling do they use?)
Researchers following this paradigm highly privilege the opinions and perceptions of study participants. They utilize inductive logic to generate 'grounded theory' based on particular realities and values of study participants, finding themes and broader patterns as they emerge from the narrative co-constructed by participants and researchers. Findings cannot be generalized broadly but might be transferable to other similar contexts.
What is the 'Constructivist' Paradigm?
The jargon-y name for the scientific method wherein a hypothesis is generated based on observations and/or prior theory and then they hypothesis/predictions are tested in order to ascertain whether and when the hypothesis can/cannot be falsified. (As opposed to the method that utilizes any and all logic/methods - inductive, deductive, abductive, dialectical, etc. - that might be most helpful to answering the research question at hand.)
What is the 'hypothetic-deductive method' (as opposed to 'mixed methods')?
Responses to the following survey question: "In your opinion, what are the top 5 benefits of trees?"
What is 'QUAL' (or narrative) data?
A table in a paper displays the averages and range (min, max) of ages for several sample groups of respondents to a survey
What is 'QUAN' analysis?
Characterized by the use of one primary data collection method, which produces one form of data (e.g., numeric data) that may analyzed at face value (e.g., using statistical methods like tests of differences) and then converted to another form (e.g., categories) for another type of analysis (e.g., comparison of qualitative, narrative profiles)
What is a 'Conversion Mixed Methods Design'?
A large team of interdisciplinary researchers from environmental science are seeking to answer a question of how different types of green spaces (small and large parks, vacant lots, schools, community gardens, large boulevards, etc.) are utilized by different people in neighborhoods throughout a large U.S. city. They gather observational data about park characteristics and visitor use activities and demographics (children and adults) during site visits to the different spaces; administer a short survey to visitors to select greenspaces and a longer household survey to the homes and apartments around all types of greenspaces; conduct semi-structured interviews with relevant park or garden managers about greenspace management practices; collect and analyzed geospatial and satellite data about the extent and types of vegetation in greenspaces and adjacent neighborhoods, and gather U.S. Census data about the Census Tracts in their study area. Combining all these different types of data will help them provide insights into greenspace vegetation management and resident use to several key city stakeholders, including the Parks District, City Sustainability Office, local Friends of the Parks nonprofit, the Community Garden Collective, several ward committees, a tree planting nonprofit, and a B-corp company that runs children's after school programs in many parks throughout the city.
What is 'Pragmatism'?
Research that aims to provide causal explanations (cause and effect) or test a hypothesis. (As opposed to research relying on the use of inductive logic to suss out the "unknown aspects of a phenomenon.")
What is 'explanatory' research (as opposed to 'exploratory' research?
After viewing a documentary at a film festival, a large number of viewers complete a short survey to assess how the film's portrayal of climate refugees has affected viewers' sense of urgency and who they believe is responsible for taking action on climate change.
What is 'QUAN' data?
Researchers read respondents' open-ended responses to the question, "How do you define your gender?" and group responses into the categories of "Male," "Female," "Nonbinary," "Something else," and "Did not respond."
What is 'QUAL' data analysis or 'open coding'?
A longitudinal study of environmental science majors is conducted to assess environmental knowledge persistence and early career experiences. Researchers recruit study participants who are freshman in an intro env. sci. class and then administer a test of environmental literacy at the beginning of this class, and then again administer the same test to the same study participants four years later in a senior capstone class. Students in the capstone class are also interviewed on the subject of career aspirations and asked questions about skills and knowledge they think they've gained during college. Five years after graduation, study participants are asked to participate in a series of focus groups about their early environmental career experiences and to reflect on their education; when study participants come in, before the focus group begins, they are asked to take again the test of environmental literacy.
What is a 'Hybrid Mixed Methods Design'? (Also acceptable, Longitudinal Mixed Methods Design)
Researchers from the fields of cultural anthropology, soil science, and agriculture collaborate to investigate issues of soil degradation, malnutrition, socio-economic conditions, and agricultural practices in two types of indigenous communities: Village 1 that has been located adjacent to a large industrial research farm and purchasing from this farm seeds engineered to withstand droughts, and Village 2 that has continued to engage in traditional agriculture practices (incl. seed saving and communal water management methods). The intent is to figure out why Village 2 appears to be better off and what lessons can be learned and applied to similar communities.
What is 'Transformativism'?
A method of selecting units for inclusion in a research study that presumes the research requires sampling units or cases with specific characteristics or for specific purposes; for example, tree-planting nonprofits are selected for potential inclusion in a research study based on the fact that they are either very large with more than 25 staff and a large budget, or very small with fewer than 3 staff and a relatively small budget. (As opposed to selecting units at random from among a large population of potential study units or participants; for instance, researchers request from the U.S. Postal Service the addresses for a random 10% of households living within the geographic boundaries of their study.)
What is 'purposive sampling' (as opposed to 'random' or 'probabilistic' sampling?
Maps that are hand-drawn by research participants to show where important community landmarks are located in relation to the participant's place of residence
What is 'QUAL' data?
The Taste Test stage of Nate Silver's Burrito Bracket.
What is both 'QUAN' and 'QUAL' integrated data analysis?