Reliability/Validity Types Application
Next, teachers were instructed to wait at least 24 hr before rating the same set of 20 informational texts a second time.
What is stability (test/retest reliability)?
The degree to which an instrument can accurately estimate the results of another measure.
What is criterion validity?
We conducted a systematic research synthesis of the literature on text structures and text features found in informational texts, with a special focus on text structures found in primary grade informational texts.
How do researchers ground their survey in the literature? (Step 2)
Description, sequential, problem-solution, compare-contrast, and cause-effect
What are text structures?
In the summer, a mother alligator builds a nest out of plants and mud. She lays about 20 to 50 eggs inside the nest. Then, she covers them up and guards them carefully. After about two months, the baby alligators begin to hatch. They use a special tooth to break out of their eggs. The mother helps them get to the water safely. Young alligators are only about 6 to 8 inches long. They eat insects, small fish, and frogs. As they grow, they begin to hunt bigger prey. They stay close to their mother for protection during the first year. Alligators grow slowly. It can take 10 to 15 years for them to become full-grown adults. Adult alligators can be over 11 feet long! They eat fish, birds, and even deer.
What does sequential text structure look like?
Initial tryouts of the pilot ITS2 instrument by three expert raters arranged into three rating pairs, all of whom were members of the research team, rated 25 randomly selected informational texts. The results yielded an average Cohen’s kappa of .86.
What is interrater reliability?
Administering validated measures of related constructs to run correlation analyses with your instrument.
What is evidence of construct validity?
We developed a new instrument, the Informational Text Structure Survey (ITS2 ), to help primary grade teachers’ identify typical text structures found in children’s informational texts.
How do researchers identify the purpose of an instrument? (Step 1)
Narrative, informational, or mixed
What are text types?
Every year, millions of tons of plastic end up in the ocean. This plastic comes from trash that people throw away, especially single-use items like bottles, bags, and straws. When it rains, plastic can wash into rivers and streams, which carry it to the sea. Once it’s in the ocean, plastic can harm sea animals. Turtles might eat plastic bags, thinking they are jellyfish. Birds and fish can get tangled in plastic and get hurt or even die.
Plastic pollution doesn’t go away easily. It can take hundreds of years to break down. While it floats in the water, it can spread to every part of the ocean. This makes it hard to clean up and dangerous for ocean life.
There are many ways people can help . First, we can use less plastic. Instead of plastic bags, we can use reusable ones. Instead of plastic bottles, we can carry refillable water bottles. Second, we can recycle more. Recycling helps keep plastic out of the ocean. Third, people can join beach cleanups to pick up trash before it reaches the water. Some scientists are even inventing machines that can collect plastic from the ocean.
What does problem/solution text structure look like?
Three nationally renowned researchers in the field of literacy were invited to review the organizational scheme to determine that no relevant dimension or category related to informational text structures had been overlooked in the specification of text structure domains.
What is content validity?
Whether items (individual questions) are correlated with each other and with the instrument as a whole.
What is internal consistency reliability?
In addition, a use protocol was also developed, which included step-by-step directions for using the ITS2 to identify text structures in children’s informational texts
How did researchers prepare the study for piloting? (Step 5)
Headings, typography, graphic displays, and signal words
What are text features?
Frogs and toads are both amphibians, which means they live part of their lives in water and part on land. They may look alike, but they have many differences too!
Frogs and toads both begin life as eggs in water. The eggs hatch into tadpoles, which swim and breathe through gills. As they grow, they change into adults with legs and lungs. Both frogs and toads eat insects and other small animals. They use their long, sticky tongues to catch food.
Frogs usually have smooth, wet skin. They live near ponds, lakes, or streams because they need to stay moist. Toads have dry, bumpy skin and can live farther from water. Frogs have long legs for jumping and swimming, while toads have shorter legs and mostly walk or hop slowly.
What does compare/contrast text structure look like?