How to solve SAT observational study and experiment questions

These questions describe a study and ask which conclusion it supports. There is almost no computation. You need to know two ideas: random sampling lets you generalize to a population, and random assignment to treatments lets you conclude cause and effect. On the Digital SAT the wrong answers usually claim more than the study design allows. Problem-Solving and Data Analysis is about 15% of the Math section, or 5 to 7 of the 44 questions. Those questions are split across seven skills, so expect about one question on it, and some tests have none.

What the questions look like

Pick the supported conclusion. A paragraph describes how subjects were chosen and what was done to them. Four conclusions follow, and you choose the one the design justifies.

Observational study or experiment. You decide whether researchers assigned treatments or just recorded what people already do, and what that means for cause and effect.

Spot bias in a sample. A survey method is described, such as asking customers at one gym or posting a poll online, and you identify why the results may not represent the population of interest.

Step-by-step method

1. Ask how the subjects were chosen. If they were selected at random from a population, results can be generalized to that population. If they were volunteers or a convenience group, results apply only to the subjects studied.
2. Ask whether the researchers assigned the treatment at random. If they did, the study is an experiment, and a difference between groups can be attributed to the treatment. If subjects chose their own behavior, or researchers only observed, the study is observational and shows only an association.
3. Combine the two answers.
   • Random sample and random assignment: cause and effect, for the population sampled.
   • Random assignment only: cause and effect, for the subjects in the study only.
   • Random sample only: association, for the population sampled.
   • Neither: association, for the subjects in the study only.
4. Check the scope of each choice. Eliminate any choice that claims causation without random assignment or extends results to a population that was not sampled.
5. Among the remaining choices, pick the one that matches the design most precisely.

Worked example

Example 1. A researcher selected 300 adults at random from the residents of a city. She randomly assigned 150 of them to take a 20-minute walk every day and told the other 150 to keep their usual routine. After 8 weeks, the walking group had a significantly lower average resting heart rate than the other group. Which conclusion is most appropriate?

A) Daily 20-minute walks are likely to lower resting heart rate for adults everywhere.
B) Daily 20-minute walks are likely to lower resting heart rate for adults in the city.
C) Adults in the city with lower resting heart rates tend to walk more, but no cause can be identified.
D) Daily walks lower resting heart rate only for the 150 adults in the walking group.

The sample was random from the city's adults, so the results generalize to that city. The walking treatment was randomly assigned, so the study supports a cause-and-effect conclusion.

• A overreaches, because adults outside the city were not sampled.
• C understates the design. Random assignment allows a causal conclusion.
• D is too narrow, because the random sample lets the result extend to adults in the city.

The answer is B.

Example 2. A survey of 500 randomly selected high school students in a state found that students who reported eating breakfast every day had higher average grades than students who did not. Which conclusion is supported?

The sample was random, so the results generalize to high school students in that state. No treatment was assigned, since students decided for themselves whether to eat breakfast. This is an observational study. The supported conclusion is that, among high school students in the state, eating breakfast daily is associated with higher grades. A claim that breakfast causes higher grades is not supported, because other factors, such as sleep or family schedules, could explain both.

Common traps

• Confusing random sampling with random assignment. Random sampling decides who is in the study. Random assignment decides who gets which treatment. Only random assignment supports cause and effect.
• Claiming causation from an observational study. Words like "causes," "leads to," or "improves" need random assignment. Without it, choose "is associated with" or "is related to."
• Generalizing beyond the sampled population. A random sample of one state's students does not support claims about students nationwide. A group of volunteers does not support claims about anyone beyond the volunteers.
• Missing a biased sampling method. Surveying people at a gym about exercise habits, or counting only people who chose to answer an online poll, produces a sample that is not representative even if it is large.
• Picking a weaker claim than the design allows. When a study has both random sampling and random assignment, the best answer usually states the causal conclusion for the sampled population, as in Example 1.