AP Stats Experimental Design: Explained Simply

In AP Statistics, experimental design is about building a study that can show cause and effect. An experiment deliberately imposes treatments on experimental units, uses random assignment to make the groups comparable, and compares the results. Get the four principles — comparison, random assignment, control, and replication — right, and you can answer almost any design question on the exam.

This guide walks through the vocabulary, the standard designs, and the exact language graders look for on free-response questions.

What is the difference between an experiment and an observational study?

This is the first distinction every AP Stats student needs to nail, because it decides what conclusions you are allowed to draw.

If a study compares people who chose their own groups (smokers vs. non-smokers, students who take AP classes vs. those who do not), it is observational, no matter how large or careful it is. That means any link you find could be explained by a confounding variable — something associated with both the explanatory variable and the response.

Key vocabulary

• Experimental units: the objects that receive treatments. When they are people, call them subjects.
• Explanatory variable (factor): the variable the researcher manipulates.
• Levels: the specific values of a factor.
• Treatment: a specific condition applied to units. With two factors, each treatment is a combination of levels.
• Response variable: the outcome you measure.

A quick example: testing three fertilizer amounts (0, 5, and 10 grams) and two watering schedules on tomato plants gives 2 factors and 3 × 2 = 6 treatments.

What are the four principles of experimental design?

Memorize these. A well-designed experiment uses all four.

1. Comparison. Use at least two treatment groups, which may include a control group that gets a placebo or no treatment. Without comparison, you cannot tell whether the treatment did anything.
2. Random assignment. Use a chance process to assign units to treatments. This creates roughly equivalent groups, so the only systematic difference between them is the treatment.
3. Control. Keep other variables the same for all groups (same soil, same light, same instructions). This reduces variability in the response and prevents those variables from becoming confounders.
4. Replication. Apply each treatment to enough units that a real treatment effect can be distinguished from chance variation.

Why random assignment matters so much

Random assignment is what lets you say "the treatment caused the difference." Because chance, not the researcher or the subjects, decided who got which treatment, lurking variables tend to balance out across groups.

Do not confuse it with random sampling. They do different jobs, and the AP exam tests this distinction constantly.

A study can have one, both, or neither. An experiment on volunteers uses random assignment but not random sampling, so you can conclude causation — but only for subjects like those in the study.

How do you describe a random assignment on the AP exam?

Graders want a method that someone else could follow exactly. A strong description has three parts: label, randomize, and assign.

Worked example. Forty volunteers will test a new study app (treatment) versus the old one (control).

Number the subjects 1 to 40. Use a random number generator to select 20 different integers from 1 to 40, ignoring repeats. Subjects with those numbers use the new app; the remaining 20 use the old app.

Alternatives that also earn credit:

• Write all 40 names on identical slips of paper, mix them in a hat, and draw 20 without replacement for the new app.
• Shuffle 40 identical cards (20 marked A, 20 marked B) and hand one to each subject.

Common mistakes that lose credit: flipping a coin for each subject without addressing unequal group sizes when equal groups were required, forgetting to say "ignore repeats," or describing the method so vaguely that it is not reproducible ("randomly split them into groups").

What is a completely randomized design?

In a completely randomized design, units are assigned to treatments completely at random — every unit has the same chance of landing in any group. It is the simplest design and works well when the units are fairly similar.

The weakness: if units differ in a way that affects the response (age, prior ability, field location), that variability gets mixed into each treatment group and can make a real effect harder to detect.

What is a randomized block design?

A block is a group of units that are similar in a way expected to affect the response. In a randomized block design, you form blocks first, then randomly assign treatments separately within each block.

Example. You want to test whether a new review method improves calculus quiz scores. Prior math background strongly affects scores. Block by whether students have taken precalculus honors or regular, then randomly assign half of each block to the new method and half to the standard one.

Why block? Blocking accounts for a known source of variability, so the comparison between treatments is more precise. Think of it this way:

• Control holds a variable constant for everyone.
• Blocking lets a variable differ, but makes comparisons within groups where it is roughly constant.
• Randomization handles all the variables you did not think of.

A favorite exam phrasing: "Blocking is to experiments what stratifying is to sampling." Both group similar units first to reduce variability.

Matched pairs design

A matched pairs design is a special case of blocking with blocks of size two. Either:

• Pair similar subjects and randomly assign one of each pair to each treatment, or
• Give each subject both treatments in a random order.

For example, each student takes one practice test with background music and one in silence, with a coin flip deciding the order. Randomizing the order protects against learning or fatigue effects. Analyze matched-pairs data using the differences within each pair.

What are placebo, blinding, and double-blind?

• Placebo effect: subjects respond to the idea of receiving treatment, even an inactive one. A placebo group gives you a fair comparison.
• Single-blind: either the subjects or the people measuring the response do not know which treatment each subject received.
• Double-blind: neither the subjects nor those who interact with them and measure the response know who got which treatment.

Blinding prevents expectations from influencing behavior or measurement. On an FRQ, always explain why blinding matters in context — for example, "a researcher who knows a patient got the new drug might rate their pain improvement more favorably."

How should you write conclusions about experiments?

Scope of inference is a near-guaranteed exam topic. Use this rule:

When results are statistically significant, the difference between groups is too large to plausibly be explained by the chance variation in random assignment alone. You will connect this to p-values once you study hypothesis testing.

Tips for full credit on experimental design FRQs

1. Always answer in context. "Control other variables" earns little; "keep the amount of daily sunlight the same for all plants" earns credit.
2. Name the specific confounder and explain both links — how it relates to the explanatory variable and to the response.
3. Justify your blocking variable. Say why it is expected to affect the response.
4. Be precise with randomization. Label, use a random process, and say how units end up in groups.
5. Match the conclusion to the design. Causal language only with random assignment.

For more on how these questions are scored, see our AP Stats FRQ guide, and keep the AP Stats formula sheet guide handy for the inference units that follow.

How Ivystar can help

Experimental design rewards precise, context-specific writing — exactly the skill our tutors drill with students one-on-one. Our AP Exam Prep program pairs you with tutors from schools like Harvard, Penn, and Stanford, and 91% of Ivystar AP students score a 5. Estimate where you stand with the AP Statistics score calculator, then book a free 20-minute strategy call.