Correlation, causation & design

START HERE

The essential idea

A correlation describes a relationship between variables. It does not by itself show that one variable caused the other. A third variable, reverse direction, or selection effects may explain the relationship.

Random assignment to conditions can strengthen causal inference in an appropriately designed experiment. Random sampling addresses a different question: how participants are selected from a population. Neither makes every possible study limitation disappear.

When reading an article, identify the question, design, sample, variables, measures, analysis, and limitations. Check whether the author’s conclusion stays within the evidence.

BUILD THE REASONING · 01

What would have happened without the intervention?

A causal question compares outcomes under alternative conditions. We cannot observe the same person at the same time both receiving and not receiving the same intervention, so a design creates a credible comparison. In an observational tutoring study, prior preparation or motivation may affect both tutoring attendance and grades. Those shared causes can produce an association even if tutoring has no effect, or distort a real effect.

Random assignment can make alternative causes comparable on average before treatment, but execution still matters: allocation concealment, adherence, missing outcomes, measurement, and selective reporting can affect the result. Blinding participants may be impractical in counseling research; independent outcome rating or standardized procedures may still reduce some bias. Each safeguard addresses a particular threat.

BUILD THE REASONING · 02

Compare designs by the claims they permit

A one-group pre/post study measures change but cannot separate intervention from history, maturation, repeated testing, or regression toward the mean. A nonrandomized comparison adds information but can retain selection differences. An interrupted time series uses repeated observations before and after a change to examine level/trend shifts; concurrent events, seasonality, and serial correlation still need attention. A carefully designed single-case multiple-baseline study staggers intervention introduction across behaviors, settings, or participants and looks for replicated changes at those introductions.

Choose a design that is feasible, ethical, and suited to the question, then state what would weaken its inference. Do not withhold needed care merely to create a comparison. Single-case experimental methods require repeated reliable measures and careful design; one person’s before/after anecdote is not a multiple-baseline experiment.

BUILD THE REASONING · 03

A worked difference-in-differences contrast

A fictional program group’s mean score rises from 50 to 62, a gain of 12. A comparison group rises from 48 to 55, a gain of 7. The difference in changes is (62−50)−(55−48) = 5 points. Reporting only the program’s 12-point gain would attribute the comparison group’s 7-point time trend to the program as well.

The 5-point contrast is not automatically causal. A difference-in-differences interpretation needs a credible account that, without the program, the groups would have followed comparable trends, along with other assumptions such as no differential simultaneous intervention and appropriate handling of composition and dependence. Multiple pre-period observations can inform the trend assessment but cannot prove the unobserved counterfactual. Report the design and uncertainty, not just the arithmetic.

FROM CONCEPT TO DECISION

The apparent training effect shrinks with a comparison

1. 1

   Calculate the observed contrast

   The workshop campus gains 12 and the comparator gains 7, leaving a 5-point difference in changes.

2. 2

   Identify shared history

   The textbook could contribute to gains at both campuses. The comparison helps account for common time-related change only if its influence and other trends are sufficiently comparable.

3. 3

   Identify differential history/composition

   Changed admissions at one campus may alter cohort preparation and undermine the comparison. Inspect who was measured and the pre-intervention trends.

4. 4

   Write a bounded conclusion

   Describe the 5-point adjusted change contrast, then state that differential admissions and unverified trend assumptions limit causal interpretation.

MAKE THE REASONING YOURS

Work through a new situation

READ WITH A PURPOSE

Best references

These readings deepen the specific models, evidence, and decisions in this lesson.

1. 01
   CONSORT 2025: Reporting Randomised Trials

   Inspect the original reporting guideline for allocation, participant flow, prespecified outcomes, analysis, and harms. Transparent reporting permits appraisal; a completed checklist does not guarantee low bias.

2. 02
   Cochrane Handbook, Chapter 25: Non-Randomized Studies

   Study confounding, selection, and intervention classification when evaluating the nonrandomized comparisons in this lesson.

3. 03
   What Works Clearinghouse: Procedures and Standards Handbooks

   Consult the official evidence-review standards for group and single-case designs; these provide a deeper check on the assumptions behind repeated and staggered comparisons.