Confidence intervals explained with a counseling example

START HERE

The essential idea

Calculate an interval from a mean, standard deviation, sample size, and supplied critical value. Explain confidence without confusing it with individual variability. Identify why a narrow interval can still support a limited conclusion.

BUILD THE REASONING · 01

Name the quantity the interval estimates

An interval estimate is meaningful only after the target quantity is clear. A mean change in a distress score, a difference between treatment-group means, a proportion attending a session, and an individual client’s likely future score are different targets. Their formulas and assumptions differ. Before interpreting a pair of endpoints, write the outcome, direction, population or comparison, and time period in ordinary language. This prevents “the interval” from becoming an impressive number with no defined meaning.

Consider an original fictional service evaluation: 36 adults have complete pre/post scores on a continuous coping scale, with higher values representing improvement when after-minus-before changes are positive. Mean change is 4 points and the sample standard deviation of changes is 6. The immediate target is the population mean change represented by the sampling/model assumptions, not each person’s change and not a causal program effect. The design is uncontrolled, so an interval cannot remove time-related alternative explanations.

The standard deviation of 6 describes spread among individual change scores. The standard error of the mean describes sampling variability of the estimated mean. Under the simple independent-observation model, SE = SD/√n = 6/√36 = 1. These quantities share the original score units but answer different questions. A service can estimate an average precisely while clients’ individual experiences remain highly variable.

BUILD THE REASONING · 02

Construct the interval one step at a time

For a normal-theory t interval for a mean, use estimate ± critical value × standard error. With 36 independent change scores, the degrees of freedom are 35. A supplied 95% two-sided t critical value is approximately 2.030. The margin of error is 2.030×1 = 2.030 points, giving an interval of 4−2.030 to 4+2.030, or 1.97 to 6.03. Report rounding appropriate to the measurement; extra decimals do not make the data more accurate.

Why use a t critical value? The population standard deviation is unknown and is estimated from the sample. The t distribution reflects that additional uncertainty under its assumptions; with larger degrees of freedom it approaches the standard normal distribution. This does not mean every counseling outcome is normally distributed. Inspect the change scores, outliers, missingness, and dependence. A small strongly skewed sample, a bounded outcome near a ceiling, or repeated clients within therapists may need another model or qualified statistical advice.

If the same SD of 6 came from only nine independent observations, SE would be 2 and the 95% critical value for eight degrees of freedom would be about 2.306. With the same mean of 4, the interval would be 4 ± 4.612, approximately −0.61 to 8.61. The wider interval reflects less information under this hypothetical comparison; it does not mean the nine-person program necessarily had more variable clients, since we held the SD constant.

BUILD THE REASONING · 03

Explain what 95% does and does not mean

In the usual frequentist interpretation, the 95% refers to the interval-producing procedure: over repeated samples under the assumptions, about 95% of intervals constructed this way would contain the fixed population mean. For one realized interval, the mean is either inside or outside; the procedure does not assign a 95% probability to that fixed value being in this particular interval. A concise practical explanation is that the estimate has this range of statistical uncertainty under the model, followed by the important design limits.

The interval 1.97–6.03 is not the range containing 95% of clients’ changes. Individual change scores may extend well beyond it, because the interval estimates an average. A prediction interval for a new observation addresses another question and includes individual variability. A confidence interval for a proportion or a treatment difference also needs its own appropriate method; the one-sample formula should not be copied merely because every result is called an interval.

For a matched two-sided test under the same model and assumptions, a 95% interval excluding zero corresponds to rejecting the zero-value hypothesis at the .05 level. That relationship is useful, but treating the interval only as a significance marker discards most of its information. Ask whether it includes effects that would lead to different practical decisions, and whether the precision is adequate for the purpose of the study.

BUILD THE REASONING · 04

Use uncertainty in a decision without hiding bias

Suppose a service defined a 5-point average improvement as worthwhile for its staffing burden before seeing data. The 1.97–6.03 interval includes changes below and above that value. The estimate suggests positive average change under the model, while its practical adequacy remains uncertain. The 9-person interval additionally includes zero and negative change. Neither result alone proves an absence of benefit or guarantees benefit for every client.

Larger samples can narrow sampling uncertainty, but they do not repair a biased process. If only clients who improved supplied post scores, a narrow interval around their mean could precisely describe an unrepresentative subset. If the scale fails to measure the intended coping construct, precision does not make the interpretation valid. Report who contributed data, who was missing, the scoring process, and plausible alternative explanations beside the interval.

A complete report names the estimate, interval, sample, model, and decision-relevant limitation. For the fictional service: “Among 36 complete pairs, mean coping-score change was 4 points, 95% CI 1.97 to 6.03. This describes observed change in the analyzed sample; incomplete follow-up and the absence of a comparison condition limit population and causal interpretation.” If follow-up was complete, remove that missingness caveat rather than copying it mechanically; limitations must match the actual study.

FROM CONCEPT TO DECISION

The director asks whether every client gained two points

1. 1

   Locate the target mismatch

   The interval estimates an average, while the director’s claim concerns the distribution of individuals. The supplied SD and the presence of declines already show why the average’s lower endpoint cannot be used as a minimum individual benefit.

2. 2

   Explain the calculation

   The standard error of 1 comes from dividing the individual SD of 6 by √36. The narrowing is about precision of the mean, not compression of each client’s experience into a two-to-six-point band.

3. 3

   Provide the right additional summary

   Show the distribution of changes and, if justified, the proportion meeting a prespecified meaningful individual-change criterion with appropriate uncertainty. Do not invent such a criterion from the mean interval’s endpoints.

4. 4

   Bound the program claim

   Even a well-estimated average change does not isolate the program’s causal contribution in an uncontrolled evaluation. The report should preserve that distinction when describing what further comparison or follow-up would help.

MAKE THE REASONING YOURS

Work through a new situation

READ WITH A PURPOSE

Best references

These readings deepen the specific models, evidence, and decisions in this lesson.

1. 01
   NIST: Confidence Intervals

   Review the repeated-sampling interpretation and why an interval for a mean is different from an interval for an individual observation.

2. 02
   NIST: Confidence Limits for the Mean

   Reproduce the mean ± t×SE calculation and inspect how sample size and variability affect interval width.

3. 03
   American Statistical Association: Statement on P-Values

   Use the six principles to distinguish a statistical model check from effect magnitude, practical value, and the probability that a hypothesis is true.

4. 04
   Cochrane Handbook, Chapter 15: Interpreting Results

   Connect uncertainty with meaningful effects and decision context rather than treating interval inclusion of zero as the entire interpretation.