Choosing a measure: population, purpose, language, and accessibility

START HERE

The essential idea

Measure selection begins with a sentence that names the construct, population, setting, decision, and consequence. 'We need a good mental-health test' is too broad. 'We need a brief, understandable measure of functional difficulty to monitor adult outpatient goals every month, alongside interview information' creates criteria for comparison. Screening for possible depression, establishing a diagnosis, comparing functioning, and selecting a career path are different uses. A measure supported for one may be inappropriate for another even if it is popular, short, and highly reliable.

BUILD THE REASONING · 01

Write the decision before shopping for a measure

Measure selection begins with a sentence that names the construct, population, setting, decision, and consequence. 'We need a good mental-health test' is too broad. 'We need a brief, understandable measure of functional difficulty to monitor adult outpatient goals every month, alongside interview information' creates criteria for comparison. Screening for possible depression, establishing a diagnosis, comparing functioning, and selecting a career path are different uses. A measure supported for one may be inappropriate for another even if it is popular, short, and highly reliable.

Specify whose perspective matters. Self-report captures the person's experience; an observer report captures what another person sees in a particular context; a performance task samples behavior under defined conditions. They are not interchangeable windows into one unquestionable truth. A caregiver's report may be helpful without replacing the young person's account. If repeated tracking is planned, ask whether the measure is responsive to the kind of change expected and whether the time frame fits the intervention. A lifetime-history measure cannot function as a weekly outcome simply because its items are clinically interesting.

Decide how the result will be used before looking at a cutoff. Will a positive screen trigger an interview, an urgent assessment, a referral, or an eligibility decision? What happens after a negative result when direct concerns remain? Who interprets the score, and what training or authorization is required? A tool is not feasible if the service cannot provide the follow-up its use requires. Include the burden to the client, privacy of data collection, availability in an appropriate language/format, and a non-test route when measurement is unsuitable or declined. These are part of the assessment design, not optional conveniences.

BUILD THE REASONING · 02

Compare the evidence for the intended interpretation

Create an evidence matrix for each candidate: construct/content, intended use, relevant population and norms, reliability/precision, validity evidence, decision accuracy when applicable, responsiveness, administration demands, access, licensing, and limitations. Validity is not a stamp that a test permanently owns. Ask what evidence supports this score interpretation for these people and this decision. Internal consistency alone does not show that a scale measures the intended construct, predicts a meaningful outcome, or works equivalently across languages. A very high coefficient can coexist with repetitive items or a narrow construct that misses the current clinical question.

For a fictional comparison, Measure A is a short symptom checklist with excellent internal consistency in English-speaking university students. Measure B assesses daily functioning, has somewhat lower but adequate precision for its intended use, and has evidence in the relevant adult outpatient population and validated language version. If the task is monthly functional monitoring for adults who need that version, B may fit better despite A's higher coefficient. That is a reasoned provisional choice, not a universal endorsement of B. Read the actual studies/manual and consider whether the difference in precision matters for individual decisions.

Screening accuracy also depends on the setting. In an invented group of 1,000 people with a 10% prevalence of the target condition, a screen with 80% sensitivity identifies 80 of 100 cases. With 90% specificity, it also flags 90 of the 900 noncases. Only 80 of 170 positive results, about 47%, correspond to cases in this example. The numbers are not characteristics of any real test. They demonstrate why a positive screen is not a diagnosis and why a result's meaning cannot be read from sensitivity alone without considering prevalence and follow-up assessment.

BUILD THE REASONING · 03

Preserve access without pretending equivalence

An accommodation aims to remove an access barrier while preserving the construct being assessed. An adaptation changes content, language, format, or context in ways that may require new evidence for interpretation. The boundary depends on the instrument and construct. Enlarging print may support access for a vision barrier without changing a self-report construct; reading aloud may alter a task intended to assess independent reading. Follow the measure's authorized guidance and document the change. Do not assume that an adjustment is harmless or invalid solely because it is an adjustment.

Translation requires more than replacing words. Meaning, cultural relevance, response options, administration, and score comparability need examination. A bilingual staff member's improvised translation may help a clinical conversation, but it does not create a validated translated instrument. The ITC guidelines address adaptation evidence and documentation. If no appropriate version exists, consider a different measure, an interview-based assessment, or carefully bounded descriptive use without unsupported normative claims. Explain this limitation rather than presenting a translated total with the same authority as a validated score.

Evaluate the client's actual experience with the tool. Can they navigate it with their device or assistive technology? Are the items understandable without a relative hearing private responses? Does the response period make sense with irregular work or caregiving? A measure may be accessible in principle but unusable in the current setting. Invite questions and record relevant administration conditions. Support should not coach the person toward a preferred answer. If the clarification changes an item's meaning, follow the manual or qualified consultation rather than silently improvising and calling the result standard.

BUILD THE REASONING · 04

Defend the choice and monitor its use

Write a short selection memo with the proposed use, compared candidates, evidence favoring the choice, known limitations, administration plan, interpretation rule, and follow-up pathway. Name the exact version and source dates. A defensible memo might state that a functioning measure was chosen because it matches the goal and language, that symptom diagnosis remains an interview task, and that scores will be interpreted with functional examples rather than as a universal eligibility cutoff. This makes the reasoning reviewable and reduces the chance that a later staff member expands the tool into an unsupported purpose.

Pilot the workflow before calling it routine. Check completion, missing items, time burden, comprehension problems, scoring accuracy, whether results reach the responsible clinician, and whether the planned follow-up happens. Evaluate patterns across relevant groups while protecting privacy and avoiding causal claims from small numbers. If one group has more missing responses, investigate language, format, relevance, and conditions before attributing the difference to motivation or ability. A technically valid instrument can still fail in an implementation that does not allow people to complete it or clinicians to use it appropriately.

Revisit the choice when the purpose, population, version, delivery method, or consequences change. A tool introduced for low-stakes discussion should not quietly become the sole gatekeeper for services. A new digital version may require evidence beyond the paper version's reputation. Track whether the measure adds useful information to decisions and whether it causes avoidable burden or misclassification. Retain a way to challenge a result using clinical evidence and the person's account. Good assessment practice is a maintained chain from purpose through evidence to action, not a one-time purchase of a respected test.

FROM CONCEPT TO DECISION

The highest coefficient is not the best match

1. 1

   Fix the intended use

   The decision concerns change in daily functioning, not ranking student symptoms or diagnosing from a total. Keep that purpose visible while comparing evidence.

2. 2

   Compare more than reliability

   A's coefficient does not repair the construct, population, and language mismatch. B's fit may justify its greater burden if its precision and evidence are adequate for the actual use.

3. 3

   Plan accessible administration

   Verify B's exact version, approved format, comprehension, privacy, scoring, and interpretation limits. Keep an alternative route for clients for whom the measure is unsuitable.

4. 4

   Pilot the decision chain

   Check missing responses and whether functional results inform the review. Record that neither candidate is being used as a standalone diagnostic test or service-denial threshold.

MAKE THE REASONING YOURS

Work through a new situation

READ WITH A PURPOSE

Best references

These readings deepen the specific models, evidence, and decisions in this lesson.

1. 01
   AERA, APA, and NCME — Standards for Educational and Psychological Testing (2014), open access

   Read the validity, reliability/precision, fairness, and score-interpretation chapters. Evidence supports a proposed interpretation and use, not a universally valid test label.

2. 02
   International Test Commission — Guidelines for Translating and Adapting Tests (2017)

   Examine cultural/linguistic adaptation, administration, evidence, and documentation. A literal translation or an improvised interpreter reading does not automatically preserve score comparability.

3. 03
   WHO — WHODAS 2.0

   Review the intended functional domains and available administration formats as an example of choosing a measure for a defined purpose.