Reading a qualitative study: credibility, context, and transferability

START HERE

The essential idea

Trace a qualitative claim from excerpts through analysis to interpretation. Evaluate sampling, researcher role, and discrepant cases in context. Judge transferability without demanding statistical representativeness.

BUILD THE REASONING · 01

Ask what kind of knowledge the study is trying to produce

A qualitative paper may describe experience, explain a social process, interpret meaning, or investigate a bounded setting. Begin with that purpose and the methodological approach. A study of how first-generation trainees experience supervision should not be judged solely by whether it estimates a population percentage. Conversely, rich interviews do not automatically support a claim that most trainees everywhere have the same experience. The form of the claim must match the sampling, context, and analysis.

Read the research question, setting, participant criteria, recruitment route, data-generation method, and researcher role before evaluating a theme. Twelve interviews can be informative or thin depending on the question, diversity of relevant experiences, depth, and analytic work. A large interview count does not guarantee insight, and a small count does not alone prove inadequacy. Ask why these participants and these encounters could illuminate the phenomenon the authors claim to understand.

Distinguish a reported participant experience from the researcher’s explanation. “Participants described withholding questions” is different from “hierarchical supervision causes withholding.” The latter introduces a mechanism and a causal tone requiring more argument. A strong paper shows the route from the account to the interpretation and acknowledges the conditions under which another explanation could fit. Your appraisal should reproduce that logic in your own words rather than merely announce that the theme sounds plausible.

BUILD THE REASONING · 02

Inspect the evidence chain, including what does not fit

For a fictional study, participants describe supervision meetings. One says, “I save questions until I can make them sound polished.” Another says, “My supervisor asks what I am uncertain about before giving feedback.” A third says, “I ask freely, but the meeting ends before we reach my cases.” Initial codes might include managing an impression, invitation to uncertainty, and time constraint. A theme such as “Permission to be uncertain is negotiated through supervisor responses and meeting conditions” offers an interpretation that connects, rather than simply lists, those codes.

Now look for the bridge: how were transcripts read, segments coded, codes compared, and candidate interpretations revised? The paper should provide enough detail and examples for readers to assess the account without pretending the entire analytic process is mechanical. A table of codes is not sufficient if the central theme makes claims absent from the data. Equally, a quotation can illustrate a theme but cannot by itself establish how common or comprehensive that theme is.

A discrepant case is analytically valuable. The third participant asks freely yet lacks time; their experience challenges an explanation based entirely on fear of evaluation. The authors might distinguish relational permission from structural opportunity, narrow the theme, or explain how the two interact. Removing the case because it is inconvenient would weaken credibility. Treat negative-case analysis as a way to improve the account, not as a ritual sentence that says the team looked for exceptions.

BUILD THE REASONING · 03

Evaluate the researcher’s role and quality procedures

Researchers help shape the data through their questions, role, relationships, assumptions, and interpretive choices. If an instructor interviews their own trainees, participants may protect their evaluation or present the program favorably. A credible account describes that relationship, recruitment safeguards, confidentiality limits, and how the team considered its effect on analysis. A statement that the researchers were objective is weaker than a concrete account of what they did to examine their influence.

Quality procedures should fit the approach. Reflexive thematic analysis, a structured codebook approach, grounded theory, and phenomenological inquiry do not make identical assumptions about coding or researcher subjectivity. Multiple coders, participant feedback, triangulation, audit trails, and detailed contextual description can each contribute, but none is a universal pass token. Ask what the procedure was intended to check, what it found, and whether it changed the interpretation.

For example, participant feedback may reveal that an interpretation misses an important distinction, but participants do not necessarily supply a single final interpretation of a collective pattern. Agreement between coders can support a defined coding procedure but cannot establish that the research question is meaningful or that the interpretation respects context. Use reporting guidance such as COREQ and APA JARS to locate needed information, then make a substantive judgment rather than scoring the number of quality words in the methods section.

BUILD THE REASONING · 04

Make transferability a reasoned comparison

Transferability asks whether a finding may illuminate another setting given its relevant similarities and differences. The reader needs sufficient description of participants, institutional arrangements, culture, relationships, and conditions to make that judgment. “The sample was not random” is not a complete critique of a study designed for depth. “The findings apply everywhere because the stories resonate” is equally weak. Identify which mechanism or experience might transfer and which contextual conditions could change it.

Suppose the study involved unpaid trainees whose supervisor also controlled their course grade, while your setting involves paid employees with a separate evaluator. Impression management may still be relevant, but the authority structure differs. Ask whether meeting time, confidentiality, evaluation stakes, and opportunities for independent consultation are comparable. A local discussion or carefully designed follow-up can explore fit without claiming that the original study measured prevalence in your service.

Write an appraisal with purpose, data/analysis strengths, a specific weakness, and a bounded application. For the fictional supervision study: its contrasting accounts illuminate both permission and opportunity to ask questions; the instructor-interviewer relationship and single-program setting limit interpretation; a local supervisor could examine invitations and meeting structure while seeking trainees’ feedback through a protected route. That conclusion uses the research without turning it into a universal recipe.

FROM CONCEPT TO DECISION

A theme that must make room for time

1. 1

   Check the breadth of the conclusion

   The accounts support a concern about evaluation for some participants. They do not establish one universal cause, especially when other participants identify a different barrier. The causal language needs a stronger explanation or a narrower claim.

2. 2

   Use the contradictory accounts

   Distinguish willingness to ask from the opportunity to ask. An improved interpretation might include both relational permission and protected meeting time, then examine where the dataset supports their interaction or separation.

3. 3

   Appraise the interview relationship

   Look for recruitment and confidentiality safeguards and an account of how the teaching role shaped questions and responses. The relationship does not automatically invalidate the study, but unexplained power effects reduce confidence in a broad interpretation.

4. 4

   Translate carefully to a new setting

   A supervisor elsewhere can examine their invitations and scheduling, but should not assume the same prevalence or cause. Local trainees may face different evaluation stakes, access, and support arrangements.

MAKE THE REASONING YOURS

Work through a new situation

READ WITH A PURPOSE

Best references

These readings deepen the specific models, evidence, and decisions in this lesson.

1. 01
   Tong, Sainsbury, and Craig: COREQ (2007)

   Use the original interview/focus-group reporting framework to inspect team/reflexivity, study design, analysis, and evidence; a checklist is a guide to appraisal rather than a substitute for it.

2. 02
   APA: Qualitative and Mixed-Methods Reporting Standards

   Read the original task-force report for methodological integrity, researcher perspective, context, and evidence supporting interpretations.

3. 03
   NIH OBSSR: Mixed Methods Research

   Compare the contribution of qualitative explanation with quantitative estimates and examine what is required when a paper integrates both.