Is the Randomized Controlled Trial Really the 'Gold Standard'?

Why You Need to Know This 'Method'

People who took a new drug got better. Was it really because of the drug, or were the people who were prescribed the drug healthier to begin with, did they visit the hospital more diligently, or did they get better on their own with time? Nearly all of evidence-based medicine hangs on this question. What we always want to know is causality (the drug made them better), but what the data shows is usually correlation (the outcomes differ between people who took the drug and those who did not).

The troublemaker that prevents us from reading correlation as causality is confounding. A third factor that affects the outcome and is also tied to the likelihood of receiving treatment, such as age, disease severity, lifestyle, and the doctor's intuition that "let's use this drug for this patient," is confounding. These factors are intertwined with the drug's effect and difficult to disentangle. Confounding and adjustment are covered in detail in "If People Who Take It Live Longer, How Were They Different to Begin With? — Confounding" and "What "Adjusted for Age, Sex and BMI" Actually Did — How to Read Confounding and Adjustment". One thing to remember from this article is that conclusions obtained from observation alone can be used to generate hypotheses, but are difficult to verify (PMID 36050978).

A randomized controlled trial (RCT) is a design that directly takes aim at this confounding by dividing participants into two groups by chance and comparing the results. Thanks to achieving this goal well, it has held the position of the gold standard of medical evidence for over half a century (PMID 38508234). This article dissects the RCT as a methodology. It covers what kind of tool it is meant to answer, why it works, how it is actually designed, and where it reveals its limitations. The goal is to acquire the eye to gauge for yourself what to believe and what to doubt when you open the Methods and Results of a paper.

How It Works: The Magic of Randomization

The heart of an RCT is the act of flipping a coin to divide people, which is randomization. Pure chance determines who goes to the treatment group (the group receiving the treatment under test) and who goes to the control group (the group serving as the comparison baseline).

To know causation perfectly, we would need to compare the same person at the same time when taking a drug and when not taking it. This hypothetical comparison is called a counterfactual, but it is impossible in reality because a person can only experience one of the two. Randomization bypasses this impossibility. If you divide a sufficiently large number of people with a coin, the two groups become, on average, a collection of almost identical people. Age and severity, as well as unmeasured factors, are evenly distributed across both groups. Therefore, if a difference arises in the outcomes of the two groups after assignment, what remains as the cause of that difference is the treatment (PMID 36050978, PMID 38320494).

This is the point that observational studies cannot imitate. An observational study can only adjust for measured confounders with statistics, and has no way to touch factors that were not measured or whose existence is even unknown. Randomization simultaneously balances known and unknown confounders regardless of whether they were measured (on average, when the sample is large enough). One review notes that randomization avoids confounding from unmeasured variables or treatment choices, enabling a causal interpretation of treatment effects (PMID 38320494). This single property is why the RCT is called the gold standard.

Actual Procedures: The Five Pillars Supporting RCTs

Randomization alone does not make a good trial. A well-designed RCT builds multiple layers of defense. The following five items are the first ones to check in a paper's Methods.

① Random assignment and allocation concealment. Random assignment is setting the groups using a chance device like a coin toss or a computer-generated random number. However, this alone is not enough. If researchers can know in advance which group the next patient will be assigned to, even unconsciously, choices like "let's put this patient in the treatment group" can creep in and contaminate randomness. Allocation concealment is a device that prevents the person in charge from seeing the assignment table beforehand, hiding the results until the exact moment assignment is finalized. It is the first lock that ensures randomization is properly maintained.

② Blinding and placebo. If allocation concealment is a device that hides things only up to the moment of assignment, blinding is a device that keeps everyone from knowing who is receiving what from after assignment until the trial ends. A placebo is a fake pill that looks identical but has no active ingredient, given to the control group. If patients know they are taking the real drug, their symptoms can improve from expectation alone (the placebo effect), and if doctors know, they might care for the treatment group more carefully or judge the outcomes more favorably (performance and ascertainment bias; a phenomenon where care or judgment varies depending on the group). Therefore, a double-blind study—where patients, doctors, and evaluators all do not know—is ideal.

When blinding breaks down, bias leaks in. In one unblinded (non-masked) device trial, unexpectedly more non-cardiovascular side effects were caught in the treatment group, leading to the trial being terminated early. The trial team suspected the rate at which the treatment group contacted the hospital on their own as the cause and analyzed it separately. In the overall cohort, the rates of spontaneous clinic visits were similar between the two groups (13.3% versus 9.7%, P=0.12), but in a post hoc analysis divided into 24 subgroups, a large difference was confirmed only in Danish participants (24.4% versus 2.3%). The authors interpret this as due to performance bias among the participants there (PMID 42129909). However, blinding is not always possible. When effects are blatantly felt, such as with hallucinogenic drugs, participants easily figure out their group, making a double-blind design practically difficult, and some point out that insisting on it as the golden standard is rather unrealistic (PMID 41424285).

③ Control group. Saying "I took the medicine and got better" is meaningless without something to compare it to. The control group is a baseline that filters out the share that would have gotten better on its own, the share that would have improved from a placebo, and the share that time would have resolved. A single-arm design observing only the treatment group without a control group is recommended to use an RCT for Phase 3 trials that finally confirm the efficacy of a new drug, and one survey asks in its title whether a single-arm Phase 3 design is an "oxymoron," concluding that when used for confirmatory purposes, justification and the potential for bias must be reported more clearly (PMID 38508234).

④ Intention-to-treat analysis (ITT). This is the question of how to count patients who stopped taking their assigned drug midway or switched to another treatment—in other words, dropouts and switchers. Intuition says, "Let's analyze only the people who actually took the drug to the end" (this is called a per-protocol analysis, meaning counting only those who followed the protocol). But the moment you do that, randomization is broken. This is because people who took the drug to the end are fundamentally different people to begin with, having no side effects or showing high compliance. ITT is the principle of analyzing patients right in the groups they were originally assigned to, regardless of whether they actually took the medication. Though it may look less impressive, it prevents the bias where dropout makes the two groups that were matched by randomization different. Trials using analyses that deviated from standard ITT showed larger treatment effects than ITT trials. This is what a meta-epidemiological study found by comparing 43 reviews and 310 trials (PMID 26016488). However, because the definition of ITT varies across literature and can be misused, a review centered in the rehabilitation field emphasizes that researchers must disclose which definition was used and how dropouts and missing data were handled (PMID 38320245).

⑤ Pre-registration and pre-specification. If you look at all the data and cherry-pick only the results that turned out significant to publish, you can turn coincidence into skill. Therefore, modern RCTs pre-register primary endpoints (the most important outcome metrics), analysis plans, and sample sizes before starting. Without this pre-specification, no matter how large a trial is, it gets tossed around by post hoc analyses and subgroup fishing. One paper summarizes that we should not excessively rely on small trials, their meta-analyses, subgroups, or post hoc analyses, but rather infer based on the results of pre-specified primary endpoints. As an example of what happens when this principle is not followed, there is criticism that the U.S. guideline for aspirin primary prevention (taken by people before experiencing cardiovascular disease for preventive purposes) set an age upper limit by unduly relying on a single trial whose results were uninformative, which led to a guideline contradicting the overall evidence. The authors view that that trial did not show the lack of benefit of aspirin in that age group. The conclusion of this paper is not that people should or should not take aspirin, but that prescriptions should be made by weighing individual benefits and risks rather than applying a uniform guideline based on a single criterion like age. Whether to take it should be discussed with your primary care physician (PMID 40165541).

How to Read Results: Effect Size, Confidence Interval, and NNT

If you understand the Methods, it is time to read the Results, which is where beginners are most often tricked. The tools used here were covered in previous articles, so their definitions are only briefly revisited.

Relative risk reduction is a way of speaking in terms of the original risk, such as "reduced risk by 50%", and absolute risk reduction is how many percentage points the risk dropped. If risk dropped from 2% to 1% it is half on a relative scale, but 1 percentage point on an absolute scale. Looking only at relative risk makes a small effect look huge (If the Risk Was Cut in Half, How Many Fewer People Got Sick? — Relative and Absolute Risk, What Does "50% Lower Risk" Mean in People? — Reading Relative Risk, Absolute Risk, and NNT).

Number needed to treat (NNT) is the inverse of absolute risk reduction, meaning how many people must be treated to benefit one person. If absolute risk reduction is 1%, the NNT is 100. It is intuitive, but since it varies depending on the baseline risk, follow-up duration, and comparator, citing the number without context leads to misreading (PMID 38876401, PMID 40839106). When the risk difference is not statistically significant, the confidence interval for NNT splits into two disconnected pieces that include unrealistic values, so caution is required in using NNT for guidelines or policy decisions. There has even been a strong argument that "it is time to gracefully retire" it (PMID 40782977).

Confidence interval (CI) shows how much a measured value can wobble using two numbers inside parentheses (How Much Does a Number Counted Once Wobble? — The Two Numbers in the Parentheses, the Confidence Interval). It is as important as the point estimate (the size of the effect) because it tells you how precise the estimate is. A confidence interval must always accompany NNT (PMID 38876401). If the confidence interval straddles the value of no effect (a difference of 0, a risk ratio of 1), the possibility of no effect cannot be ruled out.

Subgroup analysis is an analysis that splits all participants into subgroups like age or sex. "There was no effect overall, but there was in women aged 65 and older" is a typical sentence. If you slice and dice many subgroups, pieces that look significant purely by chance are bound to appear. You must not rely too heavily on subgroups or post-hoc analyses that were not specified before the trial to draw conclusions, and should judge by the results of the pre-specified primary endpoint (PMID 40165541).

Strengths, Limitations, and Debates: Do Not Treat the "Gold Standard" as Absolute

This is the power of RCTs. Now it is time to look honestly at their limitations. A well-designed RCT is second to none in internal validity (whether the cause-and-effect relationship is true within this study). However, its strengths are the very seeds of its limitations.

External Validity — Do These Results Apply to My Patient, Too. External validity is the degree to which trial results fit patients outside the trial. Trials tend to be convenience samples composed of patients enrolled sequentially under strict eligibility criteria. Therefore, even though comparisons between groups can be read as causal, the assumption that the results will carry over to other patient populations (transportability) requires separate justification. One paper explains this by pointing out that the random assignment in causal inference (dividing participants into groups) is entirely different from random sampling in surveys (drawing a representative sample from a population), and it argues that trial samples can hardly be considered representative of specific patient populations (PMID 37835368). In the era of precision medicine and real-world data, limitations regarding the generalizability, cost, and applicability of RCTs to low- and middle-income countries are also coming to the fore (DOI: 10.38106/lmrj.2026.8.1-01).

The Trap of Averages — Effect Sizes Belong to the 'Average Patient'. What an RCT yields is usually the average treatment effect. Yet behind the average, people who benefited greatly and people who were actually harmed may be mixed together. This heterogeneity of treatment effect is a core topic in the social sciences and economics as well, and methods to estimate the distribution of individual effects beyond the average are actively being developed (DOI: 10.1007/s11135-026-02615-2). An approach that attempts to answer for a single patient is the N-of-1 trial. This is a design that tests individualized treatments on a single patient (a tiny minority), and personalized gene therapies for very small patient populations are designed to be tested in this manner, requiring the natural history baseline and personalized evaluation metrics to be built into the design (PMID 38527301, PMID 39532857).

Ethics, Cost, and Rare Outcomes. Intentionally withholding a treatment that might work from half of the people inherently creates ethical tension. The use of placebos, informed consent, the appropriateness of the control group, and post-trial access to treatment are all living controversies (DOI: 10.31235/osf.io/uxst9_v2). Furthermore, large-scale RCTs are slow and expensive. There are also many questions that are difficult or outright impossible for RCTs to catch, such as very rare side effects or outcomes decades down the line.

Pragmatic vs. Explanatory — What Was Actually Asked in the First Place. Half a century ago, Schwartz and Lellouch divided trials into two streams (PMID 39396584). Explanatory trials ask, "Does this treatment work in principle under ideal conditions?", while pragmatic trials ask, "Does this treatment actually help in real-world practice?" Because the two have different purposes, how one reads them must also differ. Trial results can only be properly interpreted by understanding the choice between explanatory and pragmatic designs, and depending on the design, the meaning of a null result can become ambiguous (PMID 36566693). However, a counterargument in the digital health era is that this distinction is not a binary continuum, and modern trials are most useful when they are both pragmatic and explanatory at the same time (PMID 39396584). Registry-based randomized trials, a branch of pragmatic designs, are an attempt to retain the strengths of randomization while shedding its slow and expensive weaknesses (PMID 38320494).

Where to Ask About Causes Without an RCT. So, can we know nothing in the absence of an RCT? That is not the case. A sprawling Cochrane meta-epidemiological study comparing effect estimates from RCTs and observational studies addressing the same questions concluded that the difference between the two is either nonexistent or very small (ratio of effect ratios 1.08, 95% CI 1.01–1.15; however, the authors rated the certainty of this evidence as "low") (PMID 38174786). Yet because this value is an average, the two can diverge on individual questions. Out of 34 reviews, the estimates of the two designs generally agreed in only 23, and the discrepancy was slightly larger when looking solely at pharmacological interventions (1.12). The authors note that beyond the study design, differences in the population, intervention, comparison, and outcome must also be scrutinized (PMID 38174786).

There are also cases pointing in the opposite direction. On the issue of transporting cardiac arrest patients to specialized centers, observational studies looking at neurological prognosis all reported that specialized centers were better, whereas randomized trials failed to confirm a difference. However, that randomized trial was a single trial in a specific setting and population. While judging the certainty of the evidence to be very low, the review in question weakly recommends the use of specialized centers, stating that the impact of increased transport time or bypassing local hospitals remains unclear (PMID 39242018). The fact that benefits indicated by observational studies may not be borne out in randomized comparisons is one of the reasons why randomization is necessary.

Propensity score matching is a statistical technique that attempts to infer causality in observational data—such as registries or electronic medical records—by pairing treated and untreated groups with similar measured characteristics like age, sex, and comorbidities. What can be matched are only measured characteristics. The conclusion of the cited review is not the replacement of RCTs, but their complementary integration (PMID 38577204). The social sciences and economics feature many questions where random assignment is difficult or unethical. Of course, there are areas like crime deterrence policy where randomized field experiments are feasible, and one review reveals that out of 50 controlled evaluations, 9 were randomized field experiments (DOI: 10.1111/1745-9133.70012). For questions where random assignment is difficult, quasi-experiments (designs that lack random assignment but feature comparison groups) or natural experiments have been employed. Natural experiments refer to situations where governments, nature, or institutions—not the researchers—assign the treatment and that assignment is probabilistic depending on external factors, but it is a loosely used term and fundamentally different from randomized experiments.

The Trap of the Term "Gold Standard". As RCTs became widespread in development economics, one chapter weighed the claim that RCTs are one of the best tools for program evaluation alongside criticisms of internal and external validity (DOI: 10.2139/ssrn.6221079). In social experiments, attrition threatens both internal and external validity, and the more online and field experiments increase, the greater that threat becomes (DOI: 10.31234/osf.io/ekm95). In one economic experiment comparison dealing with agricultural policy, a trade-off between internal and external validity was confirmed between student samples and farmer samples (DOI: 10.1093/qopen/qoag008). Moreover, RCTs measure the effects of localized (individual-level) interventions when what we often want to know is the effect of comprehensive (society-level) interventions. Pointing this out, some argue that a single RCT cannot answer this and multiple imperfect pieces of evidence must be triangulated (DOI: 10.31235/osf.io/nq3b5_v2). In psychotherapy research, voices are even emerging calling to reconsider the status of the "gold standard" itself, citing biases, non-reproducibility, and the reality gap in RCTs (DOI: 10.1521/pdps.2025.53.2.157).

An Honest Postscript. Economist Angus Deaton and philosopher Nancy Cartwright are prominent debaters widely known for criticisms targeting "the attitude of placing RCTs at the top of the evidence hierarchy." However, because the abstracts used as evidence by this article do not directly contain their individual arguments, the paragraphs above do not attach their names to specific citations. Instead, the citations above demonstrate that criticisms of the same tenor—namely, the vulnerability of external validity, the trade-off between internal and external validity, and the gap between localized versus comprehensive interventions—are being raised independently across multiple fields. We clarify that the original arguments of Deaton and Cartwright fall outside the evidential scope of this article.

Five Questions for Critically Reading a Paper

When you open an RCT paper, the questions to ask yourself boil down to this.

• Was randomization, concealment, and blinding done properly? If these three collapse, the "gold standard" is just a name.
• Was the analysis done by ITT? If only actual medication-takers were selected and counted, the benefits of randomization may have leaked out.
• What is the absolute risk and NNT, not the relative risk? We also check whether the confidence interval spans the value of no effect.
• Were you not charmed by subgroup results? Post-hoc slices not decided in advance are merely hypotheses.
• How similar are these participants to my patients and my reality? High internal validity does not automatically bring external validity.

The honest conclusion is this. A randomized controlled trial is the most powerful tool for causal inference when its assumptions hold. No other design can mimic that knack for balancing even unmeasured confounding. However, it is not a panacea. In front of external validity, individuals hidden behind the average, ethics and costs, and questions that cannot be asked, the RCT falls silent or falters. The alternative we can use in the face of that silence—namely, how to ask about causes using chance created by nature when experiments are impossible—continues in "Asking About Causes Without an Experiment — Natural Experiments and Four Designs". The term "gold standard" is better read as responsibility rather than a throne. Only those who know when that power is exerted and when it crumbles can use that standard properly.

---

This article is educational content to help understand research methodology and is not medical advice for a specific treatment or drug. Individual clinical decisions must always be discussed with your primary physician, and we recommend directly checking the original text of the cited papers.

Source Papers

• "The importance of randomization in clinical research" (2022) — PMID 36050978 — Observational studies are vulnerable to confounding, selection, and information bias, making hypothesis testing difficult, whereas randomization distributes prognostic factors evenly between two groups, allowing outcome differences to be attributed to the treatment. RCTs are the gold standard for estimating average treatment effects.
• "Single-arm phase 3 designs: An oxymoron?" (2024) — PMID 38508234 — RCTs have been the gold standard for confirming new drugs since the 1950s. After analyzing 176 single-arm phase 3 designs, the paper concludes that the justification and potential for bias in using them for confirmatory purposes must be reported more clearly.
• "The Registry-Based Randomized Trial — A Pragmatic Study Design" (2024) — PMID 38320494 — Randomization avoids unmeasured variables and treatment-selection confounding, enabling causal interpretation. Pragmatic, registry-based designs mitigate weaknesses of being slow and expensive.
• "Interpreting Randomized Controlled Trials" (2023) — PMID 37835368 — Trial samples are sequentially enrolled convenience samples, making population representation impossible. It distinguishes random "allocation" from random "sampling" as a causal diagram, representing a fundamental limit to external validity.
• "Pitfalls in the reporting of clinical events in unblinded device trials..." (2026) — PMID 42129909 — Early termination of the BIO|GUARD-MI unblinded device trial. Performance bias was identified not in the overall cohort, but in the difference in spontaneous hospital visits among the Danish participant subgroup.
• "The Double-Blind Randomized Controlled Trial as the Gold Standard in Psychedelic Research: Neither Feasible Nor Desirable" (2025) — PMID 41424285 — Double-blinding is virtually impossible in psychedelic trials with distinct psychoactive effects.
• "Intention-to-Treat Analysis in Clinical Research: Basic Concepts for Clinicians" (2024) — PMID 38320245 — Organizes the concept and application of ITT (focused on the rehabilitation field) based on PRISMA.
• "Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study" (2015) — PMID 26016488 — Analyses deviating from standard ITT make treatment effects appear larger (43 reviews, 310 trials).
• "Understanding The Number Needed to Treat" (2024) — PMID 38876401 — NNT = reciprocal of the absolute risk difference. Driven by baseline severity, population, treatment intensity, duration, and comparator, and must be accompanied by confidence intervals.
• "How to interpret the number needed to treat for clinicians" (2026) — PMID 40839106 — Calculation of NNT/NNH and common pitfalls. Importance of considering baseline risk, time, and confidence intervals.
• "The number needed to treat: it is time to bow out gracefully" (2025) — PMID 40782977 — If the risk difference is not significant, the NNT confidence interval breaks and becomes uninterpretable. Limitations in policy and guideline decisions.
• "Aspirin in primary prevention: Undue reliance on an uninformative trial led to misinformed clinical guidelines" (2025) — PMID 40165541 — A case where guidelines were distorted by unduly relying on a single uninformative trial instead of the total body of evidence.
• "Understanding explanatory and pragmatic trials: Examples from randomized controlled trials on vertebroplasty" (2023) — PMID 36566693 — Distinction between explanatory and pragmatic trials, the PRECIS-2 tool, and vertebroplasty RCT examples.
• "Modern trials are most useful when they are pragmatic and explanatory — there is no continuum" (2024) — PMID 39396584 — Re-evaluating Schwartz and Lellouch's pragmatic/explanatory dichotomy in the digital health era, arguing that it is not a continuum.
• "Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study" (2024) — PMID 38174786 — Cochrane meta-epidemiology. Effect estimates from RCTs and observational studies show no difference or are very small (ratio of effect ratios 1.08, 95% CI 1.01–1.15).
• "Cardiac arrest centres for patients with non-traumatic cardiac arrest: A systematic review" (2024) — PMID 39242018 — Observational studies showed the benefits of transfer to specialized centers, but randomized trials showed no difference (an example of observational-RCT discrepancy).
• "Can propensity score matching replace randomized controlled trials?" (2024) — PMID 38577204 — Causal inference in observational data through propensity score matching. Limitations of RCTs regarding ethics and external validity versus potential and controversies of PSM.
• "N-of-1 medicine" (2024) — PMID 38527301 — Personalized trial design where each patient serves as their own control, in the context of precision medicine and digital health.
• "A framework for N-of-1 trials of individualized gene-targeted therapies for genetic diseases" (2024) — PMID 39532857 — N-of-1 trial design principles for individualized gene therapies targeting a tiny minority (sometimes a single) patient.
• "Ethical Challenges of Randomized Controlled Trials" (2025) — DOI: 10.31235/osf.io/uxst9_v2 — Ethical tensions in RCTs such as informed consent, placebos, control group appropriateness, and post-trial access to treatment (Beauchamp and Childress principles).
• "Are randomized controlled trials as gold standard for high level evidence a pragmatic approach in the era of precision medicine and real-world evidence" (2026) — DOI: 10.38106/lmrj.2026.8.1-01 — Limitations of RCT generalizability, cost, and applicability to low- and middle-income countries in the era of precision medicine and real-world evidence.
• "Beyond the average: a synthetic control approach to estimate heterogeneous treatment effects with panel data" (2026) — DOI: 10.1007/s11135-026-02615-2 — Methods to go beyond the average and estimate the distribution of individual and heterogeneous treatment effects.
• "Can Randomized Controlled Trials be a Discovery Process?" (2026) — DOI: 10.2139/ssrn.6221079 — The boom of RCTs in development economics (2019 Nobel Prize), criticisms of internal and external validity, and institutional preconditions.
• "External validity of social experiments" (2024) — DOI: 10.31234/osf.io/ekm95 — Attrition and context dependence threaten the external validity and generalizability of social experiments.
• "Trade-offs between internal and external validity in economic experiments to inform agricultural policy" (2026) — DOI: 10.1093/qopen/qoag008 — Trade-offs between internal validity and external validity in economic experiments.
• "Notes on Randomized Controlled Trials for Studying Social Media Harms" (2026) — DOI: 10.31235/osf.io/nq3b5_v2 — RCTs measure localized interventions, whereas the concern may be a comprehensive intervention. The need for triangulation of imperfect evidence.
• "Revisiting Randomized Controlled Trials: Reassessing Their Role as the Gold Standard for Evidence-Based Treatments" (2025) — DOI: 10.1521/pdps.2025.53.2.157 — Biases, reproducibility, and real-world gaps of RCTs in psychotherapy research, rethinking their status as the gold standard.
• "Focused deterrence can reduce crime: A systematic review of randomized controlled trials and quasi-experiments" (2026) — DOI: 10.1111/1745-9133.70012 — Evaluating policy effects using randomization and quasi-experiments in the social sciences (50 crime deterrence evaluations).