Same Result, Different Sentence — Where Format Changes Judgment, and Where It Doesn't

Same drug, same trial result. The only thing that differed was the sentence the doctor used to explain it to the patient.

The story begins at 23 clinics in Denmark. Thirty-four general practitioners here explained the effects of cholesterol-lowering treatment to patients. One group of clinics explained it by how much life is extended by the treatment, while the other group of clinics explained it by how much the risk is reduced. Which clinic would use which explanation was decided by cluster randomization, in other words, by drawing lots. The patients included in the analysis were 240 (112 versus 128). The people who actually took the prescription to the pharmacy and received the medicine were 6 (5.4%) in the group that heard the explanation in terms of life extension, and 32 (25.0%) in the group that heard the explanation in terms of risk reduction. There was no difference between the two formats in confidence in the decision and satisfaction with communication (PMID 24686884). They just stated the same effect in different sentences, yet the proportion of people who picked up the medicine diverged widely.

There are two things that are easy to miss when reading these results. First, both explanations accurately translate the exact same trial results. Neither side is false. Second, what this trial measured was the proportion of people who picked up the prescription (dispensing rate) and confidence-satisfaction. What happened to the patients afterwards was not measured. Therefore, this trial cannot tell us either which format leads to the right decision or how beneficial this treatment is. What diverged was human response.

This article deals with that response. It asks three things. First is how much the format changes judgment, second is where that change appears and where it does not, and third is how the sentences of a paper grow larger once more as they are transferred into press releases and news articles. The new terms to learn are four, and each term is defined at its first appearance. To state the conclusion in one line beforehand, it is this. Format changes judgment. However, it does not always win.

⚠️ The names of drugs, tests, and surgeries appearing below are all examples just to show how to read numbers. This is not a judgment on what is good or bad, and each number has meaning only within the design and subjects of that study.

Reviewing Terms Learned in the Previous Articles

This article uses the terms covered in "If the Risk Was Cut in Half, How Many Fewer People Got Sick? — Relative and Absolute Risk" and "What Does "50% Lower Risk" Mean in People? — Reading Relative Risk, Absolute Risk, and NNT" without changes. They are briefly restated here.

Baseline risk is the original rate at which an event occurs when nothing is done. How much the treatment reduced this rate can be written in two ways. If written as the percentage reduction compared to the original rate, it is relative risk reduction; if written by directly subtracting the reduced amount, it is absolute risk reduction. Risk itself is also divided into relative risk and absolute risk in the same way. NNT is the counted value of how many people must be treated for one person to benefit.

The takeaway left by these terms is one. The same result looks large when written relatively and small when written absolutely. The link connecting the two is the baseline risk of that population. This article takes on the next question. Does writing numbers differently in this way actually change the reader's judgment?

Framing — Three Things the Format Chooses

Framing is the task of choosing which sentence to write for the same fact, that is, setting the frame.

Suppose that out of 100 people who received a certain surgery, 98 were alive a month later. This fact can be written as "98 out of 100 people survive," or it can be written as "2 out of 100 people die." The two sentences point to the same table, but the impression left on the reader is not the same.

There are three things the format chooses. The first is the scale. It chooses whether to write it relatively or absolutely, as a percentage or as a number of people. The second is the time period. It chooses whether to write the risk over 10 years or the risk over the remaining lifetime. The third is what to count.

There is a separate term for the third one. The event that a study decides to count is called an event, and the proportion of people in whom that event actually occurs is called the event rate. Even for the same people, the event rate varies greatly depending on whether all things happening to the heart and vessels are counted (cardiovascular "events"), or only deaths caused by them (cardiovascular "death").

All three of these are formats, and all three change judgments. The following two sections show this in turn.

It Changes Even for Experts

Our judgment being swayed by the format happens to experts who deal with numbers, too. Dentists' understanding also varied depending on which format the same benefit was presented in (Rio de Janeiro, Brazil, 101 dentists randomly assigned).

What this trial dealt with was gum treatment using antibiotics together, and the benefit was avoiding the loss of multiple teeth. To one side, this benefit was presented as the number of people, and to the other side as relative risk reduction, and they were asked to answer five questions. On the question asking about when antibiotics were used, the group that saw it as the number of people got more right. 26 people (50%) in the number of people group and 14 people (29%) in the relative risk reduction group got it right (p = 0.04). The p-value in parentheses is the probability that a gap this wide would happen by chance assuming there is no real difference between the two groups, and the smaller it is, the harder it is to view as chance.

There was no difference in the rest. The questions asking about when antibiotics were not used were 17 people (33%) versus 12 people (25%) (p = 0.39). People who got both questions right were 16 people (31%) versus 12 people (25%) (p = 0.51).

Let us translate the authors' conclusion first. Regardless of the presentation format, most dentists did not accurately understand the treatment's benefit. It was just that slightly more people understood it accurately when it was presented as the number of people (PMID 39109767).

The format made a difference, but that difference did not solve the problem. The two conclusions of this article—that format changes judgment, yet does not always win—are both contained in this single trial.

A similar picture emerges in the ability to handle numbers. Numeracy is the ability to understand and handle numerical information like probabilities and ratios. An institution sent the same five-question survey to patients awaiting general anesthesia surgery and specialists at that institution, and 213 patients (42%) and 268 doctors (53%) responded. The median of the patients' scores (the middle value when scores are lined up) was 4 points out of 5. However, 32% of the patients scored 3 points or lower, and the patients guessed their own scores to be an average of 0.5 points higher than reality. This means there are patients who believe their numeracy is higher than it actually is. Doctors' scores were higher, and their guesses and reality did not misalign. However, the authors write that some doctors also had low numeracy, and that low numeracy was associated with failing to understand risk reduction (PMID 32304457).

Changing Time and Events Makes Perception Follow

Not only the scale, but also the time period and what is being counted change judgments. When the risk of the same hypothetical patient was written in three ways, the perceived magnitude of the risk changed significantly (PMID 30419113).

The three ways are a 10-year cardiovascular event risk of 15%, a 10-year cardiovascular death risk of 4%, and a lifetime cardiovascular event risk of 50%. All three are the risks of the same patient, differing only in the time period and what is being counted. The people who saw these numbers were 2,708 individuals (76.9% of those eligible) participating in registries at 140 US clinical practices.

The proportion who viewed the risk as "high to very high" was 70.1% when seeing the lifetime risk. When seeing the 10-year event risk, it was 31.4%, and when seeing the 10-year death risk, it was 25.7% (both P < .001). The willingness to take medication followed the same order (77.9% / 68.1% / 63.1%). This means that looking at the numbers alone can make the exact same patient seem either risky or less risky.

Participants were randomly assigned to one of three ways of viewing this risk: numbers only, bar graphs, or facial pictographs. People who saw the pictographs viewed the disease as the least serious and had the lowest willingness to consider treatment. The authors' conclusion is that format, time horizon, and outcome all influence patient perception, so they must be considered together when designing risk communication tools (PMID 30419113).

The same person, the same disease, the same data. The only things that changed were how long is measured and what is counted.

Yet Form Does Not Always Win

Reading up to this point, it seems that if you just choose the right format, your judgment will be corrected. However, there are several trials where changing the format hardly changed the judgments at all. We present three trials starting with the authors' conclusions.

The first is a trial that randomly assigned 200 patients receiving a bone density test into four groups. The four groups were created by combining two choices. One was whether to write fracture risk as the "probability of an event occurring" or the "probability of it not occurring," and the other was whether to write the treatment benefit as natural frequencies (a method of writing in terms of numbers of people, covered in the section below) or NNT. The authors' conclusion is as follows: Changing the framing had almost no effect on participants' belief that treatment was needed or on how they viewed their own fracture risk. When participants were asked above what 5-year fracture risk percentage they would consider preventive medication, the median of the answers was 50 to 60%. This value did not change significantly even after explanation, and there were no differences among the randomly assigned groups in these results.

However, other details are recorded in the same paper. Participants' initial guesses of their 5-year fracture risk were higher than the risk calculator's estimates. They guessed 2 to 3 times higher for any-site fractures and 10 to 20 times higher for hip fractures. These guesses were cut in half after explanation (though still higher than the calculator's values). And the same abstract immediately qualifies: Even after receiving information about risk and treatment benefit, participants' willingness to take medication did not change significantly. Changing the format failed to move judgments, and providing information only moved them as far as the risk estimates (PMID 28188155).

The second is visualization, which is the method of attaching pictures next to numbers. 393 individuals from the general Dutch population solved four basic problems that swapped percentages and natural frequencies. Each problem was randomly assigned one of four formats: icon array, pie chart, bar graph, or no visualization, and who received what was concealed from the investigators. 60% of respondents got all four problems right. Visualization was not associated with the correct answer rate (odds ratio 1.08, 95% CI 0.69–1.69). The odds ratio is a value comparing the odds of getting the correct answer between the two conditions; it is 1 if there is no difference, and since the confidence interval (CI), which is the range where the true value is likely to lie, spans 1, we cannot consider that pictures increased the correct answer rate. All three types of pictures showed this, and neither the format in which risk was written nor the magnitude of the risk was associated with the correct answer rate. Instead, the younger the age and the higher the education level, the higher the score. We also present the caveat attached by the authors. This result is limited to the narrow objective of literally communicating a single absolute risk. What effects visualization has on user experience, perceptions of risk communication, and subsequent patient participation have yet to be elucidated (PMID 31901707).

The third is a trial where understanding increased yet choices did not change. 600 U.S. adults read a decision aid about a skin mole called dysplastic nevus. The decision aid was randomly given in one of two formats (an A/B test): a quantitative format with risks written in numbers, or a qualitative format written in words. Those who read the quantitative format had higher self-rated understanding (80 versus 72, P = .008) and decision certainty (P = .003), and viewed their melanoma risk as lower. However, the intention to excise, meaning to remove the mole, did not change (52% versus 53%, odds ratio 1.04, P = .818). What moved intentions was the severity label attached to the diagnosis name. When written as mild it was 31%, moderate 41%, and severe 72% (P < .001), and this label did not affect understanding. This means that the "mild" or "severe" labels of the diagnosis name played a bigger role in the decision than whether numbers were understood.

The authors' conclusion is close to a summary of this entire section. While it is true that the quantitative format increases understanding, what primarily drives treatment intentions is the severity of the diagnosis name. And even with high understanding, about a third of participants made choices that differed from expert recommendations. Individual values drive treatment choices (PMID 41893389).

These three trials are easily read as evidence that humans are easily fooled by numbers. What they actually show is the scope of what formatting is responsible for. Formatting is about making sentences accurate, not making decisions on behalf of someone. Deciding differently even after reading accurate sentences is not an error, but a matter of values.

Leaving the Paper Behind — Spin

Up to this point, we have looked at the places where numbers are first written—the sentences in papers and clinics. Research results are subsequently transferred into press releases and news articles, and in that process, their shape changes once more. This chain has actually been tracked.

We need a new term here. Spin is a reporting method that writes results to make them look more beneficial than they actually are. It differs from a lie in that it does not change the numbers. The numbers are left as they are, and only where the reader's gaze is directed is chosen. For instance, when a primary outcome—the main question the study set out to answer from the beginning—does not turn out well, a well-performing secondary outcome is elevated to the conclusion. Qualifiers may also be removed. This may or may not be intentional.

The starting point of spin is often the paper itself. Following 70 randomized trials introduced via press releases, spin was present in 28 abstract conclusions (40%) and 33 press releases (47%). An abstract is a summary at the beginning of a paper, and a press release is an introductory text issued by a university or academic journal to the press. In an analysis considering multiple factors at once (multivariate analysis), the only factor associated with spin in press releases was spin in the abstract conclusions. When there was spin in the abstract conclusion, the likelihood of spin also being in the press release was 5.6 times higher (relative risk, 95% CI 2.8–11.1). Results were overestimated by reading only the press release in 19 cases (27%). Of 41 articles whose reports were verified, 21 (51%) had spin, mostly of the same type. Results were overestimated by reading only the news article in 24 cases (10%). This means that the types of spin present in the abstract conclusions were largely reflected as they were in the press releases and news articles (PMID 22984354; a cohort study selecting only two-group parallel-design randomized trials from 498 entries in a press release database from December 2009 to March 2010).

The way overstatement in press releases transfers to news articles appears in much larger data as well. This is the result of comparing 462 biomedical and health press releases and 668 related news articles from 20 major British universities in 2011 with the original papers (PMID 25498121). This study is a retrospective content analysis that gathered already published texts and compared their contents. Therefore, it shows whether the strengths of the two statements move together, but it does not establish which side is the cause.

The overstatements were of three types. 40% (95% CI 33–46) of press releases contained behavioral recommendations that went beyond the original paper. 33% contained causal statements drawn from correlational data where things merely moved together, and 36% contained inferences translating animal study results to humans. When there was overstatement in the press release, news articles were much more likely to have the same overstatement. When overstatement was present, the proportion of news articles having the same overstatement was 58%, 81%, and 86% by type, and when absent, it was 17%, 18%, and 10% (odds ratios 6.5, 20, 56).

However, there was little evidence that exaggerated press releases drew more news articles. The authors point out a spot to fix in their conclusion. Improving the accuracy of academic press releases may be a key opportunity to reduce misleading health news (PMID 25498121).

These results were subsequently verified two more times. One was a replication to see if the same association appears again, and the other was a follow-up to see if the situation improved.

The replication was done using British university data from 2014 and 2015. The association for behavioral recommendation overstatement was not replicated, but the associations for causal statements and inferences to humans were replicated. The authors' overall summary is as follows: the strength of news article statements generally goes along with the strength of press release statements, and there is no evidence that exaggerated statements significantly win more news articles (PMID 31728413).

In the follow-up, an improvement was visible. The proportion of press releases exaggerating causality from correlational data dropped from 28% (95% CI 16–45) in 2014 to 13% (6–25) in 2015. However, the decrease on the news article side was not significant, and the association between press releases and news articles remained strong. The authors specify that since this is correlational evidence, it could be due to other factors or natural variation (PMID 32500096).

The end of the chain—the judgment of people who read the news articles—also moved according to spin (PMID 31159786). 900 people recruited from patient and caregiver online communities read news articles covering the same study. One group randomly received original texts containing spin, and the other received versions rewritten to strip out the spin. They then rated "the probability that this treatment will benefit patients" from 0 to 10. There were three trials, each dealing with news articles from different research stages. In all three trials, the scores were higher for the group that read the versions with spin. For preclinical research before testing in humans, it was 7.5 versus 5.8; for early-stage clinical non-randomized studies, it was 7.6 versus 5.8; and for late-stage randomized trials, it was 7.2 versus 4.9 (mean differences 1.7, 1.8, 2.3, all p < 0.001).

It is easy to blame the media in this chain, but the starting point repeatedly pointed to by this literature is the abstract conclusions of the papers themselves.

What Is the Solution, and How Well Does It Work?

So far, we have looked at the problem. Research in this field has gone as far as proposing solutions and actually measuring their effects. We will look at three of them in turn.

① Include the Baseline Risk Together

When the baseline risk—that is, the original rate when nothing is done—was included alongside the treatment effect, comprehension increased significantly (PMID 30005747). This was a repeated-measures study that presented hypothetical clinical trial results to multiple sclerosis patients and measured their comprehension multiple times. Benefits and harms were presented as absolute expressions, relative expressions, NNT, and NNH. NNH is NNT flipped toward harm, counting how many people are treated until one person is harmed. For each method, versions with and without baseline information were distributed together.

Comparing the methods, comprehension scores were higher for absolute expressions (3.99) than for relative expressions (2.93) or NNT and NNH (2.89) (both P < 0.001). However, the largest gap within this experiment was the presence or absence of baseline information. When baseline information was attached to all methods, the score was 5.04; when it was not attached, it was 1.50 (P < 0.001). Comprehension was not associated with the degree of conflict over treatment decisions, while numeracy, intelligence, and cognitive function were significantly associated with comprehension. The authors' recommendation is to communicate risks and benefits in absolute terms, along with baseline information. They also recommend providing separate support for patients with low numeracy, intelligence, and cognitive function (PMID 30005747).

⚠️ These numbers are the result of comparing two conditions within a single experiment. Because this did not measure magnitudes across multiple improvements, it cannot be broadly read to mean that "including baseline risk together is the most effective of all improvements."

The same direction appeared when communicating drug side effects (PMID 16242904). A hypothetical scenario was given to 268 women stating that a drug's side effect risk increased. The risk increase was presented as one of relative risk, absolute risk, or NNH, and versions with and without baseline risk information were separated. Providing baseline information significantly improved risk estimation accuracy and increased satisfaction. However, the authors immediately attach a caveat. Even so, the estimates were still considerably higher than the actual risk. When baseline information was present, there was no difference among formats; when it was absent, absolute risk expression led to the most accurate performance. In their conclusion, the authors emphasize that there were no results supporting the use of NNH. They also note that more research is needed before presenting NNH or NNT to the general public.

The format in which the baseline risk is written also mattered. Baseline risk was better understood when given as frequencies rather than percentages (PMID 24803429). In four experiments involving 1,234 members of the general public, changes in relative risk were shown alongside hypothetical scenarios that lowered or raised heart disease risk. Baseline risk was given either as percentages or as frequencies. Providing it as frequencies promoted understanding of benefits and harms, whereas percentages frequently hindered understanding. For example, many participants mistook relative risk reduction for absolute risk reduction. People with higher numeracy generally did better, but all participants benefited from the frequency format. The authors disclose the limitations of hypothetical scenarios and a non-representative sample. And they add an honest caveat to their conclusion. Overall, communicating the benefits and harms of treatment in the form of relative risk changes remains problematic even when explicit baseline risk is provided alongside it (PMID 24803429).

② Natural Frequencies — Areas with Thicker and Thinner Evidence

Natural frequencies are a way of writing risks as a number of people, like "8 out of 1,000 people." They are written as if counting out 1,000 people, leaving how many people out of how many inside the sentence. Major institutions have recommended presenting test results in this format (PMID 27034447).

The evidence for natural frequencies is concentrated in one area. Every single natural frequency experiment cited in this section came from the interpretation of test results, not the size of treatment benefits. The problem these experiments were made to solve was calculating the positive predictive value. The positive predictive value is the probability that a person with a positive test result actually has the disease.

The comparator to natural frequencies is conditional probability notation. Conditional probability is the probability of an event occurring under a certain condition, such as "the probability of testing positive given that you have the disease," and writes the same fact as a percentage like "0.8%." The evidence in this area is thick. A meta-analysis combining the results of multiple studies at once integrated 226 performance estimates from 20 years of research and 35 papers. This analysis examined numeracy and expertise, incentives and scoring criteria, and the way problems are written as conditions that change accuracy rates. The conditions on the strongest side were short-menu formats and visual aids. A short-menu format is a brief notation showing overlapping cases together so that less calculation is needed. Both of these improved performance in both conditional probability formats and natural frequency formats (PMID 29048176).

Natural frequencies also helped older adults (PMID 19129155). Two screening test pieces of information were given to 115 young adults and 47 older adults. When presented as conditional probabilities, the proportion making accurate estimates on at least one task was 15% for young adults and 18% for older adults. When presented as natural frequencies, these proportions became 55% and 58%. This is a difference that appeared without training. Participants with higher numeracy scores did better, though. The conclusion of the authors is that natural frequencies help younger and older patients, including those with lower numeracy.

Natural frequencies had the greatest effect when combined with training (PMID 36747214). This is a study measuring 577 medical students before and after training. The correct answer rate for questions presented as conditional probabilities was 34% before training, and presenting the same information as natural frequencies raised it to 68% even without training. After training on how to convert conditional probabilities into natural frequencies and apply them to Bayes' rule (the rule for calculating and revising probabilities upon seeing new test results), conditional probability questions also became 64%. When training and natural frequency presentation were combined, it was 89%.

However, objections have been raised against this tool, and the debate is still alive.

One paper pointed out that natural frequency presentation benefits at most one-fifth of health service users and patient samples, and that studies reporting substantial benefits were just one single paper. This paper replicated that single paper with online respondents, and viewed the previously reported benefits as an apparent effect created by the scoring method—in other words, an artifact. It also noted that natural frequencies can induce evaluations that are normatively correct, meaning judgments that are correct in probability calculations, but conflicting. Such evaluations were paradoxically fewer in single-event probabilities, which write the probability of a single event as it is. The conclusion was to be cautious about promotion (PMID 27034447).

A counterargument to the objection also appeared (PMID 29448883). The evidence is three analyses. First, the problematic "50-split" scoring method misclassifies known errors, such as answering by looking only at a test's hit rate, as understanding. Second, in 19 out of 21 additional problems solved by multiple participant groups, that scoring criterion failed to support the original authors' results. Third, using the mean deviation scoring method makes the estimates on the natural frequency side closer to the correct answer.

Natural frequencies are not the best in every situation either (PMID 40205446). The comparison target is an odds and likelihood ratio notation, which writes out how many times test results change probabilities through multiplication. In a preregistered randomized crossover trial involving 167 medical students and 162 psychology students, the proportion correctly guessing the positive predictive value of a single test was 36.2% for natural frequencies, which was higher than the 21.6% for odds and likelihood ratios (odds ratio 2.41). However, for the positive predictive values of two consecutive tests, the order reversed, and odds and likelihood ratios at 10.6% were higher than natural frequencies at 4.9% (odds ratio 2.73). The understanding rated by the participants themselves was much higher on the natural frequency side. The authors suggest teaching odds and likelihood ratios together with natural frequencies, rather than discarding either one.

To sum up the debate: Neither side of the debate says, "just change the format and understanding follows." Even the advocating side reports that performance was best when combined with training (PMID 36747214). A meta-analysis reviewing both theoretical camps together notes that other factors, like visual aids, pull up both formats (PMID 29048176). And the very fact that this evidence came from interpreting test results is a limitation. How much better things get when the magnitude of treatment benefits is written in natural frequencies is not a question that the experiments in this section answered.

③ Writing Both Formats Together

There is also a proposal to write absolute risk in percentages and natural frequencies simultaneously. A report summarizing mortality rates after major urological cancer surgery in both formats is an example (PMID 42134078). The 30-day mortality rate across all procedures was 0.02 to 2.70%, and the 90-day mortality rate was 0.07 to 5.57%, which translates to between 1 in 5,577 people and 1 in 37 people in natural frequencies. These figures are descriptive benchmarks gathering multiple cohorts, not individual patient risks. What to look at in this case is the notation format rather than the numbers. The authors' conclusion is that the two formats are mathematically identical but complement each other, and that reporting both alongside international benchmarks can help risk communication. As limitations, they specify remaining differences between the collected data (residual heterogeneity) and the possibility of dataset duplication.

Even outside the interpretation of tests, there are statements supporting frequency notation (PMID 21931068). This is a commentary summarizing ten ways to help cancer screening, prevention, and treatment decisions. Among the methods the authors pointed out as confirmed to empirically improve comprehension or decision-making are presenting absolute risk as frequencies instead of relative risk, formats that clarify how treatment changes risk from the existing baseline level, and plain language. What the authors subsequently recommended to medical staff includes plain language, pictograms, and absolute risk instead of relative risk. However, since this is a commentary, it does not publish new data.

Looking at this field as a whole, consensus that one method is better is limited. This is the conclusion of a review examining risk communication. The same review notes that both doctors and patients find NNT harder to grasp than relative risk reduction, and summarizes that there are multiple quantitative and qualitative methods (PMID 15962835).

Why "As Long as the Paper Is Accurate" Is Wrong

It is easy to think that as long as the paper itself is accurate, everything else is someone else's problem. This is the final misconception this article addresses.

Results become simplified and inflated much more often outside the paper, but their starting point is frequently the abstract conclusion of the paper itself. The only associated factor for press release spin was spin in the abstract conclusion (PMID 22984354). Exaggeration in press releases was strongly associated with exaggeration in news articles (PMID 25498121). News articles inflated in that way caused patients and caregivers to view the benefits of treatments as higher (PMID 31159786). However, this association was only partially replicated (PMID 31728413), and after the problem became widely known, causal exaggeration in press releases decreased (PMID 32500096).

The Rules That Remain

When you encounter risk numbers in an abstract or an article, there are three things to ask. They are the exact three that format chooses, which we saw in the framing section. The first is what scale it is written on. We look at whether it is relative or absolute, a percentage or a number of people. The second is how long the number measured. The third is what was counted. We look at whether it counted "events" or "deaths." Even if just one changes, the same data is read differently.

The sentence this article wants to leave you with is one: Format is about making sentences precise, not about making decisions on our behalf. Several trials have shown that even when formats are fixed, understanding does not automatically follow, and even when understanding increases, choices still follow individual values. That does not mean format does not matter. Explaining the same result differently was enough to split the proportion of people who took home medicine between 5.4% and 25.0%.

Source Papers

Format Changes Judgments

• PMID 24686884 (2014) "ARR vs POL" — Cluster randomized, 34 Danish general practitioners and 240 patients. Dispensing rate 5.4% vs 25.0%, no difference in confidence and satisfaction.
• PMID 39109767 (2024) "Dentists' understanding" — Randomized, 101 dentists (looking at only 2 out of 5 questions + synthesis). Antibiotic use question 50% vs 29%. Conclusion=Regardless of format, most do not understand.
• PMID 32304457 (2020) "Health Numeracy & RR" — Cross-sectional survey, 213 patients and 268 physicians. 32% of patients scored 3 points or lower, some physicians also low. Conclusion=Low numeracy ↔ misunderstanding of risk reduction.
• PMID 30419113 (2018) "CV Risk Communication" — Registry 2,708 people. Awareness lifetime 70.1% · 10-year event 31.4% · 10-year death 25.7%. Conclusion=Consider format, time, and outcome together.

And cannot change it

• PMID 28188155 (2017) "Framing of fracture risk" — Randomized 200 participants into 4 groups. Conclusion = Changing the frame had almost no effect and there was no difference between assigned groups. Willingness to take the medication did not change significantly even after receiving the information.
• PMID 31901707 (2020) "Do visualizations help?" — Randomized, investigator-blinded 393 participants. Visualization OR 1.08. Caveat = Limited to conveying a single absolute risk.
• PMID 41893389 (2026) "Quantitative vs qualitative" — Randomized A/B 600 participants. Comprehension ↑, willingness to moderate unchanged (52% vs 53%), severity drives willingness. Conclusion = Personal values drive choice.

Paper → Press Release → Article

• PMID 22984354 (2012) "Misrepresentation of RCTs" — 70 cohort RCTs. Abstract conclusion spin 40%·press releases 47%. In multivariable analysis, the only associated factor is abstract conclusion spin (RR 5.6).
• PMID 25498121 (2014) "Exaggeration in news & PR" — Retrospective content analysis, 462 press releases·668 articles, OR 6.5/20/56. Conclusion=press release accuracy is the core opportunity.
• PMID 31728413 (2019) "…replication" — Replication cohort (2014·2015). Behavioral recommendations failed to replicate; causal statements and human inferences replicated.
• PMID 32500096 (2020) "Causal overstatements reduced" — Before-after comparison. Causal overstatement 28%→13%, news articles not significant. Specify that it is correlational evidence.
• PMID 31159786 (2019) "Impact of 'spin' (3 RCTs)" — 3 randomized trials, 900 participants. With vs. without spin 7.5/5.8·7.6/5.8·7.2/4.9.

Solutions and Their Effects

• PMID 30005747 (2018) "Communicating trial data (MS)" — Repeated measures. Absolute 3.99, relative 2.93, NNT/NNH 2.89, with baseline 5.04 versus without 1.50 (contrast within a single experiment). Conclusion = absolute phrasing + baseline info.
• PMID 16242904 (2006) "Medicine side effects" — Two-factor experiment with 268 participants. Baseline information improves accuracy and satisfaction (estimates remain high). Conclusion = no support for NNH.
• PMID 24803429 (2014) "RR changes with baseline" — 4 experiments with 1,234 participants. Frequencies facilitate, percentages hinder. Conclusion = relative risk changes remain problematic even when baseline risk is provided.
• PMID 29048176 (2017) "Meta-analysis: NF" — Meta-analysis of 35 papers and 226 estimates, examining two theoretical camps together. Short menu formats and visual aids improve both formats.
• PMID 19129155 (2009) "NF help older adults" — Experiment with 115 younger adults and 47 older adults. Conditional probabilities 15%/18% to natural frequencies 55%/58%, without training. Conclusion = helps including those with lower numeracy.
• PMID 36747214 (2023) "Boosting insight/agency" — Pre- and post-test with 577 medical students. 34% to 68%, after training 64%, training + natural frequencies 89%.
• PMID 27034447 (2016) "NF Do Not Foster…" — Replication experiment. Gains are at most one-fifth in healthcare user and patient samples, and prior gains are scoring artifacts. Conclusion = be cautious in promotion. (Source of major organization recommendation statements)
• PMID 29448883 (2018) "NF Do Foster…: Comment" — Commentary and re-analysis. 50-split scoring misclassifies known errors as understanding, original results unsupported in 19 out of 21.
• PMID 40205446 (2025) "LRs vs NF" — Preregistered crossover with 329 participants. Single test 36.2% versus 21.6%, two consecutive tests odds/likelihood ratio 10.6% versus 4.9%. Conclusion = both.
• PMID 42134078 (2026) "Postop mortality: % and 1-in-N" — Descriptive benchmark review. 30-day 0.02–2.70%, 90-day 0.07–5.57% = 1 in 5,577 people to 1 in 37 people. Conclusion = mathematically identical and mutually complementary.
• PMID 21931068 (2011) "Ten steps to risk communication" — Commentary. Ten empirically verified items include presenting absolute risk as frequencies, formats clarifying changes in baseline levels, and plain language inclusion. Pictograms lean toward the authors' subsequent recommendations.
• PMID 15962835 (2005) "Translating evidence into RC" — Review. Both doctors and patients find NNT harder to grasp than relative risk reduction. Conclusion = consensus on the superiority of any single method is limited.