Positive Predictive Values

We are often required to make decisions based on truth statements involving conditional probabilities. The problem? Our intuition easily fails when considering conditional probabilities, particularly when rare events are involved.

The Classic Problem

A disease affects 1% of the population. There is a test for it, and the test is “95% accurate”. Specifically: if you have the disease, it returns positive 95% of the time, if you don’t have it, it returns negative 95% of the time.

Question: What is the probability you actaully have the disease, given that you test positive?

What does your intuition tell you?

Most people [1] — including, in famous studies, a large majority of practicing physicians [2], [3], [4], [5] — grossly overestimate the probability of disease, given a positive test result. The reasoning feels intuitive: the test is 95% accurate, the test says I’m sick, therefore I’m 95% likely to be sick.

The correct answer, however, is about 16%.

A positive result from a “95% accurate” test, applied to a randomly screened person, leaves you with a five-in-six chance of being perfectly healthy. If your gut answer was anywhere north of 50%, you just experienced base rate neglect firsthand.

How Our Intuition Fails

There are three distinct cognitive errors stacked on top of each other in this problem:

1. Confusion of the Inverse

The test’s “95% accuracy” is a statement about \mathbb{P}(\text{positive} \mid \text{disease})=0.95 but the question asks about \mathbb{P}(\text{disease} \mid \text{positive})=\text{?}

These are different quantities, related but not equal — just as “most NBA players are tall” does not imply “most tall people are NBA players”. The conditional probability \mathbb{P}(A \mid B) and it’s inverse \mathbb{P}(B \mid A) can differ by orders of magnitude, and the quantity that governs the gap between them is the base rate.

Our intuition treats these two statements as interchangeable, or fails to make a distinction between the two, because natural language blurs them. “The test is 95% accurate” sounds like a promise about you, but it is actaully a promise about the test’s behavior conditioned on a state of the world you don’t know.

2. Base Rate Neglect

The 1% prevalence figure is right there in the problem, yet most often our intuition discards it. Kahneman and Tversky [6] demonstrated that when people are given both individuating evidence (a test result, a witness report, etc.) and statistical background (a base rate), the vivid, specific, case-relevant evidence dominates and the dry or mundane statistic evaporates.

The base rate feels like it’s about other people; the test result feels like it’s about you. But probabilistically, the base rate is doing most of the work. When a condition is rare, even a good test’s false positives can vastly outnumber its true positives, simply because the healthy population generating those false positives is enormous.

3. The Seduction of a Single Number

“95% accurate” compresses two different quantities — the true positive rate and the true negative rate — into a single, reassuring figure, and ignores a third (prevalence) entirely.

The answer to, “Do I have the disease?” depends on three numbers, and the phrase “the test is 95% accurate” supplies, at most, two of them. An intuition fed one number cannot compute a function of three. The question, “How accurate is the test in my case?” is not answerable in one number.

Repairing Intuition

Before reaching for any equations or algebra, here is the single most effective cognitive prosthetic known for this class of problem, due largely to research from Gigerenzer & Hoffrage [7]: convert probabilities into simple counts. Imagive 10,000 randomly screened people and just…count.

flowchart LR
    id1("**10,000**<br/>people screened") --> id2 & id3
    subgraph Section1 ["Base Rate"]
        id2("**100**<br/>diseased (1%)")
        id3("**9,900**<br/>healthy (99%)")
    end
    id2 -->|"95%"| id4
    id2 -->|"5%"| id5
    id3 -->|"5%"| id7
    id3 -->|"95%"| id6
    id4("**95**<br/>test positive<br/>*true positives*")
    id5("**5**<br/>test negative<br/>*false negatives*")
    id6("**9,405**<br/>test negative<br/>*true negatives*")
    id7("**495**<br/>test positive</br>*false positives*")

Figure 1: Test probabilities turned into natural frequencies largely resolves the cognitive gap when considering conditional probabilities.

Now let’s revisit the initial question using these natural frequencies:

“Given a positive test, what’s the chance of disease?”

This becomes trivial to solve:

\frac{\text{true positives}}{\text{all positives}}=\frac{95}{95+495}=\frac{95}{590}\approx0.161

The diseased group is tiny (100 people), so even at 95% sensitivity it can produce at most 100 true positives. THe healthy group is huge (9,900 people), so even a small 5% error rate produces a mountain of false positives — 495 of them, outnumbering the true positives five–to–one. A rare condition means the false positives come from a much larger pool than the true positives do.

We also see from the diagram above that the Base Rate is doing the bulk of the probability filtering. Our intuition naturally skips this filtering step, which is why the “I think there’s a 95% chance” vs. “There’s actually only a 16% chance” feels so jarring. Ignoring prevalence skips the biggest filter, where 99% of people are healthy and have no chance of a true positive test (only false positives).

NoteWhy natural frequencies work

The probability version of this problem asks you to hold three slippery quantities in your head at once — \mathbb{P}(+\mid D), \mathbb{P}(+\mid \neg D), and the base rate \mathbb{P}(D) — and combine them correctly. Natural frequencies do the combining for you by fixing a single reference class. Three things happen:

• One shared denominator. Every count in the tree descends from the same 10,000 people. Probabilities float free of any population; frequencies are all measured against one fixed whole, so they are directly comparable without further arithmetic.
• The base rate stops being ignorable. In the probability form the 1% is one number among several, easy to discard. In the tree it is the size of the two boxes — 100 versus 9,900 — so it visibly governs how many people each error rate acts on.
• The final comparison is a subtraction of counts, not a Bayesian update. “95 true positives against 495 false positives” is a glance; 0.95\times0.01 against 0.05\times0.99 is a calculation. Same numbers, but only one format is legible.

Formally, a natural frequency is just a conditional probability pre-multiplied by its base rate, so the normalizing constant \mathbb{P}(+) becomes the sum of two counts you can see rather than an abstract total you have to assemble. That single reframing is what drives the large gains in correct Bayesian reasoning that Gigerenzer & Hoffrage reported [7] — the audience did not get smarter, the format stopped hiding the reference class.

References

[1]

M. Bar-Hillel, “The base-rate fallacy in probability judgments,” Acta Psychologica, vol. 44, no. 3, pp. 211–233, May 1980, doi: 10.1016/0001-6918(80)90046-3.

[2]

R. C. Clarkson and N. Paruk, “Medical student syndrome: A bayesian reasoning failure,” BMC Medical Education, vol. 26, no. 1, Mar. 2026, doi: 10.1186/s12909-026-09018-9.

[3]

F. A. Araujo, C. F. Chabris, and M. N. Meyer, “The effect of on-the-job experience on base-rate neglect: Evidence from medical professionals,” Management Science, vol. 71, no. 10, pp. 8807–8822, Oct. 2025, doi: 10.1287/mnsc.2024.05461.

[4]

S. Mullainathan and Z. Obermeyer, “Diagnosing physician error: A machine learning approach to low-value health care,” The Quarterly Journal of Economics, vol. 137, no. 2, pp. 679–727, Dec. 2021, doi: 10.1093/qje/qjab046.

[5]

W. Casscells, A. Schoenberger, and T. B. Graboys, “Interpretation by physicians of clinical laboratory results,” New England Journal of Medicine, vol. 299, no. 18, pp. 999–1001, Nov. 1978, doi: 10.1056/nejm197811022991808.

[6]

A. Tversky and D. Kahneman, “Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty.” Science, vol. 185, no. 4157, pp. 1124–1131, 1974, doi: 10.1126/science.185.4157.1124.

[7]

U. Hoffrage and G. Gigerenzer, “Using natural frequencies to improve diagnostic inferences,” Academic Medicine, vol. 73, no. 5, pp. 538–40, May 1998, doi: 10.1097/00001888-199805000-00024.

---

The Counterintuitive Reality of Positive Predictive Value: Why Our Brains Fail at Bayes

As practitioners in applied mathematics and data science, we spend a lot of time building systems that make decisions under uncertainty. Whether we are training machine learning classifiers, setting up anomaly detection on a VPS, or analyzing medical diagnostics, we rely on tests to tell us the truth about a hidden state.

But there is a fundamental bug in human intuition: we are incredibly bad at processing conditional probabilities, particularly when rare events are involved. This cognitive blind spot is known as the Base Rate Fallacy, and understanding it is the key to mastering Positive Predictive Value (PPV).

The Intuition Problem: Reversing the Conditional

Imagine you have deployed an Intrusion Detection System (IDS) on your web server. The vendor boasts that the system has a 99% accuracy rate: if there is a real attack, it catches it 99% of the time (Sensitivity). Furthermore, if there is no attack, it correctly ignores it 99% of the time (Specificity).

One morning, you get a critical alert. The system has flagged an active intrusion.

What is the probability that your server is actually under attack?

If you are like most people, your brain immediately jumps to 99%. You equate the accuracy of the test with the probability of the event. Mathematically, our intuition conflates two very different conditional probabilities:

P(\text{Test}^+ \mid \text{Event}) \neq P(\text{Event} \mid \text{Test}^+)

The first is the Sensitivity (the true positive rate). The second is the Positive Predictive Value (PPV). To find the PPV, we cannot just look at the test’s accuracy; we must know how often the event happens in the first place. We need the Base Rate.

Enter Bayes’ Theorem

Let’s define our terms formally:

• D^+: The event is actually happening (e.g., a real attack, a disease).
• D^-: The event is not happening.
• T^+: The test returns a positive result (alert).
• T^-: The test returns a negative result.

We want to find the PPV: P(D^+ \mid T^+). Bayes’ Theorem provides the exact mechanism to update our belief based on the new evidence of a positive test:

P(D^+ \mid T^+) = \frac{P(T^+ \mid D^+) \cdot P(D^+)}{P(T^+)}

The denominator, P(T^+), represents the total probability of getting a positive test result, regardless of whether the event is real or not. Using the Law of Total Probability, we can expand this into the true positives and the false positives:

P(T^+) = P(T^+ \mid D^+) \cdot P(D^+) + P(T^+ \mid D^-) \cdot P(D^-)

Substituting this back in gives us the full, operational formula for Positive Predictive Value:

\text{PPV} = \frac{P(T^+ \mid D^+) \cdot P(D^+)}{P(T^+ \mid D^+) \cdot P(D^+) + P(T^+ \mid D^-) \cdot (1 - P(D^+))}

The Math Resolves the Illusion

Let’s plug the numbers from our server IDS example into the theorem. We know our sensitivity P(T^+ \mid D^+) is 0.99, and our false positive rate P(T^+ \mid D^-) is 0.01.

Now, let’s assume the base rate of a genuine, sophisticated attack on your specific IP in a given hour is very low—say, 1 in 10,000, or 0.0001.

\text{PPV} = \frac{0.99 \cdot 0.0001}{(0.99 \cdot 0.0001) + (0.01 \cdot 0.9999)}

\text{PPV} = \frac{0.000099}{0.000099 + 0.009999}

\text{PPV} \approx 0.0098

The probability that the attack is real is not 99%. It is less than 1%. This is the Base Rate Fallacy in action. When the background prevalence of an event is incredibly small, the sheer volume of “normal” traffic generates enough false positives to completely swamp the true positives, even with a highly accurate test.

Visualizing this statistically—for instance, mapping the outcomes to a 1,000-tile grid where each tile represents a trial—often breaks the intuition barrier better than pure equations. In a grid of 1,000, only a fraction of a tile represents a true positive, while 10 full tiles represent false positives.

Bayesian Updating: The Path to Certainty

So, how do we ever achieve certainty if one test leaves us at 1%? We use Bayesian Updating.

The beauty of Bayes’ Theorem is that it is iterative. Today’s posterior becomes tomorrow’s prior. If you receive that IDS alert (bringing your probability of an attack from 0.0001 to 0.0098), you don’t panic, but you do investigate. You run a second, independent diagnostic—perhaps checking your Nginx access logs for specific payload signatures.

For this second test, your new base rate P(D^+) is no longer 0.0001; it is now 0.0098. If the second test (also 99% accurate) comes back positive, the math updates dramatically:

\text{PPV}_2 = \frac{0.99 \cdot 0.0098}{(0.99 \cdot 0.0098) + (0.01 \cdot 0.9902)} \approx 0.495

Two independent positive tests bring us to about 50%. A third would push it past 98%.

By rigorously applying Bayes’ Theorem, we strip away the flaws of our innate heuristics. We learn to stop asking “How accurate is the test?” and start asking “How prevalent is the condition, and what is the math telling us to do next?”