Lecture 5: Diagnostic Testing

BIOS 600 - Spring 2026

Authors
Affiliation

Announcements

  • Updated tutoring schedule: handout posted to Canvas, course website.

  • Email bios600tutor@unc.edu to set up an individual appointment.

Announcements

  • Lab 3 Due Friday of next week.

  • HW 2 will be assigned after class, due Feb 5 at 11:59pm.

  • If more classes are canceled, will post a video lecture.

Reading

Overview

  • How conditional probabilities relate to sensitivity, specificity, & more!

  • Example with Bayes’ theorem

  • ROC Curves

Conditional probabilities

Suppose we care about the probability that someone has HIV, denoted by \(P(HIV+)\).

  • What if they have a positive HIV test?

\(P(HIV + | Test +)\)

  • What if they have a negative HIV test?

\(P(HIV + | Test -)\)

Knowing the result of the HIV test changes our estimate of their HIV probability.

TipQuestion

Which of the three probabilities would we expect to be the highest? The lowest?

Medical diagnostics

Suppose we’re interested in the performance of a diagnostic test. Let \(A\) be the event that someone has a condition of interest, and let \(B\) be the event that a test for that condition is positive.

  • Prevalence: \(P(A)\)

  • Sensitivity: \(P(B|A)\), or the true positive rate

    • probability that the test is positive given the condition is present

  • Specificity: \(P(B^C | A^C)\), or 1 minus the false positive rate

    • Probability that the test is negative given the condition is not present

Medical diagnostics (continued)

  • Positive Predictive Value (PPV): \(P(A|B)\)

  • Negative Predictive Value (NPV): \(P(A^C | B^C)\)

Write out the last two probabilities (PPV and NPV) in words.

Popularization of medical diagnostics

During COVID-19, diagnostic testing has become mainstream.

  • Testing for COVID-19 is not perfect. It may come back positive in some people who are not infected with SARS-CoV-2 and negative in some people who are.

  • Diagnostic tests are developed with both sensitivity and specificity in mind.

Sensitivity and specificity

  • The greater the sensitivity, the less likely it will miss real cases.

    • Recall: sensitivity is the probability of a positive test, given the condition is present.

  • The greater the specificity, the more likely uninfected individuals will be correctly deemed negative.

    • Recall: Specificity is the probability of a negative test, given the condition is not present.

A test’s performance can be surprisingly counterintuitive

  • Consider a scenario with COVID-19 testing in an asymptomatic or mild population with 1 in 51 people infected.

  • Assume the test is always positive in individuals with the disease (i.e., 100% sensitivity) but falsely positive 10% of the time (i.e., 90% specificity).

  • As shown in the figure (next slide), the chance that someone with a positive test result is actually infected is under 20% (1 in 6)

Illustration

Takeaway

  • What’s the takeaway here?

  • Even if a test never misses true cases (perfect sensitivity), a modest false positive rate can mess up the results, especially when the disease is rare.

    • Consequence: Most people who test positive may not actually be infected.

  • This demonstrates why specificity and disease prevalence are important as well in diagnostic testing.

Rapid self-administered HIV tests

From the FDA package insert for the OraQuick ADVANCE Rapid HIV-1/2 Antibody test,

  • Sensitivity, \(P(Test+|HIV+)\): 99.3%

  • Specificity, \(P(Test - | HIV -)\): 99.8%

From CDC statistics for 2016, 14.3/100,000 Americans aged \(\geq\) 13 are HIV+.

Suppose a randomly selected American aged \(\geq\) 13 has a positive test on this test. What do you think is the probability they are HIV+?

Steps to solve using Bayes’ Rule

  1. Define two events of interest as \(A\) and \(B\).

  2. Write the quantity of interest in terms of Bayes’ Rule.

  3. Expand denominator in terms of law of total probability.

  4. Identify quantities of interest.

  5. Plug in quantities from step 4 and solve.

Bayes’ rule

Recall Bayes’ rule below. Also recall that we can expand our denominator using Law of Total Probability.

\[ \begin{aligned} P(A | B) = \frac{P(A \cap B)}{P(B)} &= \frac{P(B | A) P(A)}{P(B)} \\ &= \frac{P(B | A) P(A) }{P(B | A) P(A) + P(B | A^C) P(A^C)} \end{aligned} \]

Let’s solve!

Step 1: Define events as \(A\) and \(B\).

Let \(A\) be the event of being HIV+ and \(B\) be testing positive.

Step 2: Write out the quantity of interest using Bayes’ Rule.

\[P(HIV + | Test +) = \frac{P(Test + | HIV +)P(HIV +)}{P(Test +)}\]

Let’s Solve! (continued)

Step 3: Expand the denominator using Law of Total Probability. \[ =\frac{P(Test + | HIV +)P(HIV +)}{P(Test + | HIV +) P(HIV +) + P(Test + | HIV - ) P(HIV -)} \]

Step 4: Identify quantities of interest.

  • \(P(Test + | HIV +)\) = True positive rate = sensitivity = .993

  • \(P(HIV +)\) = prevalence = .000143

  • \(P(Test + | HIV - )\) = False positive rate = 1 - specificity = 1-.998

  • \(P(HIV - )\) = 1 - prevalence = 1-.000143

Let’s solve! (continued)

Step 5: Plug in quantities from step 4 and solve.

\[P(HIV + | Test +)\]

\[=\frac{(.993)(.000143)}{(.993)(.000143) + (1-.998)(1-.000143)}\]

\[ = 0.066 = 6.6\%\] Is this result surprising? Why?

Pause & review

TipQuestion
  • In pairs, review the steps of solving this example. Take turns explaining each of the steps to your partner. (Person 1 explains step 1, person 2 explains step 2, etc.)

Interpretation of Result: Oraquick test

  • This calculation is surprising. We would hope that the PPV would be higher.

  • What is going on?

  • Before we answer that: what if a randomly selected adult in Botswana tested positive (HIV prevalence \(\approx\) 25%)?

  • To calculate this, we can use our same setup as above, but just swap out a new value for prevalence, \(P(HIV +)\).

PPV for Botswana

\[P(HIV + | Test +)\]

\[=\frac{(.993)(.25)}{(.993)(.25) + (1-.998)(1-.25)}\]

\[ = .99 = 99\%\]

What’s going on here?

  • The PPV is only 6.6% for US while it’s 99% in Botswana.

  • US has low prevalence of HIV while Botswana has relatively higher prevalence.

  • There will likely be a higher number of false positives relative to true positives in the place with low prevalence, causing the positive predictive value (PPV) to drop.

    • This is simply due to the nature of the test, i.e. having some nonzero false positive rate, in our case .002.

  • This is something to look out for! Specificity (i.e. FPR) is important to consider.

Pause and reflect

  • In small groups, reflect on this example and the following takeaway:
TipTakeaway

  • In the US, small prevalence + modest false positive rate is causing the PPV to be much lower than we’d want.

  • In Botswana, higher prevalence “corrects” this issue, leading to PPV being much higher (meaning more people who test positive actually have the condition.)

  • With a modest false positive rate, a higher prevalence can help our PPV be higher, yielding a more useful diagnostic test for that setting. With lower prevalence, you need to be careful with the conclusions made by a diagnostic test as it could be misleading.

Discrimination thresholds

Oral HIV tests give positive or negative results depending on levels of HIV antibodies detected in saliva.

  • If antibody levels are above a certain threshold, it is classified as a positive test.

  • Varying the threshold for a positive vs. negative test will result in a test in different characteristics.

  • At each threshold value, there is a tradeoff between sensitivity and specificity.

ROC curves

Receiver Operating Characteristic curves show how specificity and sensitivity change as the discrimination threshold changes.

  • The ROC curve was first developed by electrical and radar engineers during World War II for detecting enemy objects in battlefields, starting in 1941.

Haenssle et. al (2018)

paper link here

  • Machine learning algorithm outperformed doctors in many cases!

From Haenssle et al. (2018)

ROC curves show, for each false positive rate, what the true positive rate is corresponding to that threshold value.

Why ROC Curves?

  • By moving the threshold, we can:

    • Catch more true cases (↑ sensitivity),

    • But risk more false alarms (↑ false positive rate).

  • The ROC curve shows all possible tradeoffs at once.

  • The diagonal line = random guessing.

  • A “good” test’s ROC curve bows toward the upper-left corner (high sensitivity, low false positives).

ROC curves help us evaluate how well a test separates groups across all possible thresholds.

Recap

  • How conditional probabilities relate to sensitivity, specificity, & more!

  • Examples, how small sample size, and prevalence, affects diagnostic tests

  • Thresholds & how they affect diagnostic tests

  • ROC Curves: show tradeoff between sensitivity and specificity

Next class

  • Discrete distributions

Participation

How is lab going? How are lecture, assignments going?

Answer the 1-question survey on Canvas by tomorrow at 11:59pm. Your feedback is extremely helpful!