Bayesian Thinking During the Pandemic

Mahmut Karayel
06 Aug 2020

BAYESIAN THINKING DURING THE PANDEMIC

Last week, I almost missed my exit while driving on Highway 580 in the Bay Area. The person on the radio was saying some COVID-19 antibody tests have 40% or 50% false-positive rate. What? Should I believe this? If it is true what does it mean to get tested?

I did some research when I got home. First, they were talking about some fast and cheap (quick and dirty?) antibody test. These are not the same tests that determine whether you are currently infected (PCR tests). They are designed to detect if you had been infected in the past.

Nevertheless, the question remains: “How much should I believe the results of these tests?”
(Positive meaning infected, and Negative meaning not infected)

I will take this example as an opportunity to discuss how we interpret new data, that is the relationship between observing the data and changing our beliefs accordingly. This is what is called Bayesian thinking. Thomas Bayes was a late 18th century minister living in England who accidentally discovered the concept of updating beliefs when we see new data or new evidence. He discovered it trying to estimate the chances that a ball thrown on a table will land in a certain section of the table. I guess English priests had a lot of time on their hands, maybe because they spent less time witch hunting.

Bayes did not realize the significance of what he found and did not publish it. But, in time, the scientific community adopted his way of thinking as one of the most important methods of science. Today, Bayes formula is widely used in Medicine, AI, digital marketing, among many other fields. It is also a good philosophical framework to think about how certain cognitive biases may hinder our learning and carry us astray.

Back to COVID-19 testing. Here are some basic facts about tests:

  • No test is 100% accurate
  • Sensitivity is the true positivity rate. Some tests are not very sensitive.
    That is, many who are infected are wrongly labeled as not infected (False Negatives)
  • Specificity is the true negative rate. Some tests are not very specific.
    That is, many who are not infected are wrongly labeled as infected (False Positives)
  • How we interpret the result of the test also depends on our prior belief (hopefully, but not necessarily, with some evidence)

The last point is worth reading twice. Whether you are scientific (relatively objective) person, or a rigid believer, your interpretation of the results will depend on what you thought your chances of infection was before the test.

Interpreting the strength of our belief in a hypothesis as probability (prior probability before the observation and posterior probability after the observation) was a significant paradigm shift that allowed advancement of science in the 20th century.

This is how we usually go from data to insight!

Equation 1

The | in the formula means “given”. I put a proportion sign ~ rather than an equal sign, because there is a normalization factor that I omitted from the equation. Probabilities need to sum to 100%.

Now, suppose that I test positive, how should I adjust my belief that I am not infected?

Let us define some terms relating to the accuracy of the test and my prior belief:

Equation 2

  • Lets say that out of 1000 people, 100 are infected. But the test data is positive (test +) for only 98 of these subjects.
    Then the sensitivity (true positive rate) is 98%
  • Lets say that out of 1000 people, 900 are not infected. But the test data is negative (test -) for only 855 of these 900 subjects.
    Then the specificity (true negative rate) is 95% (855/900)
  • Prob (Infected) is the Prior. It is the probability that the Hypothesis is True (i.e., Subject is infected) given all the data before the test.
    This is our "prior" belief.

The Bayes’ formula in case of COVID-19 test where I test positive works as follows:

Equation 3

The Bayes’ formula in case of COVID-19 test where I test negative works as follows:

Equation 4

Take two tests and two risk groups:

  • Quick and Cheap test with sensitivity 90% and specificity 90%
  • Better test with sensitivity 99% and specificity 95%
  • Low risk group who social distances, masks, < 70, no pre-existing condition, with prior chances = 2%
  • High risk group who went to bars, no masks, showing mild symptoms, with prior chances = 20%

The below diagram represents the Quick and Cheap test with High Risk Group:

Figure 1

We examine the cases below. The first four cases represent adjustment of the infected Hypothesis Prior is the hypothesis value (belief) before the test.
Table 1

There are three interesting conclusions:

  • If the subject tests negative, we are confident that they are not infected, in any case.
  • If the subject tests positive with the Quick and Cheap test, we probably should administer the Better test also, especially if the Prior is a low probability. Only if they test positive again, we are confident of the result.
  • New data updates the old belief. It does not completely change beliefs or override them.

We do this routinely with our normal medical procedures. We administer better and better tests, to be sure. We get second opinions. And, we do not question good results. Why can’t we manage to do this during a pandemic and test everybody?

Please share your thoughts in the comments section.

Related posts here: