What is Bayes’ Theorem?
Bayes’ theorem is a probability theory used to calculate the likelihood of an event being true or not true based on conditions related to the event. (i.e. an equation used for calculating conditional probabilities).
An other way to say this is Bayes’ theorem is an equation for calculating the likelihood of inductive arguments that use statistical reasoning.
Given statistics like this:
- Approximately 1% of women aged 40-50 have breast cancer.
- A woman with breast cancer has a 90% chance of a positive test from a mammogram.
- While a woman without cancer has a 10% chance of a false positive result.
Bayes’ theorem can help us answer questions like:
- What is the probability a woman has breast cancer given that she just had a single positive test?”
- How many times do we have to repeat the mammogram to be 99% certain of a positive or negative answer?
TIP: See Cornell’s Bayes’ Formula page for details, or see An Intuitive (and Short) Explanation of Bayes’ Theorem from betterexplained.com for another conceptual take on Bayes without the math.
In other words, Bayes’ theory is a statistics-based theory that expresses the concept that we can compare conditional probabilities to find likely truths with a mathematic equation.
In non-math terms Bayes’ theory says, “the more variables considered, and the more certain we are of those variables, then the more certain we can be about our conclusion”.
Or, “the more data pointing to a true outcome, the higher the odds that the outcome is true.” Likewise, “the more data pointing at a false outcome, it’s less likely it is the outcome is false.”
Although we can never know for sure using the inductive method of reasoning Bayes’ theory uses, we can use the formula to increase certainty as we factor in more data and repeat tests.
The concept sounds a lot harder than it is, but in simple terms, Bayes’ theory is just “an equation for comparing probabilities to find the likelihood of truths and faleshoods based on statistical data”.
TIP: Probabilities are not certainties and correlation doesn’t necessarily imply causation. Bayes’ theory can help us be more certain of something, but it can never offer certain truth (it is a form of induction in this respect).
Bayes’ Theorem for Everyone 01 – Introduction.
FACT: The method is named after an 18th-century Presbyterian minister Thomas Bayes. Bayes published the theory in 1812.
The Actual Bayes’ Theorem Equation
With the above covered, let’s discuss the actual equation for a second.
Bayes’ theorem is stated mathematically as the following equation:
Where A and B are events.
- P(A) and P(B) are the probabilities of A and B without regard to each other.
- P(A | B), conditional probability, is the probability of observing event A given that B is true.
- P(B | A) is the probability of observing event B given that A is true.
TIP: If you want to dive into the math of Bayes’ Formula you’ll want to check out Cornell’s page on Bayesian math.
Bayes’ Theorem Examples (Without Math)
The following is a variation of the widely used “Bayes’ Theorem Cancer Example”:
If, however, we know some factors such as the person is older, lives near Chernobyl, and “smokes a pack a day” we can combine those risks factors to find the likelihood that that person has cancer.
If we add in a 4th condition, like them being poor and seldom seeking medical attention, it might increase the probability of us knowing, if we add in enough conditions and find positive or negative correlations, then at some point we can be very close to certain.
TIP: We always translate a risk factor to a real number (as we need numbers to do the math). So smoking a pack a day gets translated into a number based on risk factor analysis. This page focuses on concepts; you can see the links below to learn more about the actual math behind Bayesian equations. The gist is that we would assign values to the above variables and then use the Bayesian equation to calculate probability.
Another simple example would be that, if we want to find out if someone has cancer, we can run a test that will detect cancer correctly nine times out of ten, which we can simplify as a test with 90% accuracy. If we repeat this test, do other tests, or apply other data this will increase our effective detection rate beyond 90%. In other words, one way to increase certainty is to run more tests. Each test increases the probability that our conclusion is correct.
Why is this important?: It means that every piece of new evidence we have can improve our understanding of previously held beliefs. In practice, this allows industries like science, technology, computing, statistics, and more to better work with data. For just one example: Google’s self-driving car is literally running a Bayesian algorithm to determine the next best moves.
Explain Bayes’ Theorem – Explained as though teaching a child about probability theory.
Bayes’ Theorem: A Probability Theory
As we said above, Bayes’ Theorem is probability theory. That is, it’s a calculation of the likelihood. To clarify the terms:
- Probability works like this: It’s a number value assigned to the likelihood of something happening.
- Theories work like this: Facts point toward theory, the more facts that point to it, and the more it works as the fact itself, the more likely it is to be true. Most theories follow a similar line of reasoning to Bayesian thinking.
Bayes’ Theorem: What are the Odds? We can answer that.
Real World Applications of Bayesian Principles
Imagine I have a self-driving car and need to calculate the best next move, the car’s sensor gleans data from the environment and compares probabilities and outcomes until it knows with near certainty how to make the next safest move.
We can use Bayesian principles in insurance adjustment to calculate risks by knowing factors like age, or for healthcare by repeating tests for increased accuracy.
Bayes’ theorem also applies to computer security. Bitcoin uses Bayesian principles for security, by having many independent people confirm a transaction to the blockchain or public ledger, it increases the likelihood that the most common answer is true.
Bayesian principles have far-reaching practical applications, including helping us refine machine learning, cognitive computing, and the way we interact with the IOT. Some even theorize that the human cognition mimics Bayesian algorithms.
Can Bayes’ Theorem Calculate Absolute Truth?
Depending upon the nature of the problem, we can theoretically decrease the margin of error to almost nothing by calculating enough variables.
Using many variables works best when we have a large number of them to compare. The fewer variables, the less certain we can be.
Even with a near 0% margin of error, all probability is a thing of chance by its nature.
We can find an absolute truth using Bayesian principles, but we can’t prove it’s absolute truth using Bayesian principles.
Being 99.9% certain the woman has cancer is not a statement on whether or not the woman has cancer, but it is very useful data!