As far as the laws of mathematics refer to reality, they are not certain, and as far as they are certain, they do not refer to reality~Albert Einstein
The world we see around us is full of uncertainties. We deal with them on a day-to-day basis, in the form of Decision making. Yet, most of us don’t know how to deal with them systematically. We know that to improve on something, we need to measure it in the first place. So, how do we measure the uncertainties? The answer lies in Probability theory.
Thus, probability has been the bedrock of all statistical analysis and inference. Hence, it is imperative to define and understand probability extensively. However, take a look at the below quote:
It is unanimously agreed that statistics depends somehow on probability. But, as to what probability is and how it is connected with statistics, there has seldom been such complete disagreement and breakdown of communication since the Tower of Babel ~ Leonard Savage
After all, since we deal with uncertainty here, how can the definition be certain? Hence, there are three famous interpretations of probability.
Classification Interpretation
This definition is similar to what we have learnt in high school. It is based on the assumption that every outcome is equally likely. For instance, a tossed coin has equally possible outcomes of head and tail, each having a probability of 1/2. In general, for a process with n different equally likely outcomes, the probability for each of those outcomes is 1/n.
Here is the formal definition of the same:
While this Interpretation is simple, it is of hardly any practical use, apart from coin toss or the game of dice etc.
Frequentist Interpretation/Empirical Probability
A more practical definition or interpretation of probability is the frequentist interpretation. Suppose the experiment is repeated n times, the probability of the event A is the number of times A occurs in the same experiment. It is the proportion of the number of times an event occurs in a large number of trials. Since this is also based on empirical evidence, it is also called Empirical probability.
An obvious fallout of this definition is that this requires a large number of trials to be performed to reach a conclusion about the probability of an event. Most of the time, we don’t have that luxury.
Nonetheless, this is extremely useful in the real world, where scientists and mathematicians have observed and tabulated certain empirical probabilities. For instance, Benford’s law. Let’s try to understand it intuitively.
If you are asked what is the probability of a certain number between 1-9 being the leading digit, by the knowledge of classical probability you may say 1/9. However, by observation, according to Benford’s law, the number 1 has the highest probability, while the number 9 has the lowest. This holds true for a large number of real-life datasets. Hence, it is quite useful in anomaly detection in a lot of real-world data like stock prices, house prices etc. We encourage you to read more about it.
Subjective Interpretation
The previous two interpretations are more objective forms of probability. Having said that, in the real world, we rarely have the luxury of getting equally likely events for dealing with classical definitions. Many times, we do not even have all the required data to find empirical probabilities. So how do we proceed?
Turns out that we have another resource i.e. human belief. Yes. How many times have you heard of expert predictions about a certain event? Let’s say you are watching an India vs Pakistan cricket match. Some expert (maybe your uncle) predicts that India has an 80 per cent chance of winning, even before the toss. This is called Subjective probability.
He may have his own analysis. Yet, it is still it is an individual assessment. But, the beauty of subjective probability is that it can be updated with the arrival of new information. For instance, the new information could be the changes in the squad or the powerplay score.
This takes us to the wonderful concept of Bayesian Analysis, where probabilities are updated as the information changes. A lot of modern Data Science and AI is lying on the foundation of Bayesian Analysis.
Although it looks promising, the Subjective Probability assignment needs to follow the laws of probability. For instance, it cannot leave the bounds of 0 to 1. It should not fall into the trap of the Conjunction Fallacy. Lastly, it has to be reasonable.
Conclusion
Irrespective of the interpretation of probability you may be using, it must follow the axioms of probability theory. Read more about the axioms here. Having said that, we have referred to multiple sources while writing this article. Any misinformation or misspellings are the author’s shortcomings. This article is for information purposes. We do not claim any guarantees regarding its accuracy or completeness.
