> Home > Statistics Notes > Probability > Axioms of Probability

## Probability and Relative Frequency

The probability that a certain outcome will be observed in an experiment is the limiting frequency of that outcome after repeated experimentation. (If I believe the chance of rolling a 1 on a die is 1/6, then if I roll the die enough times, I ought to be seeing ones about one sixth of the time. I should be able to get the frequency of ones as close to one sixth of the total as I want, if I'm willing to throw the die enough times.)
For example, suppose that I perform an experiment n times, and I determine that some event A happened on a certain number of them. Call the number of times that the event A happened nA. Then the frequency of occurrence of the event A is fA=nA/n. If the event A happens every time we do the experiment, then the frequency is one; if the event A has not happened, then the frequency of A is zero. If the event A has happened on some of the experiments, but not on others, then the frequency is between 0 and 1.
If the experiment is repeated infinitely many times, then the frequency of occurrences of A should be the probability of A; the probability of A is the limiting frequency of occurrences of A. This is the frequency definition of probability.
Some authors use probability to reflect their subjective degree of belief in the truth of statements. This is a different use of the formalism of probability theory.

## The axioms of probability

Since the pioneering work of Kolmogorov, an axiomatic formulation of probability has been used; in this formalism, it is a theorem that the probability of occurrence of an event is approximately the relative frequency of occurrences of an event in a very long sequence of independent events.
Probabilities are numbers assigned to events, i.e. subsets of the sample space (for our purposes). (When dealing with certain infinite sample spaces, not just any subset of the sample space can be allowed to be an event. This fact will not be needed for this class.) The axioms of probability were inspired by the ways that relative frequencies behave; there will never be a negative probability, nor will probabilities ever be greater than one.

The first axiom is that the probability is a number between 0 and 1 which is assigned to certain subsets of the sample space, called events. It is standard to write P(A) to denote the probability of the event A.

The next is that the probability the sample space U itself, P(U), always equals 1. The sample space U may be called the certain event.
Remember that probability is meant to be the limiting frequency of occurrences of an event as the experiment is repeated more and more times. The sample space is the collection of all the outcomes of the event. Suppose I consider the event that something in the sample space occurs. For instance, suppose I roll a single die over and over again, and I ask how often I see either a 1, 2, 3, 4, 5, or a 6. Since the sample space consists of these and only these possibilities, every time I roll the die, I see one of them. The frequency is 1, always. For this reason, the probability of the sample space itself (taken as an event) is defined to be 1. This is written P( )=1.

The final axiom involves disjoint unions. Suppose you have a disjoint sequence of events like Ai for which the probability is defined (we have P(Ai) for each of them). These are disjoint events - they have no elements in common; for any two different sets Ai and Aj for which i isn't the same as j, the intersection of the two sets is the empty set. Then let A (with no subscript) be the union of all these sets. The axiom says that the probability of the disjoint union A exists and equals the sum of the probabilities P(Ai) for each of them).

On to probabilities of complementary events.