> Home > Statistics Notes > Probability > Axioms of Probability
Probability and Relative Frequency
The probability that a certain outcome will be observed in an experiment
is the limiting frequency of that outcome after repeated
experimentation. (If I believe the chance of rolling a 1 on a die is
1/6, then if I roll the die enough times, I ought to be seeing ones about
one sixth of the time. I should be able to get the frequency of ones as
close to one sixth of the total as I want, if I'm willing to throw the
die enough times.)
For example, suppose that I perform an experiment n times, and
I determine that some event A happened on a certain number
of them. Call the number of times that the event A happened
nA. Then the frequency of occurrence
of the event A is
fA=nA/n. If the event A happens every time we do the experiment, then
the frequency is one; if the event A has not happened, then
the frequency of A is zero. If the event A has happened
on some of the experiments, but not on others, then the frequency is
between 0 and 1.
If the experiment is repeated infinitely many times, then the frequency
of occurrences of A should be the probability of A; the probability of A
is the limiting frequency of occurrences of A. This is
the frequency definition of probability.
Some authors use probability to reflect their subjective degree of
belief in the truth of statements. This is a different use of the
formalism of probability theory.
The axioms of probability
Since the pioneering work of Kolmogorov, an axiomatic formulation of
probability has been used; in this formalism, it is a theorem that the
probability of occurrence of an event is approximately the relative frequency
of occurrences of an event in a very long sequence of independent events.
Probabilities are numbers assigned to events, i.e. subsets of the
sample space (for our purposes). (When dealing with certain infinite
sample spaces, not just any subset of the sample space can be allowed
to be an event. This fact will not be needed for this class.)
The axioms of probability were inspired by the ways that relative
frequencies behave; there will never be a negative probability, nor
will probabilities ever be greater than one.
The first axiom is that the probability is a number between 0 and 1 which
is assigned to certain subsets of the sample space, called events. It
is standard to write P(A) to denote the probability of the event A.
The next is that the probability the sample space U itself, P(U), always
equals 1. The sample space U may be called the certain event.
Remember that probability is meant to be the limiting frequency of
occurrences of an event as the experiment is repeated more and more
times. The sample space is the collection of all the outcomes of
the event. Suppose I consider the event that something in the sample
space occurs. For instance, suppose I roll a single die over and over
again, and I ask how often I see either a 1, 2, 3, 4, 5, or a 6. Since
the sample space consists of these and only these possibilities, every
time I roll the die, I see one of them. The frequency is 1, always.
For this reason, the probability of the sample space itself (taken as
an event) is defined to be 1. This is written
P(
)=1.
The final axiom involves disjoint unions. Suppose you have a disjoint
sequence of events like
Ai for which the probability is defined (we have
P(Ai) for each of them). These are disjoint
events - they have no elements in common; for any two different sets
Ai and Aj for which
i isn't the same as j, the intersection of the two sets is the empty set.
Then let A (with no subscript) be the union of all these sets. The axiom
says that the probability of the disjoint union A exists and equals the sum
of the probabilities P(Ai) for each of them).
)=1.