Dependent and independent events

Consider two events $E$ and $F$ of a random experiment. We already know that both events can occur at the same time if the experiment is performed. If the occurrence of one of those events changes the probability of the other event to occur, we say that $E$ and $F$ are dependent. If this is not the case, we say that the two events are independent.

Dependent events in probability are no different from dependent events in real life. Here are some examples of dependent events $E$ and $F$ :

$E=$ "robbing a bank" and $F=$ "going to jail".
$E=$ "not paying your power bill" and $F=$ "having the power cut off".
$E=$ "boarding a train first" and $F=$ "finding a good seat".
$E=$ "parking illegally" and $F=$ "getting a parking ticket".
$E=$ "buying ten lottery tickets" and $F=$ "winning the lottery".
$E=$ "driving a car" and getting in a $F=$ "traffic accident".
Selecting two balls, one after the other, without replacement from a box containing red and green balls. $E=$ "a green ball in the first selection" and $F=$ "a red ball in the second selection".

Why are "robbing a bank" and "going to jail" dependent events? The random experiment here is "selecting a person at random". The probability that a selected person goes to jail might be quite low. But if the person is robbing a bank, the probability that this person has to go to jail as well increases drastically. So the occurrence of the event "robbing a bank" changes the probability of the event "going to jail".

Here are some examples of independent events $E$ and $F$ :

$E=$ "owning a dog" and $F=$ "growing your own herb garden".
$E=$ "winning the lottery" and $F=$ "running out of milk".
$E=$ "buying a lottery ticket" and $F=$ "finding a penny on the floor".
$E=$ "in the first coin toss heads" and $F=$ "in the second coin toss tails".
Selecting two balls, one after the other, with replacement from a box containing red and green balls. $E=$ "a green ball in the first selection" and $F=$ "a red ball in the second selection".

Why is "owning a dog" and "growing a herb garden" independent events? Again, the random experiment is "selecting a person at random". Well, clearly these two events have nothing to do with each other - selecting a person who owns a dog does not increase or decrease the probability that this person grows a herb garden as well. Or is it??

Apart from our intuition, how can we tell if two events are dependent or independent? We need an exact definition:

Definition 1

Two events $E$ and $F$ of a random experiment are independent, if

p(E\vert F) = p(E)\quad\text{and}\quad p(F\vert E)=p(F)

This definition makes intuitively sense, because $p(E\vert F)=p(E)$ means that the probability for $E$ to occur does not change, no matter if $F$ is given (or has occurred) or not. So event $F$ has no influence on the event $E$ . And likewise, $p(F\vert E)=p(F)$ means that event $E$ has no influence on event $F$ . So $E$ and $F$ are independent.

Note that to show that two events are independent, it is actually enough to show that one of the equations above are valid. Indeed, the following can be shown :

Theorem 1

Consider two events $E$ and $F$ of a random experiment. If it can be shown that $p(E)=p(E|F)$ , or that $p(F)=p(F|E)$ , then the following is true:

$E$ and $F$ are independent
$E^\prime$ and $F$ are independent
$E$ and $F^\prime$ are independent
$E^\prime$ and $F^\prime$ are independent

The proof is not difficult, but quite technical and we will not show it completely. We will just prove statement (1), the other statements are shown in a similar fashion.

Exercise 1

Prove statement (1).

Solution

Assume that $p(E)=p(E|F)$ is valid. We have to show that $p(F)=p(F|E)$ .

By definition, we have

p(E|F)=\frac{p(E\cap F)}{p(F)}

Because of our assumption $p(E)=p(E|F)$ it follows

p(E)=\frac{p(E\cap F)}{p(F)}

and by multiplying both sides by $p(F)$ we get

p(E)\cdot p(F)=p(E\cap F)

Dividing both sides by $p(E)$ , we get

p(F)=\frac{p(E \cap F)}{p(E)} = p(F|E)

Done!

Here is a another useful theorem.

Theorem 2

Consider two events $E$ and $F$ of a random experiment. If $E$ and $F$ are independent, then

p(E\cap F)=p(E)\cdot p(F)

The opposite is also true. If the above equation is fulfilled, then the events $E$ and $F$ are independent.

The proof is left as an exercise:

Exercise 2

Proof the statement above.

Solution

Assume that $E$ and $F$ are independent. We want to show that
$p(E\cap F)=p(E)\cdot p(F)$
Indeed, as $E$ and $F$ are independent, it follows
$p(E)=p(E|F)=\frac{p(E\cap F)}{p(F)}$
Multiplying by $p(F)$ , we obtain
$p(E)\cdot p(F)=p(E\cap F)$
Assume now that the equation $p(E\cap F)=p(E)\cdot p(F)$ is fulfilled. We want to show that $E$ and $F$ are independent. All we have to show is that $p(E)=p(E|F)$ . But this is true, because from $p(E\cap F)=p(E)\cdot p(F)$ follows $p(E)=\frac{p(E\cap F)}{p(F)}=p(E|F)$ .

Exercise 3

Q1

Are two mutually exclusive events independent?

Q2

The experiment is flipping a coin twice. How do you show experimentally, if the events $H_1$ ="head in first flip" and $H_2$ ="head in second flip" are independent?

Q3

A box contains 3 blue and 5 red balls. We select two balls at random. Consider the two events $R_1$ ="red ball in first selection" and $R_2$ ="red ball in second selection". Are the two events independent if

selection is with replacement (that is, the first ball is returned before the second ball is selected).
selection is without replacement (that is, the first ball is not returned before the second ball is selected).

Argue with the probabilities.

Solution

A1

No. As $E$ and $F$ are mutually exclusive, it is $p(E\cap F)=p(\{\})=0$ , but $p(E)\cdot p(F)\neq 0$ (always assuming that $p(E)\neq 0$ and $p(F)\neq 0$ ). Intuitively this makes sense. Given $E$ , we know that $F$ cannot happen, as the events exclude each other from occurring. So $E$ influences the occurrence of $F$ , and thus they are dependent.

A2

Repeat the experiment many times, and determine the probabilities $p(H_1), p(H_2)$ and $p(H_1\cap H_2)$ . Then show that $p(H_1)\cdot p(H_2)=p(H_1\cap H_2)$ .

A3

The trees are shown below (left: with replacement, right: without replacement). We check if $p(R_1)\cdot p(R_2)=p(R_1\cap R_2)$ .

$p(R_2)=\frac{5}{8}\cdot\frac{5}{8}+\frac{3}{8}\cdot\frac{5}{8}=\frac{5}{8}$ . Thus
$p(R_1)\cdot p(R_2) = \frac{5}{8}\cdot \frac{5}{8}=\frac{25}{64}$ $p(R_1\cap R_2) = \frac{5}{8}\cdot\frac{5}{8}=\frac{25}{64}$
So they are independent.
$p(R_2)=\frac{5}{8}\cdot\frac{4}{7}+\frac{3}{8}\cdot\frac{5}{7}=\frac{35}{56}$ . Thus
$p(R_1)\cdot p(R_2) = \frac{5}{8}\cdot \frac{35}{56}=\frac{165}{448}=0.368...$ $p(R_1\cap R_2) = \frac{5}{8}\cdot\frac{4}{7}=\frac{20}{56}=0.357...$
So they are not independent.