Some of my favorite, simple examples demonstrating how to evaluate whether two events are independent using Probability Theory.
Example 1 - Product Rule
Example 2 - Conditionals & Intersection
Example 3 - Conditionals
Example 4 - Markov Chain
Example 1 - Product Rule
If two events are independent, then the product rule states that their intersection should equal the product of each independent probability.
p(a & b) = p(a) * p(b)
The World Values Survey is an ongoing worldwide survey that polls the world population about perceptions of life, work, family, politics, etc. The most recent phase of the survey that polled 77,882 people from 57 countries estimates that 36.2% of the world’s population agrees with the statement, “Music is not necessary to enhance one’s life.” The survey also estimates that 13.8% of people have a university degree or higher, and that 3.5% of people fit both criteria.
Does agreeing depend on level of degree? If a = “someone agrees with the statment” and b = “has a university degree or higher,” does a depend on b?
p(agree) = 0.362p(univ degree) = 0.138p(agree & univ degree) = 0.036
If they are independent, then the product rule should hold.
p(a & b) = p(a) * p(b)
Evaluate:
0.036 = 0.362 * 0.138(which does not hold)
Therefore, the two are dependent.
This example comes from a coursera class.
Example 2 - Conditionals & Intersection
If two events are independent, then the probability of one conditioning on the other should equal the probability of the original alone.
p(b | a) = p(b & a) / p(a)If a and b are independent, then
p(b | a) = p(b).
Two players are each going to role a different die. Player 1’s die is six-sided and contains the numbers 5, 5, 5, 1, 1, 1, whereas player 2’s die contains the numbers 4, 4, 4, 4, 0, 0.
Take a to be the event that the player 1’s die is 5, and take b to be the event that the sum of the dice is equal to 1.
- a = player 1 rolls a 5
- b = sum of both dice is 1
Is b dependent on a?
We can also run the same procedure but with a different event for b.
Example 3 - Conditionals
If two events are independent, then taking a and conditioning on other events (e.g., b, c, d, etc.) should not change the observed probability.
In 2013, a research group interviewed a random sample of 500 NC residents asking them whether they think widespread gun ownership protects law abiding citizens from crime, or makes society more dangerous.
- 58% of all respondents said it protects citizens
- 67% of White respondents,
- 28% of Black respondents,
- and 64% of Hispanic respondents shared this view.
Are opinion on gun ownership and ethnicity dependent?
p(agree) = 58%p(agree | white) = 67%p(agree | black) = 28%p(agree | hispanic) = 64%
Notice that conditioning on the other variables changes the probability, so opinion and ethnicity are probably dependent.
This example comes from a coursera class.
Example 4 - Markov Chains
If two events are independent, then the probability of observing one after the other should be p(a) * p(a), similar to the notion of a coin flip such that the probability of observing two heads in a row is 0.5X0.5 = 0.25. If you calculate a transition matrix and observe probabilities that differ from that original number, then the sequence is probably dependent.
Andrei Markov applied Markov chains to the poem Eugene Onegin by Alexander Pushkin. In the first 20,000 letters of the poem, he counted the number of vowels (8,638) and consonants (11,362).
p(vowel) = 0.432p(consonant) = 0.568
Then, he counted the transitions from vowel to consonant or consonant to vowel. For every vowel, the number of times the next letter was a vowel was 1,104 and the number of times the next letter was a consonant was 7,534. For every consonant, the number of times the next letter was a consonant was 3,827 and the number of times the next letter was a vowel was 7,535.
p(vowel to vowel) = 1104 / 8638 = 0.175p(vowel to consonant) = 7534 / 8638 = 0.825p(consonant to vowel) = 7535 / 11362 = 0.526p(consonant to consonant) = 3827 / 11362 = 0.474
So the transition matrix is…
transition_matrix <- matrix(c('', 'v', 'c',
'v', '0.175', '0.825',
'c', '0.526', '0.474'), 3, 3)
transition_matrix %>%
kable() %>%
kable_styling(bootstrap_options = "striped", full_width = F)
| v | c | |
| v | 0.175 | 0.526 |
| c | 0.825 | 0.474 |
If the letters were independent, then the probability of witnessing a vowel follow a vowel would be p(vowel) * p(vowel), or 0.432 * 0.423 = 0.186624. However, the observed transition probability is 0.175, so the sequence is dependent.
Note that you have to assume that the counts of vowels and consonants reflect their true propabilities (which follows from the law of large numbers). Markov showed that the law of large numbers applied to even dependent sequences.
Bo\(^2\)m =)