Probability basics
Begin with a set Ω—the sample space.
This is all the possible things that could happen.
‚ 6possiblerollsofadie.
‚ HowmanyifIhavetwodice?
ω P Ω is a sample point, atomic event.
(pngimg.com)
⃝c -Trenn, King’s College London 2
Probability basics
A probability space or probability model is a sample space with an assignment Ppωq for every ω P Ω such that:
0 ď Ppωq ď 1 ÿ
Ppωq “ 1 ω
For a typical die: Pp1q“Pp2q“Pp3q“Pp4q“Pp5q“Pp6q“1{6.
(pngimg.com) (hartsport.com.au)
⃝c -Trenn, King’s College London 3
Probability basics
An event A is any subset of Ω
PpAq “
ÿ
tωPAu
Ppωq
Again, for a regular die:
P pdie roll ă 4q “ P p1q ` P p2q ` P p3q “ 1{6 ` 1{6 ` 1{6 “ 1{2
(pngimg.com) (hartsport.com.au)
⃝c -Trenn, King’s College London 4
Random variables
A random variable is a function from sample points to some range. ‚ rainingpLondonqPttrue,falseu.
‚ temperatureplectureroomqPt0,1,…,30u.
P induces a probability distribution for any r.v. X: ÿ
PpX “xiq “
In our dice example, we could set ω “ die shows an odd number:
PpOdd“trueq “ Pp1q`Pp3q`Pp5q “ 1{6`1{6`1{6
“ 1{2
⃝c -Trenn, King’s College London
5
tω:Xpωq“xiu
Ppωq
Propositions
We describe the world in terms of propositions, which are mathematical statements such as “the die shows an odd number’ or “it is raining”.
Think of a proposition as the event (set of sample points) where the proposition is true
Given Boolean random variables A and B:
event a = set of sample ω points where Apωq “ true
event ␣a = set of sample points ω where Apωq “ f alse event a ^ b = points ω where Apωq“true and Bpωq“true
Example:
The set of sample points is Ω “ t1,2,3,4,5,6u.
Apωq “ true or simply a is the event that the number is odd, i.e., t1, 3, 5u. Bpωq “ true or simply b is the event that the number is ă 4, i.e., t1, 2, 3u. a ^ b is given by the sample points in t1, 3u.
⃝c -Trenn, King’s College London 6
Propositions
A state can be defined by a set of Boolean variables.
a ^ b _ ␣c A “ true, B “ true, C “ f alse This is then just a sample point.
⃝c -Trenn, King’s College London
7
Union and Intersection
The definitions imply that certain logically related events must have related probabilities
P pa _ bq “ P paq ` P pbq ́ P pa ^ bq
(lucidchart.com)
Example: Let a denote the event that the die shows an odd number. Let b denote the event that the die shows a number ă 4. Hence, the probability that we get an odd number or a number ă 4 is
Ppa_bq“Ppaq`Ppbq ́Ppa^bq“21 `12 ́Pproll1or3q“2{3.
⃝c -Trenn, King’s College London 8
Prior and posterior probability
Prior or unconditional probabilities of propositions
P pCavity “ trueq “ 0.1 and P pW eather “ sunnyq “ 0.72 correspond to belief before (prior) to arrival of any (new) evidence.
In contrast, P pCavity “ true | toothacheq “ 0.2 is the posterior or conditional probability. Here, we have additional evidence.
⃝c -Trenn, King’s College London 9
Conditional Probabilities
Conditional Probabilities
assuming P pbq ą 0.
Ppa|bq “ Ppa ^ bq P pbq
(lucidchart.com)
Example: Let a denote the event that the die shows an odd number. Let b denote
the event that the die shows a number ă 4. Hence, the probability that we get an
odd number given that the is number ă 4 is P pa|bq “ P pa^bq “ 1{3 “ 2{3. P pbq 1{2
⃝c -Trenn, King’s College London
10
Syntax for propositions
Propositional or Boolean random variables ‚ Cavity (do I have a cavity?)
‚ Cavity“trueisaproposition,alsowrittencavity
‚ Cavity“falseisaproposition,alsowritten␣cavityorcavity
Discrete random variables (finite or infinite)
‚ Weatherisoneoftsunny,rain,cloudy,snowu ‚ Weather“rainisaproposition
Continuous random variables (bounded or unbounded) ‚ Temp“21.6;alsoallow,e.g.,Tempă22.0.
⃝c -Trenn, King’s College London 11
Syntax for propositions
We allow arbitrary combination of logical operators (AND, OR, NOT) and comparison operators (ă, ď, “, ‰, . . . q.
E.g., T emp ă 22.0 AND Cavity “ true
⃝c -Trenn, King’s College London 12
Notation
Probability distribution gives values for all possible assignments (assumes a fixed ordering):
̈ ̨
0.72 ̊‹
̊ 0.1 ‹
PpW eatherq “ ̊‹ (1)
means
‚ PpWeather“sunnyq“0.72,
‚ PpWeather“rainq“0.1,
‚ PpWeather“cloudyq“0.08and
‚ PpWeather“snowq“0.1
Values must be exhaustive (everything covered) and mutually exclusive (no overlap)
Values have to sums to 1
Note that the book uses the notation x0.72, 0.1, 0.08, 0.1y
⃝c -Trenn, King’s College London 13
̊0.08‹ ̋‚
0.1
Quiz
There is a KEATS quiz to see if you understood. Have a look!
⃝c -Trenn, King’s College London 14