CS570 Biomedical Science & Health IT
CS544 D1
Foundations of Analytics
Lecture 3
Guanglan Zhang
1
1
Sample space
The set of all possible outcomes of a random experiment is called the sample space of the experiment.
We will denote the sample space using the letter S. For example, in the case of tossing a coin once, the sample space S={H,T}, H denoting the outcome being the head and T the outcome being the tail.
Similarly, if the coin is tossed twice, the sample space S={HH,HT,TH,TT}.
2
2
Multiplication Principle
Example
Suppose that I want to buy a iPhone 7. I can choose either a 64GB or 128GB storage capacity, and a black, grey, pink or gold cover. How many different options do I have?
Suppose that we perform r experiments such that the kth experiment has nk possible outcomes, for k=1,2,⋯⋯,r. Then there are a total of n1×n2×n3×⋯nr possible outcomes for the sequence of r experiments.
3
3
Counting
A set is a collection of things (elements).
Sampling from a set means choosing an element from that set. We often draw a sample at random from a given set in which each element of the set has equal chance of being chosen.
When we talk about sampling from sets, we can talk about 4 possibilities:
ordered sampling with replacement
ordered sampling without replacement
unordered sampling without replacement
unordered sampling with replacement
With or without replacement: when we draw multiple samples from a set, if we put each object back after each draw, we call this sampling with replacement. “With replacement” means repetition is allowed. On the other hand, if repetition is not allowed, we call it sampling without replacement.
Ordered vs. unordered: If ordering matters (i.e.: a1,a2,a3≠a2,a3,a1), this is called ordered sampling. Otherwise, it is called unordered.
https://www.probabilitycourse.com/chapter2/2_1_0_counting.php
4
4
Ordered sampling with replacement
Given n, the number of distinguishable objects, ordered samples of size k, with replacement – the number of outcomes is n⋅n⋅…⋅n(k times)=nk
There are n options. When ordering matters and repetition is allowed, the total number of ways to choose k objects from a set with n elements is n⋅n⋅…⋅n=nk
Note that this is a special case of the multiplication principle where there are k “experiments” and each experiment has n possible outcomes.
https://www.probabilitycourse.com/chapter2/2_1_1_ordered_with_replacement.php
5
5
Ordered Sampling without Replacement: Permutations
Given n, the number of distinguishable objects, ordered samples of size k, without replacement – the number of outcomes is
There are n options for the first choice, (n−1) options for the second choice (since one element has already been picked up and cannot be chosen again), (n−2) options for the third choice, … (n−k+1) options for the kth choice. Thus, when ordering matters and repetition is not allowed, the total number of ways to choose k objects from a set with n elements is
Any of the chosen lists in the above setting (choose k elements, ordered and no repetition) is called a k-permutation of the n-element set. We use the following notation to show the number of k-permutations of an n-element set:
Note that if k is larger than n, then . This makes sense, since if k>n there is no way to choose k distinct elements from an n-element set.
https://www.probabilitycourse.com/chapter2/2_1_2_ordered_without_replacement.php
6
6
combination does not take into account the order, whereas a permutation does
Unordered Sampling without Replacement: Combinations
The number of k-combinations of an n-element set is given by for 0≤k≤n.
=
I choose 3 cards from the standard deck of cards. What is the probability that these cards do not contain ace?
https://www.probabilitycourse.com/chapter2/2_1_3_unordered_without_replacement.php
7
7
This is read “n choose k.”
n! read as “n factorial.”
Unordered Sampling with Replacement
Suppose that we want to sample from the set A. If A={1,2,3} and k=2, then there are 6 different ways of doing this: 1,1; 1,2; 1,3; 2,2; 2,3; 3,3;
One way to think about this is to note that any of the pairs in the above list can be represented by the number of 1’s, 2’s and 3’s it contains. That is, if x1 is the number of ones, x2 is the number of twos, and x3 is the number of threes, we can equivalently represent each pair by a vector (x1, x2, x3), i.e.,
1,1 →(x1, x2, x3)=(2,0,0)
1,2 →(x1, x2, x3)=(1,1,0)
1,3 →(x1, x2, x3)=(1,0,1)
2,2 →(x1, x2, x3)=(0,2,0)
2,3 →(x1, x2, x3)=(0,1,1)
3,3 →(x1, x2, x3)=(0,0,2)
https://www.probabilitycourse.com/chapter2/2_1_4_unordered_with_replacement.php
8
8
This is read “n choose k.”
n! read as “n factorial.”
Unordered Sampling with Replacement
Note that here xi≥0 are integers and x1+x2+x3 =2. Thus, we can claim that the number of ways we can sample two elements from the set A such that ordering does not matter and repetition is allowed is the same as solutions to the following equation x1+x2+x3 =2, where xi ∈{0,1,2}.
The total number of distinct k samples from an n-element set such that repetition is allowed and ordering does not matter is the same as the number of distinct solutions to the equation x1+x2+…+xn=k, where xi ∈{0,1,2,3,…}.
Theorem 1
The number of distinct solutions to the equation x1+x2+…+xn=k, where xi ∈{0,1,2,3,…} is equal to =
https://www.probabilitycourse.com/chapter2/2_1_4_unordered_with_replacement.php
9
9
This is read “n choose k.”
n! read as “n factorial.”
Counting results for different sampling methods.
Assuming that we have a set with n elements, and we want to draw k samples from the set, then the total number of ways we can do this is given by the following table.
https://www.probabilitycourse.com/chapter2/2_1_4_unordered_with_replacement.php
10
10
This is read “n choose k.”
n! read as “n factorial.”
Probability
P(A or B) = P(A∪B) = P(A) + P(B) when the events A and B are mutually exclusive
If the events A and B are not mutually exclusive, adding the probabilities of the two events will add the common outcomes (A&B) twice.
P(A or B)=P(A)+P(B)–P(A&B) = P(A)+P(B)–P(A∩B), , where A and B are not mutually exclusive.
For any event E, the complement of the event is represented by not E.
P(not E)=1–P(E)
11
11
Conditional Probability
The conditional probability, P(B|A), is the probability that event B occurs given that event A occurs. It is read as the probability of B given A. Event A is called the given event.
If A and B are any two events, and P(A)>0, then the conditional probability rule applies,
The following properties apply to conditional probabilities as well. For the sample space S and any fixed event A, with P(A)>0,
P(B|A)≥0, for all events B⊂S
P(S|A)=1
If B1, B2, …, Bk are disjoint events, then P(B1 ∪ B2 ∪…∪ Bk |A)=P(B1 |A)+P(B2 |A)+…+P(Bk |A)
P(Bc|A)=1–P(B|A), where Bc is the complement of the event B
For any events A,B, and C, if B⊂C , then P(B|A)≤P(C|A)
P(B ∪ C|A)=P(B|A)+P(C|A)–P(B∩C|A) , where B and C are not disjoint.
The multiplication rule is derived from the conditional probability rule as follows:
P(A∩B)=P(A and B)=P(A)⋅P(B|A)
12
12
/docProps/thumbnail.jpeg