ISE 562; Dr. Smith
Probability and Statistics Review Part II
Decision Theory
ISE 562; Dr. Smith
Copyright By PowCoder代写 加微信 powcoder
Events may be discrete (fixed number of outcomes) or continuous (infinite outcomes)
Discrete events:
Number of truck arrivals to a receiving dock
Number of cases opened on “Deal or No Deal” out of 26 without opening the $1M case
Number of failures in a production lot of 1000 units
Continuous events:
Mean time to failure of a component
Percent contamination level in a 100 cc sample of river
Weight of quarter pound burger patties
8/21/2022 2
ISE 562; Dr. Smith
Probabilities typically represented with functions that map events to a numerical probability
Must still adhere to rules of probability
Ex: tossing 1 die (6 outcomes); event= even or odd
– P(even)=3/6=0.50
P(even) P(odd)
ISE 562; Dr. Smith
P(even) P(odd)
if x2,4,6 if x1,3,5
The function would be:
6 P(event)3
8/21/2022
ISE 562; Dr. Smith
P(even) P(odd)
1,3,5 2,4,6 odd even
The graph would be:
if x2,4,6 if x1,3,5
P(event)6 0
ISE 562; Dr. example
Number of arrivals to a shop Poisson distributed with average of 8 customers per hour
ISE 562; Dr. Smith
0 P(0) P(1)
The function would be:
𝜆𝑡𝑒 𝑃 𝑛𝑜. 𝑎𝑟𝑟𝑖𝑣𝑎𝑙𝑠 𝑥 𝑥!
= (for example) !
𝑥 0,1,2…,𝑛
ISE 562; Dr. Smith
0 P(0) P(1)
The graph format would be:
𝑃 𝑛𝑜. 𝑎𝑟𝑟𝑖𝑣𝑎𝑙𝑠 𝑥 =
(for example)
ISE 562; Dr. probability:
“Event A depends on event B”
P(A|B) = joint probability of both A and B divided by the marginal probability of B or
= P(A and B)/P(B)
Is dependence the same as not mutually exclusive? (or independence the same as mutually exclusive?)
8/21/2022 9
ISE 562; Dr. independence and dependence vs. mutually exclusive
• Given 2 events A and B, P(A|B)=P(A and B)/P(B)
• If mutually exclusive, P(A and B)=0 so P(A|B)=0 (A has nothing to do with B)
• Independence: when occurrence of one event has no bearing on the other event; P(A|B)=P(A) and P(A and B)=P(A)P(B)
8/21/2022 10
ISE 562; Dr. note: if 2 events mutually exclusive they cannot be independent; ie if A, B mutually exclusive, if A occurs, B cannot occur so
P(A|B)=0 P(A)
• Example: we choose a car at random; let A=4
cylinder engine and B=6 cylinder engine
• P(A) has some value > 0
• A and B are mutually exclusive
• But P(A|B)=0 P(A) (which is >0) so not independent. Saying car has 4 cylinders means it does not have 6.
8/21/2022 11
ISE 562; Dr. :
Random variable. A variable used to represent the events associated with a sample space
Expected value. The mean (average), weighted value of a random variable based on its probability distribution
Variance. The weighted sum of differences of all points in the sample space from the mean. The standard deviation=the square
root of the variance.
8/21/2022 12
ISE 562; Dr. probabilities by relative frequency
P(E)=nE /NS
Ex: Compute the probability of one boy in a family of 3 children
Ns=8 outcomes in sample space bbb bbg bgb bgg
gbb gbg ggb ggg
P(1 b)= 3/8
8/21/2022 13
ISE 562; Dr. probabilities-multiplication rule
Given k sets of nk items, the number of possible cases=n1 xn2 x…nk
Ex: If we want to buy a computer with 3 available monitor types, m1, m2, m3; 4 CPU speeds, c1, c2, c3, c4; and 3 hard drive capacities, h1, h2, h3; what is the probability we randomly select the combination (m2, c3, h1)?
Number of outcomes = 3x4x3=36
Prob (m2, c3, h1)=1/36
8/21/2022 14
ISE 562; Dr. probabilities-multiplication rule
Could have used this on previous example to compute number in sample space:
Three sets each with 2 possibilities (b,g)
Child 1 Child 2 Child 3 bbb ggg
Number of outcomes = 2x2x2=8
8/21/2022 15
ISE 562; Dr. : suppose the random variable for the number of proposals funded can be represented by the pdf
E ( X ) x P( X ) 1 (0.35) 2 (0.40) 4 (0.25)
i1 2.15
3 V(X)(xE(X))2P(X)(12.15)2 (0.35)
(2 2.15)2 (0.40) (4 2.15)2 (0.25) 1.3275 8/21/2022 16
Expected value and variance
P(selection)
ISE 562; Dr. binomial distribution—a discrete pdf
• Derived from the Bernoulli distribution
• Only 2 outcomes possible (success or failure)
• P(success) the same from trial to trial
• There are N trials (fixed)
• The N trials are independent
• Random variable is number of successes, r, in n trials
8/21/2022 17
ISE 562; Dr. binomial distribution
P(r) n! pr(1 p)nr r!(n r)!
p=probability of success; fixed over range of trials n=number of independent trials
r=number of successes in n trials
8/21/2022 18
ISE 562; Dr. binomial distribution
P(r) n! pr(1 p)nr r!(n r)!
Example: In a family of three children, what is the probability of 1 boy? Assuming the probability of a boy = 0.50 we want to know P(r=1) with n=3 trials:
P(1)=3!/(1!)(2!) (.5)1 (.5)2 = 0.375
(=3/8 as shown on slide 11)
8/21/2022 19
ISE 562; Dr. binomial distribution
P(r) n! pr(1 p)nr r!(n r)!
Example: suppose the probability is 0.02 that a certain lab test will fail to detect a disease. What is the probability that among 20 such tests, 2 will fail?
P(2)=20!/(2!)(18!) (.02)2 (.98)18 = 0.0528
ISE 562; Dr. application
Biomorphic explorers for Mars exploration
ISE 562; Dr. Theorem
ISE 562; Dr. Theorem
English theologian and mathematician has greatly contributed to the field of probability and statistics. His ideas have created much controversy and debate among statisticians over the years.
was born in 1702 in London, England. There appears to be no exact records of his birth date. Bayes’s father was one of the first six Nonconformist ministers to be ordained in England. Bayes’s parents had their son privately educated. There is no information about the tutors Bayes worked with. However, there has been speculation that he was taught by de Moivre, who was doing private tuition in London during this time. Bayes went on to be ordained, like his father, a Nonconformist minister. He first assisted his father in Holborn, England. In the late 1720’s, Bayes took the position of minister at the Presbyterian Chapel in Tunbridge Wells, which is 35 miles southeast of London. Bayes continued his work as a minister up until 1752. He retired at this time, but continued to live in Tunbridge Wells until his death on April 17, 1761. His tomb is located in Bunhill Fields Cemetery in London. Throughout his life, Bayes was also very interested in he field of mathematics, more specifically, the area of probability and statistics. Bayes is believed to be the first to use probability inductively. He also established a mathematical basis for probability inference. Probability inference is the means of calculating, from the frequency with which an event has occurred in prior trials, the probability that this event will occur in the future. According to this Bayesian view, all quantities are one of two kinds: known and unknown to the person making he inference. Known quantities are obviously defined by their known values. Unknown quantities are described by a joint probability distribution. Bayesian
inference is seen not as a branch of statistics, but instead as a new way of looking at the complete view of statistics. Bayes wrote a number of papers that discussed his work. However, the only ones known to have been published while he was still living are: Divine Providence and Government Is the Happiness of His Creatures (1731) and An Introduction to the Doctrine of Fluxions, and a Defense of the Analyst (1736). The latter paper is an attack on Bishop Berkeley for his attack on the logical foundations of Newton’s Calculus. Even though Bayes was not highly recognized for his mathematical work during his life, he was elected a
Fellow of the Royal Society in 1742.
8/21/2022 23
ISE 562; Dr. Theorem
Perhaps Bayes’s most well known paper is his Essay Towards Solving a Problem in the Doctrine of Chances. This paper was published in the Philosophical Transactions of the Royal Society of London in 1764. This paper described Bayes’s statistical technique known as Bayesian estimation. This technique based the probability of an event that has to happen in a given circumstance on a prior estimate of its probability under these circumstances. This paper was sent to the Royal Society by Bayes’s friend . Price had found it among Bayes’s papers after he died. Bayes’s findings were accepted by Laplace in a 1781 memoir. They were later rediscovered by Condorcet, and remained unchallenged. Debate did not arise until Boole discovered Bayes’s work. In his composition the Laws of Thought, Boole questioned the Bayesian techniques.
Boole’s questions began a controversy over Bayes’s conclusions that still continues today. In the 19th century, Laplace, Gauss, and others took a great deal of interest in this debate. However, in the early 20th century, this work was ignored or opposed by most statisticians. Outside the area of statistics, Bayes continued to have support from certain prominent figures. Both Harold Jeffreys, a physicist, and , an econometrician, continued to argue on behalf of Bayesian ideas. The efforts of these men received help from the field of statistics beginning around 1950. Many statistical researchers, such as L. J. Savage, Buno do Finetti, , and , began advocating Bayesian methods as a solution for specific deficiencies in the standard system.
However, some researchers still argue that concentrating on inference for model parameters is misguided and uses unobservable, theoretical quantities. Due to this skepticism, some are reluctant to fully support the Bayesian approach and philosophy.
8/21/2022 24
ISE 562; Dr. Theorem
Bayes is buried in Bunhill Fields in the heart of the City of London. The cemetery was used for the burial of nonconformists in the 18th century, but is now a public park maintained by the Corporation of London. Also buried in Bunhill Fields is Bayes’s friend , a pioneer of insurance, who presented Bayes’s famous paper on probability to the Royal Society in 1763, two years after Bayes’s death. Across the City Road from Bunhill Fields is Wesley’s Chapel, which has been restored in recent years. The pictures below show Bayes’s tomb with a variety of inscriptions. It was a family vault in which are laid several members of the Bayes, Cotton and West families. On the top of the tomb is an inscription saying how the tomb was restored in 1969, through public subscription from statisticians worldwide.” These photos were taken by Professor Tony O’Hagan of Sheffield University who also also provided the information about the burial place. Bunhill (probably a corruption of “bonehill”) Fields operated as a burial ground for “Dissenters” from 1665 to 1853, during which time around 123,000 burials took place. There are many notable graves, including , , , many of the Cromwell family and (mother of , the founder of Methodism, who is buried across the City Road where his chapel still stands).
8/21/2022 25
ISE 562; Dr. Theorem—derivation
First some notation:
• P(eventsAandBoccurring)=P(AB)=P(A,B) • P(AandnotB)=P(A,not_B)
• P(AgivenB)=P(A|B)
8/21/2022 26
ISE 562; Dr. Theorem—derivation
Start with 2 probability rules:
i) P(Y|X) = P(Y,X)/P(X) = P(X,Y)/P(X)
ii) P(Y)=P(Y,X)+P(Y,not_X)=P(X,Y)+P(not_X,Y)
• InadecisionproblemwearegivenXandP(X);YandP(Y|X);
• DesireP(X|Y)(posterior)asafunctionofwhatisknown
• Substituting(ii)into(i)weget:
iii) P(X|Y)=P(X,Y)/[P(X,Y)+P(not_X ,Y)]; if we don’t know the
denominator terms we can calculate them from (i):
iv) P(X,Y)=P(Y|X) P(X); and P(not_X,Y) = P(Y|not_X) P(not_X) Substituting (iv) back into denominator of (iii) yielding Bayes
P(X|Y)=P(Y|X) P(X)/ [P(Y|X)P(X)+P(Y|not_X)P(not_X)]
8/21/2022 27
ISE 562; Dr. Proof of rule ii
i) P(Y)=P(Y,X)+P(Y,not_X)=P(X,Y)+P(not_X,Y)
8/21/2022 28
ISE 562; Dr. get the discrete form, we add an index, for the state of X:
Prior distribution
(original state of knowledge about Xr)
Likelihood distribution from new information (sample)
P(Xr |Y)P(Xr)P(Y|Xr) P(Xi )P(Y | Xi )
Normalizing constant
ISE 562; Dr. Theorem—example
Three facilities supply plastic containers to a manufacturer. All
are made to same specification. However, after months of testing, records indicate the following:
Supplying facility
Fraction supplied by
Fraction defective
ISE 562; Dr. Theorem—example
The director of manufacturing randomly selects a unit, has it
tested, and finds it to be defective. Let Y=event that item is defective and Xi be the event the item came from facility i=1, 2, 3. Use Bayes rule to determine the probability the defective came from facility 1, 2, or 3 given it was defective. That is, P(X1|Y), P(X2|Y), and P(X3|Y).
𝑃𝑋|𝑌 𝑃𝑌|𝑋𝑃𝑋 ∑ 𝑃𝑌|𝑋𝑃𝑋
ISE 562; Dr. Theorem—example
P(X1 |Y) P(Y |X1)P(X1)
P(X1)P(Y |X1)P(X2)P(Y |X2)P(X3)P(Y |X3)
(0.02)(0.15) 0.24 (0.02)(0.15) (0.01)(0.80) (0.03)(0.05)
P(X3 |Y) 8/21/2022
(0.01)(0.80)
(0.02)(0.15) (0.01)(0.80) (0.03)(0.05)
(0.03)(0.05)
(0.02)(0.15) (0.01)(0.80) (0.03)(0.05)
ISE 562; Dr. (Xr) P(Y|Xr) P(Xr|Y)
New estimate of P(1,2, or 3 the source of defective)
Supplier, Xr
% supplied by company Xr
Fraction defective
Initially, company 3 looked the worst (.03 or 3 times worse than company 2). After sampling, company 3 is the best and company 2 is the worst (>5 times worse than company 3!) Why? Because company 2 accounts for 80% of the product volume—information not available in P(Xr)
8/21/2022 33
ISE 562; Dr. Example
• The following categories are from an online retail shopping site you are opening. You want to track the customer’s interest in each category so you can advertise similar items when they make future visits.
• You make the assumption that the number of searches = the level of interest in a category so you can estimate the probability the customer will be interested in that category in the future.
• Let x= buys item in category x and let y = random variable representing “interested in category y”
• Let x and y index over Arts&Crafts, Grocery, Sports, Entertainment
8/21/2022 34
ISE 562; Dr. Smith
• Next we need the Prob(buy x) but we have no data so assume a diffuse prior (uniform pdf)
Category, x P(buy x)
Arts & Crafts
Entertainment
ISE 562; Dr. Smith
• We need the likelihood of interest in x given a buy x, P(interest in x | buy x) but don’t have any buy data yet so assume independence (P(interest in x|buy x)= P(interest in x). We use the counts of searches in each category:
Category, x # searches
P(interest in x)
Arts & Crafts
Entertainment
ISE 562; Dr. Smith
• Now compute the posterior pdf of P(buy x | interest in y)
using Bayes theorem
𝑃𝑋|𝑌 𝑃𝑌|𝑋𝑃𝑋 ∑ 𝑃𝑌|𝑋𝑃𝑋
Category Prior(buy in x) Likelihood(interest in y|buy in Like x Prior Posterior(buy x | interest in y)
Arts & Crafts
Entertainment
Sum = 0.25 1
• Note the posterior is the same as the likelihoods (the only information available—but there was no information in the uniform diffuse prior!).
8/21/2022 37
ISE 562; Dr. Smith
• Now the customer buys 1 item from Arts & Crafts and 2 items from Grocery and 1 from Entertainment. This is new information to update the prior.
• We have data on buys so compute Likelihood of interest given buys in x as #buys in category x #searches in category x. (note: if # buys or # searches = zero, likelihood = 0).
Bought in Category, y # searches # buys in x
Likelihood(interest in y|buy in x)
Arts & Crafts
Entertainment
ISE 562; Dr. Smith
• Using the likelihoods and prior distributions above, compute the posterior probability for interest in x given purchases y so items in x can be prioritized for presentation on the web pages.
The 2 buys in Grocery drove the P(buy | interest) probability from 0.375 to 0.50
Category Prior(buy in x) Likelihood(interest|buy in x) Like x Prior Posterior(buy x|interest in in x)
Arts & Crafts 0.2500
Grocery 0.3750
Sports 0.3125
Entertainment 0.0625
0.250 0.0625 0.2500
0.333 0.1250 0.5000
0.000 0.0000 0.0000
1.000 0.0625 0.2500
ISE 562; Dr. Smith
• Now suppose the user returns and conducts 1 new search with another purchase in Entertainment.
Bought in Category, y # searches # buys Likelihood(buy y | interest in x)
• Now update the posterior with these new data:
8/21/2022 40
Arts & Crafts
Entertainment
ISE 562; Dr. Smith
• Now update the posterior with these new data:
Arts & Crafts
Grocery Sports
Entertainment
The second buy in Entertainment drove the
P(buy | interest) probability from 0.25 to 0.52
Posterior(interest in x | buy
0.3478 0.0000
Prior(buy in x) Likelihood(interest|buy in x)
Like x Prior 0.2500 0.250 0.0625
0.5000 0.333 0.1667 0.0000 0.000 0.0000
0.2500 1.000 0.2500 Sum = 0.4792
• The focus moves away from Grocery category to Entertainment with same number of buys but with more intense interest (2 buys in 2 searches vs. Grocery with 2 buys and 6 searches implying searches in Entertainment more productive).
8/21/2022 41
ISE 562; Dr. normal distribution—a continuous pdf
Representative of many natural processes
Standardized in tabular form
Probability defined for intervals, not points
– 8/21/2022
Requires mean and variance to calculate probability
Lookup values found using z=(x-)/ =mean; =standard deviation
ISE 562; Dr. normal distribution—a continuous pdf
• Example: The cost per patient for a particular medical procedure was determined from records to be normally distributed with mean $200 and standard deviation +/- $50.
• For a random selection of records, what is the probability the cost is less than $150?
• What is the probability the cost is between
$180 and $220 per procedure?
8/21/2022 43
ISE 562; Dr. normal distribution—a continuous pdf • P(X<150)=PN(z< (150-200)/50 )
=P(z< -1)=0.16 (from table) • P(180