DSME5110F: Statistical Analysis
Lecture 6 Probability Distributions,
Point Estimation and Sampling Distribution
Copyright By PowCoder代写 加微信 powcoder
Random Variable
• Formally, a random variable (denoted as 𝑋𝑋) is a real-valued function which assigns numerical values to the outcomes of a random trial.
• Loosely speaking, a random variable is a variable whose value is determined by the outcome of a random experiment.
–e.g., flipping coins, rolling a die, taking random samples, etc.
• There are two types of random variables: – Discrete Random Variable
– Continuous random variable
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Discrete Random Variable
• A discrete random variable is a variable whose values are discrete and countable.
• A discrete random variable can have a finite or infinite number of possible values.
• Examples:
– Number of heads obtained when flipping 10 coins.
– Number of tosses until a coin lands on heads.
– Number of women in a sample of 100 randomly selected people.
– Number of people entering a grocery store between 2:00 p.m. to 3:00 p.m.
Probability Mass Function (PMF)
• The probability mass function for a discrete random variable assigns probability to each of the values that the discrete random variable can assume.
• The probability mass function for a discrete random variable must satisfy the following rules:
1. ∑ P 𝑥𝑥 =1where𝑥𝑥 isapossiblevalue,and i𝑖𝑖𝑖𝑖
2. 0≤P 𝑥𝑥𝑖𝑖 ≤1forevery𝑥𝑥𝑖𝑖.
Example 6.1 • Let 𝑋𝑋 be the sum of rolling two dice.
• Then the probability mass function of 𝑋𝑋 is given in the following table. Note that
– Alltheprobabilitiesarelargerthan0andsmallerthan1
– the sum of all the probabilities in the second column of the table is equal to 1.
2 3 4 5 6 7 8 9 10 11 12
Expectation of
a Discrete Random Variable
• Expectation: the expected value a discrete random variable 𝑋𝑋, denoted as E(𝑋𝑋) or 𝜇𝜇, is
defined asE: 𝑋𝑋 = 𝜇𝜇 = �𝑖𝑖 𝑥𝑥𝑖𝑖P(𝑥𝑥𝑖𝑖)
• E(𝑋𝑋) is simply the “weighted average of 𝑋𝑋” with the probability associated with each 𝑥𝑥𝑖𝑖 being the weight.
Variance and Standard Deviation of
a Discrete Random Variable
• Variance: the variance of a discrete random variable, denoted as 𝑉𝑉(𝑋𝑋) or 𝜎𝜎2, is defined as:
𝑉𝑉𝑋𝑋 =𝜎𝜎2=E(𝑋𝑋−𝜇𝜇)2=�𝑖𝑖 (𝑥𝑥𝑖𝑖−𝜇𝜇)2P𝑥𝑥𝑖𝑖
– Variance measures “dispersion”
• Standard deviation: the standard deviation of 𝑋𝑋,
denoted as 𝜎𝜎, is simply the square root of 𝑉𝑉(𝑋𝑋). 𝜎𝜎= 𝜎𝜎2= 𝑉𝑉(𝑋𝑋)
Rules about Expectations and Variances of Random Variables
• For constants 𝑎𝑎 (≠ 0), 𝑏𝑏 and random variable 𝑋𝑋, then
–E𝑎𝑎𝑋𝑋+𝑏𝑏 =𝑎𝑎E𝑋𝑋 +𝑏𝑏
–𝑉𝑉 𝑎𝑎𝑋𝑋+𝑏𝑏 =𝑎𝑎2𝑉𝑉(𝑋𝑋)
• For random variables 𝑋𝑋 and 𝑌𝑌,
–E𝑋𝑋+𝑌𝑌 =E𝑋𝑋 +E𝑌𝑌
–𝑉𝑉 𝑋𝑋+𝑌𝑌 =𝑉𝑉 𝑋𝑋 +𝑉𝑉 𝑌𝑌 +2𝐶𝐶𝐶𝐶𝐶𝐶(𝑋𝑋,𝑌𝑌)
– If 𝑋𝑋 and 𝑌𝑌 are independent, then 𝐶𝐶𝐶𝐶𝐶𝐶 𝑋𝑋, 𝑌𝑌 = 0,
and𝑉𝑉𝑋𝑋+𝑌𝑌 =𝑉𝑉𝑋𝑋 +𝑉𝑉𝑌𝑌.
Example 6.1 (Continued)
• Let 𝑋𝑋 be the “sum of rolling two dice”. – E(𝑋𝑋) = 7
– 𝑉𝑉(𝑋𝑋) = 5.833
• Suppose that you are invited to play a gamble in which someone
will roll two dice. If you decide to play, – youneedtopayanentryfeeof$10and – wins2dollarsforeverydotshowing.
• Let𝑊𝑊bethenetwinningsand𝑋𝑋bethesumoftwodice,then𝑊𝑊= − 10 + 2𝑋𝑋.
• What are the expected value and variance of winnings of this game?
– E(𝑊𝑊)=E(−10+2𝑋𝑋)=−10+2E(𝑋𝑋)=−10+2(7)=4 – 𝑉𝑉 𝑊𝑊 =𝑉𝑉 −10+2𝑋𝑋 =22 ∗𝑉𝑉(𝑋𝑋)=4(5.833)=23.33
Example 6.2: Weekly Revenue
• Suppose the revenues of a restaurant in different days of a week are independently distributed. The expected value and standard deviation of revenue (in US $1,000) for each day of the week are given in the table below.
Expected Value
Stand Deviation
• What are the expected value and variance of weekly revenue?
• Using the rules, we find that
– E(weekly revenue) = Sum of daily revenues = 60.5 – Var(weekly revenue) = Sum of daily variance = 574
– SD(weekly revenue) = Var(weekly revenue)= 23.9583
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Binomial Distribution
• A random variable follows a binomial distribution if its value is about “number of successes” resulted from an experiment that involves
1. a sequence of independent and identical trials, in
each trial can only result in one of the two possible outcomes, success and failure, with probability 𝑝𝑝 and 1– 𝑝𝑝, respectively.
that 𝑝𝑝 remains the same from trial to trial.
• The fact that all trials are independent and identical implies
Which of the followings are Binomial Random Variables?
• Number of correct answers obtained when guessing the correct answers of 20 true-and-false questions randomly.
• Number of correct answers obtained when guessing randomly the correct answers of 20 multiple-choice questions:
– Setting 1: all with 4 possible answers;
– Setting 2: some with 4 but some with 5 possible answers.
Binomial Distribution
• The formula to calculate the probability of getting 𝑟𝑟 (𝑟𝑟 = 0, 1, … , 𝑛𝑛) successes from a sequence of 𝑛𝑛 independent and identical trials when each trial can only result in “success” with probability 𝑝𝑝 and “failure” with probability 1– 𝑝𝑝:
P(𝑋𝑋=𝑟𝑟)= 𝑛𝑛 𝑝𝑝𝑟𝑟 1−𝑝𝑝 𝑛𝑛−𝑟𝑟 = 𝑛𝑛! 𝑝𝑝𝑟𝑟 1−𝑝𝑝 𝑛𝑛−𝑟𝑟 𝑟𝑟 𝑟𝑟! 𝑛𝑛 − 𝑟𝑟 !
R Functions for Binomial Distribution
P(𝑋𝑋 = 𝑟𝑟)
dbinom(𝑟𝑟, 𝑛𝑛, 𝑝𝑝) pbinom(𝑟𝑟, 𝑛𝑛, 𝑝𝑝) pbinom(𝑟𝑟−1,𝑛𝑛,𝑝𝑝) 1-pbinom(𝑟𝑟−1,𝑛𝑛,𝑝𝑝) 1-pbinom(𝑟𝑟,𝑛𝑛,𝑝𝑝)
To Find This Binomial Distribution
P(𝑋𝑋 ≤ 𝑟𝑟)
P 𝑋𝑋<𝑟𝑟 =P(𝑋𝑋≤𝑟𝑟−1) P 𝑋𝑋≥𝑟𝑟 =1−P(𝑋𝑋<𝑟𝑟) P 𝑋𝑋 > 𝑟𝑟 = 1−P(𝑋𝑋 ≤ 𝑟𝑟)
Given 𝑦𝑦 = P(𝑋𝑋 ≤ 𝑟𝑟), find 𝑟𝑟: qbinom(𝑦𝑦, 𝑛𝑛, 𝑝𝑝) Generate 𝑘𝑘 random variables: rbinom(𝑘𝑘, 𝑛𝑛, 𝑝𝑝)
Example 6.3: Lee-Sanity
• It’s a basketball game, CUMBA Faculties vs. CUMBA Students. Time is running out and Faculties is still behind by 1 point. Lee (a faculty member) made the last attempt from 3-pointer zone and …… Boom, he missed it, but wait, … he was fouled. The clock shows that there is only 0.1 second left in the game.
• Now, Lee is on the foul line, having the opportunity to make 3 shoots. Based on his record, he can make each shot independently with probability 0.75.
1. What is the probability that Faculties will lose the game when the regular time ends?
2. What is the probability that the game will go into overtime?
3. What is the probability that Faculties will win when the regular time ends?
Example 6.4: Pass the Exam!
• In an exam consisting of 10 true-and-false questions, if a student guesses the correct answer of each question randomly, what is the probability that he will pass the exam if he needs a minimum of 6 correct answers to pass?
– R command: 1- pbinom(5, 10, 0.5), we find that P(X ≥ 6) = 0.3769531.
• How would the probability change if the exam consists of 10 multiple-choice questions, each with 4 choices?
– In this case, the probability of guessing each question correctly is p = 0.25 (1 out of 4). All other parameter values remain the same.
– R command: 1- pbinom(5, 10, 0.25), we find that P(r ≥ 6) = 0. 01972771.
The Expectation and Variance of
• The expectation of a binomial distribution is 𝑛𝑛𝑝𝑝 and the variance is 𝑛𝑛𝑝𝑝(1 – 𝑝𝑝).
Binomial Distributions
• If 𝑋𝑋, which represents “number of successes”, follows a binomial distribution, then 𝑃𝑃� = 𝑋𝑋/𝑛𝑛, which represents “proportion of successes”, has
–Expectation:E 𝑃𝑃� =𝐸𝐸 𝑋𝑋 =𝐸𝐸 𝑋𝑋 =𝑛𝑛𝑛𝑛 =𝑝𝑝 �𝑛𝑛𝑛𝑛𝑛𝑛
–Variance:𝑉𝑉 𝑃𝑃 =𝑉𝑉 𝑋𝑋 = 1 𝑉𝑉 𝑋𝑋 =𝑛𝑛(1−𝑛𝑛) 𝑛𝑛𝑛𝑛2 𝑛𝑛
– These have important applications in inference about the population proportion.
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Continuous Random Variable
• A continuous random variable is a variable that can assume any value in a continuous interval.
• Examples:
– The amount of time that a customer waits in a queue
– The total distance that a car travels in an hour.
– The amount of money that a foreign visitor spends in a day.
• In general, random variables that involve time, distance, volume, or money are often considered as continuous random variables.
before being served.
Probability Density Function (PDF) (Just For Your Reference)
• The probability density function defines the probability distribution for a continuous random variable.
• The probability density function (PDF), f(x), for a continuous random variable must satisfy the following rules:
– ∫∞ 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑥𝑥 = 1 where 𝑥𝑥 assumes all possible values, and −∞
– 𝑓𝑓(𝑥𝑥) ≥ 0 for every value of 𝑥𝑥.
Calculating the Probability (Just For Your Reference)
• For a continuous random variable, although the probability that the variable will assume a specific value is essentially zero, the probability that its value will fall in an interval is not zero.
• Hence, to calculate the probability that a continuous random variable will take a value between two points (say 𝑎𝑎 and 𝑏𝑏), we need to calculate “the area under the probability density function and between the two points”, as shown in the graph below.
Expectation, Variance, and Standard Deviation (Just For Your Reference)
• The expectation of a continuous random variable 𝑋𝑋, denoted as E(𝑋𝑋) or 𝜇𝜇, is defined as:
– E 𝑋𝑋 =𝜇𝜇=∫∞ 𝑥𝑥𝑓𝑓 𝑥𝑥 𝑑𝑑𝑥𝑥, where 𝑓𝑓 𝑥𝑥 is the probability −∞
density function of 𝑋𝑋.
• The variance of a continuous random variable,
denoted as 𝑉𝑉(𝑋𝑋) or 𝜎𝜎2, is defined as:
–𝑉𝑉𝑋𝑋 =𝜎𝜎2=E(𝑋𝑋−𝜇𝜇)2=∫∞(𝑥𝑥−𝜇𝜇)2𝑓𝑓𝑥𝑥𝑑𝑑𝑥𝑥, where
𝑓𝑓 𝑥𝑥 is the probability density function of 𝑋𝑋.
• The standard deviation of 𝑋𝑋, denoted as 𝜎𝜎, is simply the square root of 𝑉𝑉(𝑋𝑋).
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Normal Distribution
• Normal distribution is a continuous distribution with its probability density function 𝑓𝑓 𝑥𝑥 being bell-shaped and defined as:
–𝑓𝑓𝑥𝑥 = 1 𝑒𝑒−𝑥𝑥−𝜇𝜇2,where 2𝜋𝜋𝜋𝜋 2𝜎𝜎2
– e = the exponential constant = 2.71828… – π = 3.14159…
– 𝜇𝜇 = population mean
– 𝜎𝜎 = population standard deviation
• Itcanassumeanyvaluefrom−∞to+∞.
Why is Normal Distribution Important?
• Many real-life data follow approximately normal distribution
– e.g., weights, heights, some standard exam scores, etc.
• It can be used to approximate various discrete probability
distributions
– e.g., binomial distribution.
• It provides the basis for classical statistical inference because of its role in the central limit theorem.
– Some sample statistics, including sample means and sample proportions, follow normal distribution approximately, especially when the sample size is sufficiently large.
Family of Normal Distribution
• Each member in the family is characterized by 𝜇𝜇 and 𝜎𝜎.
• Normal distribution is actually a family of distributions.
• If 𝜇𝜇=0 and 𝜎𝜎=1, it is called the standard Normal distribution.
• The Excel file Two Normals.xlsx is designed to show how change in 𝜇𝜇 and 𝜎𝜎 affect the central location and dispersion of a normal distribution.
R Functions for Normal Distribution
𝑓𝑓(𝑥𝑥) dnorm(𝑥𝑥, 𝜇𝜇, 𝜎𝜎) P 𝑋𝑋<𝑥𝑥 =P(𝑋𝑋≤𝑥𝑥) pnorm(𝑥𝑥,𝜇𝜇,𝜎𝜎)
To Find This Binomial Distribution R Command
P 𝑋𝑋>𝑥𝑥 =P(𝑋𝑋≥𝑥𝑥)=1−P(𝑋𝑋<𝑥𝑥)=1−P(𝑋𝑋≤𝑥𝑥) 1-pnorm(𝑥𝑥,𝜇𝜇,𝜎𝜎)
Given 𝑝𝑝 = P(𝑋𝑋 ≤ 𝑥𝑥), find 𝑥𝑥: qnorm(𝑝𝑝, 𝜇𝜇, 𝜎𝜎)
Generate 𝑛𝑛 random variables: rnorm(𝑛𝑛, 𝜇𝜇, 𝜎𝜎)
Note that the default values of mean and standard deviation for pnorm(), as well as all other *norm() functions, are 0 and 1, respectively. So, for other normal distributions with different mean and standard deviation, you need to specify it as: pnorm(x, mean, stdev)
Example 6.5: Empirical Rule
• In Lecture 4, we mentioned an empirical rule about normally distributed data, which states that approximately
– 68% of the data values lie within ± 1 standard deviation from the mean,
– 95% of the data values lie within ± 2 standard deviations from the mean, and
– 99.7% of the data values lie within ± 3 standard deviations from the mean.
• We can verify the above rule with R formula below: > pnorm(1)–pnorm(-1)
> pnorm(2)–pnorm(-2)
> pnorm(3)–pnorm(-3)
Example 6.6: IQ Scores
• IQ scores are normally distributed with mean μ = 100 and standard deviation σ = 15. What is the probability someone has an IQ score of 95 or less? That is, what is P(x ≤ 95)?
• Using R command, pnorm(95, 100, 15), the probability is found to be 0.3694.
• Note that the IQ score of 95 has a corresponding Z value of -1/3:
– 𝑍𝑍= =− =−0.3333.
• Therefore, we can also find the answer with the R command, pnorm(-0.3333), without the need to specify the mean and the standard deviation.
• In fact, all normal probabilities, P(x ≤ a), can be computed in either way:
– pnorm(a, mean, stdev)
– pnorm(z), where z = (a-mean)/stdev
Example 6.6: IQ Scores (Continued)
• A high-IQ society at a large public university requires that anyone seeking to join demonstrate that they have an IQ score placing them in the highest 1% of the population. What IQ score should an aspiring member have in order to be successful?
• For this question, we need to find the 99th percentile of IQ score. This can be done with the following R command:
> qnorm(0.99, 100, 15)
– The value is found to be 134.8952.
Example 6.7: ABC Stock
• Assume the annual mean return on ABC stock is around 15% and the annual standard deviation is around 25%. Assume the annual and daily returns on ABC stock are normally distributed.
a. What is the probability that ABC will lose money during a year?
b. There is a 5% chance that ABC will earn a return of at least what value during a year?
c. There is a 10% chance that ABC will earn a return of less than or equal to what value during a year?
d. What is the probability that ABC will earn at least 35% during a year? 33
Binomial and Normal Distributions
• When 𝑝𝑝, the probability of success, is close the 0.5, the binomial distribution is very close to the normal
distribution with 𝜇𝜇 = 𝑛𝑛𝑝𝑝 and 𝜎𝜎 = 𝑛𝑛𝑝𝑝(1 − 𝑝𝑝).
– Even when 𝑝𝑝 is not close to 0.5, but 𝑛𝑛 is sufficiently large so that
𝑛𝑛𝑝𝑝(1 − 𝑝𝑝) is large, the binomial distribution will also be close to that normal distribution with 𝜇𝜇 = 𝑛𝑛𝑝𝑝 and 𝜎𝜎 = 𝑛𝑛𝑝𝑝(1 − 𝑝𝑝).
– Then, in such cases, the binomial proportion 𝑃𝑃� = 𝑋𝑋/𝑛𝑛 is close to the normal distribution with 𝜇𝜇𝑃𝑃� = 𝑝𝑝 and 𝜎𝜎𝑃𝑃� = 𝑝𝑝(1 − 𝑝𝑝)/𝑛𝑛.
• For comparison between Binomial and Normal distributions, refer to the file: Binomial and Normal.xlsm
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Sampling Terminology
• A population consists of all the elements of interest in a study that have some quality (or qualities) in common
– All students enrolled in a class
– All potential voters in a presidential election
– Population parameters are the characteristics of interest in a study. They are constants (but usually unknown).
• A sample is a subset of the population, often randomly chosen and preferably representative of the population as a whole
– Randomly select 3 students
– Opinion polls conducted by various institutions such as Gallup and Harris polls.
– Sample statistics are the characteristics of interest derived from a sample rather than a population. They are random variables (the values vary from sample to sample.).
Sample Statistics vs. Population Parameters
• Recall that
– population parameters, such as 𝜇𝜇, 𝑝𝑝, and 𝜎𝜎, are constants whereas – sample statistics, such as 𝑥𝑥̅, 𝑝𝑝̅, and 𝑠𝑠, are random variables.
• In general, statistical inference involves using (observable) sample statistics (e.g., 𝑥𝑥̅, 𝑝𝑝̅, and 𝑠𝑠) to estimate unknown population parameters (e.g., 𝜇𝜇, 𝑝𝑝, and 𝜎𝜎).
• Since sample statistics are variables, it is important to know how their possible values are distributed.
• Knowing so would allow us to have better idea about how accurate on average we can be whenever we use a sample statistic to estimate a population parameter.
Population
Probability
Statistics
Point Estimator
• A sample statistic is also referred to as a point estimator because it is the numerical value that estimates a population parameter
– 𝑥𝑥̅ = point estimator of the population mean 𝜇𝜇
– 𝑠𝑠 = point estimator of the population standard
deviation 𝜎𝜎
– 𝑝𝑝̅ = point estimator of the population proportion 𝑝𝑝
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Distribution of Sample Mean
• Suppose that the population consists of the following members: {1, 2, 3, 4, 5}
– Note that, for this population, 𝜇𝜇 = 3, 𝜎𝜎2 = 2, and 𝜎𝜎 = 1.4142.
• What are the distributions of sample mean when sample of size 𝑛𝑛 is selected randomly with replacement from this population?
– When 𝑛𝑛 = 2? – When 𝑛𝑛 = 3? – When 𝑛𝑛 = 4? – When 𝑛𝑛 = 5?
• In each of the cases above, what would “the average estimation error” be roughly (approximately) if the sample mean is used to estimate the population mean?
Distribution of Sample Mean
• The distributions of sample mean for different
values of 𝑛𝑛 are given below
Major Characteristics of the Sample Mean 𝑥𝑥̅
• In the previous examples, we observed that
– Regardless of the sample sizes, the mean of the sample
of 𝑋𝑋� quite close to normal distribution, 𝑛𝑛 gets larger.
means is always equal to the population mean. i.e., 𝜇𝜇 =
of the sample size, i.e., 𝜎𝜎 = 𝜎𝜎/ 𝑛𝑛.
– The standard deviation of the sample mean = The
distribution
population standard deviation divided by the square root
• 𝜎𝜎 is more commonly referred to as “standard error of 𝑋𝑋�”. 𝑥𝑥̅
• As the sample size increases, the standard error of the sample means decreases; the estimation becomes more accurate on
especially
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Central Limit Theorem
enough sample size (𝑛𝑛 ≥ 30), the distribution of the sample mean 𝑥𝑥̅ will be
Theorem: Regardless of the distribution of the original population, for large
𝜇𝜇 = 𝜇𝜇) and the standard error equal to the population standard deviation
divided by square root of the sample size (i.e., 𝜎𝜎 = 𝜎𝜎/ 𝑛𝑛). 𝑥𝑥̅
approximately normal with the mean equal to the population mean (i.e.,
Example 6.8
• The height of male follows approximately normal distribution with an unknown mean, but a known standard deviation of 7.5 cm.
a) If a random sample of 100 people were selected and their mean height is used to estimate the population mean, what is the probability that the estimation error will be within ± 1.5 cm?
b) If a random sample of 5625 people were selected and their sample mean height is used to estimate the population mean, what is the probability that the estimation error will be within ± 0.2 cm?
Outline • Discrete Distribution
– Binomial Distribution
• Continuous Distribution
– Normal Distribution • Point Estimator
– Sample Mean
• Central Limit Theorem
– Sample Proportion
Distribution of Sample Proportion
• The sample proportion, denoted 𝑝𝑝̅, –E𝑝𝑝̅ =𝑝𝑝and
–V𝑝𝑝̅ =𝑝𝑝(1−𝑝𝑝)/𝑛𝑛.
Theorem: For large enough sample size, sample proportion 𝑝𝑝̅ follow approximately a normal distribution with mean (of the sample proportions) equal to the true population proportion (i.e., 𝜇𝜇𝑛𝑛̅ = 𝑝𝑝) and
standard error (of sample proportion) 𝜎𝜎 = 𝑛𝑛(1−𝑛𝑛). 𝑛𝑛̅ 𝑛𝑛
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com