Logistic Regression
Course Starts at 6:35 PM
1
Logistic Regression
π(x)= eα+β1𝑋1 +β2X2 +…+βpXp 1+eα+β1𝑋1 +β2X2 +…+βpXp
2
Logistic Regression
This is a Logistic Function – Hence Logistic Regression
3
Logistic Regression – Model Odds are multiplicative
• ln (odds) = α + β 1 X 1 + β2X 2 + … + βpX p • Odds=eα+β1𝑋1 +β2X2 +…+βpXp
• = (𝒆𝜶) (𝒆β 𝟏 𝑿 𝟏 )(…)(𝒆β 𝒑 𝑿 𝒑 )
• Or: Odds = constant * exp (constant * X 1
…
* exp (constant * X p
) )
4
Logistic Regression – Model Log (Odds) are additive
• ln (odds) = α + β 1 X 1 + β2X 2 + … + βpX p
• Or: ln (Odds) = constant + (constant * X 1 )
…
+ (constant * X p )
5
Logistic Regression – Model Flexibility
• X (independent) variables can be continuous or categorical
• Interactions can be incorporated
• Coefficients are estimated by maximum
likelihood
• Most computer programs implicitly use prior probabilities estimated from sample.
6
Logistic Regression – Model Generalized Linear Model (GLM)
• Logistic regression is an example of the GLM
• Define Y = outcome = 1 (event) or 0 (not)
• E (Y|X’s) = μ = P(1|X’s)
• Find a function g(μ), called the link function, such that:
g(μ) = linear function of the X’s
• This is called the GLM
• Here we take g(μ) = ln (odds) = logit function
7
Logistic Regression – Model Estimation
• Model is: g(μ) = α + β 1 X 1 + β2X 2 + … + βpX p
• Need to estimate: α , β 1 , β2, … βp
• UseaninterativeprocesscalledIterative Weighted Least Squares:
1. Start with initial estimates of parameters
2. Evaluate the score equations (derivative of log- likelihood = 0)
3. Solve the score equations and get new estimates of parameters
4. Repeat until convergence.
8
Logistic Regression – Model Example: Depression Data Set Adjusted Risk Ratio
• RR = P(Y=1|X=1)/P(Y=1|X=0)
• P(Y=1|X) = eLC / (1 + eLC ), LC = A + B1 X1 + …
• Example: Depression, X = sex = 1 if F, age = 30, income = 10 ($10K/year)
• Find adjusted RR for F vs. M
• LC = -0.676 – 0.021 Age – 0.037 Income + 0.929 Sex
9