COMP3308/3608 Artificial Intelligence
Weeks 6 Tutorial exercises Naïve Bayes. Evaluating Classifiers.
Exercise 1. Naïve Bayes (Homework)
Suppose you want to recognize good and bad items produced by your company. You are able to measure two properties of each item (P1 and P2) and express them with Boolean values. You randomly grab several items and test if they are good or bad, obtaining the following results:
Copyright By PowCoder代写 加微信 powcoder
COMP3308/3608 Artificial Intelligence, s1 2022
Y Y Y N N N N Y Y N N N
good bad good bad good good
to predict the class, good or bad, of the following new item: P1=N, P2=Y. If there
E is P1=N and P2=Y; E1 is P1=N, E2 is P2=Y
We need to compute P(good|E) and P(bad|E) and compare them.
P(good | E) = P(E1 | good)P(E2 | good)P(good) P(E)
Use Naïve Bayes
are ties, make a random choice.
P(good)=4/6=2/3 P(E1|good)=P(P1=N|good)=2/4=1/2 P(E2|good)=P(P2=Y|good)=1/4
P(bad)=2/6=1/3 P(E1|bad)=P(P1=N|bad)=1/2 P(E2|bad)=P(P2=Y|bad)=1/2
111 1 P(bad|E)=223= 12
112 1 P(good|E)=243= 12
The two probabilities are the same. To resolve the tie we randomly choose between the 2 classes =>
e.g. class good. Exercise 2. Naïve Bayes
Why is the Naïve Bayesian classification called “naïve”?
Answer: Naïve Bayes assumes that the values of the attributes are independent of each other and that all attributes are equally important. These assumptions are unrealistic, that’s why it is called “Naïve”.
Exercise 3. Applying Naïve Bayes to data with both numerical and nominal attributes
Given is the training data in the table below (the weather data with some numerical attributes, play is the class). Predict the class of the following new example using the Naïve Bayes classification: outlook=overcast, temperature=60, humidity=62, windy=false.
COMP3308/3608 Artificial Intelligence, s1 2022
temperature humidity windy play
sunny 85 85 overcast 80 90 overcast 83 86 rainy 70 96 rainy 68 80 rainy 65 70 overcast 64 65 sunny 72 95 sunny 69 70 rainy 75 80 sunny 75 70 overcast 72 90 overcast 81 75 rainy 71 91
false no true no false yes false yes false yes true no true yes false no false yes false yes true yes true yes false yes true no
First, we need to calculate the mean and standard deviation values for the numerical attributes, for each class (yes and no) separately.
Xi (Xi −)2
=i=1 2 =i=1
Xi, i=1..n – the i-th measurement, n-number of measurements
You can use Excel, Python, Matlab etc. to calculate these values.
_temp_yes=73, _temp_yes=6.2; _temp_no=74.6, _temp_no=8.0 _hum_yes=79.1, _temp_yes=10.2; _hum_no=86.2, _temp_no=9.7
Second, to calculate f(temperature=60|yes), f(temperature=60|no), f(humidity=62|yes) and f(humidity=62|no) using the probability density function for normal distribution:
1 −(x−)2 f(x)= 2e 22
𝑓 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 = 60 𝑦𝑒𝑠 =
1 − 60−73 2
𝑓 𝑡𝑒𝑚𝑝𝑒𝑟𝑎𝑡𝑢𝑟𝑒 = 60 𝑛𝑜
1 − 60−74.6 2
= 0.0071 = 0.0094
1 − 62−79.1 2
𝑒 2 ∗10.22 = 0.0096 1 − 62−86.2 2
𝑒 2∗9.72 =0.0018
Third, we can calculate the probabilities for the nominal attributes:
COMP3308/3608 Artificial Intelligence, s1 2022
𝑓 h𝑢𝑚𝑖𝑑𝑖𝑡𝑦 = 62 𝑦𝑒𝑠 =
𝑓 h𝑢𝑚𝑖𝑑𝑖𝑡𝑦=62𝑛𝑜 =
P(yes)=9/14=0.643 P(outlook=overcast|yes)=4/9=0.444 P(windy=false|yes)=6/9=0.667
P(no)=5/14=0.357 P(outlook=overcast|no)=1/5=0.2 P(windy=false|no)=2/5=0.4
Fourth, we can calculate the final probabilities:
P(yes|E)= 0.444*0.0071*0.0096*0.667*0.643 =12.97*10−6 P(E) P(E)
𝑃 𝑛𝑜𝐸 =0.2∗0.0094∗0.0018∗0.4∗0.357=4∗10−7 𝑃𝐸 𝑃𝐸
Therefore, the Naïve Bayes classifier predicts play=yes for the new example.
Exercise 4. Bayes Theorem (Advanced only)
Suppose that the fraction of undergraduate students who smoke is 15% and the fraction of graduate students who smoke is 23%. If 1/5 of the University students are graduate students and the rest are undergraduates, what’s the probability that a student who smokes is a graduate student?
Hint: Use the Bayes Theorem; you will need to calculate the denominator using the law of total probability, see its Wikipedia description.
Suppose that:
X represents if the student smokes: {smoker, non-smoker} or abbreviated {S, NS} and Y represents the type of student: {undergraduate, graduate} or abbreviated {UG, G}
Given: P(G)=1/5=0.2, P(UG)=4/5=0.8, P(S|UG)=0.15, P(S|G)=0.23 P(G|S)=?
P(G | S) = P(S | G) P(G) P(S)
To calculate P(S) we will use the law of total probability. If {Y1, Y2, …, Yk} is the set of mutually
exclusive and exhaustive outcomes of Y, then:
P(X) = P(X,Y ) = P(X |Y )P(Y )
iii i=1 i=1
=> P(S)=P(S|G)P(G)+P(S|UG)P(UG)=0.23*(1/5)+0.15*(4/5)=0.166 =>P(G|S)=(0.23*0.2)/0.166=0.277
Exercise 5. Using Weka – Comparing Classifiers
1. Load the iris dataset
2. Choose “Percentage split” mode for evaluation: 66% training set, 33% testing set
3. Run the Naïve Bayes and review Weka’s output
4. For comparison, also run k-neatest neighbor with k=1 and 3 (IB1 and IBk), OneR and ZeroR. Which is the most accurate classifier?
5. Change the test mode to “Cross validation”. Apply 10-fold cross validation instead of percentage split as evaluation mode and compare the classifiers.
• Which classifier produced the most accurate classification?
• Which evaluation strategy (percentage split or 10-fold cross validation) produced better results?
• Which evaluation strategy, percentage split or cross validation, is more statistically reliable and
6. Apply leave-one-out cross validation. Tip: You need to specify the number of folds in the WEKA’s cross validation box.
7. Check the confusion matrix printed by WEKA for one of the classifiers, e.g. Naïve Bayes, and verify the accuracy, recall, precision and F1 measure. Note: Weka shows recall, precision and F1 for each class separately.
Answer: Leave-one-out is n-fold cross validation where n is the number of examples. Thus, the number of folds should be set to 150 for iris data.
Additional exercises to be done at your own time:
Exercise 6. Naïve Bayes with Laplace correction
As in exercise 1, but now suppose that you are able to measure 3 properties of each item (P1, P2 and P3) and the data is as follows:
P1 P2 P3 result
Y Y Y good Y N N bad N N Y good N Y N bad Y N Y good N N N good
COMP3308/3608 Artificial Intelligence, s1 2022
Use Naïve Bayes
use the Laplace correction.
P(bad)=2/6 P(P1=N|bad)=1/2 P(P2=Y|bad)=1/2 P(P3=Y|bad)=0/2=0
to predict the class of the following new example P1=N, P2=Y P3=Y. If necessary
P(good)=4/6 P(P1=N|good)=2/4 P(P2=Y|good)=1/4 P(P3=Y|good)=3/4=3/4
We need to apply the Laplace correction: +1 to the nominator, + number attribute values to the denominator.
All probabilities with the Laplace correction P(bad)=(2+1)/(6+2)=3/8 P(P1=N|bad)=1/2=(1+1)/(2+2)=2/4 P(P2=Y|bad)=1/2=(1+1)/(2+2)=2/4 P(P3=Y|bad)=0/2=(0+1)/(2+2)=1/4
2213 3 0.023 𝑃 bad E = 4 4 4 8 = 128 =
3245 5 0.069 𝑃 good E = 6 6 6 8 = 72 =
P(bad|E)
COMP3308/3608 Artificial Intelligence, s1 2022
P(good)=4/6=(4+1)/(6+2)=5/8 P(P1=N|good)=2/4=(2+1)/(4+2)=3/6 P(P2=Y|good)=1/4=(1+1)/(4+2)=2/6 P(P3=Y|good)=3/4=(3+1)/(4+2)=4/6
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com