Coursework 2 MN-3024
The ‘Hepatitis’ Data set (provided in arff. format is available on the Blackboard) contains information about patients affected by the Hepatitis disease. The task is to predict if these patients have or have not hepatitis (Histology: Yes or No).
You should use the Weka data mining package, which is installed in the university computers and also available to download from: http://www.cs.waikato.ac.nz/~ml/weka/
You should hand in a report covering the following:
- Select a suitable tree building algorithm and build a model. Describe how you split the data for training and testing purposes. Explain the splitting method.
(9 points)
- Interpret the output results:
- The accuracy rates;
- Which attributes were used to make the predictions;
- How many nodes and leaves you obtained;
- Include a visual tree diagram showing the structure of the model that you built.
(12 points, with a greater attention to the accuracy rate interpretation)
- Give a detailed technical description of the classification model:
- What tree induction method is utilised;
- Which attribute selection criteria is used;
- Give an example of how the attributes were selected for growing the tree.
(20 points)
- Change the confidence factor to 35%, report any change in the model accuracy, explaining reasons behind the change.
(5 points)
- Set the ‘REP’ parameter (Reduced Error Pruning) to ‘TRUE’. Explain the meaning of this operation. Report and explain any change in the model accuracy.
(7 points)
- Set the parameter ‘unpruned’ to ‘TRUE’, Report and explain any change in the model accuracy and in the tree structure. Explain which pruning method for this algorithm is used. Carefully explain how pruning was performed.
(11 points)
- Report on the model’s comparative ability to other 2 models of your choice (for example, neural networks or SVM or Bayesian network etc.) to predict the class variable. Which model classified data most accurately and what are the possible reasons of its prevalence?
(20 points)
- Show a confusion matrix for the model and interpret it. Show a ROC curve and a Lift chart for the decision and interpret them.
(6 points)
- Generate a set of rules along the subtree path: Ascites – Class – Spiders – Bilirubin – Sex – Class ‘No’. What would you recommend to reduce the number of rules in the set? Hint: speculate about Support and Confidence.
(10 points)
Note: the allocated points are given as tentative benchmarks only. The report will be assessed based on overall understanding of the data mining process as well, i.e. relating to the domain of the problem, cross-referencing between the points in the questions, notifying anything interesting etc. The report structure, quality of references will also be evaluated.