程序代写代做代考 scheme flex algorithm Excel database decision tree Learning in the presence of Class Imbalances

Learning in the presence of Class Imbalances

*
Inductive Learning from Imbalanced Data Sets

*
Standard Assumption
The data sets are balanced: i.e., there are as many positive examples of the concept as there are negative ones.
Example: Our database of sick and healthy patients contains as many examples of sick patients as it does of healthy ones.

*
The Standard Assumption is not Always Correct
There exist many domains that do not have a balanced data set.
Examples:
Helicopter Gearbox Fault Monitoring
Discrimination between Earthquakes and Nuclear Explosions
Document Filtering
Detection of Oil Spills
Detection of Fraudulent Telephone Calls

*
But What is the Problem?
Standard learners are often biased towards the majority class.
That is because these classifiers attempt to reduce global quantities such as the error rate, not taking the data distribution into consideration.
As a result examples from the overwhelming class are well-classified whereas examples from the minority class tend to be misclassified.

*
Significance of the problem for Machine Learners/Data Miners
For the past 16 years, there has been a lot of research on the problem.
There are excellent review articles on the subject as well as comparative studies.
Some of the dominant approaches are SMOTE (and its many variants) and bagged random undersampling, but there are many others as well.
The problem occurs in many domains.

*
Several Common Approaches
At the data Level: Re-Sampling
Oversampling (Random or Directed)
Undersampling (Random or Directed)
SMOTE
One class learning (ignore the small class altogether)
At the Algorithmic Level:
Adjusting the Costs
Adjusting the decision threshold / probabilistic estimate at the tree leaf

*
Analysis and Approaches
Fundamental
What domain chara-cteristics aggravate the problem?
Class imbalances or small disjuncts?
Are all classifiers sensitive to class imbalances?
Which proposed solutions to the class imbalance problem are more appropriate?

Some Approaches
SMOTE
Specialized Resampling: within-class versus between-class imbalances
One class versus two-class learning
Multiple Resampling

*
Part I: Fundamentals
What domain characteristics aggravate the problem?
Class Imbalances or Small Disjuncts?
Are all classifiers sensitive to class imbalances?
Which proposed solutions to the class imbalance problem are more appropriate?

*
I. I What domain characteristics aggravate the Problem?
To answer this question, I generated artificial domains that vary along three different axes:
The degree of concept complexity
The size of the training set
The degree of imbalance between the two classes.

0
1

+ – + – + – + –

*
I. I What domain characteristics aggravate the Problem?
I created 125 domains, each representing a different type of class imbalance, by varying the concept complexity (C), the size of the training set (S) and the degree of imbalance (I) at different rates (5 settings were used per domain characteristics).
I ran C5.0 [a decision tree learning algorithm] on these various imbalanced domains and plotted its error rate on each domain.
Each experiment was repeated 5 times and the results averaged.

*
I. I What domain characteristics aggravate the Problem?
S= 1

Error
rate
Large
Imbal.
Full
balance

*
I.I What domain characteristics
aggravate the Problem?
Error
rate
S= 5
Large
Imbal.
Full
balance

*
I. I What domain characteristics aggravate the Problem?

The problem is aggravated by two factors:
An increase in the degree of class imbalance
An increase in problem complexity – class imbalances do not hinder the classification of simple problems (e.g., linearly separable ones)
However, the problem is simultaneously mitigated by one factor:
The size of the training set – large training sets yield low sensitivity to class imbalances

*
I.II: Clas Imbalances or Small Disjuncts?
Studying the training sets from the previous experiments, it can be inferred that when i and c are large, and s, small, the domain contains many very small subclusters.
These were also the conditions under which C5.0 performed the worst.
To test whether it is these small subclusters that cause performance degradation, we disregarded the value of s and set the size of all subclusters to 50 examples.

*

Previous
Experiment

This
Experiment
High Concept Complexity: c=5
I.II: Clas Imbalances or Small Disjuncts?
Error
rate
Large
Imbal.
Full
balance

*
I.II: Class Imbalances or Small Disjuncts?

When all the subclusters are of size 50, even at the highest degree of concept complexity, no matter what the class imbalance is, the error is below 1%  It is negligible.
This suggests that it is not the class imbalance per se that causes a performance decrease, but rather, that it is the small disjunct problem created by the class imbalance (in highly complex and small-sized domains) that cause that loss of performance.

I.III Class Overlap
Another factor that was studied by [Batista et al.] along with the other ones is the question of class overlap.
They showed that as the class overlap increases, classifiers become more and more sensitive to the class imbalance problem.
Overlap, they showed, is a very significant factor that cannot be overlooked since it is present in most real-world domains.

*

*
I.IV Are all classifiers sensitive to class imbalances?

Neural Net (MLPs)
A5
A2 A7
+ A5 – – +

+ –

T
F
> 5
5
< 5 T F F T Decision Tree (C5.0) + + + + + - - - - - - Support Vector Machines (SVMs) * I.III Are all classifiers sensitive to class imbalances? Error rate S = 1; C = 3 Large Imbal. Full balance * I.III Are all classifiers sensitive to class imbalances? Decision Tree (C5.0) C5.0 is the most sensitive to class imbalances. This is because C5.0 works globally, not paying attention to specific data points. Multi-Layer perceptrons (MLPs) MLPs are less prone to the class imbalance problem than C5.0. This is because of their flexibility: their solution gets adjusted by each data point in a bottom-up manner as well as by the overall data set in a top-down manner. Support Vector Machines (SVMs) SVMs are even less prone to the class imbalance problem than MLPs because they are only concerned with a few support vectors, the data points located close to the boundaries. * I.IV Which Solution is Best? Random Oversampling Directed Oversampling Random Undersampling Directed Undersampling Adjusting the Costs * I.IV Which Solution is Best? Err. rate S = 1; C = 3 Large Imbal. Full Bal. * I.IV Which Solution is Best? Three of the five methods considered present an improvement over C5.0 at S=1 and C=3: Random oversampling, Directed oversampling and Cost-modifying. Undersampling (random and directed) is not effective and can even hurt the performance. Random oversampling helps quite dramatically at all complexity. Directed oversampling makes a bit of a difference by helping slightly more. On the graph of the previous slide, Cost-adjusting is about as effective as Directed oversampling. Generally, however, it is found to be slightly more useful. I.IV Which Solution is Best? However, note that: The results obtained using random oversampling (directed or not) have been discarded in later research (See Drummond and Holte, 2003: http://www.site.uottawa.ca/~nat/Workshop2003/drummondc.pdf) on more general domains. On more general domains, it was shown that random oversampling has a tendency to cause classifiers to overfit the data. When simple approaches are sought, which do not generate artificial data, it is now believed that random undersampling is preferable. * * Part II:More sophisticated approaches SMOTE (Chawla et al., 2002) Specialized Resampling: within-class versus between-class imbalances One class versus two-class learning Multiple Resampling II.I SMOTE * * II.II: Within-class versus Between-class Imbalances Idea: Use unsupervised learning to identify subclusters in each class separately. Re-sample the subclusters of each class until no within-class imbalance and no between-class imbalance are present (although the subclusters of each class can have different sizes) * II.II: Within-class versus Between-class Imbalances Symmetric Case Asymmetric Case 5 * II.II: Within-class vs Between- class Imbalances: Experiments Imbalances Random Oversampling Between class imbalance eliminated Guided Oversampling I (# Clusters Known) Use prior knowledge of classes to guide clustering Guided Oversampling II (# Clusters Unknown) Let clustering algorithm determine the number of clusters * II.II: Within-class vs Between- class Imbalances: Letters Subset of the Letters dataset found at the UCI Repository Positive class contains the vowels a and u Negative class contains the consonants m, s, t and w. All letters are distributed according to their frequency in English texts. * II.II: Within-class vs Between- class Imbalances: Letters Method Precision Recall F-Measure Imbalanced 0.905 0.818 0.859 Random Oversampling 0.905 0.818 0.859 Guided Oversampling I (# Clusters Unknown) 0.923 0.914 0.919 Guided Oversampling II (Using Known Clusters) 0.935 0.877 0.905 * II.I:I Within-class vs Between- class Imbalances: Text Classification Reuters-21578 Dataset Classifying a document according to its topic Positive class is a particular topic Negative class is every other topic * II.II: Within-class vs Between- class Imbalances: Text Classification Method Precision Recall F-Measure Imbalanced 0.617 0.394 0.455 Random Oversampling 0.580 0.545 0.560 Guided Oversampling I (# Clusters Unknown) 0.650 0.510 0.544 Guided OversamplingII (Using Known Clusters) 0.601 0.751 0.665 * II.II: Within-class versus Between-class Imbalances Results: On letter and text categorization tasks, this strategy worked better than the random over-sampling strategy. Noise in the small subclusters, however, caused problems since it got too magnified. This promising strategy requires more study.. * II.III One-Class versus Two-Class Learning DMLP RMLP A5 A2 A7 + A5 - - + + - T F > 5
5
< 5 T F F T Decision Tree (C5.0) * II.III One-Class versus Two-Class Learning Error Rate * II.III One-Class versus Two-Class Learning One-Class learning is more accurate than two-class learning on two of our three domains considered and as accurate on the third. It can thus be quite useful in class imbalanced situations. Further comparisons with other proposed methods are required. * II.IV Multiple Resampling Idea: Although the results reported here suggest that undersampling is not as useful as oversampling, other studies of ours and others (on different data sets) suggest that it can be  It shouldn’t be abandoned Further experiments of ours (not reported here) suggest that rather than oversampling or undersampling until a full balance is achieved may not always be optimal  A different re-sampling rate should be used * II.IV Multiple Resampling Idea (Continued): It is not possible to know, a-priori, whether a given domain favours oversampling or undersampling and what resampling rate is best. Therefore, we decided to create a self-adaptive combination scheme that considers both strategies at various rates. * II.IVMultiple Resampling … … Output Oversampling Expert Undersampl. Expert Oversampling Classifiers (sampled at different rates) Undersampling Classifiers (sampled at diff.rates) * II.IV Multiple Resampling The combination scheme was compared to C4.5-Adaboost (with 20 classifiers) with respect to the FB-measures on a text classification task (Reuters-21578, Top 10 categories) The FB-measure combines precision (the proportion of examples classified as positive that are truly positive) and recall (the proportion of truly positive examples that are classified as positive) in the following way: F1  precision = recall F2  2 * precision = recall F0.5 precision = 2 * recall * II.IV Testing the Combination Scheme  Results F- Measure In all cases, the mixture scheme is superior to Adaboost. However, though it helps both recall and precision, it helps recall more. Chart1 C=1 C=1 C=1 C=1 C=1 C=2 C=2 C=2 C=2 C=2 C=3 C=3 C=3 C=3 C=3 C=4 C=4 C=4 C=4 C=4 C=5 C=5 C=5 C=5 C=5 I=1 I=2 I=3 I=4 I=5 C 0 0 0 0 0 50 2.5 0.9 0.4 0.9 50 50 2.1 1.5 0.5 50 50 50 50 2.3 50 50 50 50 50 Sheet1 c s I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 1 0 1 0 0.4 0 0.3 0.4 0.4 0.2 0.2 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 0 0 0.2 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0.8 0 0.7 0.8 0.8 0 0.2 0 0.2 100 0 2 0 2 100 0 1 0 1 100 0 0 0 0 100 0 1 0 1 5 100 0 0 0 0 0 0 0 0 0 0 0 0.2 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 2 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 3 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 3 100 0 0 0 0 0.2 0 0.2 0 0.2 100 0 0 0 0 100 0 0 0 0 100 1 0 1 0 100 0 0 0 0 4 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 5 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 1 4 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 5 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0.2 0 0.2 0 0.2 100 0 0 0 0 100 1 0 1 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 1 1 200 100 0 50 0 50 0 0 0 50 50 4.2 1 0.8 0.7 200 100 0 50 0 200 100 0 50 0 200 100 0 50 0 200 100 0 50 0 2 200 7 0 3.5 0 2.5 0.9 6.625 0.55 3.4 3.4 1.7 1 0.5 0.1 200 13 2 6.5 1 200 3 4 1.5 2 200 2 1 1 0.5 200 0 2 0 1 3 200 0 0 0 0 0.9 0.6 0.675 1.175 1.5 1.5 1.2 0.2 0 0.1 200 3 0 1.5 0 200 1 1 0.5 0.5 200 1 0 0.5 0 200 4 5 2 2.5 4 200 2 3 1 1.5 0.4 0.9 0.175 0.425 1.3 1.3 0.6 0.2 0 0.2 200 1 3 0.5 1.5 200 0 2 0 1 200 0 0 0 0 200 1 1 0.5 0.5 5 200 0 1 0 0.5 0.9 0.6 1.05 0.175 1.5 1.5 0.3 0.3 0.4 0.5 200 4 1 2 0.5 200 0 2 0 1 200 1 0 0.5 0 200 4 2 2 1 2 2 1 200 2 0 1 0 4.1 0.1 11.3 0.05 4.2 200 0 0 0 0 200 15 1 7.5 0.5 200 12 0 6 0 200 12 0 6 0 2 200 2 1 1 0.5 1.5 0.2 2 0.075 1.7 200 6 0 3 0 200 0 0 0 0 200 1 1 0.5 0.5 200 6 0 3 0 3 200 1 1 0.5 0.5 0.9 0.3 0.425 0.075 1.2 200 2 0 1 0 200 0 1 0 0.5 200 3 0 1.5 0 200 3 1 1.5 0.5 4 200 0 1 0 0.5 0.3 0.3 0.45 0.075 0.6 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 200 3 1 1.5 0.5 5 200 0 0 0 0 0 0.3 0 0.075 0.3 200 0 1 0 0.5 200 0 1 0 0.5 200 0 0 0 0 200 0 1 0 0.5 2 3 1 200 7 0 3.5 0 1 0 2.125 0 1 200 2 0 1 0 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 2 200 0 0 0 0 1 0 1 0 1 200 3 0 1.5 0 200 1 0 0.5 0 200 1 0 0.5 0 200 5 0 2.5 0 3 200 0 1 0 0.5 0.1 0.1 0.05 0.05 0.2 200 1 0 0.5 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 4 200 0 0 0 0 0.1 0.1 0.05 0.05 0.2 200 1 0 0.5 0 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 5 200 0 0 0 0 0.2 0.1 0.2 0.05 0.3 200 0 0 0 0 200 0 0 0 0 200 2 0 1 0 200 0 1 0 0.5 2 4 1 200 0 0 0 0 0.8 0 0.45 0 0.8 200 1 0 0.5 0 200 3 0 1.5 0 200 1 0 0.5 0 200 3 0 1.5 0 2 200 2 0 1 0 0.5 0 0.25 0 0.5 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 200 2 0 1 0 3 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 4 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 5 200 0 3 0 1.5 0 0.4 0 0.425 0.4 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 200 0 0 0 0 2 5 1 200 1 0 0.5 0 0.7 0 1.075 0 0.7 200 0 0 0 0 200 0 0 0 0 200 5 0 2.5 0 200 1 0 0.5 0 2 200 0 0 0 0 0.1 0 0.05 0 0.1 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 3 200 0 0 0 0 0.1 0 0.05 0 0.1 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 4 200 0 0 0 0 0.2 0 0.075 0 0.2 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 200 1 0 0.5 0 5 200 1 0 0.5 0 0.5 0 0 0 0.5 200 1 0 0.5 0 200 1 0 0.5 0 200 1 0 0.5 0 200 1 0 0.5 0 3 1 1 400 200 0 50 0 50 0 0 0 50 50 50 50 50 1.3 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 50 50 50 10.8 0.65 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 14 2 3.5 0.5 2.1 1 1.01875 0.25 3.1 3.1 2.55 0.95 0.4 0.35 400 8 6 2 1.5 400 3 4 0.75 1 400 7 6 1.75 1.5 400 10 2 2.5 0.5 4 400 2 2 0.5 0.5 1.5 1.3 0.96875 0.7 2.8 2.8 0.85 0.65 0.4 0.55 400 3 2 0.75 0.5 400 7 6 1.75 1.5 400 6 6 1.5 1.5 400 12 10 3 2.5 5 400 4 3 1 0.75 0.5 0.85 0.25 0.39375 1.35 1.35 1.5 0.25 0.1 0.25 400 0 6 0 1.5 400 4 6 1 1.5 400 2 1 0.5 0.25 400 0 1 0 0.25 3 2 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 6 3 1.5 0.75 1.9 0.65 0.7375 0.33125 2.55 400 9 6 2.25 1.5 400 13 3 3.25 0.75 400 5 1 1.25 0.25 400 5 0 1.25 0 4 400 1 5 0.25 1.25 0.35 0.5 0.08125 0.25 0.85 400 1 1 0.25 0.25 400 3 0 0.75 0 400 2 3 0.5 0.75 400 0 1 0 0.25 5 400 3 6 0.75 1.5 0.55 0.95 0.10625 0.325 1.5 400 4 1 1 0.25 400 1 6 0.25 1.5 400 2 4 0.5 1 400 1 2 0.25 0.5 3 3 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 3 0 0.75 0 0.85 0.1 0.39375 0.01875 0.95 400 0 1 0 0.25 400 7 0 1.75 0 400 4 1 1 0.25 400 3 0 0.75 0 4 400 0 0 0 0 0.4 0.25 0.1125 0.09375 0.65 400 3 1 0.75 0.25 400 1 3 0.25 0.75 400 3 0 0.75 0 400 1 1 0.25 0.25 5 400 2 0 0.5 0 0.2 0.05 0.04375 0.0125 0.25 400 1 0 0.25 0 400 1 0 0.25 0 400 0 1 0 0.25 400 0 0 0 0 3 4 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 10.75 0.05 481.875 0.0125 10.8 400 4 1 1 0.25 400 8 0 2 0 400 2 0 0.5 0 400 1 0 0.25 0 3 400 0 0 0 0 0.35 0.05 0.3 0.0125 0.4 400 0 0 0 0 400 0 0 0 0 400 5 0 1.25 0 400 2 1 0.5 0.25 4 400 0 0 0 0 0.2 0.2 0.04375 0.2 0.4 400 2 4 0.5 1 400 0 0 0 0 400 1 0 0.25 0 400 1 0 0.25 0 5 400 0 1 0 0.25 0.05 0.05 0.0125 0.0125 0.1 400 0 0 0 0 400 0 0 0 0 400 0 0 0 0 400 1 0 0.25 0 3 5 1 400 14 0 3.5 0 1.25 0.05 2.03125 0.0125 1.3 400 7 0 1.75 0 400 0 1 0 0.25 400 1 0 0.25 0 400 3 0 0.75 0 2 400 3 0 0.75 0 0.65 0 0.08125 0 0.65 400 3 0 0.75 0 400 2 0 0.5 0 400 1 0 0.25 0 400 4 0 1 0 3 400 2 0 0.5 0 0.3 0.05 0.075 0.0125 0.35 400 2 1 0.5 0.25 400 2 0 0.5 0 400 0 0 0 0 400 0 0 0 0 4 400 0 0 0 0 0.35 0.2 0.05 0.04375 0.55 400 1 0 0.25 0 400 2 1 0.5 0.25 400 2 2 0.5 0.5 400 2 1 0.5 0.25 5 400 0 1 0 0.25 0.15 0.1 0.1125 0.01875 0.25 400 3 0 0.75 0 400 0 1 0 0.25 400 0 0 0 0 400 0 0 0 0 4 1 1 800 400 0 50 0 50 0 0 0 50 50 50 50 50 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 50 50 50 50 1.7 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 400 0 50 0 50 0 0 0 50 50 50 50 0.9 0.675 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 4 800 400 0 50 0 50 0 0 0 50 50 50 1.275 0.4 0.4 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 5 800 25 15 3.125 1.875 2.3 2.175 0.5515625 0.4578125 4.475 4.475 2.275 0.85 0.5 0.2 800 15 21 1.875 2.625 800 22 17 2.75 2.125 800 20 24 2.5 3 800 10 10 1.25 1.25 4 2 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 7 14 0.875 1.75 0.825 1.45 0.1375 0.1296875 2.275 800 2 14 0.25 1.75 800 6 11 0.75 1.375 800 10 7 1.25 0.875 800 8 12 1 1.5 4 3 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 4 800 5 1 0.625 0.125 0.875 0.4 0.1875 0.0421875 1.275 800 2 4 0.25 0.5 800 8 2 1 0.25 800 10 4 1.25 0.5 800 10 5 1.25 0.625 5 800 1 8 0.125 1 0.3 0.55 0.0203125 0.0828125 0.85 800 3 5 0.375 0.625 800 2 2 0.25 0.25 800 4 3 0.5 0.375 800 2 4 0.25 0.5 4 4 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 6 1 0.75 0.125 0.875 0.025 0.2734375 0.003125 0.9 800 13 0 1.625 0 800 9 0 1.125 0 800 2 0 0.25 0 800 5 0 0.625 0 4 800 2 4 0.25 0.5 0.225 0.175 0.034375 0.04375 0.4 800 4 2 0.5 0.25 800 2 0 0.25 0 800 0 1 0 0.125 800 1 0 0.125 0 5 800 2 4 0.25 0.5 0.225 0.275 0.0109375 0.034375 0.5 800 1 2 0.125 0.25 800 2 0 0.25 0 800 1 3 0.125 0.375 800 3 2 0.375 0.25 4 5 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 12 0 1.5 0 1.625 0.075 0.4609375 0.0125 1.7 800 15 1 1.875 0.125 800 13 0 1.625 0 800 5 0 0.625 0 800 20 2 2.5 0.25 3 800 3 2 0.375 0.25 0.525 0.15 0.1203125 0.0265625 0.675 800 0 1 0 0.125 800 5 0 0.625 0 800 6 3 0.75 0.375 800 7 0 0.875 0 4 800 1 0 0.125 0 0.25 0.15 0.0625 0.0421875 0.4 800 5 1 0.625 0.125 800 3 1 0.375 0.125 800 0 0 0 0 800 1 4 0.125 0.5 5 800 2 0 0.25 0 0.125 0.075 0.0078125 0.0125 0.2 800 1 0 0.125 0 800 0 1 0 0.125 800 1 2 0.125 0.25 800 1 0 0.125 0 5 1 1 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 1.2875 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 50 50 50 1.7875 1.125 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 800 0 50 0 50 0 0 0 50 50 50 1.7625 1.375 1.125 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 2 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 3 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 7 12 0.4375 0.75 0.825 0.9375 0.166796875 0.01953125 1.7625 1600 22 16 1.375 1 1600 11 18 0.6875 1.125 1600 8 15 0.5 0.9375 1600 18 14 1.125 0.875 5 4 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 24 6 1.5 0.375 1.3625 0.425 0.283984375 0.030078125 1.7875 1600 11 11 0.6875 0.6875 1600 16 8 1 0.5 1600 25 5 1.5625 0.3125 1600 33 4 2.0625 0.25 5 1600 9 5 0.5625 0.3125 0.625 0.75 0.095703125 0.0703125 1.375 1600 6 15 0.375 0.9375 1600 11 14 0.6875 0.875 1600 6 15 0.375 0.9375 1600 18 11 1.125 0.6875 5 5 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 26 1 1.625 0.0625 1.1125 0.175 0.1296875 0.0125 1.2875 1600 18 3 1.125 0.1875 1600 20 5 1.25 0.3125 1600 11 4 0.6875 0.25 1600 14 1 0.875 0.0625 4 1600 18 3 1.125 0.1875 0.925 0.2 0.09453125 0.0046875 1.125 1600 15 3 0.9375 0.1875 1600 9 2 0.5625 0.125 1600 21 3 1.3125 0.1875 1600 11 5 0.6875 0.3125 5 1600 5 6 0.3125 0.375 0.475 0.65 0.028515625 0.1046875 1.125 1600 6 4 0.375 0.25 1600 7 16 0.4375 1 1600 12 12 0.75 0.75 1600 8 14 0.5 0.875 Sheet1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sheet2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sheet3 c s I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0 0 0 0 0 0 0 2 100 0 1 0 1 0 0.4 0 0.3 0.4 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0.8 0 0.7 0.8 5 100 0 0 0 0 0 0 0 0 0 1 2 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 1 3 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 3 100 0 0 0 0 0.2 0 0.2 0 0.2 4 100 0 0 0 0 0 0.2 0 0.2 0.2 5 100 0 0 0 0 0 0.2 0 0.2 0.2 1 4 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 1 5 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0.2 0 0.2 0 0.2 5 100 0 0 0 0 0 0 0 0 0 2 1 1 200 100 0 50 0 50 0 0 0 50 2 200 7 0 3.5 0 2.5 0.9 6.625 0.55 3.4 3 200 0 0 0 0 0.9 0.6 0.675 1.175 1.5 4 200 2 3 1 1.5 0.4 0.9 0.175 0.425 1.3 5 200 0 1 0 0.5 0.9 0.6 1.05 0.175 1.5 2 2 1 200 2 0 1 0 4.1 0.1 11.3 0.05 4.2 2 200 2 1 1 0.5 1.5 0.2 2 0.075 1.7 3 200 1 1 0.5 0.5 0.9 0.3 0.425 0.075 1.2 4 200 0 1 0 0.5 0.3 0.3 0.45 0.075 0.6 5 200 0 0 0 0 0 0.3 0 0.075 0.3 2 3 1 200 7 0 3.5 0 1 0 2.125 0 1 2 200 0 0 0 0 1 0 1 0 1 3 200 0 1 0 0.5 0.1 0.1 0.05 0.05 0.2 4 200 0 0 0 0 0.1 0.1 0.05 0.05 0.2 5 200 0 0 0 0 0.2 0.1 0.2 0.05 0.3 2 4 1 200 0 0 0 0 0.8 0 0.45 0 0.8 2 200 2 0 1 0 0.5 0 0.25 0 0.5 3 200 0 0 0 0 0 0 0 0 0 4 200 0 0 0 0 0 0 0 0 0 5 200 0 3 0 1.5 0 0.4 0 0.425 0.4 2 5 1 200 1 0 0.5 0 0.7 0 1.075 0 0.7 2 200 0 0 0 0 0.1 0 0.05 0 0.1 3 200 0 0 0 0 0.1 0 0.05 0 0.1 4 200 0 0 0 0 0.2 0 0.075 0 0.2 5 200 1 0 0.5 0 0.5 0 0 0 0.5 3 1 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 14 2 3.5 0.5 2.1 1 1.01875 0.25 3.1 4 400 2 2 0.5 0.5 1.5 1.3 0.96875 0.7 2.8 5 400 4 3 1 0.75 0.5 0.85 0.25 0.39375 1.35 3 2 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 6 3 1.5 0.75 1.9 0.65 0.7375 0.33125 2.55 4 400 1 5 0.25 1.25 0.35 0.5 0.08125 0.25 0.85 5 400 3 6 0.75 1.5 0.55 0.95 0.10625 0.325 1.5 3 3 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 3 0 0.75 0 0.85 0.1 0.39375 0.01875 0.95 4 400 0 0 0 0 0.4 0.25 0.1125 0.09375 0.65 5 400 2 0 0.5 0 0.2 0.05 0.04375 0.0125 0.25 3 4 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 10.75 0.05 481.875 0.0125 10.8 3 400 0 0 0 0 0.35 0.05 0.3 0.0125 0.4 4 400 0 0 0 0 0.2 0.2 0.04375 0.2 0.4 5 400 0 1 0 0.25 0.05 0.05 0.0125 0.0125 0.1 3 5 1 400 14 0 3.5 0 1.25 0.05 2.03125 0.0125 1.3 2 400 3 0 0.75 0 0.65 0 0.08125 0 0.65 3 400 2 0 0.5 0 0.3 0.05 0.075 0.0125 0.35 4 400 0 0 0 0 0.35 0.2 0.05 0.04375 0.55 5 400 0 1 0 0.25 0.15 0.1 0.1125 0.01875 0.25 4 1 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 400 0 50 0 50 0 0 0 50 5 800 25 15 3.125 1.875 2.3 2.175 0.5515625 0.4578125 4.475 4 2 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 400 0 50 0 50 0 0 0 50 5 800 7 14 0.875 1.75 0.825 1.45 0.1375 0.1296875 2.275 4 3 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 5 1 0.625 0.125 0.875 0.4 0.1875 0.0421875 1.275 5 800 1 8 0.125 1 0.3 0.55 0.0203125 0.0828125 0.85 4 4 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 6 1 0.75 0.125 0.875 0.025 0.2734375 0.003125 0.9 4 800 2 4 0.25 0.5 0.225 0.175 0.034375 0.04375 0.4 5 800 2 4 0.25 0.5 0.225 0.275 0.0109375 0.034375 0.5 4 5 1 800 400 0 50 0 50 0 0 0 50 2 800 12 0 1.5 0 1.625 0.075 0.4609375 0.0125 1.7 3 800 3 2 0.375 0.25 0.525 0.15 0.1203125 0.0265625 0.675 4 800 1 0 0.125 0 0.25 0.15 0.0625 0.0421875 0.4 5 800 2 0 0.25 0 0.125 0.075 0.0078125 0.0125 0.2 5 1 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 800 0 50 0 50 0 0 0 50 5 2 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 800 0 50 0 50 0 0 0 50 5 3 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 7 12 0.4375 0.75 0.825 0.9375 0.166796875 0.01953125 1.7625 5 4 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 24 6 1.5 0.375 1.3625 0.425 0.283984375 0.030078125 1.7875 5 1600 9 5 0.5625 0.3125 0.625 0.75 0.095703125 0.0703125 1.375 5 5 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 26 1 1.625 0.0625 1.1125 0.175 0.1296875 0.0125 1.2875 4 1600 18 3 1.125 0.1875 0.925 0.2 0.09453125 0.0046875 1.125 5 1600 5 6 0.3125 0.375 0.475 0.65 0.028515625 0.1046875 1.125 s c I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Positive: S=1 1 1 2 100 0 1 0.0 1.0 0.0 0.4 0.0 0.3 0.4 C=1 C=2 C=3 C=4 C=5 1 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=1 0.0 50.0 50.0 50.0 50.0 1 1 4 100 0 0 0.0 0.0 0.0 0.8 0.0 0.7 0.8 I=2 0.0 2.5 50.0 50.0 50.0 1 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=3 0.0 0.9 2.1 50.0 50.0 1 2 1 200 100 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.4 1.5 50.0 50.0 1 2 2 200 7 0 3.5 0.0 2.5 0.9 6.6 0.6 3.4 I=5 0.0 0.9 0.5 2.3 50.0 1 2 3 200 0 0 0.0 0.0 0.9 0.6 0.7 1.2 1.5 1 2 4 200 2 3 1.0 1.5 0.4 0.9 0.2 0.4 1.3 1 2 5 200 0 1 0.0 0.5 0.9 0.6 1.1 0.2 1.5 1 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 3 3 400 14 2 3.5 0.5 2.1 1.0 1.0 0.3 3.1 1 3 4 400 2 2 0.5 0.5 1.5 1.3 1.0 0.7 2.8 1 3 5 400 4 3 1.0 0.8 0.5 0.9 0.3 0.4 1.4 1 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Positive: S=1 : S=2 1 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 1 4 4 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 4.1 50.0 50.0 50.0 1 4 5 800 25 15 3.1 1.9 2.3 2.2 0.6 0.5 4.5 I=2 0.0 1.5 50.0 50.0 50.0 1 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.9 1.9 1.9 50.0 1 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.3 0.4 0.4 50.0 1 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.0 0.6 0.6 0.8 1 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 5 5 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2 1 2 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 2 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2 1 4 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Positive: S=3 2 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 C=1 C=2 C=3 C=4 C=5 2 2 1 200 2 0 1.0 0.0 4.1 0.1 11.3 0.1 4.2 I=1 0.0 1.0 50.0 50.0 50.0 2 2 2 200 2 1 1.0 0.5 1.5 0.2 2.0 0.1 1.7 I=2 0.0 1.0 50.0 50.0 50.0 2 2 3 200 1 1 0.5 0.5 0.9 0.3 0.4 0.1 1.2 I=3 0.2 0.1 0.9 50.0 50.0 2 2 4 200 0 1 0.0 0.5 0.3 0.3 0.5 0.1 0.6 I=4 0.0 0.1 0.4 0.9 50.0 2 2 5 200 0 0 0.0 0.0 0.0 0.3 0.0 0.1 0.3 I=5 0.0 0.2 0.2 0.3 0.8 2 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 3 3 400 6 3 1.5 0.8 1.9 0.7 0.7 0.3 2.6 2 3 4 400 1 5 0.3 1.3 0.4 0.5 0.1 0.3 0.9 2 3 5 400 3 6 0.8 1.5 0.6 1.0 0.1 0.3 1.5 2 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Positive: S=4 2 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 2 4 4 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.8 50.0 50.0 50.0 2 4 5 800 7 14 0.9 1.8 0.8 1.5 0.1 0.1 2.3 I=2 0.0 0.5 10.8 50.0 50.0 2 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.0 0.4 0.9 50.0 2 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.0 0.2 0.2 1.4 2 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.0 0.1 0.2 0.6 2 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 5 5 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3 1 2 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 3 1 3 100 0 0 0.0 0.0 0.2 0.0 0.2 0.0 0.2 3 1 4 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 False Positive: S=5 3 1 5 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 C=1 C=2 C=3 C=4 C=5 3 2 1 200 7 0 3.5 0.0 1.0 0.0 2.1 0.0 1.0 I=1 0.0 0.7 1.3 50.0 50.0 3 2 2 200 0 0 0.0 0.0 1.0 0.0 1.0 0.0 1.0 I=2 0.0 0.1 0.7 1.6 50.0 3 2 3 200 0 1 0.0 0.5 0.1 0.1 0.1 0.1 0.2 I=3 0.0 0.1 0.3 0.5 1.1 3 2 4 200 0 0 0.0 0.0 0.1 0.1 0.1 0.1 0.2 I=4 0.2 0.2 0.4 0.3 0.9 3 2 5 200 0 0 0.0 0.0 0.2 0.1 0.2 0.1 0.3 I=5 0.0 0.5 0.2 0.1 0.5 3 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 3 3 400 3 0 0.8 0.0 0.9 0.1 0.4 0.0 1.0 3 3 4 400 0 0 0.0 0.0 0.4 0.3 0.1 0.1 0.7 3 3 5 400 2 0 0.5 0.0 0.2 0.1 0.0 0.0 0.3 3 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=1 3 4 4 800 5 1 0.6 0.1 0.9 0.4 0.2 0.0 1.3 C=1 C=2 C=3 C=4 C=5 3 4 5 800 1 8 0.1 1.0 0.3 0.6 0.0 0.1 0.9 I=1 0.0 0.0 0.0 0.0 0.0 3 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=2 0.4 0.9 0.0 0.0 0.0 3 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.6 1.0 0.0 0.0 3 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.8 0.9 1.3 0.0 0.0 3 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.6 0.9 2.2 0.0 3 5 5 1600 7 12 0.4 0.8 0.8 0.9 0.2 0.0 1.8 4 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 2 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 4 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 2 1 200 0 0 0.0 0.0 0.8 0.0 0.5 0.0 0.8 4 2 2 200 2 0 1.0 0.0 0.5 0.0 0.3 0.0 0.5 4 2 3 200 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 2 4 200 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Negative: S=2 : S=2 4 2 5 200 0 3 0.0 1.5 0.0 0.4 0.0 0.4 0.4 C=1 C=2 C=3 C=4 C=5 4 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.1 0.0 0.0 0.0 4 3 2 400 200 0 50.0 0.0 10.8 0.1 481.9 0.0 10.8 I=2 0.2 0.2 0.0 0.0 0.0 4 3 3 400 0 0 0.0 0.0 0.4 0.1 0.3 0.0 0.4 I=3 0.0 0.3 0.7 0.0 0.0 4 3 4 400 0 0 0.0 0.0 0.2 0.2 0.0 0.2 0.4 I=4 0.0 0.3 0.5 0.0 0.0 4 3 5 400 0 1 0.0 0.3 0.1 0.1 0.0 0.0 0.1 I=5 0.0 0.3 1.0 1.5 0.0 4 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 4 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 4 4 3 800 6 1 0.8 0.1 0.9 0.0 0.3 0.0 0.9 4 4 4 800 2 4 0.3 0.5 0.2 0.2 0.0 0.0 0.4 4 4 5 800 2 4 0.3 0.5 0.2 0.3 0.0 0.0 0.5 4 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=3 4 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 4 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.0 0.0 0.0 0.0 4 5 4 1600 24 6 1.5 0.4 1.4 0.4 0.3 0.0 1.8 I=2 0.2 0.0 0.0 0.0 0.0 4 5 5 1600 9 5 0.6 0.3 0.6 0.8 0.1 0.1 1.4 I=3 0.0 0.1 0.1 0.0 0.0 5 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=4 0.2 0.1 0.3 0.4 0.0 5 1 2 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=5 0.2 0.1 0.1 0.6 0.9 5 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5 1 4 100 0 0 0.0 0.0 0.2 0.0 0.2 0.0 0.2 5 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5 2 1 200 1 0 0.5 0.0 0.7 0.0 1.1 0.0 0.7 5 2 2 200 0 0 0.0 0.0 0.1 0.0 0.1 0.0 0.1 5 2 3 200 0 0 0.0 0.0 0.1 0.0 0.1 0.0 0.1 5 2 4 200 0 0 0.0 0.0 0.2 0.0 0.1 0.0 0.2 False Negative: S=4 5 2 5 200 1 0 0.5 0.0 0.5 0.0 0.0 0.0 0.5 C=1 C=2 C=3 C=4 C=5 5 3 1 400 14 0 3.5 0.0 1.3 0.1 2.0 0.0 1.3 I=1 0.0 0.0 0.0 0.0 0.0 5 3 2 400 3 0 0.8 0.0 0.7 0.0 0.1 0.0 0.7 I=2 0.0 0.0 0.1 0.0 0.0 5 3 3 400 2 0 0.5 0.0 0.3 0.1 0.1 0.0 0.4 I=3 0.0 0.0 0.1 0.0 0.0 5 3 4 400 0 0 0.0 0.0 0.4 0.2 0.1 0.0 0.6 I=4 0.0 0.0 0.2 0.2 0.4 5 3 5 400 0 1 0.0 0.3 0.2 0.1 0.1 0.0 0.3 I=5 0.0 0.4 0.1 0.3 0.8 5 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 5 4 2 800 12 0 1.5 0.0 1.6 0.1 0.5 0.0 1.7 5 4 3 800 3 2 0.4 0.3 0.5 0.2 0.1 0.0 0.7 5 4 4 800 1 0 0.1 0.0 0.3 0.2 0.1 0.0 0.4 5 4 5 800 2 0 0.3 0.0 0.1 0.1 0.0 0.0 0.2 5 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=5 5 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 5 5 3 1600 26 1 1.6 0.1 1.1 0.2 0.1 0.0 1.3 I=1 0.0 0.0 0.1 0.0 0.0 5 5 4 1600 18 3 1.1 0.2 0.9 0.2 0.1 0.0 1.1 I=2 0.0 0.0 0.0 0.1 0.0 5 5 5 1600 5 6 0.3 0.4 0.5 0.7 0.0 0.1 1.1 I=3 0.0 0.0 0.1 0.2 0.2 I=4 0.0 0.0 0.2 0.2 0.2 I=5 0.0 0.0 0.1 0.1 0.7 Total: S=1 C=1 C=2 C=3 C=4 C=5 I=1 0.0 50.0 50.0 50.0 50.0 I=2 0.4 3.4 50.0 50.0 50.0 I=3 0.0 1.5 3.1 50.0 50.0 I=4 0.8 1.3 2.8 50.0 50.0 I=5 0.0 1.5 1.4 4.5 50.0 Total: S=2 C=1 C=2 C=3 C=4 C=5 I=1 0.0 4.2 50.0 50.0 50.0 I=2 0.2 1.7 50.0 50.0 50.0 I=3 0.0 1.2 2.6 50.0 50.0 I=4 0.0 0.6 0.9 50.0 50.0 I=5 0.0 0.3 1.5 2.3 50.0 Total: S=3 C=1 C=2 C=3 C=4 C=5 I=1 0.0 1.0 50.0 50.0 50.0 I=2 0.2 1.0 50.0 50.0 50.0 I=3 0.2 0.2 1.0 50.0 50.0 I=4 0.2 0.2 0.7 1.3 50.0 I=5 0.2 0.3 0.3 0.9 1.8 Total: S=4 C=1 C=2 C=3 C=4 C=5 I=1 0.0 0.8 50.0 50.0 50.0 I=2 0.0 0.5 10.8 50.0 50.0 I=3 0.0 0.0 0.4 0.9 50.0 I=4 0.0 0.0 0.4 0.4 1.8 I=5 0.0 0.4 0.1 0.5 1.4 Total: S=5 C=1 C=2 C=3 C=4 C=5 I=1 0.0 0.7 1.3 50.0 50.0 I=2 0.0 0.1 0.7 1.7 50.0 I=3 0.0 0.1 0.4 0.7 1.3 I=4 0.2 0.2 0.6 0.4 1.1 I=5 0.0 0.5 0.3 0.2 1.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 C False Pos Percent Avg False Positive: S=1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 False Negative: S = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 Total: S=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 Chart5 C=1 C=1 C=1 C=1 C=1 C=2 C=2 C=2 C=2 C=2 C=3 C=3 C=3 C=3 C=3 C=4 C=4 C=4 C=4 C=4 C=5 C=5 C=5 C=5 C=5 I=1 I=2 I=3 I=4 I=5 0 0 0 0.2 0 0.7 0.1 0.1 0.2 0.5 1.25 0.65 0.3 0.35 0.15 50 1.625 0.525 0.25 0.125 50 50 1.1125 0.925 0.475 Sheet1 c s I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 1 0 1 0 0.4 0 0.3 0.4 0.4 0.2 0.2 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 0 0 0.2 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0.8 0 0.7 0.8 0.8 0 0.2 0 0.2 100 0 2 0 2 100 0 1 0 1 100 0 0 0 0 100 0 1 0 1 5 100 0 0 0 0 0 0 0 0 0 0 0 0.2 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 2 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 3 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 3 100 0 0 0 0 0.2 0 0.2 0 0.2 100 0 0 0 0 100 0 0 0 0 100 1 0 1 0 100 0 0 0 0 4 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 5 100 0 0 0 0 0 0.2 0 0.2 0.2 100 0 0 0 0 100 0 0 0 0 100 0 1 0 1 100 0 0 0 0 1 4 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 1 5 1 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 4 100 0 0 0 0 0.2 0 0.2 0 0.2 100 0 0 0 0 100 1 0 1 0 100 0 0 0 0 100 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 100 0 0 0 0 2 1 1 200 100 0 50 0 50 0 0 0 50 50 4.2 1 0.8 0.7 200 100 0 50 0 200 100 0 50 0 200 100 0 50 0 200 100 0 50 0 2 200 7 0 3.5 0 2.5 0.9 6.625 0.55 3.4 3.4 1.7 1 0.5 0.1 200 13 2 6.5 1 200 3 4 1.5 2 200 2 1 1 0.5 200 0 2 0 1 3 200 0 0 0 0 0.9 0.6 0.675 1.175 1.5 1.5 1.2 0.2 0 0.1 200 3 0 1.5 0 200 1 1 0.5 0.5 200 1 0 0.5 0 200 4 5 2 2.5 4 200 2 3 1 1.5 0.4 0.9 0.175 0.425 1.3 1.3 0.6 0.2 0 0.2 200 1 3 0.5 1.5 200 0 2 0 1 200 0 0 0 0 200 1 1 0.5 0.5 5 200 0 1 0 0.5 0.9 0.6 1.05 0.175 1.5 1.5 0.3 0.3 0.4 0.5 200 4 1 2 0.5 200 0 2 0 1 200 1 0 0.5 0 200 4 2 2 1 2 2 1 200 2 0 1 0 4.1 0.1 11.3 0.05 4.2 200 0 0 0 0 200 15 1 7.5 0.5 200 12 0 6 0 200 12 0 6 0 2 200 2 1 1 0.5 1.5 0.2 2 0.075 1.7 200 6 0 3 0 200 0 0 0 0 200 1 1 0.5 0.5 200 6 0 3 0 3 200 1 1 0.5 0.5 0.9 0.3 0.425 0.075 1.2 200 2 0 1 0 200 0 1 0 0.5 200 3 0 1.5 0 200 3 1 1.5 0.5 4 200 0 1 0 0.5 0.3 0.3 0.45 0.075 0.6 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 200 3 1 1.5 0.5 5 200 0 0 0 0 0 0.3 0 0.075 0.3 200 0 1 0 0.5 200 0 1 0 0.5 200 0 0 0 0 200 0 1 0 0.5 2 3 1 200 7 0 3.5 0 1 0 2.125 0 1 200 2 0 1 0 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 2 200 0 0 0 0 1 0 1 0 1 200 3 0 1.5 0 200 1 0 0.5 0 200 1 0 0.5 0 200 5 0 2.5 0 3 200 0 1 0 0.5 0.1 0.1 0.05 0.05 0.2 200 1 0 0.5 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 4 200 0 0 0 0 0.1 0.1 0.05 0.05 0.2 200 1 0 0.5 0 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 5 200 0 0 0 0 0.2 0.1 0.2 0.05 0.3 200 0 0 0 0 200 0 0 0 0 200 2 0 1 0 200 0 1 0 0.5 2 4 1 200 0 0 0 0 0.8 0 0.45 0 0.8 200 1 0 0.5 0 200 3 0 1.5 0 200 1 0 0.5 0 200 3 0 1.5 0 2 200 2 0 1 0 0.5 0 0.25 0 0.5 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 200 2 0 1 0 3 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 4 200 0 0 0 0 0 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 5 200 0 3 0 1.5 0 0.4 0 0.425 0.4 200 0 0 0 0 200 0 0 0 0 200 0 1 0 0.5 200 0 0 0 0 2 5 1 200 1 0 0.5 0 0.7 0 1.075 0 0.7 200 0 0 0 0 200 0 0 0 0 200 5 0 2.5 0 200 1 0 0.5 0 2 200 0 0 0 0 0.1 0 0.05 0 0.1 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 3 200 0 0 0 0 0.1 0 0.05 0 0.1 200 0 0 0 0 200 0 0 0 0 200 0 0 0 0 200 1 0 0.5 0 4 200 0 0 0 0 0.2 0 0.075 0 0.2 200 0 0 0 0 200 1 0 0.5 0 200 0 0 0 0 200 1 0 0.5 0 5 200 1 0 0.5 0 0.5 0 0 0 0.5 200 1 0 0.5 0 200 1 0 0.5 0 200 1 0 0.5 0 200 1 0 0.5 0 3 1 1 400 200 0 50 0 50 0 0 0 50 50 50 50 50 1.3 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 50 50 50 10.8 0.65 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 14 2 3.5 0.5 2.1 1 1.01875 0.25 3.1 3.1 2.55 0.95 0.4 0.35 400 8 6 2 1.5 400 3 4 0.75 1 400 7 6 1.75 1.5 400 10 2 2.5 0.5 4 400 2 2 0.5 0.5 1.5 1.3 0.96875 0.7 2.8 2.8 0.85 0.65 0.4 0.55 400 3 2 0.75 0.5 400 7 6 1.75 1.5 400 6 6 1.5 1.5 400 12 10 3 2.5 5 400 4 3 1 0.75 0.5 0.85 0.25 0.39375 1.35 1.35 1.5 0.25 0.1 0.25 400 0 6 0 1.5 400 4 6 1 1.5 400 2 1 0.5 0.25 400 0 1 0 0.25 3 2 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 6 3 1.5 0.75 1.9 0.65 0.7375 0.33125 2.55 400 9 6 2.25 1.5 400 13 3 3.25 0.75 400 5 1 1.25 0.25 400 5 0 1.25 0 4 400 1 5 0.25 1.25 0.35 0.5 0.08125 0.25 0.85 400 1 1 0.25 0.25 400 3 0 0.75 0 400 2 3 0.5 0.75 400 0 1 0 0.25 5 400 3 6 0.75 1.5 0.55 0.95 0.10625 0.325 1.5 400 4 1 1 0.25 400 1 6 0.25 1.5 400 2 4 0.5 1 400 1 2 0.25 0.5 3 3 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 3 400 3 0 0.75 0 0.85 0.1 0.39375 0.01875 0.95 400 0 1 0 0.25 400 7 0 1.75 0 400 4 1 1 0.25 400 3 0 0.75 0 4 400 0 0 0 0 0.4 0.25 0.1125 0.09375 0.65 400 3 1 0.75 0.25 400 1 3 0.25 0.75 400 3 0 0.75 0 400 1 1 0.25 0.25 5 400 2 0 0.5 0 0.2 0.05 0.04375 0.0125 0.25 400 1 0 0.25 0 400 1 0 0.25 0 400 0 1 0 0.25 400 0 0 0 0 3 4 1 400 200 0 50 0 50 0 0 0 50 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 400 200 0 50 0 2 400 200 0 50 0 10.75 0.05 481.875 0.0125 10.8 400 4 1 1 0.25 400 8 0 2 0 400 2 0 0.5 0 400 1 0 0.25 0 3 400 0 0 0 0 0.35 0.05 0.3 0.0125 0.4 400 0 0 0 0 400 0 0 0 0 400 5 0 1.25 0 400 2 1 0.5 0.25 4 400 0 0 0 0 0.2 0.2 0.04375 0.2 0.4 400 2 4 0.5 1 400 0 0 0 0 400 1 0 0.25 0 400 1 0 0.25 0 5 400 0 1 0 0.25 0.05 0.05 0.0125 0.0125 0.1 400 0 0 0 0 400 0 0 0 0 400 0 0 0 0 400 1 0 0.25 0 3 5 1 400 14 0 3.5 0 1.25 0.05 2.03125 0.0125 1.3 400 7 0 1.75 0 400 0 1 0 0.25 400 1 0 0.25 0 400 3 0 0.75 0 2 400 3 0 0.75 0 0.65 0 0.08125 0 0.65 400 3 0 0.75 0 400 2 0 0.5 0 400 1 0 0.25 0 400 4 0 1 0 3 400 2 0 0.5 0 0.3 0.05 0.075 0.0125 0.35 400 2 1 0.5 0.25 400 2 0 0.5 0 400 0 0 0 0 400 0 0 0 0 4 400 0 0 0 0 0.35 0.2 0.05 0.04375 0.55 400 1 0 0.25 0 400 2 1 0.5 0.25 400 2 2 0.5 0.5 400 2 1 0.5 0.25 5 400 0 1 0 0.25 0.15 0.1 0.1125 0.01875 0.25 400 3 0 0.75 0 400 0 1 0 0.25 400 0 0 0 0 400 0 0 0 0 4 1 1 800 400 0 50 0 50 0 0 0 50 50 50 50 50 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 50 50 50 50 1.7 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 400 0 50 0 50 0 0 0 50 50 50 50 0.9 0.675 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 4 800 400 0 50 0 50 0 0 0 50 50 50 1.275 0.4 0.4 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 5 800 25 15 3.125 1.875 2.3 2.175 0.5515625 0.4578125 4.475 4.475 2.275 0.85 0.5 0.2 800 15 21 1.875 2.625 800 22 17 2.75 2.125 800 20 24 2.5 3 800 10 10 1.25 1.25 4 2 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 7 14 0.875 1.75 0.825 1.45 0.1375 0.1296875 2.275 800 2 14 0.25 1.75 800 6 11 0.75 1.375 800 10 7 1.25 0.875 800 8 12 1 1.5 4 3 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 4 800 5 1 0.625 0.125 0.875 0.4 0.1875 0.0421875 1.275 800 2 4 0.25 0.5 800 8 2 1 0.25 800 10 4 1.25 0.5 800 10 5 1.25 0.625 5 800 1 8 0.125 1 0.3 0.55 0.0203125 0.0828125 0.85 800 3 5 0.375 0.625 800 2 2 0.25 0.25 800 4 3 0.5 0.375 800 2 4 0.25 0.5 4 4 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 3 800 6 1 0.75 0.125 0.875 0.025 0.2734375 0.003125 0.9 800 13 0 1.625 0 800 9 0 1.125 0 800 2 0 0.25 0 800 5 0 0.625 0 4 800 2 4 0.25 0.5 0.225 0.175 0.034375 0.04375 0.4 800 4 2 0.5 0.25 800 2 0 0.25 0 800 0 1 0 0.125 800 1 0 0.125 0 5 800 2 4 0.25 0.5 0.225 0.275 0.0109375 0.034375 0.5 800 1 2 0.125 0.25 800 2 0 0.25 0 800 1 3 0.125 0.375 800 3 2 0.375 0.25 4 5 1 800 400 0 50 0 50 0 0 0 50 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 800 400 0 50 0 2 800 12 0 1.5 0 1.625 0.075 0.4609375 0.0125 1.7 800 15 1 1.875 0.125 800 13 0 1.625 0 800 5 0 0.625 0 800 20 2 2.5 0.25 3 800 3 2 0.375 0.25 0.525 0.15 0.1203125 0.0265625 0.675 800 0 1 0 0.125 800 5 0 0.625 0 800 6 3 0.75 0.375 800 7 0 0.875 0 4 800 1 0 0.125 0 0.25 0.15 0.0625 0.0421875 0.4 800 5 1 0.625 0.125 800 3 1 0.375 0.125 800 0 0 0 0 800 1 4 0.125 0.5 5 800 2 0 0.25 0 0.125 0.075 0.0078125 0.0125 0.2 800 1 0 0.125 0 800 0 1 0 0.125 800 1 2 0.125 0.25 800 1 0 0.125 0 5 1 1 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 50 50 50 50 1.2875 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 50 50 50 1.7875 1.125 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 800 0 50 0 50 0 0 0 50 50 50 1.7625 1.375 1.125 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 2 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 3 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 5 1600 7 12 0.4375 0.75 0.825 0.9375 0.166796875 0.01953125 1.7625 1600 22 16 1.375 1 1600 11 18 0.6875 1.125 1600 8 15 0.5 0.9375 1600 18 14 1.125 0.875 5 4 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 4 1600 24 6 1.5 0.375 1.3625 0.425 0.283984375 0.030078125 1.7875 1600 11 11 0.6875 0.6875 1600 16 8 1 0.5 1600 25 5 1.5625 0.3125 1600 33 4 2.0625 0.25 5 1600 9 5 0.5625 0.3125 0.625 0.75 0.095703125 0.0703125 1.375 1600 6 15 0.375 0.9375 1600 11 14 0.6875 0.875 1600 6 15 0.375 0.9375 1600 18 11 1.125 0.6875 5 5 1 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 2 1600 800 0 50 0 50 0 0 0 50 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 1600 800 0 50 0 3 1600 26 1 1.625 0.0625 1.1125 0.175 0.1296875 0.0125 1.2875 1600 18 3 1.125 0.1875 1600 20 5 1.25 0.3125 1600 11 4 0.6875 0.25 1600 14 1 0.875 0.0625 4 1600 18 3 1.125 0.1875 0.925 0.2 0.09453125 0.0046875 1.125 1600 15 3 0.9375 0.1875 1600 9 2 0.5625 0.125 1600 21 3 1.3125 0.1875 1600 11 5 0.6875 0.3125 5 1600 5 6 0.3125 0.375 0.475 0.65 0.028515625 0.1046875 1.125 1600 6 4 0.375 0.25 1600 7 16 0.4375 1 1600 12 12 0.75 0.75 1600 8 14 0.5 0.875 Sheet1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sheet2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Sheet3 c s I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0 0 0 0 0 0 0 2 100 0 1 0 1 0 0.4 0 0.3 0.4 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0.8 0 0.7 0.8 5 100 0 0 0 0 0 0 0 0 0 1 2 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 1 3 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0.2 0 0.2 0.2 3 100 0 0 0 0 0.2 0 0.2 0 0.2 4 100 0 0 0 0 0 0.2 0 0.2 0.2 5 100 0 0 0 0 0 0.2 0 0.2 0.2 1 4 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0 0 0 0 0 5 100 0 0 0 0 0 0 0 0 0 1 5 1 100 0 0 0 0 0 0 0 0 0 2 100 0 0 0 0 0 0 0 0 0 3 100 0 0 0 0 0 0 0 0 0 4 100 0 0 0 0 0.2 0 0.2 0 0.2 5 100 0 0 0 0 0 0 0 0 0 2 1 1 200 100 0 50 0 50 0 0 0 50 2 200 7 0 3.5 0 2.5 0.9 6.625 0.55 3.4 3 200 0 0 0 0 0.9 0.6 0.675 1.175 1.5 4 200 2 3 1 1.5 0.4 0.9 0.175 0.425 1.3 5 200 0 1 0 0.5 0.9 0.6 1.05 0.175 1.5 2 2 1 200 2 0 1 0 4.1 0.1 11.3 0.05 4.2 2 200 2 1 1 0.5 1.5 0.2 2 0.075 1.7 3 200 1 1 0.5 0.5 0.9 0.3 0.425 0.075 1.2 4 200 0 1 0 0.5 0.3 0.3 0.45 0.075 0.6 5 200 0 0 0 0 0 0.3 0 0.075 0.3 2 3 1 200 7 0 3.5 0 1 0 2.125 0 1 2 200 0 0 0 0 1 0 1 0 1 3 200 0 1 0 0.5 0.1 0.1 0.05 0.05 0.2 4 200 0 0 0 0 0.1 0.1 0.05 0.05 0.2 5 200 0 0 0 0 0.2 0.1 0.2 0.05 0.3 2 4 1 200 0 0 0 0 0.8 0 0.45 0 0.8 2 200 2 0 1 0 0.5 0 0.25 0 0.5 3 200 0 0 0 0 0 0 0 0 0 4 200 0 0 0 0 0 0 0 0 0 5 200 0 3 0 1.5 0 0.4 0 0.425 0.4 2 5 1 200 1 0 0.5 0 0.7 0 1.075 0 0.7 2 200 0 0 0 0 0.1 0 0.05 0 0.1 3 200 0 0 0 0 0.1 0 0.05 0 0.1 4 200 0 0 0 0 0.2 0 0.075 0 0.2 5 200 1 0 0.5 0 0.5 0 0 0 0.5 3 1 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 14 2 3.5 0.5 2.1 1 1.01875 0.25 3.1 4 400 2 2 0.5 0.5 1.5 1.3 0.96875 0.7 2.8 5 400 4 3 1 0.75 0.5 0.85 0.25 0.39375 1.35 3 2 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 6 3 1.5 0.75 1.9 0.65 0.7375 0.33125 2.55 4 400 1 5 0.25 1.25 0.35 0.5 0.08125 0.25 0.85 5 400 3 6 0.75 1.5 0.55 0.95 0.10625 0.325 1.5 3 3 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 50 0 0 0 50 3 400 3 0 0.75 0 0.85 0.1 0.39375 0.01875 0.95 4 400 0 0 0 0 0.4 0.25 0.1125 0.09375 0.65 5 400 2 0 0.5 0 0.2 0.05 0.04375 0.0125 0.25 3 4 1 400 200 0 50 0 50 0 0 0 50 2 400 200 0 50 0 10.75 0.05 481.875 0.0125 10.8 3 400 0 0 0 0 0.35 0.05 0.3 0.0125 0.4 4 400 0 0 0 0 0.2 0.2 0.04375 0.2 0.4 5 400 0 1 0 0.25 0.05 0.05 0.0125 0.0125 0.1 3 5 1 400 14 0 3.5 0 1.25 0.05 2.03125 0.0125 1.3 2 400 3 0 0.75 0 0.65 0 0.08125 0 0.65 3 400 2 0 0.5 0 0.3 0.05 0.075 0.0125 0.35 4 400 0 0 0 0 0.35 0.2 0.05 0.04375 0.55 5 400 0 1 0 0.25 0.15 0.1 0.1125 0.01875 0.25 4 1 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 400 0 50 0 50 0 0 0 50 5 800 25 15 3.125 1.875 2.3 2.175 0.5515625 0.4578125 4.475 4 2 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 400 0 50 0 50 0 0 0 50 5 800 7 14 0.875 1.75 0.825 1.45 0.1375 0.1296875 2.275 4 3 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 400 0 50 0 50 0 0 0 50 4 800 5 1 0.625 0.125 0.875 0.4 0.1875 0.0421875 1.275 5 800 1 8 0.125 1 0.3 0.55 0.0203125 0.0828125 0.85 4 4 1 800 400 0 50 0 50 0 0 0 50 2 800 400 0 50 0 50 0 0 0 50 3 800 6 1 0.75 0.125 0.875 0.025 0.2734375 0.003125 0.9 4 800 2 4 0.25 0.5 0.225 0.175 0.034375 0.04375 0.4 5 800 2 4 0.25 0.5 0.225 0.275 0.0109375 0.034375 0.5 4 5 1 800 400 0 50 0 50 0 0 0 50 2 800 12 0 1.5 0 1.625 0.075 0.4609375 0.0125 1.7 3 800 3 2 0.375 0.25 0.525 0.15 0.1203125 0.0265625 0.675 4 800 1 0 0.125 0 0.25 0.15 0.0625 0.0421875 0.4 5 800 2 0 0.25 0 0.125 0.075 0.0078125 0.0125 0.2 5 1 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 800 0 50 0 50 0 0 0 50 5 2 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 800 0 50 0 50 0 0 0 50 5 3 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 800 0 50 0 50 0 0 0 50 5 1600 7 12 0.4375 0.75 0.825 0.9375 0.166796875 0.01953125 1.7625 5 4 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 800 0 50 0 50 0 0 0 50 4 1600 24 6 1.5 0.375 1.3625 0.425 0.283984375 0.030078125 1.7875 5 1600 9 5 0.5625 0.3125 0.625 0.75 0.095703125 0.0703125 1.375 5 5 1 1600 800 0 50 0 50 0 0 0 50 2 1600 800 0 50 0 50 0 0 0 50 3 1600 26 1 1.625 0.0625 1.1125 0.175 0.1296875 0.0125 1.2875 4 1600 18 3 1.125 0.1875 0.925 0.2 0.09453125 0.0046875 1.125 5 1600 5 6 0.3125 0.375 0.475 0.65 0.028515625 0.1046875 1.125 s c I cases fPos fNeg fPosPer fNegPer fPosPerAvg fNegPerAvg fPosPerVar fNegPerVar fPerAvg 1 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Positive: S=1 1 1 2 100 0 1 0.0 1.0 0.0 0.4 0.0 0.3 0.4 C=1 C=2 C=3 C=4 C=5 1 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=1 0.0 50.0 50.0 50.0 50.0 1 1 4 100 0 0 0.0 0.0 0.0 0.8 0.0 0.7 0.8 I=2 0.0 2.5 50.0 50.0 50.0 1 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=3 0.0 0.9 2.1 50.0 50.0 1 2 1 200 100 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.4 1.5 50.0 50.0 1 2 2 200 7 0 3.5 0.0 2.5 0.9 6.6 0.6 3.4 I=5 0.0 0.9 0.5 2.3 50.0 1 2 3 200 0 0 0.0 0.0 0.9 0.6 0.7 1.2 1.5 1 2 4 200 2 3 1.0 1.5 0.4 0.9 0.2 0.4 1.3 1 2 5 200 0 1 0.0 0.5 0.9 0.6 1.1 0.2 1.5 1 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 3 3 400 14 2 3.5 0.5 2.1 1.0 1.0 0.3 3.1 1 3 4 400 2 2 0.5 0.5 1.5 1.3 1.0 0.7 2.8 1 3 5 400 4 3 1.0 0.8 0.5 0.9 0.3 0.4 1.4 1 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Positive: S=1 : S=2 1 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 1 4 4 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 4.1 50.0 50.0 50.0 1 4 5 800 25 15 3.1 1.9 2.3 2.2 0.6 0.5 4.5 I=2 0.0 1.5 50.0 50.0 50.0 1 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.9 1.9 1.9 50.0 1 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.3 0.4 0.4 50.0 1 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.0 0.6 0.6 0.8 1 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 1 5 5 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2 1 2 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 2 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2 1 4 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Positive: S=3 2 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 C=1 C=2 C=3 C=4 C=5 2 2 1 200 2 0 1.0 0.0 4.1 0.1 11.3 0.1 4.2 I=1 0.0 1.0 50.0 50.0 50.0 2 2 2 200 2 1 1.0 0.5 1.5 0.2 2.0 0.1 1.7 I=2 0.0 1.0 50.0 50.0 50.0 2 2 3 200 1 1 0.5 0.5 0.9 0.3 0.4 0.1 1.2 I=3 0.2 0.1 0.9 50.0 50.0 2 2 4 200 0 1 0.0 0.5 0.3 0.3 0.5 0.1 0.6 I=4 0.0 0.1 0.4 0.9 50.0 2 2 5 200 0 0 0.0 0.0 0.0 0.3 0.0 0.1 0.3 I=5 0.0 0.2 0.2 0.3 0.8 2 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 3 3 400 6 3 1.5 0.8 1.9 0.7 0.7 0.3 2.6 2 3 4 400 1 5 0.3 1.3 0.4 0.5 0.1 0.3 0.9 2 3 5 400 3 6 0.8 1.5 0.6 1.0 0.1 0.3 1.5 2 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Positive: S=4 2 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 2 4 4 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.8 50.0 50.0 50.0 2 4 5 800 7 14 0.9 1.8 0.8 1.5 0.1 0.1 2.3 I=2 0.0 0.5 10.8 50.0 50.0 2 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.0 0.4 0.9 50.0 2 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.0 0.0 0.2 0.2 1.4 2 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.0 0.1 0.2 0.6 2 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 2 5 5 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3 1 2 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 3 1 3 100 0 0 0.0 0.0 0.2 0.0 0.2 0.0 0.2 3 1 4 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 False Positive: S=5 3 1 5 100 0 0 0.0 0.0 0.0 0.2 0.0 0.2 0.2 C=1 C=2 C=3 C=4 C=5 3 2 1 200 7 0 3.5 0.0 1.0 0.0 2.1 0.0 1.0 I=1 0.0 0.7 1.3 50.0 50.0 3 2 2 200 0 0 0.0 0.0 1.0 0.0 1.0 0.0 1.0 I=2 0.0 0.1 0.7 1.6 50.0 3 2 3 200 0 1 0.0 0.5 0.1 0.1 0.1 0.1 0.2 I=3 0.0 0.1 0.3 0.5 1.1 3 2 4 200 0 0 0.0 0.0 0.1 0.1 0.1 0.1 0.2 I=4 0.2 0.2 0.4 0.3 0.9 3 2 5 200 0 0 0.0 0.0 0.2 0.1 0.2 0.1 0.3 I=5 0.0 0.5 0.2 0.1 0.5 3 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 3 2 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 3 3 400 3 0 0.8 0.0 0.9 0.1 0.4 0.0 1.0 3 3 4 400 0 0 0.0 0.0 0.4 0.3 0.1 0.1 0.7 3 3 5 400 2 0 0.5 0.0 0.2 0.1 0.0 0.0 0.3 3 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 3 4 3 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=1 3 4 4 800 5 1 0.6 0.1 0.9 0.4 0.2 0.0 1.3 C=1 C=2 C=3 C=4 C=5 3 4 5 800 1 8 0.1 1.0 0.3 0.6 0.0 0.1 0.9 I=1 0.0 0.0 0.0 0.0 0.0 3 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=2 0.4 0.9 0.0 0.0 0.0 3 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=3 0.0 0.6 1.0 0.0 0.0 3 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=4 0.8 0.9 1.3 0.0 0.0 3 5 4 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=5 0.0 0.6 0.9 2.2 0.0 3 5 5 1600 7 12 0.4 0.8 0.8 0.9 0.2 0.0 1.8 4 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 2 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 4 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 2 1 200 0 0 0.0 0.0 0.8 0.0 0.5 0.0 0.8 4 2 2 200 2 0 1.0 0.0 0.5 0.0 0.3 0.0 0.5 4 2 3 200 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4 2 4 200 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 False Negative: S=2 : S=2 4 2 5 200 0 3 0.0 1.5 0.0 0.4 0.0 0.4 0.4 C=1 C=2 C=3 C=4 C=5 4 3 1 400 200 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.1 0.0 0.0 0.0 4 3 2 400 200 0 50.0 0.0 10.8 0.1 481.9 0.0 10.8 I=2 0.2 0.2 0.0 0.0 0.0 4 3 3 400 0 0 0.0 0.0 0.4 0.1 0.3 0.0 0.4 I=3 0.0 0.3 0.7 0.0 0.0 4 3 4 400 0 0 0.0 0.0 0.2 0.2 0.0 0.2 0.4 I=4 0.0 0.3 0.5 0.0 0.0 4 3 5 400 0 1 0.0 0.3 0.1 0.1 0.0 0.0 0.1 I=5 0.0 0.3 1.0 1.5 0.0 4 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 4 4 2 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 4 4 3 800 6 1 0.8 0.1 0.9 0.0 0.3 0.0 0.9 4 4 4 800 2 4 0.3 0.5 0.2 0.2 0.0 0.0 0.4 4 4 5 800 2 4 0.3 0.5 0.2 0.3 0.0 0.0 0.5 4 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=3 4 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 4 5 3 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 I=1 0.0 0.0 0.0 0.0 0.0 4 5 4 1600 24 6 1.5 0.4 1.4 0.4 0.3 0.0 1.8 I=2 0.2 0.0 0.0 0.0 0.0 4 5 5 1600 9 5 0.6 0.3 0.6 0.8 0.1 0.1 1.4 I=3 0.0 0.1 0.1 0.0 0.0 5 1 1 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=4 0.2 0.1 0.3 0.4 0.0 5 1 2 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 I=5 0.2 0.1 0.1 0.6 0.9 5 1 3 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5 1 4 100 0 0 0.0 0.0 0.2 0.0 0.2 0.0 0.2 5 1 5 100 0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5 2 1 200 1 0 0.5 0.0 0.7 0.0 1.1 0.0 0.7 5 2 2 200 0 0 0.0 0.0 0.1 0.0 0.1 0.0 0.1 5 2 3 200 0 0 0.0 0.0 0.1 0.0 0.1 0.0 0.1 5 2 4 200 0 0 0.0 0.0 0.2 0.0 0.1 0.0 0.2 False Negative: S=4 5 2 5 200 1 0 0.5 0.0 0.5 0.0 0.0 0.0 0.5 C=1 C=2 C=3 C=4 C=5 5 3 1 400 14 0 3.5 0.0 1.3 0.1 2.0 0.0 1.3 I=1 0.0 0.0 0.0 0.0 0.0 5 3 2 400 3 0 0.8 0.0 0.7 0.0 0.1 0.0 0.7 I=2 0.0 0.0 0.1 0.0 0.0 5 3 3 400 2 0 0.5 0.0 0.3 0.1 0.1 0.0 0.4 I=3 0.0 0.0 0.1 0.0 0.0 5 3 4 400 0 0 0.0 0.0 0.4 0.2 0.1 0.0 0.6 I=4 0.0 0.0 0.2 0.2 0.4 5 3 5 400 0 1 0.0 0.3 0.2 0.1 0.1 0.0 0.3 I=5 0.0 0.4 0.1 0.3 0.8 5 4 1 800 400 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 5 4 2 800 12 0 1.5 0.0 1.6 0.1 0.5 0.0 1.7 5 4 3 800 3 2 0.4 0.3 0.5 0.2 0.1 0.0 0.7 5 4 4 800 1 0 0.1 0.0 0.3 0.2 0.1 0.0 0.4 5 4 5 800 2 0 0.3 0.0 0.1 0.1 0.0 0.0 0.2 5 5 1 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 False Negative: S=5 5 5 2 1600 800 0 50.0 0.0 50.0 0.0 0.0 0.0 50.0 C=1 C=2 C=3 C=4 C=5 5 5 3 1600 26 1 1.6 0.1 1.1 0.2 0.1 0.0 1.3 I=1 0.0 0.0 0.1 0.0 0.0 5 5 4 1600 18 3 1.1 0.2 0.9 0.2 0.1 0.0 1.1 I=2 0.0 0.0 0.0 0.1 0.0 5 5 5 1600 5 6 0.3 0.4 0.5 0.7 0.0 0.1 1.1 I=3 0.0 0.0 0.1 0.2 0.2 I=4 0.0 0.0 0.2 0.2 0.2 I=5 0.0 0.0 0.1 0.1 0.7 Total: S=1 C=1 C=2 C=3 C=4 C=5 I=1 0.0 50.0 50.0 50.0 50.0 I=2 0.4 3.4 50.0 50.0 50.0 I=3 0.0 1.5 3.1 50.0 50.0 I=4 0.8 1.3 2.8 50.0 50.0 I=5 0.0 1.5 1.4 4.5 50.0 Total: S=2 C=1 C=2 C=3 C=4 C=5 I=1 0.0 4.2 50.0 50.0 50.0 I=2 0.2 1.7 50.0 50.0 50.0 I=3 0.0 1.2 2.6 50.0 50.0 I=4 0.0 0.6 0.9 50.0 50.0 I=5 0.0 0.3 1.5 2.3 50.0 Total: S=3 C=1 C=2 C=3 C=4 C=5 I=1 0.0 1.0 50.0 50.0 50.0 I=2 0.2 1.0 50.0 50.0 50.0 I=3 0.2 0.2 1.0 50.0 50.0 I=4 0.2 0.2 0.7 1.3 50.0 I=5 0.2 0.3 0.3 0.9 1.8 Total: S=4 C=1 C=2 C=3 C=4 C=5 I=1 0.0 0.8 50.0 50.0 50.0 I=2 0.0 0.5 10.8 50.0 50.0 I=3 0.0 0.0 0.4 0.9 50.0 I=4 0.0 0.0 0.4 0.4 1.8 I=5 0.0 0.4 0.1 0.5 1.4 Total: S=5 C=1 C=2 C=3 C=4 C=5 I=1 0.0 0.7 1.3 50.0 50.0 I=2 0.0 0.1 0.7 1.7 50.0 I=3 0.0 0.1 0.4 0.7 1.3 I=4 0.2 0.2 0.6 0.4 1.1 I=5 0.0 0.5 0.3 0.2 1.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 C False Pos Percent Avg False Positive: S=1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 False Negative: S = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 Total: S=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I=1 I=2 I=3 I=4 I=5 0.0 10.0 20.0 30.0 40.0 50.0 60.0 C=1C=2C=3C=4C=5 C I=1 I=2 I=3 I=4 I=5 0.0 10.0 20.0 30.0 40.0 50.0 60.0 C=1C=2C=3C=4C=5 I=1 I=2 I=3 I=4 I=5 0 5 10 15 20 25 30 35 40 45 50 s=1s=5All Subs = 50 I=1 I=2 I=3 I=4 I=5 0 5 10 15 20 25 30 35 40 45 50 C5.0MLPSVM I=1 I=2 I=3 I=4 I=5 0 5 10 15 20 25 30 35 40 45 50 Imb.R.OvD.OvR.UnD.UnCost I=1 I=2 I=3 I=4 I=5 0 5 10 15 20 25 30 35 Helicop.PromotersSonars RMLP DMLP C4.5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 B=1B=2B=0.5 Adaboost Mixture-of- Experts