CSc 352: Testing and Code Coverage
Testing and test cases
int main() {
read x;
if (x is odd) {
compute payroll data; }
else {
delete all files;
send rude email to boss; crash computer;
} }
make sure you use at least fifty different test inputs
1, 3, 5, 7, 9, 11, 13, 15, 17, 19, …
2
Testing and test cases
int main() {
read x;
if (x is odd) {
compute payroll data; }
else {
delete all files;
send rude email to boss; crash computer;
} }
make sure you use at least fifty different test inputs
1, 3, 5, 7, 9, 11, 13, 15, 17, 19, …
It isn’t enough to have a lot of test cases. We have to make sure our tests “cover” the program adequately.
3
gcov: a code coverage analyzer
• testcoverageprogram
– indicates how many times each line was executed
– marks code that did not get executed – cumulative over a set of tests
• helpsyouunderstand
– how effectively your current test cases “cover” the code
– what additional test inputs you need in order to get better coverage
• needs the program to be compiled with additional gcc options
4
An example
input
input input
input input
additional compiler flags
testing didn’t execute all of the code
5
An example
input
input input
input input
testing didn’t execute all of the code
6
An example
input
7
An example
input
8
An example
input
9
Code coverage and testing
• Just because every line has been executed does not mean the program has been tested thoroughly
– we may want to test the same line of code under different conditions
• e.g.: a loop should be tested with values that cause 0, 1, and “many” iterations
• However, if some lines are not executed the program is definitely not thoroughly tested
– gcov helps us identify and fix this
• exception: “system errors” that may be difficult to create
10
Example of not enough testing
This is (almost) the program we wrote in class to convert all lower case letters in a string to upper case.
11
Example of not enough testing
Compile and test it. 100% of code is executed and the result is correct. So the code has no bugs, right?
12
Example of not enough testing
There were still errors.
Compile and test it. 100% of code is executed and the result is correct. So the code has no bugs, right?
Oops!
13
gcov: summary
• codecoveragetestingtool
– works with gcc; needs additional compiler flags
• gcc –fprofile-arcs –ftest-coverage …
• shows execution frequency of each line of code
– reports % of lines covered by tests • coverage values are cumulative
• delete *.gcda file to start afresh
– how many times each line was executed – highlights lines not executed
14