DUMMY EXAM PAPER
BSc, MSci, MSc
6CCS3SMT, 7CCSMASE
Software Measurement and Testing, Advanced Software Engineering
January 2015 (Period 1)
Rubric ANSWER ALL THREE QUESTIONS.
Books, notes or other written material may not be brought into this
examination.
ANSWER EACH QUESTION IN A SEPARATE ANSWER BOOK AND WRITE ITS NUMBER ON THE COVER
Calculators Calculators may be used. The following models are permitted: Casio fx83 / Casio fx85
Degree Programmes
Module Code Module Title
Examination Period Time Allowed 2 hours
King’s College London © 2014
6CCS3SMT Software Measurement and Testing
Question 1 Part A
Give the definitions of three measurement scale types and their averages.
[7 marks]
Full answer (choose three of these): There are five types of scales:
• Nominal scale: (classification) – eg. blood groups, program language, colour
• Ordinal scale: (ordering) – eg. {Excellent, Very Good, Good, Satisfactory}
• Interval Scale: (quantifying differences) – eg. Date, temperature in Farhenheit or Centigrade
• Ratio scale: (ratios and zero are meaningful) – eg. Length, Mass, temperature in Kelvin (zero is meaningful)
• Absolute scale: (counting) – E.g. Number of faults in a system The averages:
• For Nominal – only Mode makes sense (the most frequently appearing value)
• For Ordinal – Median: The midpoint of the data, when it is arranged in increasing order. It divides the data into two equal parts.
• For Interval – we can use Arithmetic Mean (a1+a2+…+an)/n
• For Ratio – we can use either Arithmetic or Geometric mean
(geometric mean is the product of all values in the power 1/n)
2
SEE NEXT PAGE
6CCS3SMT Software Measurement and Testing • For Absolute scale – any average can be used.
Part B
What is commonly used as a measure of effectiveness of the testing process? Give two advantages and two disadvantages of this measure.
[7 marks]
Answer:
A possible answer – the number of faults found in a unit of time (day/week/month) is commonly used as a measure of effectiveness of the testing process.
Advantages:
– easy to use, if the faults are properly recorded
– indication of progress: if the number of faults found is reduced, we might hope that the software is becoming better
Disadvantages:
– there are many different quality requirements – some of them are easier to check than others; just counting the number of faults does not give an accurate picture
– some faults are more critical than others
– a reduced number of faults found can be due to a low quality of testing or due to a high quality of software
– the faults are unevenly distributed; the fact that there are no more faults found in a particular part of software says nothing about other parts
Part C
(a) What type of testing can we do for third-party components (when
the source code is unknown)? (4 marks)
3
SEE NEXT PAGE
6CCS3SMT Software Measurement and Testing
(b) If a source code is not known, but the control flow graph of the program is known, what type of testing can we do? Describe one suitable coverage criterion and ways to achieve maximal coverage. (4 marks)
(c) Is 100% statement coverage enough, in general? If yes, explain. If no, give an example (pseudo-code) where 100% statement coverage misses a fault. (4 marks)
(d) Give the definition of an appropriate coverage criterion for your test suite that ensures that the fault is caught.
(4 marks)
(e) Give a minimal testing suite that catches the fault in your example. (4 marks)
[20 marks]
Answer:
(a)We can do black-box testing – specification-based testing is suitable for checking third party components. In this type of testing, we construct a test suite that is based on the specification.
(b) We can do structural testing – to construct a test suite which checks the control flow graph exhaustively based on our coverage criteria. Common coverage criteria for the structural testing include statement coverage, branch coverage, condition coverage, MC/DC coverage (elaborate on the definitions). The way to achieve 100% state or statement coverage is to design the test suite that visits all states or statements.
(c) No, 100% state coverage is not enough. If not all branches are covered, we can miss a fault. Example: {x is a non-negative integer} if(x>0) x++; a = 1/x; In this pseudocode example, the control flow graph has a condition state x>0, if yes, then x++ is
4
SEE NEXT PAGE
6CCS3SMT Software Measurement and Testing
executed. In both cases, a is assigned to 1/x. Now consider the case where x=0. The condition x>0 is false, hence x stays 0. But now the program will crash on division by zero. This bug will not be revealed by the test x=1, even though it has 100% state coverage. (Draw a control flow diagram if you see a question like this in the exam).
(d) For this example, the appropriate coverage criterion is branch coverage. Since x is a non-negative integer, in order to cover all branches we need to consider the case where x>0 is true and the case where x>0 is false – hence x would be 0. The second case will catch the fault. Both test cases together will give a test suite with 100% branch coverage.
(e) A minimal test suite is {x=1; x=0}.
Question 2 Part A
(a) What is abstraction? Are all faults found in an abstract model real ones? (5 marks)
(b) Give the definition of Dataflow testing. What is it used for? Give one example of a fault that can be found using dataflow testing. (5 marks)
(c) Give the definition of regression testing. Where is it used and why is it important? (5 marks)
(d) Give the definition of combinatorial testing. Describe two possible coverage criteria for combinatorial testing. (5 marks)
[20 marks]
5
SEE NEXT PAGE
6CCS3SMT Software Measurement and Testing
Answer:
(a)Abstraction is a simplification of the model by ignoring some of the parameters. It is an over-approximation of the original program and contains more behaviours. Some of the behaviours can be spurious – that is, not existing in the original program.
(b) Dataflow testing is testing that is concerned with the flow of the data in the program. It is used to find errors that result from incorrect assignment or computation of data. One example: assume that x is an input to a function, and that y is later computed as, say, x+1 and is returned from the function. Assume that the value of y is an error (out of range). Dataflow testing construct definition- use pairs and will find a source of this error (either incorrect formula for y, or incorrect value of x given as an input).
(c)Regression testing is testing to make sure that the system did not lose any of its previous functionality after an upgrade. A regression test suite is maintained and updated after every bug fix or an update.
(d) Combinatorial testing tests a variety of possible combinations of values of variables. One possible (and widely used) coverage criterion is pairwise coverage – have all pairs of values been tested? This criterion can be extended to all triples, quadruples, etc. In the extreme, we test all possible combination of values. In practice, it is rarely possible, since the number of all possible combinations is exponential.
Part B
(a) Write a formal specification in linear temporal logic for the property φ = “always if (x>0) then sometimes in the future (y>0)”.
Draw an infinite execution (with a loop) that satisfies this property and
an infinite execution (with a loop) that is a counterexample. (7 marks)
6
SEE NEXT PAGE
(7 marks)
Answer:
[14 marks]
6CCS3SMT Software Measurement and Testing
(b) Write a formal specification in temporal logic for the property
π = “h (height) is always a non-negative number.” How is this type of property called? How does a counterexample to this property look like? Draw a counterexample and a trace that satisfies the property.
(a)φ = G( (x>0) -> F (y>0) ). An infinite execution that satisfies this property can be an execution where x is always less or equal to zero. Another option is an infinite execution where x>0 and y>0 occur on the loop (draw an execution if asked in the exam).
(b) π = G (h >= 0). This is an invariant. A counterexample to this property is a finite execution that ends in h<0. An infinite
execution that satisfies this property is an execution where h>=0 in all states (draw an execution if asked in the exam).
Question 3
Consider the following program.
1. 2. 3. 4. 5. 6. 7.
main() {
int x = input();
if (x > 0){ draw(circle);
}
7
SEE NEXT PAGE
8. 10. 11. 12. 13. 14. 15. 16. 17. 18.
else{
draw(triangle); }
if (x > 0){ draw(ellipse);
} else{
draw(square); }
return; }
Part A
(a) Draw the Control Flow Graph for the program. (b) What is the cyclomatic complexity of the graph?
Answer:
(5 marks) (4 marks)
[9 marks]
6CCS3SMT Software Measurement and Testing
(a)Draw a control flow graph. It will have a shape of two diamonds one following the other.
(b) The cyclomatic complexity is 3 (the number of decision points +1).
Part B
How many paths does the graph have? Do all of them correspond to feasible executions of the program? Give one test case for each feasible path and for each infeasible path explain why it is infeasible.
[10 marks]
Answer:
The graph has 4 paths. Only two of them correspond to feasible executions of the program. Two test cases: { x=1; x=-1 } cover both feasible paths. The rest of the paths are infeasible because the value of x is not changed inside the condition. Hence, either both if branches are taken, or both else branches are taken.
8
SEE NEXT PAGE
6CCS3SMT Software Measurement and Testing
Part C
(a) Is it always possible to achieve 100% statement coverage? Explain your answer. (7 marks)
(b) Is it always possible to achieve 100% branch coverage if 100% state coverage is achievable? Explain your answer. (7 marks)
[14 marks]
Answer:
(a)It is not always possible to achieve 100% statement coverage. The impossibility can be due to a dead code (code that is never executed) or defensive programming, where impossible conditions are checked for explicitly.
(b) No, it is not always possible to achieve 100% branch coverage even if 100% state coverage is achievable. Example: if(True) x++; a = 1/x; In this example, there are two branches from the if statement – one for True and one for False. However, the False branch is never taken, because the condition is always True.
9
S E E N EFXI NT APLA GP AE G E