Introduction to Testing and NUnit
Cpt S 321 Washington State University
Testing – what is this all about?
• Testing
• Finding inputs that cause the software to fail.
• Concentrates on the product (i.e., the software); testing versus Quality Assurance (QA).
• Automated, repeatable, systematic
• Exhaustive testing:
• Can we do it?
• Do we want to do it?
• Dijkstra, 1972: “Program testing can be used to show the presence of bugs, but never to show their absence”
• No absolute certainty can be gained from testing
• Must be integrated with other activities such as code reviews and quality assurance. More on code reviews later in the semester.
Testing – what does it take?
• Oracle, i.e., the gold set: a set of data to compare the output against
• Specification (when available)
• Implementation (when available)
• Different types of testing:
• Ex. white-box versus black-box
• Ex. unit testing versus integration testing versus system testing
Three simple steps when testing
1. Setupanythingthatneedstobesetup
• Not always necessary
• Can be done in the ‘setup’ method if the setting is common to all tests or in the beginning of the test otherwise
2. Callthemethodthatyouwanttotest
3. Checkiftheresultsobtainedfromcallingthemethod match the expected results.
We use assertion statements for that.
Note: Steps 1 and 2 are typically what you would do in a normal program.
What are assertions?
• The Classic Model allows you to use the assert statements
defined in the NUnit.Framework.Assert class, for example:
• Assert.True
• Assert.False
• Assert.Null
• Assert.NotNull
• Assert.IsNotEmpty • Assert.AreEqual
• Assert.AreNotEqual • Assert.AreSame
• Assert.AreNotSame • Assert.Contains
• Assert.Greater
• Assert.Less
• Assert.IsInstanceOf
• Assert.IsNotInstanceOf • Assert.Throws
•…
Note: See also StringAssert, CollectionAssert, FileAssert, DirectoryAssert classes.
How does a test file look like?
Attributes that signals what the entities is to NUnit
Target method
Assertions (cont.)
• Constraint Model
• Using the Assert.That method • And specifying constraints
• Ex.: Assert.That(myString, Is.EqualTo(“Hello”));
• Examples of constraints: • Is.EqualTo
• Is.Not.EqualTo
• Is.All.InstanceOf
•…
• Check here for a quick set of examples
NUnit – let’s get started (Tasks 1-6)
• First, install Nunit (you can also check the installation manual on BB).
1. Create a new NUnit Test Project HelloWorldTests in the same solution
as our HelloWorld project.
2. We want to test a Math class that we will implement in namespace HelloWorld.Math. To this end, create the class Math and a method Add. Let’s not worry about the algorithm yet:
public static int Add(int a, int b){return 0;}
3. Create a new test file and call it TestMath; define it in namespace HelloWorld.Math.Tests. (You can use the example from the previous slides as a starting point.)
4. Add the HelloWorld project as a reference to your HelloWorldTests. (Expand HelloWorldTests and right-click on Dependencies -> Add Project Reference…)
5. Create a test method for method Add; call it TestAdd and add assertion statements to test method Add. What should we test?
6. Run the tests: they should ALL (or almost all) FAIL!
What did just happen?
• We created the skeletal code for a class Math and a method Add without implementing any algorithm yet.
• The method body could/should have been empty if the return type was void.
• When there is a return type other than void, some sort of default implementation is
needed in order for the code to compile.
• We created a class TestMath that we will use to test class Math.
• We created a method TestAdd to test method Add.
• We tested method Add! I.e., we prove that the method is not working properly! • Next, we can start working on the implementation of method Add until our
tests pass.
• This process is called Test Driven Development (TDD)
What is Test Driven Development (TDD)?
User Story
Story Tasks
No
Story Test Cases
Run Tests
Fail
“Just Enough” Code
Pass
Refactor?
Yes
Refactor Existing code
https://www.refactoring.com/catalog/
TDD in this class
• For 1.
2.
3.
4.
5.
6. 7.
HWs and in-class exercises:
Write skeletal code and tests. Tests should FAIL!
Commit skeletal code and tests
Write implementation (i.e., the algorithms). ALL tests should PASS!
Commit the implementation
Refactor code if needed and commit. The number of passing tests should be the same, i.e., refactoring should not break your code!
Add more tests if needed and commit. If the tests in 6 do not pass: work on the
implementation and commit
Iterate multiple times over 3-5 if a feature is complex. In that case, in every iteration more tests should be passing!
=> I need to see at least 2 commits per feature/task that you decide to tackle.
Let’s practice TDD
• Implement method IsLeap in class Date by following TDD. Recall that:
• A year that is divisible by 4 is a leap year
• A year not divisible by 4 is a common year
• A century year not divisible by 400 is a common year • A century year divisible by 400 is a leap year
• Remember:
• Use the process from the previous slide!
White Box Testing and code coverage in VS
•How does the VS Code coverage work? • Check the documentation
• What is the coverage of our tests? • Date.IsLeap?
• Math.Add?
•What is a good coverage? • As high as possible (>80%)