Items that are intended to stay in as part of your document are in bold; explanatory comments are in italic text
Software Test Plan Pretty Good Project team (PGPt)
Software Test Plan
Video Stream Scheduling System
Client
Mike Dixon
School of Information Technology
Murdoch University
Pretty Good Project team (PGPt)
Supervisor: Shri Rai
Version 1.2
5/11/2004
Table of Contents
3Member Contribution
41 Introduction
52 Objectives
63 Testing Strategy
94 Scope
95 Reference Material
96 Definitions and Acronyms
107 Test Items
127.1 Program Modules
127.2 Job Control Procedures
137.3 User Procedures
137.4 Operator Procedures
138 Features to be tested
138.1 Database Software Feature
148.2 Dynamic Web Page Feature
158.3 Scheduler Feature
158.4 Administration GUI Feature
169 Features not tested
1610 Approach
1910.1 Component Testing
1910.2 Integration Testing
1910.3 Conversion Testing
1910.4 Job Stream Testing
1910.5 Interface Testing
1910.6 Security Testing
1910.7 Recovery Testing
2010.8 Performance Testing
2010.9 Regression Testing
2010.10 Acceptance Testing
2010.11 Beta Testing
2111 Pass/Fail Criteria
2111.1 Suspension Criteria
2111.2 Resumption Criteria
2211.3 Approval Criteria
2212 Testing Process
2412.1 Test Deliverables
2412.2 Testing Tasks
2512.3 Responsibilities
2612.4 Resources
2612.5 Schedule
2713 Environmental Requirements
2713.1 Hardware
2813.2 Software
2813.3 Security
2813.4 Tools
2913.5 Publications
2913.6 Risks and Assumptions
3014 Change Management Procedures
3115 Plan Approvals
3116 Configuration Management
Member Contribution
All sections of this document created by Michelle Lister. Formatting for final version by David Patullo
1 Introduction
The Software Test Plan (STP) is designed to prescribe the scope, approach, resources, and schedule of all testing activities. The plan must identify the items to be tested, the features to be tested, the types of testing to be performed, the personnel responsible for testing, the resources and schedule required to complete testing, and the risks associated with the plan.
This document is intended to assist the Pretty Good Project Team to adequately test the Video Scheduling System. The test plan provides guidance for the projects test strategy. Supporting documentation such as the Unit Test Plans and System Integration Plans will also be produced to assist in the testing of the system. The results of testing will be recorded in these documents and will form the basis of evidence needed to provide assurance that the video scheduling system is ready for User Acceptance Testing and Production Deployment.
The purpose of the project is to develop a system that allows users to view streaming video over a Local Area Network (LAN). The system will use the following existing open source technologies:
· VLC (Server and Client)
· MySQL (Database Management System)
This project will develop additional capability that will interface to these existing technologies. Its primary purpose will be to control the delivery of a video stream (multicast and unicast) to end-users. It will provide a GUI to allow administrators to schedule when programs are to be delivered and a viewing guide which users can view using a web browser.
The following capability will be developed by this project:
· MySQL database to store information about VLC servers, schedule of streaming sessions (similar to tv guide), location of video resources and user information for registered users.
· A Graphical User Interface (GUI) for use by system administrators.
· A web-based GUI for use by end-users.
· A scheduler to control video stream delivery from multiple servers.
The intended use of the system is as a LAN streaming solution for university lecture material.
2 Objectives
The project will endeavour to comprehensively test the Video Scheduling System. The project approach to testing will be to follow structured testing procedures these will include Unit Testing, Systems Integration Testing and User Acceptance Testing. The project team will be responsible for completing the Unit and Systems Integration testing phases and these stages will be completed by the end of October 2004. There are five human resources for this project – all members will be involved in some aspect of the testing.
The objectives of the testing phase are:
· To find defects and resolve problems,
· Ensure that all system modules fit together correctly,
· Configuration of the system meets user requirements,
· The data contained in the database is valid and correct,
· Verify that all Graphical User Interfaces are working appropriately,
· The system integrates well with the VLC server and client and the videos are streamed at the required times.
· Test system performance prior to user acceptance testing.
The products to be delivered contain the following components:
· Administrator GUI which is a user interface which will enable easy maintence for the database application.
· Dynamic Web Page to display Scheduling Information and request videos to be scheduled
· A Scheduler daemon which uses the information stored on the database to invoke the Video Lan Client.
· A relational database with a set of tables that holds information used by the three previous components, as the underlying data structure that contains all the data that each component requires
The above components will interface with other product and software components although the team will ensure that the components will work with this software detail testing of these products is outside of the scope of this project.
As each piece of software is developed, the developer will unit test the component. The entry and exit criteria of each phase will be described later in this document. There is a high importance that the unit testing for each component be completed by mid October once the exit criteria is met the Systems Integration testing will begin. Once the System Integration testing has been completed this will be passed to the Client for User Acceptance Testing.
Due to the strict timeline of this project there is no room for the team to correct major errors found in User Acceptance Testing therefore the other stages of testing need to be comprehensive and validated and verified against the user requirements.
3 Testing Strategy
Testing is the process of analysing a software item to detect the differences between existing and required conditions and to evaluate the features of the software item.
Software testing has become the most critical element in software quality assurance. Every component of the software passes through the distinct stages as per the Quality Life Cycle shown below:
Figure 1.1 Quality Life Cycle
The approach to development is to focus sharply on the first two stages of the above cycle, so that only a minimal number of defects reach the third stage of testing. The product has to be put through carefully planned test cycles to ensure quality. Testing only releases the defects but does not prove the absence of defects.
The following testing strategies will be utilized to test the video scheduling software system in the various stages of the software development lifecycle.
Verification: Refers to the set of activities that ensure that the software correctly implements a specific function, imposed at the start of that phase. Testing activity focuses on verifying the correct implementation of business requirement and customer requirement.
Validation: Refers to the set of activities, which ensure the software that has been built is traceable to customer requirements. Validation includes activities like Code-walkthroughs to ensure that the software conforms to set standards.
Unit testing: Is the first level of software testing. It addresses the scope of a single program, module, object, or method and does so from both an external functional (black-box) perspective and an internal technical (white-box) perspective. Unit testing is normally the responsibility of the individual who wrote the program, module, object, or method.
Integration Testing: Is the testing in which software units of an application are combined and tested for evaluating the interaction between them. Black box test case designs are most prevalent during integration, though white box testing techniques like Control flow graphing and Execution tracing are also carried out.
System Testing: Verifies each individual work product (such as application software or technical infrastructure) still meets requirements when integrated with the rest of the software and the business system. If the system as a whole meets its requirements, the system is likely to pass the subsequent acceptance testing. System testing includes the following types of tests: application recovery test, application performance test, security testing and performance testing. The team plans and executes system integration testing to meet these key objectives. However, an additional objective is to make certain that the system is ready and able to pass acceptance testing. Therefore, system testing should include tests similar or identical to those that will be used for subsequent acceptance testing.
Acceptance Testing: allows the system users or their designated representatives on the acceptance team to satisfy themselves that the release components have been built or configured to their specifications. It is a formal and thorough process carried out in accordance with an Acceptance Test Plan. Acceptance testing occurs in the development location, to demonstrate that the as-built components perform their intended functions. Acceptance testing may be performed in pieces, as specific products become ready. The system user is responsible for acceptance testing and although the Pretty Good Project Team is not directly responsible for acceptance testing our aim is for our software to meet the user requirements.
Test Level
Test Type
Development
· Unit Tests
System Integration
· Integration Test
· System Test
· Performance Test
Acceptance
· Acceptance Tests
Test Case documentation is carried out at the various stages of testing. The test data is used and refined during the various stages of testing of the product. Testing during the maintenance phase can now reuse the test data prepared and documented for every product feature during development.
The activities involved in the testing process of development and maintenance phases are illustrated below:
Figure 1.2 Quality Life Cycle
The test strategy will be applied to the Video Scheduling Software that the Pretty Good Project Team is producing.
4 Scope
Testing will be performed at several points in the life cycle as the product is constructed. Testing is a very ‘dependent’ activity. As a result, test planning is a continuing activity performed throughout the system development life cycle. Test plans must be developed for each level of product testing.
This test plan describes the Unit level tests and System Integration Tests. Although acceptance testing is discussed within this document this is not the responsibility of the Pretty Good Project Team to perform and is therefore not within the scope of the test plan. Operational tests and evaluation are out of scope of this test plan due to time constraints. In addition, the testing of subsequent releases of the Video Scheduling System is out of scope for this test plan.
5 Reference Material
· Project Management Plan
· User Requirements Document
· Design Documentation
· Unit Test Plan
· System Integration Test Plan
6 Definitions and Acronyms
The following definitions, acronyms, and abbreviations are being used within this Project definition.
Definitions, Acronyms and Abbreviations
GNU
General Public License (GPL) a general license for open-source software that has been developed in the open domain. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing the license is not allowed (patenting GNU software)
Multicast
With a multicast design, applications can send one copy of each packet and address it to the group of computers that want to receive it. This technique addresses packets to a group of receivers rather than to a single receiver, and it depends on the network to forward the packets to only the networks that need to receive them.
PGPt
Pretty Good Project Team – Team name and our group number is group 4
VLC
Video LAN Client is a complete software solution for video streaming. Video LAN is designed to stream MPEG videos on high bandwidth networks.
GUI
Graphical User Interface – a user interface (displayed as part of a program) which uses graphical images (pictures, icons. etc.) to interact with the end user
MySQL
A popular open source DBMS
DBMS
Database Management System – a software application to manage databases
SIT
System and Integration Test
System Test
Testing of an entire system to ensure that it meets its functional and performance requirements.
Test Plan
A Test Plan is a plan for uncovering defects in the functionality of software. The test plan document identifies test cases for various functional areas (test criteria).
Test Cases
Test Case is a collection of test sequences with expected outcomes for testing the identified functionality. Test sequences include sample test data to be used to uncover defects. The test case also lists the expected outcome for every test sequence, enabling the developer to verify execution of test cases as planned.
Unit Test
Testing of a component (e.g., a program) in isolation, i.e. by minimal interaction with other system components.
7 Test Items
The baseline testing for this project is determined by the functional needs of the system in the requirements and design specifications. Test Phase documents and the User Guides can be found in the current repository for system information. The documents are:
· Analysis Document
· User Requirements Document
· Software Requirements Document
· Design Document
· Technical and System User Guides
The Video Scheduling System contains the following items to be tested:
· Administrator application: This will allow the administrator to schedule when particular videos should be streamed across the network. This should be easy to navigate and intuitive to use and learn so that those not associated with the project can use the system. The administrator can add, delete and modify scheduling information stored in the database. The administrator should be able to add, delete students to restrict viewing to students enrolled in a particular unit. This will be created and in Java.
· Dynamic Web Page to display Scheduling Information: This page should display a ‘TV-guide’ style format displaying scheduling information that this student (unit) can view. The page should only display the streams that the student is able to view i.e. not the entire table, and the web page cannot modify data in the database. The web page will give details of the multicast stream, but will not actively connect the client, they must invoke the VLC on their pc manually (they must give the VLC the server address). The dynamic web page will be served from an Apache Web Server and written in Perl.
· Scheduler: A scheduling daemon which uses the information stored on the database to invoke the VLC on the server with the correct commands so that they desired stream is started at the correct time. The scheduler runs continuously checking with the database whether there are any tasks to invoke on the VLC. The Scheduler shall be written in C and shall reside on a Linux machine, as will the rest of the system.
· Database: A relational database structure with a set of tables that holds information used by the three previous components, an underlying data structure that contains all the data that each component requires, only the Administrator application will be able to change data in the database. This will be implemented in MYSQL software.
Technical and System user guides will be developed and all steps of these documents will need to be walked through to ensure the steps and/or information is valid.
Defect Identification and removal as well as Verification and validation items will be identified within the testing phase documentation.
7.1 Program Modules
The following outlines the program modules to be tested:
Module
Type
Title/Description
Database
Database
A MySQL relational database will act as the central repository of data for the system. It shall hold information regarding videos that can be streamed, users, streaming channels and other data. It shall reside on a single server in the system and will be used by the Administrator GUI, dynamic Web Page and Scheduler daemon described below. This database shall provide the underlying data structure of the system and will perform an integral role in the operation of all applications the project team will create.
Scheduler
Program
The scheduler is to initiate VLC on the server-end whenever a stream is scheduled to take place. For each time slot the database is queried and for each stream (on separate channels) scheduled for that slot, a separate VLC process is created/forked to deliver a multicast stream to the channels network destination
Administrator Graphical User Interface
Program
Screens
The Administrator GUI connects to a MySQL database via the menu and maintains this connection until the program is exited or the user chooses to disconnect. The Administration GUI sends SQL queries via database APIs and processes the results.
Dynamic Web Page
Program
Screens
The Administrator GUI connects to a MySQL database via the menu and maintains this connection until the program is exited or the user chooses to disconnect. The Administration GUI sends SQL queries via database APIs and processes the results.
7.2 Job Control Procedures
This section is not applicable.
7.3 User Procedures
Technical and StudentUser Guides will be produced for this project. We aim to have a member of the project team other than the author of the document will complete testing by walking through all the procedures contained in the document to ensure that they are clear, concise and correct.
7.4 Operator Procedures
This project will be tested from the university on machines with the same configuration and network as will be run in the production system. The project team will not have responsibility for setting up support features or bug fixes after the initial handover of the system to the client Mike Dixon.
8 Features to be tested
8.1 Database Software Feature
The database is the central repository for all data in the Video Scheduling System. This database shall provide the underlying data structure of the system. Each component of the system interacts with the database this will need to be extensively tested to ensure that the data is verified and correct.
The database will be tested in the unit and system integration phases of the project. The other combinations of software will be tested in the system integration testing stage. A listing of the Software Features to be tested with the database is listed below:
Combination Features for Database
Test Phase/Specification
Title/Description
Dynamic Web Page
System Integration
The MySQL database server will interact with the web interface, which will display data stored in the database, and to a limited extent, update data in the database.
Scheduler
System Integration
Generally the scheduler works by checking the database every half hour and starting a VLC process for each scheduled stream.
Administrator Graphical User Interface
System Integration
The Administrator GUI connects to a MySQL database via the menu and maintains this connection until the program is exited or the user chooses to disconnect. The Administration GUI sends SQL queries via database APIs and processes the results.
8.2 Dynamic Web Page Feature
The Dynamic web page is a graphical user interface to the users of the system and will contain a login page; this is verified with the details in the database. A main page will present available channels and time slots for users as well as videos that already have been scheduled. Users will be able to carry out four tasks from the main page:
· Change the date for which time slots are currently shown to any date up two 14 days into the future
· “Reserve” a movie against a specific time slot and specific channel by use of a “Reserve” hyperlink adjacent to each respective time slot
· “Delete” a reservation that they have made effectively clearing a timeslot for use.
· “Modify” a reservation that they have made
The database and web browser are the applications that interrelate with the dynamic web page and will be tested in systems integration. A description of the software feature is listed below:
Combination Features for Dynamic Web Page
Test Phase/Specification
Title/Description
MySQL Database
System Integration
The MySQL database server will interact with the web interface, which will display data stored in the database, and to a limited extent, update data in the database.
Web Browser Application
System Integration
The web browser will display the information obtained from the database. To be viewed by the users.
8.3 Scheduler Feature
The scheduler has been built to initiate the VLC on the server-end whenever a stream is scheduled to take place. For each time slot the database is queried and for each stream (on separate channels) scheduled for that slot, a separate VLC process is created/forked to deliver a multicast stream to the channels network destination.
The scheduler will be unit tested. The scheduler networks with both the database and the Video LAN Client, this interaction will be analysed in the system integration test. A description of the software feature is listed below:
Combination Features for Dynamic Web Page
Test Phase/Specification
Title/Description
MySQL Database
System Integration
The Scheduler will query the MySQL database to extract the scheduled streams on each channel.
VLC
System Integration
The scheduler will invoke the VLC to stream the videos that are scheduled within the time slot
8.4 Administration GUI Feature
The job of the administration graphical user interface is to enable the administrator a high level of control over the database in a user-friendly manner. The GUI places general logical constraints upon the administrator. The main form allows the administrator to see each table within the database with the option of removing, adding or editing its contents. The other forms are mainly used for entering record details and checking the validity of such requests before they are made.
The administration GUI will be unit tested. The system integration phase will examine the relationships between the GUI and the database. A description of the software feature is listed below:
Combination Features for Dynamic Web Page
Test Phase/Specification
Title/Description
MySQL Database
System Integration
The administration GUI is designed to maintain the data within the MySQL database.
9 Features not tested
VLC will be tested within the scope of this project in that the scheduler will start the VLC but no other VLC functionality will be tested. This piece of software is already being utilised and any testing problems not related to the video scheduler software are outside the scope of this project.
The listed hardware and software is the only available testing platforms for this project any other types of browsers, operating systems or hardware are outside the capacity of the project.
10 Approach
The major testing tasks for this project will be the component or unit testing as well as the system integration testing. The client will also perform the user acceptance testing.
The testing tasks will have all resources allocated for two weeks to complete testing, retesting and redevelopment of any section as necessary. The developer will complete most of the unit testing; all team members will perform the systems integration testing. Any problems found in the components will need to be recorrected; unit tested and then progressed back to the system integration phase.
The testing will utilise standard testing techniques to perform these task as listed below:
White Box Testing Techniques
Basis Path Testing: This is a white box control structure testing technique that enables the definition of “basis set” of execution paths. Test cases derived to exercise the “basis set” ensure that every statement in the program is executed at least once during testing.
Branch Testing: Testing designed to execute each outcome of each decision point in a program.
Condition Testing: This is a white box control structure testing technique that exercises the testing of each logical condition contained in a program.
Data Flow Testing: This is a white box control structure testing technique that aims at generating test cases to satisfy the execution control of program depending upon the data values and sequences of operation.
Black Box Testing Techniques
Back-to-back testing: Testing in which two or more variants of a program is executed with the same inputs, the outputs are compared and errors analysed in case of discrepancies.
Big Bang Testing: A type of integration testing in which software components of an application are combined all at once into an overall system.
Bottom-up Testing: A type of integration testing that begins with construction and testing of atomic modules (components at the lowest level in the application structure) and moves up through the structure to integrate and test the entire application.
Top-down Testing: A type of integration testing that employs depth-first integration or breadth first integration to start with the controlling module of the application and integrate test the rest of the modules. This testing employs test drivers and stubs for testing and relies on Regression Testing to ensure testing completion.
Regression Testing: Regression Testing refers to the selective re-testing of a system or component to verify that modifications have not caused unintended effects and the system component still conforms to the specified requirements.
Performance Testing: Performance testing is a type of system testing that aims to determine whether a system meets the performance requirements within the physical limitation of the computing environment (CPU process speed, memory capacity and number of users etc.) of the system.
Recovery Test: This System test examines how a system recovers from hardware failure, circuit errors, power blackouts and other problems caused by program errors within a pre-specified time.
Security Test: Security features are essential to protect data from unauthorized access and modifications. Security testing is a system testing technique that aims at verifying the protection mechanisms built into the system to protect it from improper penetration.
Equivalence Class Partitioning: This is a black box testing method that divides the input domain of a program into classes of data from which test cases can be prepared. In a unit-testing situation, for a field level validation criteria, the equivalence class represents a set of valid or invalid data types to check the validity of inputs.
Boundary Value Analysis: This Black box technique relies on the rule that a greater number of errors tend to occur at the boundaries of the input domain than in the “centre”. This leads to developing test case with test criteria as boundary conditions wherein the appropriate inputs are chosen to study the system behaviour.
Implementing software testing in a structured manner involves preparation of well designed Test Plans and Test Cases for checking the functionality of the software. The critical success factor of effective testing lies in the test plan and test case design to meet the objectives of initial system requirements.
A necessary part of every test case is a description of expected output of results. The video scheduling system will require a number of test cases to be developed for the differing types of software components. In our system we have an Administrator GUI which mainly interacts with the database, a Dynamic Web Page for users to schedule videos and to determine what videos are already scheduled; we also have a scheduler daemon which invokes the VLC software to start streaming the videos. Each of these components requires different testing techniques to best suit the software. Each test will be validated against the expected results and any unexpected results will be then be scrutinized and faults fixed. This process will involve thoroughly inspecting the results of each test using the following methods:
· Ensuring an activity in which a component is executed under specified conditions is recorded and the results are observed and evaluated with the expected outcome or target.
· Expected outcomes may not be available for every test sequence step. However, the structuring of test sequences should be such that expected outcomes are definable.
· Test sequences should cover both valid and invalid outcomes.
· Some of the test sequences may involve a direct execution of a software component and a check for the expected outcome, while others may involve a sequence of entry and a subsequent analysis / comparison of action.
Software components can be declared as tested if and only if all test cases defined are executed to meet the expected outcomes.
10.1 Component Testing
Component testing will be performed on all modules of the Video Scheduling Software. The details of the component testing can be found in the Unit Test Plan. As stated above the developer of the component will perform the testing in this phase.
10.2 Integration Testing
System Integration testing will be performed on the whole system to ensure that all components work together to achieve the system requirements. For more detail on this testing phase please refer to the System Integration Test Plan.
10.3 Conversion Testing
Conversion Testing will not be performed in this project. Any data conversion requirements will be the responsibility of the client.
10.4 Job Stream Testing
Job Stream Testing will not be performed within this Project.
10.5 Interface Testing
The interface testing will be included in the unit and system integration test phases. The interfaces into the Video Scheduling system are the VLC application, the Web Browser and Human Interfaces.
10.6 Security Testing
System control features such as logins to the system will be tested within the component and integrations phases of testing.
10.7 Recovery Testing
Specific recovery testing will not be tested.
10.8 Performance Testing
Performance testing for this project will be aimed at the scheduled videos starting on time as well as customer expectation such as the portability onto different software platforms. Performance testing will also ensure that the response time of the system is within a reasonable amount of time. These issues will be included in the system integration test.
10.9 Regression Testing
In the event that application components have problems the team members will correct the error perform the required unit testing ensuring the test results have not changed. The change will then be moved to the system integration environment and this phase of testing will also be repeated. Regression testing will not have an individualised plan but will be included in the unit and system testing documentation.
10.10 Acceptance Testing
The customer will perform acceptance testing and a plan will be outlined for the client that addresses the user requirements to complete user acceptance testing although the team will assist in building a plan the testing is not the responsibility of the project team. Any errors discovered in the acceptance testing will try to be resolved as quickly as possible as the timeline of the project is very tight and close down of the project will occur soon after the acceptance testing is completed.
10.11 Beta Testing
Beta Testing will not be performed due to a tight schedule and that the user will perform user acceptance testing in place of beta testing.
11 Pass/Fail Criteria
As Tests, Test Procedures, and Test Steps are completed, they are graded on a Pass/Fail Basis. The table below lists the criteria for determining the Pass/Fail status of Testing.
Status
Level
Description
Pass
Step
Expected result achieved
Pass
Procedure
Successful completion of all test steps
Pass
Test
Successful completion of all procedures comprising the test
Fail
Step
Expected result not achieved
Fail
Procedure
Failure of one or more test steps
Fail
Test
Failure of one or more procedures comprising the test
11.1 Suspension Criteria
The project team could suspend testing if it is determined that it is no longer valuable or practical to continue testing.
Any time there is a suspension of testing, it is likely to have an impact on the schedule. Depending on how much time is lost during the suspension, and how much retesting must be done on items that have already been completed, this could have a significant impact on the implementation schedule. The team will escalate the issue and will evaluate the impacts of the assessed.
Suspension could occur if the software does not meet the Entrance Criteria for the next scheduled phase of testing. At that time it would have to be determined if it is practical to waive or extend a portion of the entrance criteria and begin testing limited portions of the system. Otherwise, the team will suspend testing until the test phase Entrance Criteria are met.
Testing could be suspended if resources are not available. The resources could be computers, test area, network access, or testers.
11.2 Resumption Criteria
If testing is suspended, upon completion of the necessary corrective actions, the Video Scheduling System will be rescheduled and testing will proceed.
The correction of the problem will be documented and the fix will have to pass unit testing before proceeding to system testing. The PGP Team will retest appropriate test cases to verify that the fix passes all former-testing scenarios. Retesting must be completed prior to the completion of SIT.
A strategy for resumption will have to be developed for differing resource failures. If the resource is hardware, the strategy must take into account time to repair, acquire, configure, load and regression test as appropriate. If the resources are human, this strategy should take into account the extra time needed for other team members to complete testing.
If there is a suspension for any reason, the team will endeavour to rectify the situation as quickly as possible so the project team will stay on schedule. This is of the utmost importance, as the project is required to stay on schedule, as the team will be disbanding at the end of semester.
11.3 Approval Criteria
There is no formal approval process for the testing phases. Once the exit criterion of each phase is completed and the test results are successful the testing will be deemed completed.
12 Testing Process
The software development lifecycle begins with the identification of a requirement for software and ends with the formal verification of the developed software against that requirement. To accomplish this primary goal, the testing process is structured into several phases, where each is conducted with the intention to broaden the scope of the testing in a structured effort to identify and remove defects. Testing Phases are typically broken down into the following stages:
· Unit Testing
· Application or System Testing
· Integration Testing
· Acceptance Testing
For each testing phase, the testing process follows a consistent and structured process for inputs, entrance criteria, tasks, verification, validation, outputs, and exit criteria. Specific work products are used in the execution step of the test. In turn, each phase updates or produces work products, which in turn may be used in subsequent testing phases.
This project will use a standard testing process as detailed below in figure 1.3 to test the video scheduling software. The process flow can be used for any testing phase and all of the testing phases follow a standard format.
Figure 1.3 Standard Testing Phases Process Flow
12.1 Test Deliverables
The documents that will be delivered from the testing process are:
Software Test Plan (This document)
Unit Test Plan and Results
Systems Integration Test Plan and Results.
12.2 Testing Tasks
To prepare for and perform testing the following list of tasks will need to be completed below:
Test Tasks
Test Plan
Develop Test Cases/Procedures Unit Test Plan
Prepare Test Environment
Migrate components to test environment
Verify Entry Criteria for Unit Testing is met
Unit Test Components & Document Results
Fix and Re-Test as required
Validate & Verify Results of testing
Verify Exit Criteria is met
Ensure all component testing is completed
Integrate all components in the test environment
Completed Systems Integration Plan
Verify Entry Criteria for System Integration testing
Perform System Integration Testing & Document Results
Fix and Re-Test as required
Validate & Verify Results of testing
Verify Exit Criteria is met
Request User Acceptance Testing
Testing Phase Closes
12.3 Responsibilities
The testing responsibilities are listed below:
Test Responsibilities
Resource
Software Test Plan
Michelle Lister
Unit Test Plan
Michelle Lister
Systems Integration Test Plan
Michelle Lister
Unit Testing
Pretty Good Project Team
System & Integration Testing
Pretty Good Project Team
Documentation Walkthroughs
Pretty Good Project Team
Rework & Retesting
Pretty Good Project Team
User Acceptance Testing
Mike Dixon
Each member of the team holds a different role within the testing phase of the project these roles and the general responsibilities are listed below:
Role
Available
Name
Responsibility
Test Manager
100%
Cliffe Schreuders
Test Management
Conduct Reviews
Defect Tracking and Reporting
Issue Reporting and Tracking
Suspension and Escalation of testing
Test Review Meetings
Conduct of Test Completion Review
Test Coordinator
100%
Michelle Lister
Test Planning
Test Documentation
Test Case Creation
Test Coverage Analysis
Test Result Analysis
Defect Tracking and Reporting
Conduct Reviews and Test Completion Review
Tester
100%
David Patullo
Install & Configure
Test Case Creation
Test Case Execution
Defect Reporting to the group
Resolution of the problems and regression testing
Tester
100%
Martin Sulzynski
Install & Configure
Test Case Creation
Test Case Execution
Defect Reporting to the group
Resolution of the problems and regression testing
Tester
100%
Tim Radbone
Install & Configure
Test Case Creation
Test Case Execution
Defect Reporting to the group
Resolution of the problems and regression testing
12.4 Resources
The members of the project team will test the software. The individual developers of the component will perform the unit tests. All team members will be involved in the system integration test phase to ensure that all the components of the system are integrated and functional. The user acceptance testing will be completed by the client and is not the responsibility of this team, although guidance will be given if required.
Resources available for testing:
Test Type
Component
Resource
Unit
Scheduler
David Patullo
Web Page
Martin Sulzynski
Database
Martin Sulzynski
Administration GUI
Cliffe Schreuders/Tim Radbone
System Integration
Scheduler
PGPt
Web Page
PGPt
Database
PGPt
Administration GUI
PGPt
Acceptance
All Components
Mike Dixon
12.5 Schedule
In the development phase of the project the unit tests will be performed to ensure that the test cases are completed.
Once all the individual components have been developed the team will test the overall system integration of each of the components to ensure that the system meets the user requirements as defined in the requirements document.
User acceptance testing will be performed before the presentation at the end of the semester to ensure the software meets client expectations and requirements.
Task
Start
Finish
Test Plan
11/09/2004
25/09/2004
Develop Test Cases/Procedures Unit Test Plan
26/09/2004
02/10/2004
Prepare Test Environment
12/09/2004
18/09/2004
Migrate components to test environment
03/10/2004
09/10/2004
Verify Entry Criteria for Unit Testing is met
03/10/2004
09/10/2004
Unit Test Components & Document Results
03/10/2004
09/10/2004
Fix and Re-Test as required
10/10/2004
16/10/2004
Validate & Verify Results of testing
10/10/2004
16/10/2004
Verify Exit Criteria is met
10/10/2004
16/10/2004
Ensure all component testing is completed
10/10/2004
16/10/2004
Integrate all components in the test environment
10/10/2004
16/10/2004
Systems Integration Test Plan
10/10/2004
16/10/2004
Verify Entry Criteria for System Integration testing
17/10/2004
23/10/2004
Perform System Integration Testing & Document Results
17/10/2004
23/10/2004
Fix and Re-Test as required
24/10/2004
30/10/2004
Validate & Verify Results of testing
24/10/2004
30/10/2004
Verify Exit Criteria is met
24/10/2004
30/10/2004
Request User Acceptance Testing
31/10/2004
05/11/2004
Fix and Re-Test as required
31/10/2004
05/11/2004
Testing Phase Closes
31/10/2004
05/11/2004
13 Environmental Requirements
The hardware and software environment needed for the test phase is described in the following sections. A room will be made available at the University for the purpose of testing the Video Scheduling System. The room will contain computers that will be supplied by our client to perform the testing phases. The hardware and software environment will replicate the University setting so that the team can ensure the Project software will perform on all types of machines.
13.1 Hardware
Access to Linux and specialized hardware such as Multicasting switches will be required in our implementation and testing phases. Our client has agreed to grant us access to such hardware when we require it. GNU/Linux is available via the GPL public license agreement.
Characteristic
Specification
CPU type
Intel Pentium 4 Architecture
CPU speed
2Ghz +
Mb of Random Access Memory
512Mb (will be upgraded to 1Gb)
Hard Disk Space
80Gb
Characteristic
Specification
Hardware
PC, Mac
13.2 Software
System Component
Software Development Requirements
Administrator GUI
Sun Java J2SE 1.4.1 SDK and Java API docs
TkCVS (CVS GUI)
End User web interface
Apache 1.3.31
PHP 4.3.6 or Perl 5.8.3
MS Internet Explorer, Mozilla FireFox, Opera
VLC Client
TkCVS
Scheduler
gcc 3.3.2
VLC Server
TkCVS
Database
MySQL
Operating System
Windows, Mac OS, Unix (includes all variants)
13.3 Security
The group will be accessing the software from a secured room at Murdoch University, which has been set-up specifically for testing this project. The Project Coordinator is responsible for the key and security of this room.
13.4 Tools
Test Drivers and Stubs may be used to assist in the unit test phase.
Test Driver: A test driver is a simulation of a control module. It performs the two functions of repeatedly calling the lower module being tested and passes input data to the lower module in each call.
Test Stub: A test stub is a dummy program module used by the driving / controlling units. It gets invoked by a function call and posts an action to notify execution.
13.5 Publications
No relevant publications for this project.
13.6 Risks and Assumptions
Assumptions
· Video Scheduling Components are all developed.
· Data is available in the MySQL database.
· Required Hardware and Software is available for testing configuration should be the same, as the live system will be required to run on.
· Resources will be available for testing
· User Acceptance Testing will commence after the Unit and SIT has been completed.
Resource availability needs to be determined to ensure the Unit & SIT phase is performed according the testing schedule. Project members must be involved if greater demands must be placed on their time.
Risk Type
Risk Description
Probability
Effects
1
Human Resources
Group Members unable to complete scheduled testing.
Moderate
Tolerable
2
Human Resources
Lack of experience in testing procedures
Moderate
Tolerable
3
Human Resources
Team lacks experience to implement one or more components of the project causing delays.
Moderate
Tolerable
4
Technology
One or more components not being developed on time. This risk could have serious consequences on the overall testing process especially the SIT. If one of the components is not completed the testing maybe delayed, time limitation may cause defects not to be found and a quality product may not be delivered.
Low
Serious
5
Technology
Hardware Failure inability to complete unit or SIT testing
Low
Serious
6
Technology
Compatibility of software this will affect the progress and completion of SIT
Low
Serious
7
Technology
Changes due to testing and the impact on the Unit and SIT
Moderate
Serious
The serious risks for this project are the completion of individual components of the system.
As unit testing is performed on individual components the risks will not be as serious for this as each individual component can be unit tested as it is completed.
The System Integration will be seriously affected if some of the components are not developed and unit tested on time. As all the components rely on another component in the system anyone of these will affect the testing process. The work around for this situation is to test as many of the components that can be tested within the limitations.
Hardware Failure as the client has access to these resources the replacement of the hardware will be available but the time delays in this process may affect the degree in which the system is fully tested before handover.
If one of the components of the system is not integrating correctly these issues will need to be addressed as soon as possible and research will need to be undertaken to resolve the issue.
The project team has five members of the group and each of these team members are committed to the success of the project, even though they’re maybe a lack of experience among the group in testing process. As there is no opportunity for the group to secure further resources, if any of the group members cannot complete part of the testing schedule another group member may need to take on the responsibility.
14 Change Management Procedures
All changes to hardware and software in the test environment have to be documented and approved by the Project Team Coordinator. After every change of the hardware or software the system has to be retested appropriately.
Error Correction and Reporting Procedures
All defects found during testing will be documented via email to the project account and to the rest of the team member. The Test Manager will be responsible for notifying team members of the development of the defect and tracking the defect to closure. If a number of defects are detected an excel spreadsheet may need to be created to track the progress and status of the reported defects. The Test Manager will report on the outstanding status of defects by priority at team meetings.
Redelivery Procedures for Problem Fixes
The developers will fix the problems in accordance with the priority and deliver the files, and technical documentation to the Test Manager. The Test Manager or developer will record the fix of the project to the project email account and update the files on Gryphon as well as loading the files on the Test Bed. Regression testing of the fix will be performed and upon a successful pass of the testing the defect will be considered closed.
15 Plan Approvals
There are no formal testing plan approvals required from the client for this project. The PGPt will agree to abide by the test plan strategies to ensure that the team produces a quality software product.
16 Configuration Management
Version
Date
Author
Description of change
1.1
19/09/2004
Michelle Lister
Initial Software Test Plan
1.2
05/11/2004
David Patullo
Formatted / Final Version
PAGE
Software Test Plan Page 2 of 31 17/04/2014
_1158322085.vsd
_1158324450.vsd
Acceptance
Testing
Acceptance
Testing
SDLC
Stages
Coding
Testing
Maintenance
Integration
Test Plan
Unit Test Plan
Unit Tests
Unit, Integration &
SystemTesting
Unit Testing
& Test Data
Testing
Methods
Test Plans
Prepared
System Testing
& Test data
SystemTest
Plan
Integration
Testing &
Test Data
Requirements
Analysis
High level
Design
Low Level
Design
Integration &
System tests
Complete all output
documentation
Complete testing
activities
Begin Testing Phase
Are all
Entrance
Criteria
in Place?
Review
Test Plan
and
other
inputs
needed.
Complete all input
documentation for
successful testing.
Verify
the results with
reviews
Have all
Exit Criteria
been achieved?
Testing Phase
Complete
Initiate Tests
Complete overlooked
activities or
documentation as
required
Not Ready to Begin
No
Yes
Yes
No
Validate
the
Results with
Reviews