A Model For Describing And Executing Test cases Using XML and Java
The task is to develop a model for test case execution that allows automated testing and to implement part of the model as a case study.Problem Definition
Test cases are documented in test instructions (one or more test cases per test instruction).
The test cases are executed by manually starting the appropriate tool and feeding it with test instructions.
If the test cases could be modeled in XML it should be possible to have a Java program parse the test cases and then execute the test instructions in an automated way.
My task is to evalute the test case execution process and suggest a model that could automate this process.
The model has to involve several parts:
- a way of describing test cases in XML
- a tool that parse XML data and makes it available for others.
- an investigation of how interaction with the different tools used for testing can be achived in an automated way.
- the test cases have to generate a meaningful result and the expected result should be included in the test instruction.
I will also implement a prototype of the suggested model in incremental steps.
The development cycles contain:
- describe test cases in XML.
- implement adapters that handle communication with specific test tools.
- implement a program that parses the XML and calls the adapter corresponding to the tool that will be used for execution of that test case.
The XML documents should relate to existing documents in some specified way. Simplified overview of documents related to testing:
| FS | Function Specification: description of the included functions. |
| TS | Test Specification: - background/description (text, images). - configuration needed, hardware/software components that should be included. - Environment, other hardware/software that has to be set up. - code (start-up script files). - test cases. |
| (TI) | Test Instruction: code, occasionally plain text for manual execution. |
| TC | Test Case: subset of the Test Specification, contains description plus start-up scripts |
Objective
The motivation of the project is mainly to increase the efficieny of the test process, which would help increase the quality of the products. Every time extensions have been made to a product the same tests have to be performed to ensure that all of the old functionality still works as expected. This is a very time consuming process since there are a large amount of test cases. Still, the testing is necessary to be able to guarantee the quality of the product. To automate these tests would therefore be a way to help ensure high quality of the products. A long term goal could be to make a system that can be started in the evening and then executes test cases all night and produces a report in time for the work to start in the morning. Although this reaches beyond the scope of my thesis project.
The objective of my thesis project is to develop an automated model and to implement a prototype as a case study. I will also evaluate the prototype and suggest incremental steps on implementation.
Timetable
| Task: | Time: |
| Studies | 4 weeks |
| Designing a model | 4 weeks |
| Implementation and testing of prototype | 7 weeks |
| Write report | 4 weeks |
| Oral presentation | 1 week |
| See timeplan for a more detailed view. | |
Litterature
The litterature that I will focus on is XML related. Mainly consisting of webdocuments, articles and specifications. Since I am going to remote control existing systems I will have to collect neccesary information about those as well.
Methods
Since this system is depending on a lot of existing, and even unexisting, parts and documents the development is best performed in incremental steps. This will make it easier to get response from not directly involved parties. My intention is to get a limited working prototype up as soon as possible and then extend that prototype throughout the project. This implies that the foundation is general and extendable from the start, which is good since theese characteristics are required anyway.
The system (the prototype) will mainly consist of two parts, one part that parses the XML documents and one part that handles the test case execution.
XML parsing app: will parse the XML and validate the structure against a DTD or XML Schema. The parsing will be performed by a standard ,e.g. SUNs, XML parser and if the structure of the input XML is similar to the output structure SAX is preferably used, since it is very simple.
Integration and Verification Test Model (IVTM) app: a set of specialized components that inherits a general component making it easy to extend it by adding specialized components that, e.g., contains new features needed.
Fig.1, Furthest down is the system that needs to be tested. Above are the tools used for testing (FTDriver etc.). Above that are the adapters that I will have to make to interface the test tools. At the top, then, are the IVTM prototype and the XML parsing app found.
Main focus:
- Usability, it should not be "easier" to do things by hand.
- Usefulness, the system should be general and flexible allowing extensions.
- Robustness, the system must not fail. High availability. A central issue is how exceptions will be handled, what happens when one part, e.g. a related system, fails? Where should things like this be specified? In test case program code or in XML?
- Structure, the system must be easy to extend, both regarding xml and interfaces to test tools.
Out of focus:
- performance, the time of execution will mainly depend on the various test tools anyway.
- portability among different platforms, UNIX is the chosen platform since remote access to the system as well as stability are vital concepts.