We need Test Scripts because of the following points:
Acceptance testing is a complex and painstaking process that requires
great attention to detail in order to ensure that the software is adequately
exercised and that it meets the original requirements.
The objective of testing is to answer a binary decision: does the system
pass the test or not. To make that decision there must be a defined result
that must be achieved for each specified test ; and for the overall system
there is usually an acceptance level defined in the contract which defines
what number of individual failures may be acceptable.
To decide whether a test has been passed, there must be clear criteria
that it must meet. This may be a value or set of values to be displayed on a
screen, some results printed out, some specific changes in a database etc.
In some cases it will be easy to check that the necessary result has been
achieved (e.g. a picture on a screen) in other cases an extra action might
be required (e.g. printing out the before and after state of a database
In order for overall testing to be accurately carried out, there will
usually be a need to set up base data - perhaps by conversion of some
existing computer data. Where this data can be set up and manipulated by the
system it would be usual to use the system to do so - in fact to define the
set up process as part of the overall test sequence.
To ensure that the correct steps are carried out, and in the correct
sequence, it is important to put the process in writing. Where the sequence
of the tests is important, for example where one sets up data for another,
it is particularly important to have a defined "running order".
The function of the test script is to define what input should be made
to the system, and what output should be received. Usually the output will
simply be text or perhaps a graphic on a workstation screen, sometimes there
may be a printed report of some nature. A formal script defines (or
identifies) several things:
Any data which should pre-exist and on which this test will rely,
which may have been input by an earlier step in the script or crated by
an entirely separate process - this data may be data held in the
computer memory, a simple data file or a set of records in a database;
The steps that must be taken to run the test - for example keyboard
input. This may of course be quite complex like filling in a long form
defining a ship and its cargo capacity/facilities.
Any data that is created by the test, with steps to take to check
that it has been properly created. In some instances this will be the
next step in the test sequence, in others it will require the use of a
separate piece of software to (say) examine a database.
The "pass" criteria which will be applied to the test.
A most important criterion for a test script is that it must be
repeatable. Provided the environment is set up correctly, the test should
produce the same result each time it is executed. But please note the
dependence on the environment - usually data but potentially other factors
as well such as computer or network configuration.
There is a risk of "false negatives", i.e. tests said to have been
deemed to have failed but where the failure is due to extraneous errors. For
example, there was an error in the script and the "pass" criteria was not
obtainable, there was a mistake in typing in data (say a wrong selection of
"yes" or "no" - simple but devastating) or required pre-set data did not
exist (the tests had been run out of sequence). It is of course also
possible to make a mistake in the "pass" criteria - hence the script is
faulty not the system.
All "failed" tests should be investigated to establish a reason for
failure. Because of the complexity of testing, in many cases it is a "false
negative", and the root cause needs to be identified so that the test can be
re-run properly. Occasionally investigations will indicate a "false
positive" from another test, which had apparently passed but the system had
failed to carry out some action or other during that test on which this test
depended. Whilst the result might be still one test that failed (albeit a
different one) it may affect the classification of the failure (and hence
overall acceptability). On the other hand it may again simply be that the
test script for that test failed to ensure that the action was carried out -
i.e. a script error not a system error. There are then a string of tests to
be repeated, or if time is not available, discounted.