Automated Test Concepts

Automated Test Concepts

---

Types of Tests

The type and amount of testing done by software companies varies greately, depending on the size of the application and on how much the software company affords to spend. Here is a list of some of the types of tests usually performed:

• Unit Test: tests done for the smallest reasonable programming units, for example classes or global functions. Most frequently done by the developer using the debugger, usually during the coding phase.
• Component Test: tests on the external interfaces of deployable components, for example DLLs, COM components an so on.
• Regression Test: tests done on the whole system before a release, to check if previous functionality is still working.
• Integration Test: tests done on all components working as a whole.
• Stress Test: overload done on a system to test limit conditions, scalability and fault tolerance.
• Business Test: checks done on a stable system to determine if the original requirements are fully implemented.
• User Acceptance Test / Alpha Test: application is exposed to a number of 'friendly' users, which were not involved in development or previous testing.
• User Acceptance Test / Beta Test: application is assumed to have nearly-commercial quality so it is exposed to 'non-friendly' users. The information gathered is fed into a final build of the system which becomes the commercial version.

Few companies actually perform the whole set of tests, and even they will skip some tests for small applications. Small to medium companies and individual developers will typically compress the testing phases in the following way:

• The Unit and Component Tests are done as part of programming, in a fast compile-run-check cycle.
• The Integration, Business and Alpha Tests are done at the end of development as a single Functional Test, but usually without a formal test plan.
• The Regression Test should duplicate the above test for each patch or new version, but this is difficult without a test plan to follow.
• The Stress and Beta Tests are only done for larger applications.

Automated Tests

Many companies are recognizing the importance of automating the work of testers and including the auto-test as part of the regular build process. The results of the automatic test are seen as a measure of the current quality of the software. Combined with a code coverage tool it gives the answer to the all-elusive question: "How much of my code is currently running ok?"

Automated tests are not meant to completely replace manual testing. They cannot answer questions regarding the program's ease-of-use or user experience, and they cannot be used on small components during development. However they are far superior when it comes to Regression and Functional Testing, which as you've seen above are pretty much the only tests run in the real world.

Here are some of the advantages of having automated test scripts which can be run after each new build of the application:

• Low Running Cost: running an automated test script before each release of a new version, patch or bugfix is a lot cheaper than a manual test.
• Better Quality: especially for individual developers and small companies who would not employ a tester and will perform all testing themselves.
• Consistency: the test script will perform the same checks every time it is run. A manual test will be affected by human error and it will tend to skip certain areas believed to be stable.
• Speed: a script will execute many times faster than a manual test, giving you a full report on the quality of your product in a few minutes.
• Formal: No more "testing-by-ear". A code coverage tool (if you wish to use one) can tell you how much code is tested. The test scripts can then tell you if what you test works fine. The result is the exact percentage of the code which is guaranteed to work fine.
• Compactness: you can perform a full compatibility check by simply copying the application together with the test scripts on all the platforms where you believe it should work. It can give you the confirmation that all functionality works indeed as expected. Maybe in the not-so-distant future all applications will come with a "minimal self-test script" so that the user can be confident that his/her installation works as expected.

Continued.....