Test Data Generation for Large Software Systems

Test Data Generation for Large Software Systems

Abstract

Software testing is an arduous and incomplete process. Indeed the famous maxim goes, Testing confirms the presence of errors, not their absence !. Testing usually occupies more than 50% of the time and effort over the entire software development life cycle. Therefore, in practice, testing is taken up with a purpose in mind and, when the purpose is accomplished, testing is usually terminated. A number of techniques have appeared in the literature, which test software systems to various degrees and coverage levels. In this seminar, we look at the testing problem in terms of generation of data in order to achieve (near-)complete test coverage. We use a combination of symbolic computing and program instrumentation and convert the test data generation problem into a non-linear numerical constraint optimization problem. The solution to this problem yields the right amount of test data for comprehensive testing of the given software. Extension of the fundamental concepts to domain testing has also been addressed. With appropriate conventions and practices for software development, it is found that full coverage can be achieved in linear time. The entire system, called ADTEST, is over 70K lines of code and was originally developed for the Ada programming language. The techniques have since been extended to other languages such as Pascal and C. Whilst this approach has been applied extensively to large software systems, particularly for unit testing and system testing, it is currently being re-designed for testing OO software also. Some extensions to class testing and class integration testing will also be presented in the seminar. An off-shoot of the instrumentation process was the development of a software visualization system. The visualiser offers a number of views (which can themselves be tailored to particular needs of a customer) to viewing the software. These include: interface views, systems views, version control views, tracing views and more. Current works in this area include (particularly for large scale information systems) automatic detection of languages, view instrumentation, view-rule distribution, distributed software management and maintenance. Some applications to parallel and distributed software debugging are also envisioned (which was actually the motivation for this work). An aspect of the visualiser has also been extended to visualizing of WWW sites and contents. A part of the above work was sponsored by the Australian Defense Force (ADF). An early part of the work appeared as a large paper in the IEEE Transactions on Software Engineering, Sept.1997.