Regression Test and Automation
As a web developer, I am always frustated with the fact that when I made a modification to a system application, something else broke. This is especially true in large application. Recently, I was made aware of Regression Test.
Regression testing intends to ensure that software changes do not introduce new problems, or failures, in a software. Also, it ensures that you don’t reintroduce previously fixed bugs. The introduction of new problems is especially common if the software has many dependencies to third-party components and libraries, like XML processing APIs or database abstraction APIs like JPA. In these cases, a subtle change in the database schema might cause a behavior change in the application that is difficult to predict. If the software is inherently complex, one small change in a common piece of code can cause unforeseen side effects.
A scenario that requires continued support of previous versions of a specific middleware software, or different platforms, sharpens the probability that you might introduce problems during maintenance or development. A fix or function that works in one version of the database middleware might have a disastrous outcome in a different version. Also, when you add functions, an API available only for newer versions of the middleware might break clients that use previous versions. When you support different platforms, thoroughly test functions dependent on command-line interfaces or scripts. Otherwise different behaviors can arise in different operating systems like Windows® and Linux®.
Regression tests aim at the software as a whole, and consider its interdependencies with other components or applications. During development, focus on the specific modules or functions under development. You can incrementally test modules or functions as you implement them, and sometimes use stubs that refer to other functions not yet implemented. If incremental tests are sufficiently structured and independent that you can repeat them given a certain environment (such as required middleware software, machines, or configuration), you can reuse them in future regression tests. This structure and its independence reduce rework and improve testing quality as every piece of code is tested with the same scrutiny used when it was developed.
Consider automated incremental tests when you test the function under development on a set of architectures, operating systems, and middleware versions. For example, imagine a command-line interface that is supported for Linux, AIX®, Windows 7 and Windows XP. During development, the execution of tests in each one of these operating systems might take considerable resources and time. In some cases, the time that is required to test the new feature can overcome the time necessary to develop it. Considering that many features are concurrently developed in a project by different developers, this scenario becomes even more difficult. An added complication is that there are normally a limited number of machines, even if virtual ones, for testing. Often, tests require a pre-configured environment, including database servers, application servers, and more. The more machines that are dedicated for testing, the more time spent updating and configuring each of them. With automated incremental tests run by a tool in a set of environments, or test bed, you save the time that is needed to manually run a test on each environment, and allow better sharing of resources.
Automated regression tests can use the same test bed as incremental tests. By setting regression tests to run once a day, for example, you do not consume resources that you might otherwise use for development or incremental testing. You can set lengthy regression tests to run at longer intervals, such as during weekends.
Source: IBM Developer Works
The IBM Developer Works excerpt above actually details the use of a framework to help with regresstion testing.
What I am more interested is actually in the fact regression test is a great tool to help developer to make modification without introducing new bugs into the system. Even with a human tester testing out the system thoroughly, there are going to be missed bugs.
The act of repetition and tedious tasks are better left for computers. Thus automation.
Right now, I am still new to this concept and have not found a suitable regression test and automation system. I wonder if anyone can point me in the right direction ?