When Extreme
Programming (XP) was first introduced to the VCAPS project there were
no automated acceptance tests. It took a while to add coverage. We
added tests for all new functionality and old functionality as it
required changes. After about a year we had an estimated 40% covered by
tests and the trouble ticket count dropped by 50% as well. We don't
consider this a coincidence.
But
the customers had noted a different effect. By trapping bugs before
they reached the production environment there were far fewer emergency
production releases. Previously, it was not uncommon to release to
production a couple times a day for a couple days due to bugs that
required
an immediate fix. The acceptance tests improved the quality of the
system to a point where a production release was rarely re-released
because an emergency fix was required.
The
customers experienced this as a system with a far greater feeling of
stability. They had more confidence in the system and us. The customers
also noticed that with fewer releases there was a large drop in
spurious bugs often caused by quick and dirty fixes.
Next Lesson