Developing trends in software quality

Technology is designed to make life easier. But does it?

The software industry spends billions of dollars each year to field millions of technical support calls. A 1996 Better Business Bureau report ranks computer-related complaints seventh on the organization's Top 10 list. That emphasizes the need for better quality and leads us to ask why better quality assurance practices and tools are not employed. Imagine the time and money that could be saved if the number of technical support calls alone -- not to mention the amount of money that goes into application maintenance, rework and writing/distributing software patches -- throughout the industry were cut by even as little as 10%. Savings would easily be in the millions of dollars.

Consider software testing as it related to centralized mainframe systems. Once developed, applications were pushed to a tester who would manually punch variations of key strokes in an attempt to "break" the application. Next, consider an event-driven, decentralized client-server environment in which applications were deployed on multiple hardware platforms in conjunction with various other applications and the "points of failure" increased.

Now, think about the additional layer of complexity involved in the deployment of Web applications. Multiple software variables -- browser types (even different versions of the same browser), HTML, XML, JavaScript and Java applets, Microsoft Active Server Pages (ASP), ActiveX controls, communication protocols, operating systems, back-end databases -- combine with different hardware platforms and network structures. The exponential growth of variables elevates points of failure. In addition, Web applications say something about the integrity of business operations to a large audience of external users (i.e. customers, business partners). Combine the widening circle of application users with ever increasing time-to-market demands, growing expectations and complex deployment environments and it becomes clear why the need for quality controls within the application development life cycle are becoming increasingly vital.

Combating the quality challenge is the pain experienced in learning from past mistakes. In previous Newport Group case studies, IT managers have indicated that lines were drawn in the sand between development and QA. According to current research, however, that dynamic is changing.

A case in point -- Unit Parts

As the largest U.S. remanufacturer of automobile starters and alternators for discount auto stores, Unit Parts (Oklahoma City, OK) uses disciplined testing processes and automated testing tools to ensure applications meet business demands the first time.

"A lot of people don't realize that the development cycle is actually shorter if you test during development to find defects early in the process," said Charles Moore, software development manager at Unit Parts. Moore and his team have used automated testing tools from Cyrano and Rational Software over the past two years. Specifically, Moore uses Cyrano WinScope and Cyrano Standards -- tools designed to specify coding rules and optimize the application development process -- and Rational's SQA TeamTest for defect tracking and automated regression testing. Those tools are used to test the Microsoft client portions of many of its PowerBuilder-based applications. Unit Parts' mission-critical applications run on Windows NT servers between the company's three sites in Oklahoma.

Using actual time spent as a basis of measurement, Moore devotes a develop/test ratio of 2.2 to 1 hours respectively. Committing to and measuring the testing effort has allowed Unit Parts to obtain the national average of 85% to 95% defect removal efficiency. That is a measure of the number of defects removed from an application within the first six-month post-deployment time frame, and also serves to manage the cost-justification for the tools.

One reason for the testing success is the development routine that Moore and his team members follow, which promotes communication and involvement among all project members. Unit Parts also supports a corporate culture dedicated to quality not only in the remanufacturing of its auto parts, but in the software systems that support the business. That routine includes having a test engineer sit in during the development planning cycle, and having a software engineer present during the test planning stage. The complementary working relationship between QA and development has not always been the case. Moore remembers when QA and development spent most of their time throwing project responsibilities back and forth to one another.

Moore has found that communication is the key. After each application is deployed, he does a review with his developers and test engineers. Communication is one of the major issues that routinely surfaces, even when applications are deployed smoothly. Moore said the increased interest in communication is "a result of pain". Developers and QA staff realize they can eliminate headaches during the application life cycle if both departments can effectively share knowledge regarding status, concerns, bugs, etc. Communication alone, however, is not the sole factor of efficiency. Implementation of automated testing tools has made the software testing process much easier for Unit Parts. Moore estimates the company currently tests less than 5% of its code manually because it is too tedious and time-consuming.

Clearly, the ability for QA to work closely with development allows Unit Parts to be more efficient in deploying applications and running the business. Incorporating testing plans into the early design phase of development and working hard to establish clear, continuous lines of communication can pay tremendous dividends by eliminating the time and increased expense of fixing problems after deployment. It also saves in maintenance and technical support costs. Fortunately, lessons learned the hard way, combined with maturing automated quality tools, has fostered a more synergistic working relationship between QA and developers while reducing software development inefficiencies.