Paper 141-30Software Testing Fundamentals—Concepts, Roles, and TerminologyJohn E. Bentley, Wachovia Bank, Charlotte NC  

ABSTRACTSAS® software provides a complete set of application development tools for building stand-alone, client-server, andInternet-enabled applications, and SAS Institute provides excellent training in using their software. But making it easyto build applications can be a two-edged sword. Not only can developers build powerful, sophisticated applications,but they can also build applications that frustrate users, waste computer resources, and damage the credibility of boththe developer and SAS software. Formal testing will help prevent bad applications from being released, but SASInstitute offers little guidance related to software testing. For those unfamiliar with the topic, this paper can serve as aprimer or first step in learning about a more formal, rigorous approach to software testing. The paper does notaddress any specific SAS product and may be appropriate for even experienced application developers.

INTRODUCTIONWith SAS and Java, talented developers can do incredible, wonderful things. AppDev Studio™ provides a full suite ofapplication development tools for building client-server and Internet-enabled applications. With Base SAS software,SAS/Connect®, and SAS/Share® as a foundation, developers can easily build a 'thick-client' application usingSAS/AF® and SAS/EIS® to access data and share information across a LAN. For 'thin-client' Internet-enabledapplications SAS provides web/AF™, web/EIS™, SAS/IntrNet®, and SAS® Integration Technologies. 

In the hands of less talented developers, these same tools can still do incredible things but not always wonderfulthings. Everyone has used poorly-designed, clumsy, frustrating applications that are only barely able to get the jobdone. This is not an indictment of entry-level developers—everyone was a rookie at one time, and often it's not theirfault anyway. This author believes that the finger is better pointed at those responsible for training the entry-leveldeveloper and, even more so, at those responsible for overseeing the testing and release of their work. In manycases, however, the problem may be a systemic or scheduling failure—overly aggressive schedules to documentrequirement, design, build, test, and release software may leave too little time for adequate testing and forcedevelopers to release code that isn't ready

Assuming that a project has fully collected and clearly documented its business and technical requirements (whichmight be a stretch, but let's pretend), a primary cause of failed application software development is lack of a formalrequirements-based testing process. "Formal requirements-based testing" may conjure up an image of a lengthy,involved, and minutely detailed process but it doesn't necessarily have to be like that, although in organizations withmature software engineering practices or at CMM level 3 it probably is. In many if not most organizations, formalsoftware testing can easily be tailored to the application being examined and has only two real prerequisites.

♦ Business requirements and design documents that allow development of a test plan 

♦ People who understanding how to write and carry out a test plan 

Collecting and documenting business requirements is outside the scope of this paper, so here we will say only thatclear, concise, and measurable requirements are essential not only to developing the application itself and creating atest plan but also gauging if the final product meets the users' needs.

James Whittaker, Chair of the software engineering program at the Florida Institute of Technology, has noted thatdespite the countless hours that go into code development and seemingly endless code reviews, bugs and defectsstill are found in the production release. Why? A big part of his answer is a lack of understanding of software testingand, consequently, inadequate software testing processes and procedures. The material in this paper may begin toremedy this situation by presenting some concepts and terms related to software testing.In this paper, the terms application, program, and system are used rather interchangeably to describe 'applicationssoftware', which is "a program or group of programs designed for end users to accomplish some task".