Testing Patterns

bullet1 Requirements are

  • well defined and stable
  • moving, driven by market and competition
  • fuzzy, e.g. new technology, new market

Once upon a time in the history of software development (i.e. some years ago), requirements were well defined and documented in detail, and they remained stable until release. Today they keep on moving and were driven by customers, market and competition. Furthermore in the age of command line interfaces it was relatively simple to write perfect requirements: you just had to describe and to test all switches, parameters and operands of a CLI command - a straightforward job which was easy to automate. With the new interactive graphical user interfaces the software is directly confronted with a fuzzy and unpredictable opponent - the common end user ! And if there is a new product, for a new market, with new technology, the requirements will be as clear as a foggy day in November.

One approach to overcome these obstacles can be a more iterative software development process, such as the Unified Process or the highly iterative agile methods (Crystal, XP, ..). This requires a better cooperation of developers and testers and their tools - the days of huge separated (and sometimes hostile) development and testing departments are fading away. If the requirements are fuzzy, moving or poorly documented, Exploratory Testing will be the best choice to start with.

I want to finish with some additional remarks, which scratch at the surface of this complex subject:

  • Are the requirements testable ? For example requirements written by marketing won't often be testable ("The product will leverage the customer benefit in a competitive environment and add tremendous value ....")  ;-)
  • We put requirements through a number of translations - analysis, design, coding, test design - and each of those translations make the requirements more difficult to change and may spoil the original information. That's a typical contradiction: to transfer the information precise and accurate more effort will be required (and usually more paperwork), which make changes more difficult.
  • Test cases may not only be derived from requirements: requirements may be written as test cases. That means test cases ARE the requirements. This concept comes from test-driven development, but it may also be used in a broader context.
  • Another important topic, which has to be addressed by the requirements, is testability. Testability addresses design and architecture of the product (some architectures are easier to test than others), log and trace information and interfaces for test automation. Testability is often neglected because the testers aren't involved in writing and reviewing the requirements and the design.
  • Finally there is a more philosophical, but crucial question, should the testers take over the underlying models and scenarios, which bias the requirements, from development or should they try to develop fresh and independent models and scenarios on their own ? If they take over the context from development, they only will see bugs within this context; if they develop an own context, they will find unexpected bugs. But to create models and scenarios on their own, the testers requires additional sources of information, for example information about competing products. This is also something to keep in mind if you think about model-driven software development.