One of my favourite mathematical stories is of the 7 year old Carl Friedrich Gauss (born 1777); his teacher was looking forward to spending a few hours catching up with paperwork and so set the class a problem. The problem wasn’t a difficult one but it was likely to be a time-consuming one. The problem was to sum all the natural numbers from 1 to 100.
Traditional methods would suggest that you start at 1 and add the next value (2) to get 3; you then add the next number (3) to the running total to get 6 and continue in this vein until you finally add 100 to find the answer.
This is the approach that the teacher assumed the class would take and so he thought he was in for a few hours of peace and quiet. However he hadn’t reckoned on pure genius of Gauss and was astounded when, within seconds of setting the problem, the young Gauss announced that the answer was 5050. The teacher couldn’t understand how a child of 7 could know the answer to this and demanded an explanation. The young Gauss calmly explained that if you sum pairs of numbers starting with the first and the last (1 and 100), then the second and second from last (2 and 99) and so on, each pair of numbers sums to 101 and since there are 50 such pairs of numbers then the answer must be 50 x 101 = 5050.
Whether this story is true or not (and I like to think it is) is not really the point, the point is that when faced with a problem we can approach it in the same mechanical way that we always have done or that we that we have been taught (e.g. start from 1 and successively add the next number to the running total) or we can stop and think about how we might be able solve the problem in a more efficient or effective way. It may be that after thinking about the problem, the best way to solve the problem is actually the traditional way but without that period of reflection and thought you will never know.
The approach of adding each number in turn to a running total is not wrong; on the contrary it is a structured and systematic approach to summing a list of numbers that, if followed correctly, will provide us with an answer. It is just that depending on the context there are different ways of approaching the problem.
Consider for a moment that instead of the teacher asking the class to sum the first 100 integers, the problem was to sum a list of 100 random numbers. In this case, Gauss’s approach wouldn’t have worked but by stopping and thinking about the problem rather than blindly going with tradition, Gauss may have found other ways to simplify the problem such as first adding together numbers which end in 0 (e.g. 10, 120 and so on) would make the calculations a bit easier and likewise finding pairs of numbers whose last digits add up to 10 (such as 87 and 23) might also make the calculation easier. He might decided that the teacher probably didn’t know the answer so any reasonable estimate would do rather than the definitive answer. Alternatively he might have looked at the numbers and decided that his best strategy was to add each number to a running total (i.e. the traditional way).
In many ways, testing is like this; we are asked to test (whatever that happens to mean to you) a product and often we follow a structured method to achieve this. This structure comes in many forms and can include, always splitting testing into neat test phases, always writing detailed test scripts, running the same set of tests for each release, insisting on test independence, using boundary analysis or other test design techniques. As with the structured way of summing sets of numbers, none of these practices are inherently wrong but their contribution to reaching our goals is highly dependent upon the context they are applied.
For many this structure fairly very rigid in terms of how testing is approached; however in blindly following this structure we are falling into a trap: the mechanical testing trap.
This is a trap whereby we accept the structure that is placed upon us in order to solve a problem. The structure may solve the problem but it may do so in a way that is inefficient or ineffective. In today’s competitive world, inefficient and ineffective are two words you don’t want to be associated with.
Take for example the widely adopted practice of writing detailed test scripts (e.g. test case steps); this type of structure is placed upon test teams citing the belief that:
- It ensures anyone can execute the tests.
- It ensures that nothing important is missed.
- It ensures that testing is repeatable.
- They are easier to check for completeness, audit or review.
This type of structure is frequently cited in books, courses and standards related to software testing as a best or at least a good practice and many teams adopt this approach without thinking critically about whether this structure is best practice for them. Don’t get me wrong, it’s not that writing detailed test cases (or any other widely used practice) is wrong – in your context it may be the right approach. It is more the case that we, as testers, should think more critically about what we do how we test and decide what, in this context, is the best way of achieving our outcomes rather than the blind acceptance that leads to the mechanical testing trap.
We can avoid this trap by thinking more critically and asking some key questions such as:
- Do we want just anyone executing our tests? Are we really comfortable with that?
- Do our testers need detailed test cases to do their work?
- Will having detailed test cases improve the effectiveness of our testing?
- How can we be confident that our detailed test cases are correct and up to date?
- Can we actually write our tests to level where there is no ambiguity in what we expect?
- Do we only want to know when the product behaves differently to what the script says?
- How do we know that our testers will follow the steps to the letter?
- It is possible to ever repeat the same test?
- What is the value to us of being able to run the same test twice?
- What value does having detailed test cases have to our organisation?
- What are we trying to achieve and how does having detailed test cases help us achieve this?
- Can we achieve the same results in different ways? If so which do we think is best for us?
- Does this practice constrain our testing to only tests that we can script in advance?
The answers to questions such as these depends on the context in which they are asked and no book, training course or standard can answer those questions for you without first understanding the context under which you are operating.
Thinking critically about what we are doing is only part of the solution; a key part of the Gauss story is he was able to articulate how he arrived at the solution and convince his teacher that his reasoning was correct (it’s not known whether the teach knew the answer was 5050 or not).
Within any context going against tradition is not easy, particularly if you are not in a position of authority so being able to explain your reasoning is key to convincing others to accept change. Having a great idea is just that ‘an idea’, but being able to communicate the idea and convince others takes leadership.
Whenever I think I’m in danger of falling into the Mechanical Testing Trap I always ask myself : “What would Gauss do?”