We were supposed to have a winter storm last Thursday. But as it turned out, we just had some light snow instead. The two inches accumulation was a far cry from the eight inches that was forecasted the night before.
Everybody would probably agree it is the butterfly effect that leads to this uncertainty. Weather forecast involves models that contain hundreds if not thousands of parameters, and many of these parameters can not be well formulated or predicted. Given a non-linear system this big, any minute deviation of a parameter from its predicted value could produce a totally different result.
This made me think of the complexity of today’s software. Any non-trivial software system is very complex in nature. And any complex software would almost certainly contain bugs. How would these bugs magnify themselves in real world? And how would you catch them before a product is shipped?
The quality aspect of software development is a big topic by itself, and I shall not go into too much detail in this post. But there are some very fundamental questions one would ask. And one such question is: should the people performing the QA tests have a good knowledge of how the system is implemented? There are some different thoughts on this question. Some think that the less the QA people know about the system, the less likelihood that their tests would be negatively impacted by the design and implementation of the system. Others believe that the more knowledge you have on a system, the better chance you would have to find all the potential problems.
I would definitely agree with the latter. Suppose I have a function called Add(a,b) that returns the sum of a and b. How would you test it? Ok, most people would probably find many valid cases. For example, you could pass in some positive numbers and negative numbers and compare your own results against that returned from the Add(a,b) function. You could even find cases in which the results would overflow or underflow and you would be able to test whether Add(a,b) handles it correctly or not. So suppose you have tested all the extreme cases and all the results suggest that the implementation of Add(a,b) is correct, would you be a hundred percent sure that it works as expected?
The answer is clearly no. Suppose somewhere in the function I have something like
If ((a == 3.141592654) && ((b == 9999) || (b == 1.4142)) return 0;
Chances are that you wouldn’t be able to catch it through your normal reasoning.
Ok, this is an overly simplified case, and no one would implement a simple Add function that way. But in reality when the system becomes complex, some seemingly benign functions could return some totally unexpected results, without the original programmer’s intention to create such twists like what I did before. And clearly, any intimate knowledge of the system would greatly facilitate testing.
