|
|
 | | From: | Edward Berard | | Subject: | Re: Every software defect should be able to reproduce?? | | Date: | Tue, 11 Jan 2005 16:54:43 -0600 |
|
|
 | In article <1105318437.384248.112490@c13g2000cwb.googlegroups.com>,
wrote:
> If the testers find a bug (unexpected behavior) in software, but that
> bug cannot be reproduced every single time for the same input, or maybe
> because of the environment setting problem, we may consider it as an
> intermittent bug? But we should still consider this as a bug?
>
> I think every bug should be able to reproduce, there may be other
> variables to consider: configuration setting, environment, OS, etc...
> If we can match all those variables, the bug is able to produce.
> Otherwise, this is not a bug.
There are some other issues to consider, for example:
-> Suppose that you are dealing with a non-deterministic algorithm.
* Consider, for example, an encryption algorithm that just
happens to use the value for the second character in the
"seconds" field each time the algorithm is invoked. At a
high level, the algorithm is accomplishing the same
general behavior each time it is invoked, but at lower,
more detailed levels, the behavior appears to be "different,"
but not necessarily incorrect, each time.
Of course there could be problems with the algorithm, depending,
for example, on the checksum, or with even (as opposed to odd)
values in the "seconds" field.
* Any algorithm that makes use of a random-number generator is
potentially non-deterministic.
* Any concurrent algorithm, or paralell processing situation
(e.g., any distributed application), is, by definition,
non-deterministic. (Non-deterministic does not necessarily
mean incorrect.)
In such situations, the error may be reproducible, but the
exact combination of conditions may only occur once every
ten years. (Avoid the temptation to make the algorithm
more (or completely) deterministic purely for the sake of
testing. You must test what you are going to ship, not some
artificial variant.)
-> Suppose that, if the algorithm is behaving correctly, it should
be impossible to reproduce (produce) the error. However, no one
realizes that the original occurrence of the symptom, and its
reproducibility, is indeed an indication of the presence of
an error.
One example of this is the phenomenon frequently occurs in
compilers. A piece of code that is semantically/syntactically
incorrect (and, thus, should not compile) actually does compile
"cleanly." Further, the resulting code executes exactly as its
author expects. It is only when a new version of the compiler
is installed, or the code is ported to a new platform, that
symptoms of the real problem begin to emerge.
(Another similar error is "pseudo-compilation," i.e.,
situations where the compiler generates code whose semantics
do not match what was expressed in the original source code.)
I hope this helps.
-- Ed
--
Edward V. Berard | Voice: (901) 309-1912
The Object Agency, L.L.C. | Fax: (901) 755-5622
2965 Cane Creek Drive | E-Mail: ed@toa.com
Germantown, Tennessee 38138 | WWW: http://www.toa.com
|
|
| | From: | Bjorn Reese | | Subject: | Re: Every software defect should be able to reproduce?? | | Date: | Thu, 13 Jan 2005 21:12:04 +0100 |
|
|
| Edward Berard wrote:
> -> Suppose that you are dealing with a non-deterministic algorithm.
Or a learning algorithm (think: Artificial Intelligence).
--
mail1dotstofanetdotdk
|
|
|
