mj writes:
> I'm a big believer in practical testing. That means
>you test the HELL out of stuff you know from real life experience
>is likely to break.
and:
> Testing where you know applications typically have
>problems is really really important because that is where
>you get your bang for your buck.
To which I must take exception! The unpleasant truth is that the
engineer who wrote the thing hasn't got the foggiest idea where the bugs are.
It's just like performance tuning, you might be aware of some problematic
areas, but you can't really know unless you are a Scientist and Find Out
where the problems are. The only way I know of to do this is to do testing
with some form of complete coverage -- lines of code, customer viewable
behaviors, something.
Eric Allman has been testing sendmail for apparently centuries now,
and he's probably well aware of what he thinks are problem areas in sendmail,
but he's never actually gotten it really right.
Now, I suspect that what Marcus is *really* saying is that you should
design so that the scary bits are more or less provably some small subset of
the actual code, and then exhaustively test these scary bits. Test the
non-scary bits too, of course, but if you know that sendmail can't hurt you
because you have it penned up by some very well tested code, you're in fine
shape, and don't need to test anything beyond 'yeah, sendmail gets mail
,' probably. This I agree with.
Andrew
|
|
