Testing is Like Fishing
The longer I do it, the more I realize that testing software is a lot like fishing.
Now I’m no expert on fishing, but I do know some things about fish:
- different species of fish like certain kinds of bait
- some fish live only in certain areas
- fish have a lifespan and will only be available at certain times of the year
- particular fish require different equipment to catch them properly
All four of these points need to be considered in order to have a successful catch. I don’t know, maybe there are more than 4. But let’s consider an example: fishing for marlin.
A marlin is a deep-sea fish. Deep-sea means salt water.
They’re also a big honking fish, so it requires a deep-sea fishing rig.
There’s probably a good time to catch them–a time when they’re at peak maturity, or mating, or closer to the surface or something.
Being big fish, they eat other sizable animals. Google says they like tuna.
Huh. I just realized, an acronym can be used for this. BELT, for Bait, Equipment, Location and Timing. That’s pretty cool. We’ll come back to that later.
Anyway: All four conditions have to be met in order to catch this fish. If even one is wrong, the chances are slimmer that there will be a successful catch. Fishing in a freshwater lake, using a wimpy fishing pole, fishing at the wrong time of year or fishing with worms instead of tuna, will zonk the success rate.
This guy? He’s prepared. And he’s probably got a marlin on the end of that thing he’s holding in his hands.
This guy is prepared but is missing the critical component of fishing at the right time of year. He may not catch a marlin today.
In the same sense, when you begin the hunt for a bug, consider the following:
- not all bugs are equal–there are certain kinds, or species, of bugs. What kind do you want to catch?
- not all bugs exist everywhere–bugs of one ilk are easy to find in one place, but rare in others. Where are you looking for bugs?
- some bugs only exist at particular times. Are you looking at the right time?
- some bugs won’t reveal their presence unless you provide the right stimulus. Are you using the right tools and data for the job?
So back to the BELT acronym–Bait, Equipment, Location and Timing. And, time for another example.
Let’s say the species of bug you’re looking for is a crash in a C++ application. Crashes can be triggered when bad or unitialized data is passed between functions, or if the application reads something wonky into itself.
What would be the bait in this case? How will you provide a stimulus for a bug of this sort to show its face? Does the application read data in from the command line? A file? A database? A network?
What would be the equipment? What tools will you use to assist in finding this bug? Maybe a script to create some garbage input. Maybe you can catch problems even earlier by using a code analyzer, or by using different compilers set to be as strict as possible.
Where would an ideal location be? Where would you expect to find this type of bug? At the application’s entry (such as initially reading a file), or deeper in the code (arguments passed between functions)? The approach would be different depending on where you’re fishing (deep sea vs. a freshwater creek).
When would you expect to find these issues? Is there a particular time you might find a crash? Does your application behave differently at certain times of day, or when certain other events are happening?
Hmm… Intriguing. What are you testing right now that could benefit from this approach?
I can’t believe it took me this long to notice and then apply this principle to testing. I mean, if you’re a software tester, how many times have you noticed that sometimes we just run tests, and the tests run and none fail, and everything’s great? It’s great! Awesome! Ship it! Probably plenty, right? We like when tests pass–it means the product may be pretty good. Not perfect, but it’s at least less broke than it was.
But think about this: running a set of tests, and not having particular types of bugs in mind, is like dumping a tackle box into the ocean. If you don’t know what you’re fishing for, how will you be able to catch it? Mind. Equals. Blown.
Hopefully this has helped–consider what kind of bug you’re looking for, and don’t forget your fishing BELT.
Happy fishing!
Ready to get a strategy on how to catch the kinds of bugs slowing your company down? Schedule a free 15-minute consultation call here.