By "Users as testers" I don't mean the way some companies throw out software which isn't quite finished, without admitting that, and let people find the bugs.
I'm talking about normal human beings, as opposed to IT people, who get to use technology that previously would have been unavailable or require too much specialist skill. This article is for you normal human beings, and says what you need to know about testing stuff.
Thinks break by themselves
I remember someone who had set up some sort of automatic process and was surprised when it stopped working, without them realising. He said, "I assumed that once it was set up it would keep on working". It would be really nice if that was the case, but unfortunately it's not. The trouble is with processes like this, like something that sends you an email if some rare event happens
- something somewhere along the line gets changed and things don't fit together and they stop working
- something was a paid service and someone stopped paying the bill, or a credit card expires and no-one notices
- a computer gets upgraded and the thing is no longer possible with the upgraded system
- someone leaves, their email gets closed down, and then no-one gets notified
So if a process is important to you, then it's worth checking regularly that it works. How regularly? Ask yourself how soon would you like to know that it stopped working.
If you change something then you have to check it
Many years ago I was working on a system for controlling gas pipelines. Something asked me to make a very small change. I didn't check that the change worked, as they were in a hurry. They took the tape the next day (it was many, many years ago) and went to the control room and installed the new version. It didn't work, so their whole day was wasted.
Even if you're not a programmer you might:
- write a spreadsheet formula
- create a form
- use some fancy website tool which links two things together
If you change something then you have to check that it works. The thing you change might be something as simple as adding a new thing to a list. There are several ways that just adding the thing may not work out:
- There are in fact two lists, and you need to add the new thing to the other list too
- You may have typed something in that's too long, and it doesn't work
- There are a maximum of five items in the list and you just added the sixth
So if you change something, then check that whatever you changed has the effect that you want.
The other check
The other thing to check is that everything else works as it should before. As I said above, just because something was working doesn't mean it will carry on. In making a change in one area something else may now not work, as a side effect of what you've just done. It's not going to be practical to check every aspect, but you can check two things:
- If you have an idea how things work underneath (and you may not, some things are very mysterious) then you can check the things that are linked to what you changed.
- The important things - do they still work?
How to check
I can't give much specific advice on this, because a lot depends on whatever it is you're changing. Here are some general tips though:
- Check things two ways, such as adding up numbers from two different sources that should be the same
- Assume you know nothing about how it works (which may be true - see "mysterious" above), check that it does what it should be doing
- Check the extremes - the biggest or smallest numbers
- Put in rubbish - if a number is expected then how does it handle ?Elephant99!
- Try different combinations. If you have something that can be A or B and something else that can be X or Y then try AX, AY, BX and BY in case there's some interlink