Multi-user testing has been on my mind recently. I’ve tested three different applications within the last two months for MU issues and in all three applications I’ve found defects. One book I’m currently reading is The Little Book of Semaphores . I just discovered the book although its been around awhile. Its helping me generate ideas about MU testing. The book holds a collection of stories created over the years to describe concurrency issues and related problems and possible solutions.
Edsger Dijkstra created the first story in the series. He used the story format to explain and teach in a tale widely known as the dining philosophers problem. This story works well for me because concurrency is a topic of interest and I enjoy a good story. I like the dining philosophers problem because it’s so accessible. And I admire how Dijkstra stepped outside of any specific programming language and computer terms to build a story that illustrates a problem and possible solutions. Deadlocks, race conditions, and starvation of resources are makings for a good testing story.
I first learned about MU testing from a previous boss of mine, (thanks again Bob). And I continue to learn about MU testing. These are some of my unrelated thoughts about MU testing.
I don’t hear deadlocks or concurrency testing discussed very often even thought as recently as within the last two months, I’ve found issues in this category. How odd because I would think any category of testing that reaps bugs would be discussed more often.
I suspect, although I have no proof is when the word concurrency is raised, we leap to thinking about performance testing and volumes of users. In my mind and in my MU testing, it’s not about volume. It’s about timing. After all in the dining philosophers problem, there are only five philosophers mentioned – not a dozen or hundreds of philosophers and yet it’s enough resources to cause problems.
Another thought that comes to mind is how easy this form of testing is. Forget scripting, two keyboards and solo testing can cover a cycle of MU testing. I guess that’s my opinion but I think it’s a short fairly easy cycle to cover.
Very specifically the testing I do in this category can be reflected as 3-2-2. There are three operations I focus on: add, update, delete. There are two timings I focus on: same time and staggered timing. And two users because that’s all the users I need to simulate in order to find a problem.
Another reason the topic is on my mind is that I’ve written an article about MU testing which should be out early in ’08 as long as my editor is ok with what I’ve written.
Something I don’t know but would like to figure out is – are deadlocks more likely or less likely to take place with faster processing PCs and servers? My first inclination is no because the speed of grabbing and releasing a lock on an object might be faster. But then the more I think about it, perhaps with faster processing, collusion is more likely. I just don’t know. All I do know is when I grab two keyboards and test with frequently used objects in the applications I’m testing, I can usually hit some type of error. Good enough for me.