No preparation for tests = easiest way of project failure

Not long ago my friend cursed his P2P-like program as it seemed to be an endless well of bugs (multiple clients synchronization issues). I’ve talked with him, and this is basically how the talk went:

Me: So why didn’t you test (i.e. automated unit tests) it step by step, module by module with different number of stations?
Him: It’s easy to say for you. It will be very hard to test the network stuff.
Me: Aren’t there any tools to facilitate such tests?
Him: There are, but they are quite expensive and still require a lot of work.
Me: But why are you so inclined on testing the protocol (TCP/IP)? You should rather test the logic.
Him: Logic is dependent on the protocol so how can I do it?
Me: Don’t you have good interface/implementation separation?
Him: … nope.

As you may guess he continued with some manual testing until he found the causes of failures, at least most of them.

Basing on this story I understood why so many teams still don’t test their software enough. It’s because test-unfriendly architecture requires a ton of resources to test. Any by testing I don’t mean some rough manual tests to find if the code builds and doesn’t throw an exception in the first 5 … make it 3 … seconds! I mean automated unit tests run every day, every hour, every commit, all the time.

First, read the all popular Hevery’s guide on writing testable code. Remember: testing is easy, writing testable code is hard.

Second: take a look at the following diagram:

This is my personal ranking of stuff which is hard/easy to test/work with. The lower the element that you are interested to test is, the harder it will be to do it.

In other words, it’s very hard to test thread schedulers. You will only be able to create random tests that will basically play around with various thread counts, affinities and Yields. This will at least give you some security against pesky concurrent issues. On the other hand, if your application deals a lot with files, you will probably not have many problems with it. You will just create a set of test files, and then pump it through your logic.

Let’s say you identified that dealing with web communication will be a considerable part of your application. So what to do now?

  1. Treat the communication protocol as the biggest national secret that no class should be aware of.
  2. Describe the functionality of your code via abstracted interfaces (top-down).
  3. Verify (prototype) that you can create tests basing on the abstracted interface.
  4. Verify (prototype) that low-level communication protocol can be operated via interface (bottom-up).
  5. Make necessary fixes keeping the interface as abstracted as possible.

It’s obligatory that you prepare tests for your application, and I strongly recommend that you treat testability as one of architectural priorities. Ask yourself: “How will I test it?“, and have a solid answer on that.


Tags: , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: