Programblings

Rambling about programming and life as a programmer

An easy way to make your code more testable

Posted by webmat on December 13, 2007

Update: This article has has a new home on programblings.com. Go to the article.
Comments are closed here but still open there :-)

James Golick wrote a very good article about testing a while ago. In it he dissects (and refutes) the too often heard arguments where people say they don’t write automated tests because they don’t have the time.

In the comments, some people concluded that yes, they should try to write more tests, but didn’t know where to start. In this post I won’t suggest frameworks, or specific tutorials. I’d just like to give one very first step that will help you write code that is easier to test. You’ll benefit from it even if you don’t use a testing framework yet.

As the title suggests, what I’m suggesting is pretty simple. Write side effects free methods/functions. Simple isn’t it? The rest of this article is just about explaining my point. So if the light bulb went off already, you can stop reading now.

I’m kidding, of course! So let’s not take anything for granted, instead let’s make sure we’re on the same page and define “side effect free”. It means that a function (substitute with “method” if you like) receives parameters, spits out a result and has not touched anything outside of it. There are two key parts to this definition:

  1. The function does not depend on anything else than it’s parameters: it does not expect a variable to be set outside of it to work properly (at the object, class or global level).
  2. It does not modify anything. Its result can be entirely observed either from the return value or from the exception thrown.

Of course you often have to modify the state of the application as a result of a computation. What I’m suggesting is not a substitute for that. What I’m suggesting is simply to put the juicy bits of your computation in a side effect free function. This part will be trivial to test, but you’ll still have the code that uses this function. That other part, which modifies the state of the app will need unit tests or other higher level testing.

Trivial examples of side effect free functions can be found in any good math library supplied with a language. Of course no modern language expects you to set a global variable in order to compute a square root. Those who want to follow the Ruby examples can do so on Try Ruby (fear not, you’ll be able to follow along even if you don’t know any Ruby).

Math.sqrt(4)

=> 2.0

And

Math.sqrt(-1)

Errno::EDOM: Numerical argument out of domain - sqrt
from (irb):6:in `sqrt'
from (irb):6

So there we have it. The result of squirt is observed either from the return value or from the exception thrown. We don’t expect any state to have been modified anywhere else in the application. I’ll present a less trivial example in a bit, but first let me just say that the direct consequence of writing this kind of code is that you can test it trivially, whatever your technique of choice.

  • If you’re in your debugger or in your interactive console, you can call it as many times as you want, with different parameters and check out if its behavior is what you expect.
  • If you use unit tests, you can code the interesting scenarios and verify their expected outcome, only in a repeatable manner.

Now since we all have a Math library of some sort in our language of choice, let’s look at another example: analyzing the parameters that will dictate the execution of your program. This is valid for configuration files with a bit more work, but let’s keep the example simple and just analyze command-line parameters.

Most languages provide us with some kind of array of parameters when entering our main function. The common way of dealing with them is to slap a big if or switch statement somewhere at the beginning of your program, which sets the state of the application accordingly, before actually starting to work on the application’s main task.

A side effect free approach would be to split the process in two parts:

  1. parse the parameters
  2. set the state / do some work

For example we could define a function that accepts an array and returns a hash (a Dictionary for .Net folks, a Map for Java folks) of the execution parameters:

{

'arg1' => 'val1',

'arg2' => 'val2'

}

So let’s say we start with the following method to analyze our arguments (to see the readable, indented version of this code, look at it on Pastie) :

def analyze_args(arg_list)
parsed_args = {}
#We check that each argument is in the form 'arg=value'
arg_list.each { |arg|
key, value = arg.split '='
if (key.nil? || value.nil?)
raise "Some arguments are not in the 'arg=value' format"
end
parsed_args[key] = value
}
return parsed_args
end

Now we can trivially test the parsing of the command-line params:

analyze_args(["mom"])

RuntimeError: Some arguments are not in the 'arg=value' format
from (irb):23:in `analyze_args'
from (irb):20:in `each'
from (irb):20:in `analyze_args'
from (irb):29
from :0

analyze_args(["mom=food"])

=> {"mom"=>"food"}

analyze_args(["mom=food", "dad=car"])

=> {"mom"=>"food", "dad"=>"car"}

Now that this part is taken care of, I can test it with whatever input I want, trivially.

To reiterate, in order to make your code easier to test, just extract the juicy bits of your program in side effect free functions and keep them apart from the rest of your program, which in turn makes sense of the result. At least the side effect free parts will be trivial to test.

This is just the beginning of the testability and automated tests journey, however. Of course you still have the rest of the program to test, preferably in an automated fashion.

About these ads

25 Responses to “An easy way to make your code more testable”

  1. pete said

    Maybe I am stupid but I see only one use case for a test – that is to ensure that your application is running as expected on other machines as well. I dont believe that anyone who does intensive testing wasn’t a professional coder who has to use tests anyway for various reasons.

    But when I actually go and read test stuff for a programming language, they seem to rather focus their minds on the whole testing aspect.
    Do they not allow that, if you use objects for a long time in their “expected” use area, these object will specifically work very well, simply because author(s) improved them over time?

    Recently a few of us (3) finished a small game project. It was inspired by a python blog, where someone wrote about a 48 hours game project (ours was only 6 hours but it was still interesting).

    There were 0 tests involved, but the game works (has only 12 .rb files and about 16 classes).

  2. James said

    basically – what you are saying is “in order to unit test your program, use a functional programming language”. The whole idea of OOP is to take into account the state of an object within methods. In fact, methods on objects are useless unless they take into account that state (otherwise you would make them static class functions).

    Testing OOP languages is almost useless (IMO) since every method by definition should either alter state or return different results based on state. Anyone who has tried to test a sufficiently complex class will find that the amount of work to set-up the internal state of an object to test a single use-case for a method prevents even the most keen programmer from doing so.

  3. @James

    “Anyone who has tried to test a sufficiently complex class will find that the amount of work to set-up the internal state of an object to test a single use-case for a method prevents even the most keen programmer from doing so.”

    Sounds like poor class design or laziness on the part of the developer. The reason most developers don’t want to create repeatable tests (and I am guilty of this as well) is that it’s boring. Not testing stuff because “it’s a lot of work to set it up” is the same as a little kid stamping his foot on the ground and saying “I don’t wanna!” when asked to clean his room.

    If writing tests in OOP languages is so useless, why do so many people do it?

  4. Danno said

    It’s also easier to test when you take testing into account from the beginning. Leads to writing objects with less complex internal state that is more easily setup with mocks and stubs.

  5. localhost said

    @Pete

    “Maybe I am stupid but I see only one use case for a test – that is to ensure that your application is running as expected on other machines as well. I dont believe that anyone who does intensive testing wasn’t a professional coder who has to use tests anyway for various reasons.”

    On a large application with a lot of developers, you’d be surprised how easy it is to change a single function and break your application in completely unexpected places, many of which you’ve probably never even seen before. Having a strict automated nightly build process in conjunction with unit tests is basically the only way you can control this with any confidence – it is impossible to re-test everything by hand every time a change gets made. Writing decent tests ma seem like a waste of time if you’re working in a small team, but it can save weeks/months on larger projects.

  6. Sam Kong said

    Interesting article.

    I once tried this approach myself but eventually gave it up.

    In OOP language, side-effect-free methods break the rule of “low coupling, high cohesion”.
    Methods in an object should be cohesive to each other, which means that they should share instance variables.
    Math.sqrt is a static method and doesn’t have to be cohesive.

    More concretely, if methods are not cohesive, they will unnecessarily have more parameters like in procedural language.

  7. webmat said

    @James, @Chris
    I agree with Chris, in that if testing a given class is so tiedous, the design is probably at fault to a certain extent.
    For very central classes, sometimes it’s very difficult to reduce the coupling, however. But there are also techniques such as mocking (to stub, or fake some dependencies) which can help in these cases.
    I don’t agree with the “so many people do it” part, however ;-)

  8. webmat said

    @Pete, @Localhost,
    Automated testing has many goals, here are 2:
    – The most important reason for automated testing for me is to have tests that check continually (with no effort on my or my team’s part) that everything still works as before. This ties in to what Localhost is saying.
    – In a TDD approach, you define the expected behavior, then implement it. The benefits are often debated. I’m interested in it, but haven’t had a chance to use the approach extensively yet. If you want to read more on the approach, just search Google or read more of Jame’s articles (mentioned at the beginning of the post).

  9. webmat said

    Oh, and Pete, if you implemented a working game in 6 hours, I can guarantee you that your premise is wrong: you’re not stupid ;-)

  10. webmat said

    @James, @Sam Kong
    I’ll have to make a follow-up post on using static methods to advertise that code is side effect free (only to a certain extent, static members and other objects can still be affected). It’s a method I’ve been using and I’d like to talk about it in more details.
    I especially like it because it offers a small guarantee, but it saves me from totally abstracting it from the current class and putting it in a library. Not all code should be made shareable, especially not up front, when there is no immediate benefit.
    This method allows you to separate it without too much effort, and only when you realise it could be made available elsewhere you can come back, extract it and generalise it a bit more.

  11. I did not find this very helpful. Yes, it is trivial to test side-effect free functions such as sqrt. But IMHO you fail to explain how to actually extra “the juicy bits” of my program into a side-effect-free function?

    For example, if you want to test an implementation of a stack, I don’t quite see, how you can implement this side-effect free. Unless, you use a functional programming language in the first place, of course.

  12. Great article, Mat!

    I am often asked: “How can I test this method – it’s untestable/too hard to test?” The implication, there, is that testing is too hard or impossible in certain situations. While the asker of said question is often right, the (un)testability of the code in question is nearly always a result of problematic design. When code is written without consideration for testing, it becomes easy to get trapped in a mess of untestable code. It makes sense – if you’re not thinking about it, why would your code turn out testable?

    Testing is a necessity (no space to prove that statement here), so testability is a requirement. If your code is untestable, you haven’t met the testability requirement. No matter what you think of it, test driven development is a near sure fire way to ensure testability. Because you write your tests first, you can’t (or are far less likely to) trap yourself in a an untestable situation.

    As far as side effects are concerned, with respect to classes, just treat the internal state of the class as though it were the method’s parameters, and return values. Whether the input comes from a class-scope variable, or is passed in to the method through arguments is irrelevant. Find a way to reproduce each potential situation in isolation (typically using mocking). Mock out any external dependencies. Then, you’ll have a side-effect free method (effectively).

  13. Patrick said

    Sam Kong made an excellent point. Objects should share instance variables and thus be far from side-effect free. Making code more testable is more about function breakup than anything else. When I’m testing my classes, I make sure I can figure out (manually) the state of my object after a given function call. That way, all I have to do is pass the object, run the test, and compare the object to truth. Done.

  14. @Andreas,

    A stack seems fairly easy to test; perform operation, check element on top and bottom of stack, and count of items in stack. That would reasonably assure that your stack is working correctly. Am I missing something?

    Overall I agree that creating a side-effect-free non-trivial object is harder than isolated functions or FP-oriented code.

  15. […] An easy way to make your code more testable James Golick wrote a very good article about testing a while ago. In it he dissects (and refutes) the too often heard […] […]

  16. Stu said

    Sam and Patrick are correct… in OOP, an object is data + behavior, which means that methods answer state and/or modify state. In a way, good OOP is all about side effects. Thinking about coupling and cohesion is a GOOD thing.

    I would assert that, contrary to the static method example given as a “good idea”, static methods and instances are a BAD idea in OOP, especially when it comes to testing. Static methods cannot (easily) be substituted by the test code to force certain code paths to be tested, and static variables are difficult to properly test.

    Testing in an OOP language should be easy – create the object, set it to a known state, invoke the method, and inspect the new state. Problems with testing tend to arise in a couple of areas: (1) creating the object is a laborious and tedious task, generally because the object is complicated and/or poorly designed; (2) the object exhibits poor cohesion and widespread coupling; and (3) the object’s state is difficult or impossible to determine or interrogate.

    (1) is an effect of leaving testing to a later part of the development cycle; testing early in the development cycle encourages the developer to make their life easy when it comes to creating an object with a known state. Sometimes what is required is a builder-pattern, where a complicated configuration string (structured-key-value-pairs or XML or S-expressions or P-lists) can be provided to a helper class that spits out the appropriate object.

    (2) is quite simply a design problem, often due to (in my experience) insufficient foresight or excessive fondness for the code. (Sometimes, you just have to throw the first one away. If the cookies give you indigestion, frosting won’t help.) Foresight is wonderful, but not dependable; it can warn you away from problematic paths, but it can’t guide you towards the minimum-effort-maximum-success path in the general case.

    (3) is a natural consequence of (2), but it also a natural consequence of hurried development, sloppy development, poor or inexperienced developers, and (what I call) demo-driven implementation. When you’re trying to get some code out the door yesterday, it’s natural to not think about testing, documentation, cohesion, coupling, appropriateness or lack of same of the “static” keyword, etc. It’s natural. But it also causes problems down the line. The way to deal with this is to have a strong aesthetic (if you puzzle out what a method does, do you write a summary comment for that method? If not, why not — should the next programmer have to duplicate all of your effort? Why?) and to, naturally, never forgo testing.

    I just wish I were better at paying attention to my own advice.

  17. Kabarety said

    “So let’s say we start with the following method to analyze our arguments (to see the readable, indented version of this code, click here) :”

    Thanks, great article… ;]

  18. Rob Whelan said

    About coupling, testing in OOP, etc.
    @Andreas — the stack example is a good one. You shouldn’t try to somehow implement it as static methods (bad OOP.. it naturally has state), but if you cleanly separate your stack functionality from your *other* code, that’s both good OOP and easily testable.

    The side-effects of the Stack methods are simple and restricted to the instance, and hence straightforward to test.

    The bad OOP approach (and the test-killing approach) is to just put a List in your business logic class, and *treat* it like a stack — pushing items into one end, pulling them back out and deleting them…. Moving *that* code into a clean stack object puts a bunch of code into the realm of easy test coverage.

    [Of course, it’s even better just use the stack impl likely built into your language of choice.. but hey, we’re talking concepts here.]

  19. […] recente artigo An easy way to make your code more testable, no blog Programblings, me levou a outro artigo […]

  20. Nick said

    Quite right. Command-Query separation is needed.

    My personal pet hate is functions like this

    x = rand()

    Its both a command and a query, and difficult to unit test.

    In reality, it should be an iterator

    x = random.value()
    random.move_next()

    Then you can test it.

    The other advantage of command query is you can write code in this way.

    if x.invalid():
    do_something()
    else:
    x.calculate()

    instead of

    try:
    x.calculate()
    except:
    do_something()

    I personally don’t like exceptions being use for flow control, unless you really, really need to. e.g. Inverting a matrix where the query on the matrix, is_inveritable() is very computationaly expensive.

    Microsoft are awful at not having Command-Query separation.

    Next stop, a decent design by contract implementation in dot net.

    Nick

  21. […] your code into a set of smaller functions that can be tested individually. I find that code that is easier to test is also easier to […]

  22. […] Side effect free functions also make testing & refactoring easier (less state to worry about etc) […]

  23. […] your code into a set of smaller functions that can be tested individually. I find that code that is easier to test is also easier to […]

  24. save gas said

    save gas

    An easy way to make your code more testable « Programblings

  25. low-priced nissan parts

    An easy way to make your code more testable « Programblings

Sorry, the comment form is closed at this time.

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: