Brendan Enrick

Daily Software Development

Time-Tested Testing Tips - Part 5

Welcome back for another exciting tip for those developers writing unit tests. Today we will be looking at assertions in unit tests.

Only Assert On One Case Per Test

When people start writing unit tests they do the natural thing with their Assert statements; they write a bunch of them in the test. This makes a bunch of sense, because you don't want to repeat yourself in your tests. This is not bad as long as all of the asserts are asserting the same expectation with different values. If you are testing different cases then they need to be separate test.

The reason for this is that unit testing frameworks tend to abandon a test as soon as an assertion has failed, so you get less information if you have grouped many tests cases into one test method. By making sure that every test method contains only assertions dealing with the case in question, we will garner valuable information from each test. Maybe we will discover that the cases with negative numbers and the edge case, zero were the ones to fail. If we had grouped the assertions into one method, we would have only had one failed assertion and wouldn't have gotten all of this info. For example we might have only had zero fail if its assert was first, and then we would be thinking this was a problem with the edge case when it was really a problem with all numbers less than one.

Group Your Asserts Together

If you can, try to keep your asserts together at the end of the method. This will make them much easier to read and keep track of. I've seen some tests where someone wanted to assert before changing a value. A better way to handle this is to store the value you need to hang on to in a variable and assert at the end with everything else. This will make your tests a lot more maintainable, and someone down the road editing the test will sure thank you.

Keep Your Assertions Concise

You should have no logic in your assertions. Don't ever access a property or call a method in an assert statement. These are what someone looks at to try and figure out what is going on. Use simple, concise variable names which explain things. If assertions are cluttered and confusing, your tests will be very difficult to use.

Remember to be careful with your asserts. They should be clean, easy to read, and managed nicely. Their maintenance is extremely important to your tests.

Writing Testable, Maintainable Code

At our company we’ve had a few interns start this summer, so we ran a little workshop with the development team to help teach the team a few things about writing tests before writing other code. In our test driven workshop we did some simple problems in teams. We looked at different problems from Project Euler, and solved them in pairs. We all worked out our own tested solutions to the problems and brought them back to discuss. While looking at these we discussed what was good in each one and what could have been done better. We were pairing more experienced testers with the new guys for this exercise.

I think this went very well actually. These might not be production level challenges to work, but the variety of designs gives great insight into what people were thinking and how they were approaching the problem. I was delighted to see that there were small bits of code very similar between designs, but overall none of them really looked similar. Class names here and there might have been the same, but the implementations of classes with the same name could vary wildly.

After we finished with the exercise, we jumped back into teams to continue working. A while later I got into a discussion with one of our full-time developers and an intern where I ended up explaining one of the main reasons we like testing code. There is the obvious security of having the tests, but there is more than just that. One added bonus that not everyone seems to realize is that writing testable code creates some interesting properties in the code.

Why don’t we take a look at a few of the properties of testable code. I will certainly not cover all of them, but I will try to get enough to demonstrate my point. (these are in no intentional order)

A Few Properties of Testable Code

  • Keeps dependencies to a minimum – Having fewer dependencies means less mocking, faking, and stubbing.
  • Follows Single Responsibility – This keeps things as small pieces which makes for smaller easier to understand and maintain tests.
  • Programming is done against interfaces – This allows you to mock and fake these objects, because your code only knows about the interface.
  • Dependencies are injected into classes – Without doing this unit testing is impossible and only integration tests could be written.
  • The code is well documented through the tests – The tests themselves describe how the code works, and this documentation stays up to date.

The neat part about this is the fact that these properties are present in maintainable code as well as testable. I'll sit here a moment while that sinks in..... Yes, you heard right that in order for code to be testable it must also be maintainable. If you ask someone how to write maintainable code, they might be able to spout off some information about theoretically how to do it. What is much harder is actually implementing solutions which are maintainable. This is why testing is important, because if you want to even be able to write the tests, the code first needs to be written in nice decoupled, well-organized ways.

I don't know about you, but I think it is very cool when you see that the properties of testable code seem to coincide with best practices for writing maintainable code.

Time-Tested Testing Tips - Part 4

Rather than spending the time with needless introduction, I think I’ll just jump right in today. I’ve got a few tips I am going to post today.

Reproduce Bugs Using Unit Tests

Yes, this is another test driven development method. When you’re looking for a bug, it is very common to try to reproduce it. What you might try to do instead of doing this is reproduce the bug manually, fix the bug, and then write a test to prevent it. Well I think that is a pretty bad way of doing things. Reason number one is DRY; don’t repeat yourself. Why did you reproduce it once manually and once automatically? I also wonder how you know you’ve written the test correctly. If it never failed you can’t be certain you’re testing the bug.

If you write this test first you jump right to finding the bug. Seeing the red of the test failing tells you that you’ve found the bug. Having this test will also let you run the buggy code multiple times, which can be useful if you can’t tell right away what is causing the issue. This becomes a faster, easier process. Since you managed to get the failing test, you know that once you’ve fixed it that you’ve at least fixed the bug you tested. Going the other route you were left to assume that you fixed the bug. We all know what happens when we assume… Yes, that is correct, we are sometimes not accurate.

Keep Dependencies to a Minimum

Yes, this is another situation where we are thinking about both the test code and the production code. If you’re finding that the objects you’re testing are requiring a lot of set up you probably need to simplify things. Get some of these tests in place first. These will help you to clamp down the behavior you want to maintain while you refactor. You’ll want to observe plenty of principles while doing this especially the single responsibility principle.

For example if you have a method which takes in a parameter called BankAccount, but all it really needed was the AccountNumber you should alter your method so it takes the AccountNumber instead. This will make testing easier and it will also lessen your dependence. These are great things to do.

Do not pass an object used to get another. Pass only what is required for something to achieve what it needs to. You need to keep things minimal in this sense. It will make testing as well as maintenance much easier.

Make Things Work Before Making Them Right

It is difficult for a lot of us to follow this, because we know so well how bad some types of code are. We try to avoid having ugly code by spending lots of time and effort trying to get things right the first time. Then we aren’t even sure they work.

Plenty of times in the past, I’ve had programming partners who wanted me to write my code better than I did the first time. Sometimes I made the foolish mistake of listening to them. If you’re dealing with some complicated logic you need to write it the ugliest easiest way you can. This will let you get your tests passing. Once your tests are passing you are free! By knowing that the test and the code are working you have freed yourself up to refactor. You don’t have to think quite so hard since you have this safety net in place. Try something. If it does not work, just revert back and try something else. You will quickly find a design that you like and keeps those tests passing.

Sometimes just seeing the working code will lead you to seeing a better design for it. This situation is much better than if you had spent a lot of time coming up with some way of designing it. The design might not even have worked the first time, and then you would really be wasting time.

I hope you’ve enjoyed these tips. Remember that testing takes a lot of practice. You’ll never see any benefits from testing if you don’t keep writing tests.

Time-Tested Testing Tips - Part 3

Context switching is very costly and this same issue can be seen when writing tests, but I would argue most importantly in maintaining and reading tests.

There are three main parts to a test. The first part sets everything up, the second part takes some action, and the third part expects certain results given the the start and the action taken. In this tip I'll be talking about the first part, the setup.

Keep Test-Relevant Details Visible

I've said in the past to treat test code just like any other code. However, there are a few reasons to break from this rule. One is the context switching which will be caused by extracting information which is important to a test. Let me explain what I mean with two examples.

Example Using Helper Method

[Test]
public void CalcInterestRoundsToTenthPennies()
{
    decimal initialMoney = 100.00;
    decimal expectedInterest = 33.333
    InterestCalculator ic = GetTestInterestCalculator();
    
    decimal actualInterest = ic.CalcInterest(initialMoney);
    
    Assert.AreEqual(expectedInterest, actualInterest, 
      "CalcInterest did not round to a tenth of a penny correctly");
}

Example Without Helper Method

[Test]
public void CalcInterestRoundsToTenthPennies()
{
    decimal initialMoney = 100.00;
    decimal interestRate = 0.3333333333;
    decimal expectedInterest = 33.333
    InterestCalculator ic = new InterestCalculator(interestRate);
    
    decimal actualInterest = ic.CalcInterest(initialMoney);
    
    Assert.AreEqual(expectedInterest, actualInterest, 
      "CalcInterest did not round to a tenth of a penny correctly");
}

Notice here that you can tell a lot more about what is going on in the second one, because you know what percentage is being used to calculate. You cannot tell that the first one is accurate since you can't see the percent. I recommend trying to keep the amount of values to a minimum and if you're writing your code well you'll have few enough dependencies that you will not have a problem. These helper methods as you can see make it so you will need to go to another location to see what was initially created. In some instances they are useful, but use them sparingly as they might hide away details.

Do Not Unit Test Third Party Frameworks

Make sure that what you're testing is part of your code and not someone else's. If you're testing code you have no access to you better not be keeping that around as an automated test. You're wasting your time if you're creating automated tests for this code. If you find a bug in the code you can tell someone about it, but you probably can't fix the code. If it is an open source project you're testing then go test the code in their test library and fix issues you find. You will be doing them and yourself a favor. Don't clutter your own test library.

Write Automated Tests Whenever You Are Curious

If I ever wonder how some piece of code or a class works or something, I could go write a simple application and test something out. I certainly do not waste my time though. It tends to be much faster just to write a test to answer my question or learn something new. If I want to try something out I just write a test. If it is an internal piece of code I go look at the existing tests, and if it is not I write a temporary one just to try something out.

Unit tests are quite useful for this sort of thing, and since I have keyboard shortcuts mapped to run them they're very fast. If you really want to test a lot, you need to start using them all the time. They become very easy to come up with and write once you practice them enough.

Time-Tested Testing Tips - Part 2

In the first part of this series of testing tips, I mentioned a couple of tips I believe to be quite useful. I am going to continue this series today by writing about a couple of more ways to write better tests. I also plan to give reasons for testing and describe different benefits of the practice as I go.

Test Driven Development

Yes, I am going to bring this up. Plenty of people have latched onto the idea that testing code helps make code better. A lot of people even believe that it makes the process of writing the code faster. I see a lot of people who are reluctant still to use test driven development. I know I am throwing around a buzzword… or maybe I mean buzzphrase since that’s actually three words.

I don’t plan on doing TDD justice here since I am trying to keep these tips relatively short. I just hope this is enough to inspire people to go read, learn more, and try some test driven development. I am going to give some quick reasons why developers testing their code should write the tests first.

  • You will not want to go back and write it later. Sometimes you are going to go back and write the tests for the code you have already written, but what happens if you decide you’re going to write a lot of code and then go back and add the tests. Now you’re talking about a lot of testing all at once. There is a good chance you’ll cut corners and maybe skip entire aspects of the testing. I know I would be tempted.
  • How do you know what kind of interface your new code needs if you’ve never used it? If you start by trying to use some code that does not exist yet, you’ll be deciding how you would want to use it. Now you’re probably thinking you could do that anyway. If you write code to use some new feature you will know the design is easy to use and work with. You just made it up and used it in the test. If you didn’t write the test you had to guess what interface you will want later.

Organize, Refactor, and Take Care of Your Tests

I recommend you repeat this daily, “My test code is just as important as my production code”. I think this is a very good point to remember when you’re writing tests. All of those principles you apply when writing production code should be followed with test code. DRY, SOLID, YAGNI, etc. are all important even with the testing code.

Obviously duplicating code can make your tests difficult to maintain. What if some business logic changes? If you were repeating yourself you now have the fun task of going through a dozen tests changing each one, but if you had not repeated the same code you might have been able to update one location in the test code. A lot of people are concerned when the line count of a single file gets large and they will refactor it into multiple manageable files. This same policy should apply to test classes. If you’ve ever gone into a test class with way too many classes, you probably know how difficult it can be to maintain.

Tests exist to make development easier, and if they become difficult to maintain then something needs to change. I certainly don’t advocate spending large amounts of time refactoring the tests, but since they are supposed to increase the longevity of the application they must also be maintained.

Adjust Your Style to Your Test Framework

I obviously recommend that everyone use a testing framework. There are plenty of them out there, and they supply a great deal of the tools you’ll need for testing. They come in all languages, flavors, and colors. You might look into these NUnit, MSTest, JUnit, CppUnit. What is important is that you make sure you know how your tools work so you can test best with them.

I will elaborate on a couple of examples and perhaps down the road I’ll give some more specific examples.

Some tools show you a list of test names, and only minor details about why the test failed. For this you usually need to go the a description view or something similar. In these cases it is important to name your tests effectively.

As an example if I have a method called Add and it takes two parameters a and b. I might write a test for that method. If I name my test TestAdd, and that test fails you know something is wrong in that method, but you do not know what failed. If I had instead made a few more specific methods you could glean more information from the test having failed. Some examples of tests I might create are AddTwoZerosShouldBeZero, AddNumberToZeroShouldBeTheNumber, AddPositiveNumbersTest, AddNegativeNumbersTest, etc.

Some parts I would lump together like positive numbers and negative numbers. It is important to handle a couple of different scenarios as well as the edge cases. I could have done negative and natural numbers and that would cover all numbers, but I wanted to make sure the edge case, zero, was handled correctly, so I test it separately.

People can argue back and forth all day long about whether you should have a lot of small tests or group them together, but this is what has worked well for me in the past and I hope it works well for you also.

Time-Tested Testing Tips - Part 1

These days more and more people seem to be testing. I admit I am one of the developers who has been writing unit, integration, acceptance, regression, and other tests. The idea of testing has been around for a long time, but it seems lately there has been a surge of people beginning to use them. Some are having great success and some are seeing their success fall away. Simply writing tests does not suddenly make great code.

Something I think everyone should know is that testing code is desired because it allows us a few benefits; the confidence that everything is working, the confidence that a bug has been eliminated, ease of maintainability, and extra documentation of a system written in code.

Tests as Documentation

I believe I've said plenty of times in the past that the tests you write become excellent documentation which can be used to demonstrate how different aspects of your system should be used. This is very useful for someone joining a project. If it is well-documented in its tests, learning how to work with the code is as easy as reading through the tests and seeing how the different objects are intended to interact. If you can't do this, you're probably not testing well.

A good measure of your application's testing is to have someone learn to use the system based solely on the tests. If the tests are written well enough, someone will be able to figure out how everything works and interacts.

This is one of my primary goals when testing an application. I want to make sure that things are clear, because when I come back to this code a month from now I will be that new developer on the project. I will need to know how to use the classes, interfaces, etc. and having it documented in working cases goes a long way.

How many tests should I write?

This is one of the most common questions asked by new testers. For now I will take the cowardly approach and say, "it depends". I've read posts from people touting the number of tests they've written. This is a new metric as if "line count" wasn't already bad enough. There is a balance here that we are looking for. Just with the line count.

If your line count is small you're application probably doesn't do very much and if it is large then your application might be cumbersome and difficult to maintain. When you don't have many tests your code is more prone to errors, but if you've written too many tests maintainability disappears. But wait! Didn't I say earlier that tests make code more maintainable. Well, yes I did, but if you write too many it becomes difficult to change the code. If you test every single possible minute little thing in your application you're going to have a heck of a time changing anything.

When you're writing tests make sure you have just enough to give yourself confidence that your code is working.

I highly recommend against ever setting code coverage or test count goals. If you set goals for these you're just creating incentives to write more tests than is required. Too many tests can create the same problems as too much code. Why? Because tests are code.

You need to treat your test code well. It needs to follow a lot of the same rules as the rest of what you write.

Converting an Array of Integers int[] to an Array of Strings string[]

I was working on a code snippet I plan to put into a blog post, and while writing it I needed to convert and array of integers to an array of strings. I try to think of a quick easy way to do this and none comes to mind. Well, I of course know that I could just loop through them and convert them from ints to strings, but that is really lame especially when I want the code to be small so I can put it in a blog post.

So there are a few options that present themselves when you are looking for a method to help you out.

  • Write your own method to solve the problem. (I try to only do this as a last resort. I try to use existing code when possible.)
  • Ask Google (or any other search engine) for answers posted on forums, blogs, articles, documentation, etc.
  • Go ask/search on a forum or a Question and Answer site.
  • Use intellisense and check some related classes which may hold the answer.

In this instance I used the last one. I checked out some classes using intellisense to see what methods were available. I was working with arrays, so the one I checked is the Array class. Array.ConvertAll is what I needed.

var intArray = new[] {1, 2, 3, 4, 5};
string[] stringArray = Array.ConvertAll(intArray, i => i.ToString());

That is roughly how I am using it in my code. Notice the nice lambda way of handling the individual object conversion. You can also solve the problem using a full-fledged method call. I just chose this to keep things more compact.

Here is an example using a method to hide the lambda and do the conversion.

static void Main()
{
    var intArray = new[] {1, 2, 3, 4, 5};
    string[] stringArray = Array.ConvertAll(intArray, IntToStringConverter());
}

private static Converter<int, string> IntToStringConverter()
{
    return i => i.ToString();
}

As usual there are still many other ways to do this, but I'll show you one that just has way too much unnecessary code.

I don't recommend using this example since it really just specifies way more than it needs to. I am just including this to illustrate the many ways to have solved this problem using this same method.

static void Main()
{
    var intArray = new[] {1, 2, 3, 4, 5};
    string[] stringArray = Array.ConvertAll<int, string>(intArray, 
        new Converter<int, string>(IntToStringConverter));
}

private static string IntToStringConverter(int n)
{
    return n.ToString();
}

Why don't I really like that one? Well to be honest I intentionally made it bad. The "new Converter<int, string>()" part is completely useless. You can remove it completely and the code still works, because we specified already that we are doing int to string. We alternatively could have removed the first <int,string>.

Although if I were using an older version of C# I would be forced to use the last one, or I could also use a combination of the second and third one.

Well, we're out of time here today. I hope you've enjoyed this edition of Finding Stuff Using Intellisense. This is your host, Brendan Enrick. We've been glad to have you with us today. Join us next time when we....  Find Stuff Using Intellisense. {queue the ending theme music}

Testing Private Methods

In a previous post about cutting large classes down in size I mentioned that

Sometimes there are methods kept private in a class. Some calculations are kept private because nothing should be calling those methods on this class. This is a good hint that the method belongs somewhere else. If the method is kept private because it doesn't make sense for a user of this class to use it, it belongs somewhere else.

Private methods are a common occurrence in classes. Sometimes they should be moved into another class, because they were only private because they didn't make sense in that class. Other times they are part of the internal workings of the class. At the end of the day it is always up to the developer how he is going to structure his code.

If you don't want to move the method and don't want to make it public you still have a couple of options to test it.

  • You can sometimes test a method through the public methods that call it. (Can be difficult sometimes because it is harder to control what is being passed to the method.)
  • You can write a public method which passes through to it, and prefix the name with "Test". (This is a bit hacky and should only ever be done with internal code that will not ever be in an API.)
  • You can change the method to protected and write a Test version of the class that inherits, and then exposes the method publicly on the test class. (This option works well because the test class can be kept with the tests so it doesn't dirty the production code. Only do this if you will not be subclassing this class already.)

Some people discourage testing private methods, because there really shouldn't be much logic in private methods that really needs to be tested. If it really needs tests it probably belongs in another class. My opinion is that if people are going to keep the code in the private method anyway, they might as well at least be testing it.

Structure your code how you like. Just don't let your classes get out of hand. If it becomes an issue then refactor it.

Friends don't let friends perform premature optimization.

PrematureOptimization

As Donald Knuth said, "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."

I think it is safe to say this should apply to refactoring as well. Don't go on refactoring binges. Only refactor code you're currently working with.

Keeping Large Classes in Line

One thing that seems to get out of control quickly on a lot of projects I've seen is a tendency for classes to grow to enormous size. This is one problem that seems to turn up at least once in a lot of projects. Why does it happen? Because programmers are lazy. Yes, I said it. We are. I think it is a good thing in a lot of cases. We are "efficiency experts". We pay an up-front cost writing a program so we will have less work to do to solve a problem. By writing the code we save ourselves time. This is one example of this laziness.

When we get lazy we tend to group things together. Since the foo class already has access to types x, y, and z, we can just put this new code in foo. It makes sense, but we're really just making our classes larger. This is why the single responsibility principle is so important. This is one rule important enough that developers everywhere should know about it.

Classes should do one, and only one, task. That is their responsibility. If you need to generate some value, write a class to do that. Don't reuse an existing class. I've seen classes which have gotten so large people don't even know what is in them. I would be scared to make modifications to files like that.

Keep your classes small. It is easier to test individual parts. This is why unit tests are called "unit tests". They test a small piece of the code. If you keep classes small and separate responsibilities among classes it is easier to test. You can break dependencies on the other classes using interfaces, abstract classes, or other things.

Private Methods to discourage use

Sometimes there are methods kept private in a class. Some calculations are kept private because nothing should be calling those methods on this class. This is a good hint that the method belongs somewhere else. If the method is kept private because it doesn't make sense for a user of this class to use it, it belongs somewhere else.

Breaking Classes Apart

The best way to cut large classes down to size is to go through and find pieces which don't belong and are grouped together. Often looking for the collapsible regions of code helps. If you've grouped methods into a region there is a good chance those are related methods and probably belong in a different class.

When you create the new class where you will put these methods, the first thing you should do is test it. This new piece should be very testable since you're pulling it out of the beast. If you test this one piece you'll be able to keep things stable while you pull apart the large class. If you aren't testing each part as you break things up, there is a very good chance you will create bugs in the process.

In my opinion, tests serve two very important purposes. They test to make sure that things work as expected, and they help keep code stable. This stability clamps things down and allows changes to be made. Often times after breaking off separate pieces many refactorings which were hard to find in the large cluttered class become quite obvious. This is why it is so very important to break apart large classes.

Returning From Inside a Using Statement

A while back I wrote a blog post regarding this topic. In that post I explain that it is safe to use a return statement inside a using block, because the using block will handle the disposing of the defined IDisposable object. This is one truly great ability of the IDisposable interface. It makes it very important to use in my opinion. The using block in combination is very nice, because it handles the disposal for you.

One of my commenter on that post asked me if I was certain that the return statement was safe. He asked this because he couldn't find it in the documents. He also asked me if I had any code showing that this behavior works as I claim it does. I quickly wrote this simple little piece of code which works to demonstrate this behavior.

In my example I have a few things. I have a class which implements IDisposable and all it does to dispose is to write to the console. I then wrote 3 methods which each create and instance of this class in a using statement. One class is normal; it returns after the using statement is complete. The other two end abruptly. One of them returns from within the using statement, and the other throws an exception within the using statement.

 

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Application Starting\n");

        Console.WriteLine("Before normal method");
        NormalMethod();
        Console.WriteLine("After normal method\n");
        
        Console.WriteLine("Before return method");
        ReturnFromMethod();
        Console.WriteLine("After return method\n");

        Console.WriteLine("Before Throw method");
        try
        {
            ThrowFromMethod();
        }
        catch (Exception ex)
        {
            Console.WriteLine("Caught the Exception: " + ex.Message);
        }
        Console.WriteLine("After Throw method\n");

        Console.WriteLine("Application Ending");
    }

    private static void NormalMethod()
    {
        using (MyDisposable myDisposable = new MyDisposable())
        {
            // do nothing
        }
        return;
    }

    private static void ThrowFromMethod()
    {
        using (MyDisposable myDisposable = new MyDisposable())
        {
            throw new Exception("This is the exception");
        }
    }

    private static void ReturnFromMethod()
    {
        using (MyDisposable myDisposable = new MyDisposable())
        {
            return;
        }
        Console.WriteLine("DO NOT WRITE THIS!!!");
        return;
    }

    class MyDisposable : IDisposable
    {
        #region IDisposable Members

        public void Dispose()
        {
            Console.WriteLine("I am disposing!");
        }

        #endregion
    }

}

This code returns the following result. In this screen shot you should notice that each time it calls the dispose method of the MyDisposable object.

 UsingStatementReturn

Give the code a shot in a console application, and you will see that the using statement cleans up disposable objects pretty nicely.

Download this sample code as a small C# console application here.

I hope you enjoyed this post. I would also like to thank Lambros for the good comment. I hope to get more feedback from you in the future.