Thursday 10 June 2010

Unit Testing .Net Automatically

When I am working on a project, I try to keep the tools we use for building and testing in the same language as any production code uses. So, if I am working on a .Net project, I will endeavour to use .Net wherever possible. The same can be said for things like Java, although I rarely develop in this language, I will try and make sure that the tools used to build, test and deploy the code are based in Java.

Recently though, I have been working on pure c# projects. Since becoming a contract developer, I have encountered a number of lead or senior developers that simply turn around and say "no unit testing, just make it work". It makes me a little sad when I hear comments like this. As far as I am concerned, unit testing is one of the most useful processes a developer can exploit. It cuts down the amount of logic fails in testing and in production. For a relatively short amount of work prior to development, both analysts and developers can agree the unit tests that support the specification - once the test is defined for each unit of code it never has to be changed again (or rarely changed). This cuts down the amount of fails picked up in testing. Some lead developers I have worked with recently have turned down unit testing flat simply because they do not understand what a unit test actually is. You find this situation in teams that are under resourced and over time/budget. The embattled lead developer, possibly dealing with a bit of cynicism, just wants to make the deadline. But, one thing I have seen is that if you take the time to explain the benefits of unit testing, they normally come round and embrace the concept.

These days though, Visual Studio comes along with built in unit testing support. I think it is about time this has happened, even though I am not a fan of what is bundled. My unit testing tool of choice is MbUnit. These days it is part of a much larger test automation platform called Gallio. I like MbUnit for one reason you can test against a DB and roll back any changes once the test is complete, fail or not. I know that typically this is the way you should use unit testing with a DB, but for small things its fine.

Most unit testing tools will come with a test runner that you can use to run your tests, there are also VS add-ins that will do this for you - but I am not going to teach anyone to suck eggs here. Instead, I am going to examine just how unit testing can be automated as part of a larger build process. Some unit testing frameworks will come with an example NAnt or MSBUILD task that can be slotted straight into your build process. There are some out there that will not do this and/or you have the need to execute tests with a test runner outside of NAnt or MSBUILD. Another bit of win for Gallio is that it has very good support for NAnt. There is a task provided by Gallio that allows you to use the test framework automatically, there is a really good example of how this can be done here. The fragment of xml on that site is enough to demonstrate how Gallio can be used with NAnt, however in practise you may find that it is too simplistic, especially for larger builds.

The trick comes down to maintaining your unit tests. IMHO, creating a requirement for naming unit test assemblies is a must, for instance calling your unit tests someting like foo.tests.mbunit is very important for managing the fixtures that will carry out your tests. Personally, I like to provide unit tests within the project it tests. However, I suppose that you could create a separate solution containing all of the test fixtures, I am just of the mind that if you have a project then the fixture should go in that project to maintain encapsulation.

Anyway, regardless of where you keep your fixtures, the naming convention is pretty important as you can make NAnt loop through all assemblies that use a certain naming convention. You could, of course, use NAnt to run an external tool to do this for you etc.

One complaint I used to here from developers when they were just getting used to unit testing was the fact that they needed to run a command on the CLI or use a test runner to prove their code. Again, there is a simple solution to this problem called Test Driven. They provide a fantastic little tool that allows you to test fixtures from within VS itself. I know this may seem like teaching some people to suck eggs but describing this, but there are many new developers who are new to .Net or development in itself who may not know about some of the tools. I am not going to describe any NAnt code here to explain how it can be done. There are lots of NAnt scripts I have written for other bits and pieces I have done in the past, these coupled with the support docs provided by Gallio, NAnt and et al will show you how this can be done.