Just about .Net

… and all I got was this lousy Christmas tree.

I like TDD and I like test automation. It is great to see the application growing with each successful test and to be sure that it will work even if a lot of changes are made on the code base. On the other I see (especially) TDD as a practice which is hard to master and I can understand people who drop any ambition because it feels intricate and some times frustrating in real projects.

One of these things is for example the advice to use baby steps to drive the implementation. I thing this advice is essentially to TDD and makes, from my point of view, the difference between Test First and Test Driven Development. Baby steps means, that you are not allowed to implement more functionality than needed by your tests. If the test expects a 1 as return value of your method, than your first step would be to simply return a 1. Your algorithm will also grow with each test you write and that’s why it is called Test Driven Development. This has also the advantage that the your tests get more robust, they are easier to change and to read, at least in theory.

Test Automation Pyramid & Ice-Cream Cone

I know that this way of working leads me to a lot small tests as well as highly specialist classes and I really appreciate that. It improves my detailed design and allows me to work without even using the debugger because every failed test points exactly to the part of the program which does not work or was changed in an unexpected way.  That is one of the reasons why it is commonly accepted that most of the automated tests should be unit tests and if you do TDD right, it just will be that way.

2009 Mike Cohn described the concept of the Test Automation Pyramid in his book „Succeeding with Agile“ and since then it became a widely known best practice. The foundation of this pyramid are the unit tests because of  the points I already mentioned. The Integration Tests (he wrote “tests of the service level”) are less in number because they include a larger amount of code and check the interaction with resources like data bases. System Tests check if everything is working from top to bottom by entering values in the UI and verifying if they reach the database. (Cohn used „UI“ but I prefer System Test because we need end-to-end tests to check if the application works completely.) They are the tip of the pyramid because they take long to execute, complex to create and maintain, check a lot of code at the same time and thus will be less often automated.

An anti pattern which was derived from this is the ice-cream cone. This happens in projects where manual regression tests are „converted“ into automated System and Integration Tests.  In these projects automated tests are mainly used to reduce the amount of work for manual testers, they don’t drive the implementation. This is bad because you will have a lot of work just to keep the tests running. It is also not possible to execute the tests often enough to get a good feedback to your changes and thus bugs will be found later and can cause more damage.

Unit Tests vs. Unit(?!?) Tests

These patterns and anti-patterns are great and I absolutely appreciate their guidance. In books and during dojos, every thing sounds so easy and absolutely consistent,  but the reality seems (a bit more) unfriendly. I think  I know a lot of the theories behind TDD and Unit Testing and also did many of the Katas, but some of this stuff does not seem to fit my daily workflow.

The reason for this post is, that I some times feel doomed if I don’t follow the rules of TDD and common definition of the term Unit Test. If you use for example the definition that a Unit Test should only check one class and should not use external resources like databases or the file system than the Test Automation Pyramid will not work if you have an application with a small business layer which simply combines a lot of frameworks.

Today there are a lot of great frameworks with which you can reduce the work you have to do to a minimum, at least in typical business applications. You often just have to take data out of some storage or from a web service and put it into the UI where the user can change it. After he clicks a button, everything is validated and goes back into the storage.

In this case you do not really have a business layer. Ok, it is necessary to make some view handling and converting the data between data base and UI but compared with the functionality hidden in the frameworks, it will need only a few lines of code and thus most of the code of your application is simply not written by yourself.

This example might be a bit extreme but often you end up with a lot of UI stuff you cannot unit test, a lot of data base stuff you need integration tests for (sorry but I can’t let in memory data bases count for unit tests) and a lot of framework stuff you actually don’t need to test because it should be tested by the guys who wrote it. In this case it is much more interesting if the integration of the frameworks is correct and this can’t be done by unit tests which follow the common definition because it often involves more than one component or needs external resources.

In such an environment I have two choices: Either I encapsulate all the frameworks and test my wrappers or I just don’t care about the definition of Unit Tests and define for me that I will use Component Tests or Façade Tests. That means instead of poking around until I have some kind of unit to test, I just grab a bunch of classes or frameworks and check their communication without any bit of encapsulation. In reality it is actually a combination of both.

Red-Green-Implement?

Another thing I often don’t do in the way it might be expected,  is the Red-Green-Refactor cycle. It usually says: Implement a failing test, make it pass with as less effort as possible and then refactor. Stop! Refactor? This is the part where it gets interesting.

TDD is TDD because the tests drive the implementation. This is done by writing one test after another which defines what code is actually needed for the requested functionality. That’s why the refactor phase should  be used for code clean up, renaming and so on but not for implementing the complete production code.

Sounds good but ends up normally in: Red-Green-Implement. I saw it often during dojos and do it often by myself. Instead of seeing the green phase as the part where I get the code I need in production, defined by the tests I write, I see it just as a check if my test can succeed. So if my test can fail (Red) and succeed (Green) I am confident that I can rip my production code apart and start with the “real” work.

Practical that means, that I get often lesser tests than I would get if I would use a method like “The Transformation Priority Premise” of Robert C. Martin. I actually write another test when I feel not confident during development, have an if-statement in my code or if there are additional requirements I have to check.

One reason why I do this is that if I just convert data from A to B or if I only need to call an API to get things done, then the code transformation looks a bit oversized. I know it might fit for implementing algorithms like it is done in most of the katas but in the environment I work, it feels like implementing endless code. Who knows, may be I just did not understand the concept…

With my approach it does not feel like I would drive the implementation with tests.  I just write my tests first. That’s all and that’s why I stopped saying for some time that I would do TDD.

Conclusion

I am doing test automation now for around four years and all I ended up is a Christmas Tree instead of a pyramid when ever the projects get bigger and more people are involved. Even I do not have a situation with lots of frameworks it feels easier to combine some classes and create a component instead of a unit test. I think this even increases if you start with BDD or ATDD.

For me, tests are more like a safety net. I have to choose their granularity depending on the particular situation or project. It is good to have them because they help to stay focused on the behavior of the software instead of its code. So,  even they will not prevent from each and every accident they can show which required parts work and which does not.

May be the Christmas Tree is just symbol showing teams they are near to the pyramid and should stop thinking to much on definitions and best practices of different teachers. Some times it is important to find an own way to understand which are the dead ends.