Писалось для корпоративного блога, поэтому на английском. Движок блога коряв, поэтому форматирование текста слетает с завидным упорством. Если кому-то захочется версию, годную для печати, можно взять по ссылке Effective Testing.
Effective Testing
In this blog post I’m going to talk about testing, or more precisely, what makes testing effective. Probably every developer nowadays knows that certain test coverage is required, but not everybody has a clear understanding how to make these tests really effective and helpful.
If one is asked what are tests for, the answer usually is: “They verify that my code works”. This is true but is not everything about it. This mentality usually drives people to write their tests after the actual code is complete just to make sure that it can run and does something. This is better than nothing but still not as good as it could be. There’re several reasons for that.
First, if a developer writes the test after writing the code, he/she has never had the chance to see the test fail and then subsequently pass (once the code is written). Never seeing a test fail may mean that it is just working by coincidence. The developer sees the green bar and assumes everything is good, but that green bar is actually masking a bug.
Second, it’s not a rare case that the written code is not testable. That is, it can have lots of dependencies that make it very hard to write a good test case for it, so if somebody wants to test it, he/she will have to spend quite an amount of time breaking these dependencies (but quite often people just give up on that and live with the code being completely untested).
Third, this approach leads to a trouble when a code base grows over a certain size. Developers can easily forget to update some test cases or fix the failing tests in an inappropriate way just to make them pass. This makes a test codebase rot which is a huge problem. After a while the amount of effort that one has to spend to keep a rotten test code base in sync with the actual system will be more then he/she has to spend on the actual code changes and after a while it can slow down the whole process so much that a team will have to abandon their tests completely. This means that they are in big trouble. Now they can’t be sure that their changes do not break anything, so no refactoring can be done and they have to do manual regression testing after each change.
Last but not least – by writing tests after the code you usually test just methods but not behaviour. This is not effective because tests should focus on a valuable behaviour that the class under tests provides. So one needs to test the features of the code to make sure that it works as expected and can be used efficiently in collaboration with its clients instead of just covering all methods with tests one by one – quite often you need to call several methods of your class in a row to collaborate with other objects.
On the other hand, if you write tests first by following the TDD principles you get a whole bunch of benefits: first, you give up on treating your tests just as something that verifies your code. Your tests become a runnable specification that describes what your code does and not how it does it. That is, you start thinking in terms of features and this helps us to maintain a necessary level of abstraction.
Test first also makes the test readable and maintainable – you think about a certain scenario and this allows you to keep your test clean from unnecessary details. Writing tests first also enforces you to think about the dependencies. Because you have to pass all the dependencies to the object under test you will get a clear indicator as you are coding that your design is bad. Another indicator is if your test becomes too large and complicated – this points you to the fact that the class under test is doing too much and has too many responsibilities and you probably need to decompose it.
So what should you do to make your test code base effective and helpful besides writing your tests first? There’re several best practices and pieces of advice that can be used. This is not the complete list of course, but it will show you the way to go.
The key factor that makes your tests effective is their readability. If the test is easy to read it is easy to maintain it. This means that you should apply the same code standards to your test code as you apply for production code. You should not treat the test as something that can be written in a mess. Follow the common clean code rules: no magic numbers, good and explanatory variable naming, no nested if-s, etc. Please refer to [1] as a great guide how to write clean and readable code. If you do not follow these standards, it’s just a matter of time until the test codebase starts to rot.
Test methods should be small, ideally 10-15 lines and contain just a few asserts (ideally one). It should be very clear what the test does and what atomic scenario is being executed. If your test is long and has lots of assertions inside you will have to spend a lot of time just figuring out what it actually does. Also, the test method should have a very clear and descriptive name that states what feature is being tested:
public void testCheckout1() // NO
public void testCheckoutWithOneCartItemAndDiscountAppliedGivesFreeShipping() //YES
Don’t be afraid of long method names – you are not going to call these methods anywhere, but you will be able to understand what kind of scenario is being tested at a glance.
Do not duplicate creation of test objects inside your tests – use test object builders that will help you easily create and maintain families of such object instead [2, 3]. By using the builders you will greatly increase readability and remove duplication from your tests. An example can look like following:
List movies = Arrays.asList(
MovieBuilder.movie().withTitle("Blade Runner") // <- here's the builder being used
.withAddedActor("Harrison Ford")
.withAddedActor("Rutger Hauer")
.build(),
MovieBuilder.movie().withTitle("Star Wars") // <- ... and also here
.withAddedActor("Carrie Fisher")
.withAddedActor("Harrison Ford")
.build());
There’s an Eclipse plugin called “Fluent Builders” that allows you to generate these builders automatically from any class you wish and saves your time greatly [4].
All of your test should have the same canonical structure:
1) Setup (prepare context and environment);
2) Execution (trigger the tested behaviour);
3) Verification (check that results are what we expect);
4) Teardown (clean up everything that can influence other tests and release all resources)
This means that all the tests should be completely independent from each other. This is very important, because if you have chained tests and the first test in the chain fails – all other tests will also fail. This is really bad because your test code base becomes very brittle and uninformative – you don’t know if tests are failing because of chaining or because of incorrect behaviour of the code that’s under test. So make your tests completely decoupled and independent.
Do not load data from hardcoded location in the file system, this will ensure that your test suite can be run in different environments under different operation systems and this will increase the maintainability of the test suite.
/home/ynovikov/workspace/project/src/test/resources/config.properties //NO. Absolute path is used
../test/resources/config.properties //YES. Relative path is used
Always use explanatory assertion messages, so that you can easily understand the reason of failure.
assertEquals(“Price discount hasn’t been applied”, expectedPrice, actualPrice);
Assertions should be precise – assert only those results that are triggered by test scenario and are not covered by other tests. This will make your test bas more robust.
Try to use mocks only for something that you can’t change, don’t use them as stubs. There’s a good article by M. Fowler about that [5].
Make sure that the code under test satisfies SOLID principles [6]. If you do that together with writing your test first, you are very likely to get the code that is easier to test, change and maintain.
Very important: your tests (not only unit, but acceptance and integration tests as well) should be fully automated so that you can make running your test as a part of continuous integration process. This will ensure that your build is always in a good and working state.
One more thing to remember – effective unit tests should be fast. This will allow you to execute them very often, may be after every single change you make to the code. This will provide you fast feedback if your changes broke something so that you can very quickly locate and fix the newly introduced bug.
If you start developing a user story, write acceptance test(s) first and don’t forget to follow TDD when you start implementing the feature itself. Each acceptance test suite should include a happy path scenario (when everything works as expected) and tests for cases when something goes wrong or for alternative behaviour.
Write integration tests to ensure that your code works correctly with some third-party libraries or services that you can’t change. All integration points with such things like payment gateways, geoip providers, persistent mechanisms etc. should be covered by integration tests.
If you’ve just started a project from scratch, the first thing you should do is to build a “walking skeleton” – the minimal possible configuration of your product that can be built, deployed and tested in a continuous integration environment against all kind of tests (unit, acceptance, integration). For example, if you develop a web application, it can just show you a page that reads a couple of records from database and displays them (and you create all kinds of tests for this functionality). This will ensure that your test environment and test frameworks are up and running, so that you can be sure that they are working correctly and ready for serious work.
I hope this article has been helpful, but don’t forget to check out the books and links in the list below. I have found them extremely useful and I hope you do too.
Bibliography and references:
1. “Clean Code: A Handbook of Agile Software Craftsmanship” by Robert C. Martin;
2. “Growing Object-Oriented Software, Guided by Tests” by Steve Freeman, Nat Pryce;
3. http://www.betriebsraum.de/blog/2010/02/08/better-tests-with-test-data-builders/;
4. http://code.google.com/p/fluent-builders-generator-eclipse-plugin/;
5. http://martinfowler.com/articles/mocksArentStubs.html;
6. http://en.wikipedia.org/wiki/Solid_(object-oriented_design)
Метки: testing
