Tuesday, July 30, 2013

Non-transactional tests with Spring and H2

The general paradigm for testing Spring based application back-end is to use a sort of unit testing which loads the whole application (the application context) and performs certain tests on it. Spring provides all sort of helper classes among others AbstractTestNGSpringContextTests and AbstractTransactionalTestNGSpringContextTests. I don't know when that decision happened but we always used the transactional one hence every test method run in a separate transaction which was rolled back after each method. It seems to me that such an approach is prevalent in current development yet it is as well flawed.

I bet you encountered cases when the test cannot be transactional, you want to test that your fix for a concurrent race condition works, the locking on the DB does really prevents another user to access something, etc. So you write a non-transactional test class, annotate it with the Spring annotation @DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD) and you're done. The context is reloaded several times for the start so it's not a big deal. But then you write tens of non-transactional tests and all of a sudden build with tests takes time.

That happened to us and made us thinking, do we need to recreate application context so often? The only reason we do it is that the DB is polluted by the non-transactional tests data. I started the investigation. We use in-memory H2 for our tests. I found that H2 has a wonderful feature which let's you drop its content easily: All you need is the following method in your test parent class:

So no more context recreation. It sped our test build from 1:30 to 55 seconds, boohoo. But than a heretic thought began to creep in our brains. What if we don't need to wrap each test method in a transaction? What if we don't gain much speed there? What if we actually oversee a bug here and there as our tests don't work the same as when client works with the application?

We gave it a try and made all tests non-transactional. The build takes the same time as before as populating and dropping in-memory DB is apparently not an issue, we would do the same in separate transactions anyway. The tests now mimic exactly use cases we want to test as often many transactions happen before the app is in certain state.

We found three bugs straight away. I hope you find some too.