There are a lot of tools and options out there for java unit testing, replacing JUnit or on-top of JUnit. Let's do some order in things.
1. TestNG vs. JUnit4
TestNG came out to add features that were missing in JUnit 3.x.
It did quite a good job and some may say that TestNG 5.10+ is still better than JUnit 4.7+
TestNG has better data providing mechanism.
TestNG has test dependency definition, missing in JUnit.
TestNG has group level with fixtures on group. And it also has the ability to create automatic run file for all failed tests.
All that said, JUnit has more extending libraries using its ability to extend JUnit TestCase and JUnit TestRunner.
Both have very good IDE support, as well as Ant and Maven support.
Coming to choose, both are good. I stick with JUnit.
Note that JUnit 4 made the mistake of integrating another library into its jar, that is the hamcrest core. They should have known better than that... The problem is that when you get the full hamcrest lib (to get the really powerful matchers that you need in your tests) it gets confused with the lib that came with JUnit. Solution is simple: put the full hamcrest that you bring to be first in the classpath, before the JUnit jar.
2. Load
JUnitPerf do the job with its ability to run tests in several threads.
It's easy and useful, for integration stage and for any piece of code that may act differently under load.
The following code was tested with JUnitPerf and it catches the bug of not synchronizing the addIfAbsent:
public class MyList extends ArrayList
// we test with the synchronized and without
public synchronized boolean addIfAbsent(Object o)
boolean absent = !super.contains(o);if(absent) {
super.add(o);
}
return absent;
}
}
The test that catches the bug when the synchronized is deleted:
public class MyListTest extends TestCase {
private MyList myList = new MyList();
private int count = 0;
MyListTest(String name) {
super(name);
}
TestSuite suite = new TestSuite();
Test testToRun = new MyListTest("testMyList");
int numUsers = 1000;
int iterations = 5;
suite.addTest(new LoadTest(testToRun, numUsers, iterations));
return suite;
}
public void testMyList() {
if(myList.addIfAbsent(count)) {
// there is a gap here that may cause error on correct
// implementation, in case of thread switch at this point
synchronized (this) {
++count;
assertEquals(myList.size(), count);
}
}
}
}
This solution is not bullet proof.
Theoretically it may give errors on correct scenario, and of course it can always miss bad implementation, as it’s a matter of timing.
However, in reality it does catch the problem!
JUnitPerf is simple and gets the job done.
3. Thread testing
It could be nice if we could have time the behavior of the code under test, to mimic race conditions and check how the code operates. The idea is to deliberately create race condition then to check that a synchronization lock is working as expected, ensure that unlocked blocks are fine, check for deadlocks and see if there data corruption.
With such ability we could have create the race condition in the above code with a deterministic approach instead of using load.
Suggested tools:
- thread-weaver (version 0.1)
allows the test to time different threads reach code points in a certain order, either explicitly or “semi-automatically” - MultiThreadedTC (version 1.01)
allows to set time tickers on objects in the test itself, but NOT on the tested code, thus less relevant for most testing purposes
Both tools are not highly supported, recommending still not to rush into it…
4. Mock Objects
There are 3 cases where you need a Mock Object in your test:
(a) Tested code gets a complicated object as parameter
Solution: Mock the parameter with a Stub/Mock implementation
(b) Tested code invokes a static call on some resource
Solution: Replace the static call with a Stub/Mock implementation
(c) Tested code creates complicated objects
Solution: Replace the created objects with a Stub/Mock implementation
The Mock utilities rely on one of the following technologies: java.reflection.Proxy, replacing the ClassLoader, byte code instrumentation and AOP weaving (which relies by itself on replacing the ClassLoader or on byte code instrumentation).
Possible tools: Mockito, JMock, JMockit, EasyMock, JEasyTest and many others…
There are differences between the tools in syntax and abilities.
Tools that rely on Proxy cannot mock static behavior and object creation.
The two possibile combinations that came to my final round were:
(a) JMockit 0.998
(b) EasyMock 3.0 + PowerMock .1.3.8
JMockit seems powerful and well documented, but in one of my tests the test failed without a reason and when I debugged it I saw that some exception is thrown within the JMockit code itself. Probably I did something wrong in the test, but still this is not what I expect for. Maybe the 1.0 version would be better...
My selection here would be EasyMock + PowerMock
5. Other tools
Other tools in this domain worth mentioning:
- DBUnit
- HttpUnit (not only for tests)
- Selenium (it is a good tool for Web UI tests in general, and you can operate it from your java unit tests if you'd like)
