It's fairly obvious how each of the assert methods work from their name but what's not so obvious is how they report failures.
All assertions signal verification failure by throwing a java.lang.Error subtype. This allows JUnit to provide the location of the failed assertion in the stacktrace as feedback. Here's assertTrue()
failing:
java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) ... the line of your test code where assertTrue was called from ...
But where's the error message? We can guess what the actual and expected values were but only once we visit the test method that failed and see assertTrue()
. This adds an extra step (and perhaps a non-trivial amount of time) to the information gathering effort, making it harder to figure out what went wrong and how to fix it.
Here are the error messages generated by pre-Hamcrest JUnit assert methods (JUnit classic?). Most of these messages are not helpful, some are missing (you just get the AssertionError) and some are inconsistent (assertNull()
/ assertNotNull()
reporting isn't symmetrical)
Assertion | Generated Message |
assertArrayEquals( array("one", "two", "three"), array("one", "three", "four")) | arrays first differed at element [1]; expected:<t[wo]> but was:<t[hree]> |
assertArrayEquals("message", array("one", "two", "three"), array("one", "three", "four")) | message: arrays first differed at element [1]; expected:<t[wo]> but was:<t[hree]> |
assertEquals("aaa", "bbb") | expected:<[aaa]> but was:<[bbb]> |
assertEquals("message", "aaa", "bbb") | message expected:<[aaa]> but was:<[bbb]> |
assertFalse(true) | |
assertFalse("message", true) | message |
assertNotEquals("aa", "aa") | Values should be different. Actual: aa |
assertNotEquals("message", "aa", "aa") | message. Actual: aa |
assertNotNull(null) | |
assertNotNull("message", null) | message |
assertNotSame("aa", "aa") | expected not same |
assertNotSame("message", "aa", "aa") | message expected not same |
assertNull("aa") | expected null, but was:<aa> |
assertNull("message", "aa") | message expected null, but was:<aa> |
assertSame("aa", "bb") | expected same:<aa> was not:<bb> |
assertSame("message", "aa", "bb") | message expected same:<aa> was not:<bb> |
assertTrue(false) | |
assertTrue("message", false) | message |
Even if an optional description parameter is provided to the assert in an attempt to improve the reporting, the description is used inconsistently. Though always appended added before the generated part of the error message, the wording of the error message makes constructing a useful description highly dependent on the assert. It's better not to add descriptions in order to compensate for inadequate assertion reporting, anyway.
Compare that with the assertThat()
equivalents which have better and more consistent failure messages.
Assertion | Generated message |
assertThat( array("one", "three", "four"), is(array("one", "two", "three"))) | Expected: is ["one", "two", "three"] but: was ["one", "three", "four"] |
assertThat("bbb", is("aaa")) | Expected: is "aaa" but: was "bbb" |
assertThat(false, is(true)) | Expected: is <true> but: was <false> |
assertThat("aa", is(not("aa"))) | Expected: is not "aa" but: was "aa" |
assertThat(null, is(notNullValue())) | Expected: is not null but: was null |
assertThat("aa", is(not(sameInstance("aa")))) | Expected: is not sameInstance("aa") but: was "aa" |
assertThat("aa", is(nullValue())) | Expected: is null but: was "aa" |
assertThat("bb", is(sameInstance("aa"))) | Expected: is sameInstance("aa") but: was "bb" |
assertThat(false, is(true)) | Expected: is <true> but: was <false> |
And because assertThat()
leans heavily on Hamcrest to do the comparison, there's no need for custom static assertion methods (e.g. ) to be generated for different operations on different types and the reporting is consistent.
Assertion | Generated message |
assertThat(3, is(greaterThanOrEqualTo(4))) | Expected: is a value equal to or greater than <4> but: <3> was less than <4> |
assertThat(3, is(greaterThan(4))) | Expected: is a value greater than <4> but: <3> was less than <4> |
assertThat(3, is(lessThanOrEqualTo(2))) | Expected: is a value less than or equal to <2> but: <3> was greater than <2> |
assertThat(3, is(lessThan(2))) | Expected: is a value less than <2> but: <3> was greater than <2> |
assertThat("foobar", containsString("baz")) | Expected: a string containing "baz" but: was "foobar" |
assertThat(new Point(0, 1), hasProperty("x", equalTo(2.2))) | Expected: hasProperty("x", <2.2>) but: property 'x' was <0.0> |
...and collections. Prior to Hamcrest, collection verification looked something like:
List<String> _123 = asList("one", "two", "three"); assertEquals(3, _123.length()); assertTrue(_123.contains("one")); assertTrue(_123.contains("two")); assertTrue(_123.contains("three"));
Often people were driven by their dislike of the assertTrue()
feedback to do:
assertEquals(3, _123.length()); Collections.sort(_123); assertEquals(_123.get(0), "one"); assertEquals(_123.get(1), "three"); assertEquals(_123.get(2), "two");
Either way, it required N+1 assertions for collections of size N. One assertion to check the expected size and one for each expected elements. Aside from being cumbersome and ugly, this is an impractical way of checking collections with more than a few elements.
Hamcrest makes this much easier.
assertThat(_123, containsInAnyOrder("three", "one", "two"));
Verification can be expressed as a single assertion without requiring a specific element ordering (though order can also be verified), can match subsets of specific items and in each case gives pretty decent feedback.
Assertion | Generated message |
assertThat(_123, containsInAnyOrder("one", "two")) | Expected: iterable over ["one", "two"] in any order but: Not matched: "three" |
assertThat(_123, containsInAnyOrder("one", "two", "three", "four")) | Expected: iterable over ["one", "two", "three", "four"] in any order but: No item matches: "four" in ["one", "two", "three"] |
assertThat(_123, containsInAnyOrder("one", "three", "four")) | Expected: iterable over ["one", "three", "four"] in any order but: Not matched: "two" |
assertThat(_123, contains("one", "two")) | Expected: iterable containing ["one", "two"] but: Not matched: "three" |
assertThat(_123, contains("one", "two", "three", "four")) | Expected: iterable containing ["one", "two", "three", "four"] but: No item matched: "four" |
assertThat(_123, contains("one", "three", "four")) | Expected: iterable containing ["one", "three", "four"] but: item 1: was "two" |
assertThat(_123, contains("one", "three", "two")) | Expected: iterable containing ["one", "three", "two"] but: item 1: was "two" |
assertThat(_123, hasItems("one", "four")) | Expected: (a collection containing "one" and a collection containing "four") but: a collection containing "four" was "one", was "two", was "three" |
Choosing the wrong assertion just makes things harder to diagnose test failures. There isn't any reason to use the old 'JUnit classic' assertions.