home / blog / JUnit 5 – a New Java Unit Testing Method

JUnit 5 – a New Java Unit Testing Method

August 17, 2017 Piotr Szerszeń

Tags:JavaJava 8JUnittestingtestsunit testing

Testing your application should be a development step just as important as programming. Of course, testing everything manually would be very time-consuming and tiring. When you think of “unit testing”, I bet you think of JUnit. The new, fifth version of this framework caught my attention some time ago and so I decided to share a few facts and code samples.

The first thing worth mentioning is that JUnit 5 is, technically speaking, a new framework rather than just another version of the one we already know. This is actually good news. The previous version is decent but it still has a few serious flaws, which cannot be alleviated with simple fixes. There was no comfortable way to test exceptions. We could not use more than one Runner at a time and there was no good way to filter tests that need to be run.

We needed a new approach – now we have it. Most of the equivalent annotations have different names and are in another package. This allows JUnit 3/4 and JUnit 5 to be used in parallel. Therefore, any project currently using JUnit 4 can start using JUnit 5, and developers can start writing new tests with the new framework. Legacy tests can be (i.e. should be) migrated gradually.

This article teaches you how to set up your project to be able to use JUnit 5, exposes the biggest differences between JUnit 4 and JUnit 5, and presents a few new features and techniques, which have been made available in the new version.

Set up:

To use both JUnits in an Apache Maven project, you need to add org.junit.jupiter dependencies in your pom.xml. See example. Take another look at the Surefire Plugin configuration; you have to specify JUnit dependencies there as well. Otherwise, tests will not be triggered while building the project with Maven.
This kind of configuration lets you run JUnit 4 as well as JUnit 5 tests:

1
2
3
4
5
6

import org.junit.Test;
public class VintageTest {
    @Test
    public void testSomething() {
    }
}

What’s new? What’s different?

Methods can be package-private:

Yes, no need to type “public” all the time any more.

1
2

@Test
    void testSomething() {}

Parameter order:

If an error message shows up now, it appears in the last parameter, not the first.

1

        assertEquals(expected, actual, "This is assertion message");

Message supplier:

Creating error messages can sometimes be expensive and time-consuming. You can now add a supplier so that the message will only be constructed if needed.

1

assertEquals(expected, actual, this::getAssertionMessage);

New lifecycle names:

Annotation names have been changed. New names seem to be more accurate and human-friendly. In addition, this makes it much easier to sense the difference when a project contains tests using both JUnits.
Here is how it changed:

  • @BeforeClass –> @BeforeAll
  • @Before –> @BeforeEach
  • @After –> @AfterEach
  • @AfterClass –> @AfterAll

Nested tests:

The @Nested annotation lets you have an inner class that is essentially a test class, allowing you to group several test classes under the same parent (with the same initialisation). Nested tests inherit @BeforeAll, @BeforeEach, @AfterEach, and @AfterAll.
The code in E03LifecycleInheritanceDemoTest.java produces the following output:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

15:32:13.504 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - beforeAll
15:32:13.514 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - beforeEach
15:32:13.517 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - testing	
15:32:13.524 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - afterEach
15:32:13.535 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - beforeEach
15:32:13.535 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	another nested beforeEach
15:32:13.535 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	another nested testing
15:32:13.536 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	another nested afterEach
15:32:13.536 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - afterEach
15:32:13.543 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - beforeEach
15:32:13.544 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	nested beforeEach
15:32:13.544 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	nested testing
15:32:13.544 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - 	nested afterEach
15:32:13.544 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - afterEach
15:32:13.547 [main] DEBUG com.pgs.junit5.examples.E03LifecycleInheritanceDemoTest - afterAllDisconnected from the target VM, address: '127.0.0.1:61610', transport: 'socket'

Group assertion:

We can put multiple assertions in one assertAll() method. This forces one to execute all the assertions in the group, instead of exiting after the first fail. Thanks to this, we can now have an aggregated report of failing assertions. This feature certainly saves time when rerunning and debugging failing assertions, as the first fail will not force the end of the test run.

Test class:

1
2
3
4
5
6
7
8
9
10

public class E04AssertAllTest {
    @Test
    public void shouldReturnCollectionOfSizeWithFixedFirstElement() {
        List actual = ImmutableList.of("AAA", "anything");
        assertAll("collection",
                () -> assertEquals(3, actual.size(), "Size must be 3"),
                () -> assertEquals("AZZ", actual.get(0), "AAA element must be first")
        );
    }
}

Output:

1
2
3

org.opentest4j.MultipleFailuresError: collection (2 failures)
	Size must be 3 ==> expected: <3> but was: <2>
	AAA element must be first ==> expected: <AZZ> but was: <AAA>

A new approach to testing exceptions:

We do not need to “tribute” a whole method for testing a single exception. Now, exceptions are tested by using the assertThrows() method, which returns a caught Throwable. A new @Test annotation does not allow us to specify an expected Exception or timeout. Everything is asserted within the method body. This makes it much easier to test error messages and, thanks to that, track the spot where the exception was added, in case the same exception can be added in multiple places in a method. I hope having better conditions to test all the edge cases will make developers more eager to do so.

1
2
3
4
5
6
7
8

public class E05ExceptionsTestingTest {
    @Test
    public void shouldThrowExceptionWithMessage() {
        Throwable throwable = assertThrows(IllegalArgumentException.class,
                () -> new Generator().generate(null, 0));
        assertEquals("parameter must not be null", throwable.getMessage());
    }
}

Custom test names:

If we wish, we can add a custom name for a test with the @DisplayName annotation so that the provided value will be displayed as a name instead of a method name.

Test factory:

Dynamic test generation is useful when you need to run the same set of tests on many different input values or configurations. A method annotated with @TestFactory should return Stream, Collection, Iterable or Iterator of DynamicTest instances. Each dynamic test can be treated as a single test case run lazily. This way, we can parameterise only specific test methods.

In previous JUnits, Runners allowed one to introduce additional behaviour while running tests. For example, MockitoJUnitRunner initialised mock fields and let us inject them into tested objects before running each test. Parameterized set field values from parameter values and ran tests for every parameter set.

We can think of test factories as a replacement for the Parameterized Runner from JUnit 4. In fact, in JUnit 5 there are no Runners at all.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

public class E06DynamicTest {
    @TestFactory
    Stream fibonacciTestFactory() {
        return Stream.of(
                params(0, 0),
                params(1, 1),
                params(2, 1),
                params(3, 2),
                params(4, 3),
                params(5, 5),
                params(6, 8))
                .map(this::callTest);
    }
    private DynamicTest callTest(Pair params) {
        return test(params.getKey(), params.getValue());
    }
    private DynamicTest test(final int input, final int expected) {
        return dynamicTest(format("input={%d}, expected={%d}", input, expected),
                () -> assertEquals(expected, Fibonacci.compute(input)));
    }
    private Pair params(int input, int expected) {
        return ImmutablePair.of(input, expected);
    }
    @Test
    @DisplayName("Test not needing parameters.")
    public void test2() {
    }
}

Extensions:

As I said before, there are no Runners in JUnit 5. Extensions are their general replacement. Why are they better, you ask? They are far easier to implement. A developer is not forced to think about the whole flow while introducing a new extension. One extension can be responsible for one functionality, for example, for deciding if a test should be skipped. Extensions actually let you to do many interesting tricks. You are able to create an Extension from the following interfaces:

  • ContainerExecutionCondition – is evaluated to determine if all tests in a given container (e.g., a test class) should be executed based on the supplied ContainerExtensionContext
  • TestExecutionCondition – is evaluated to determine if a given test method should be executed based on the supplied TestExtensionContext
  • BeforeAllCallback – provides additional behaviour to test containers before all tests are invoked
  • AfterAllCallback – provides additional behaviour to test containers after all tests are invoked
  • BeforeEachCallback – provides additional behaviour to tests before each test is invoked
  • AfterEachCallback – provides additional behaviour to tests after each test is invoked
  • BeforeTestExecutionCallback – provides additional behaviour to tests immediately before each test is executed
  • AfterTestExecutionCallback – provides additional behaviour to tests immediately after each test is executed
  • TestInstancePostProcessor – post-process test instances; common use cases include injecting dependencies into the test instance and invoking custom initialisation methods on the test instance, etc.
  • ParameterResolver – dynamically resolves parameters at runtime. If a test constructor or a @Test, @TestFactory, @BeforeEach, @AfterEach, @BeforeAll, or @AfterAll method accepts a parameter, the parameter must be resolved at runtime by a ParameterResolver.
  • TestExecutionExceptionHandler – handles exceptions, which appear during test execution.

ParameterResolver:

Allows you to define parameters that are later passed directly to methods annotated with any of the following – @Test, @TestFactory, @BeforeEach, @AfterEach, @BeforeAll, @AfterAll. It does so in order for the test shown in the previous snippet to be made with a little bit more finesse:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

public class E06ParametrizedTest {
    @TestFactory
    @ExtendWith(FibonacciParameterResolver.class)
    Stream test(Stream> dataSet) {
        return dataSet.map(pair -> test(pair.getKey(), pair.getValue()));
    }
    private DynamicTest test(final int input, final int expected) {
        return dynamicTest(format("input={%d}, expected={%d}", input, expected),
                () -> assertEquals(expected, Fibonacci.compute(input)));
    }
    @Test
    @Special
    @DisplayName("Test not needing parameters.")
    public void test2() {
    }
    private static class FibonacciParameterResolver implements ParameterResolver {
        @Override
        public boolean supports(final ParameterContext parameterContext, final ExtensionContext extensionContext) throws ParameterResolutionException {
            return parameterContext.getParameter().getType().equals(Stream.class);
        }
        @Override
        public Object resolve(final ParameterContext parameterContext, final ExtensionContext extensionContext) throws ParameterResolutionException {
            return Stream.of(
                    params(0, 0),
                    params(1, 1),
                    params(2, 1),
                    params(3, 2),
                    params(4, 3),
                    params(5, 5),
                    params(6, 8));
        }
        private Pair params(int input, int expected) {
            return ImmutablePair.of(input, expected);
        }
    }
}

The supports() method determines whether the ParameterResolver should be used and the resolve() method returns the value of the parameter.

*Callback interfaces:

Allow you to define generic behaviour before and/or after specific methods and to reuse their annotation. The structures of implemented methods deserve some attention – each method provides access to the TestExtensionContext, within which you can add and take objects. The simple extension below measures the time of each test execution.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public class BenchmarkExtension implements BeforeTestExecutionCallback, AfterTestExecutionCallback {
 
    @Override
    public void afterTestExecution(final TestExtensionContext context) throws Exception {
        long elapsed = System.currentTimeMillis() - getStore(context).remove(context.getTestMethod().get(), long.class);
        String message = String.format("Test %s took %d ms.%n", context.getTestMethod().get().getName(), elapsed);
        context.publishReportEntry(
                createMapWithPair("Benchmark", message));
    }
 
    @Override
    public void beforeTestExecution(final TestExtensionContext context) throws Exception {
        getStore(context).put(context.getTestMethod().get(), System.currentTimeMillis());
    }
 
    private Map createMapWithPair(String key, String message) {
        return Collections.singletonMap(key, message);
    }
 
    private ExtensionContext.Store getStore(TestExtensionContext context) {
        return context.getStore(ExtensionContext.Namespace.create(getClass(), context));
    }
}

A test annotated with this extension will produce a log similar to this:

1

timestamp = 2017-02-23T22:33:04.132, Benchmark = Test shouldGenerateCombinations took 163 ms.

ExecutionConditions:

Allow you to decide if a test should be run or not. For example, I created an annotation that lets you decide on which days of the week the test or test container should be skipped.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

public class DisabledOnCondition implements ContainerExecutionCondition, TestExecutionCondition {
    private static final ConditionEvaluationResult ENABLED = ConditionEvaluationResult.enabled("@DisabledOn is not present");
    @Override
    public ConditionEvaluationResult evaluate(ContainerExtensionContext context) {
        return evaluate(context.getElement());
    }
 
    @Override
    public ConditionEvaluationResult evaluate(TestExtensionContext context) {
        return evaluate(context.getElement());
    }
 
    private ConditionEvaluationResult evaluate(Optional element) {
        return findAnnotation(element, DisabledOn.class)
                .map(DisabledOn::value)
                .filter(dayOfWeeks -> ArrayUtils.contains(dayOfWeeks, LocalDateTime.now().getDayOfWeek()))
                .map(dayOfWeeks -> element.get() + " is disabled on " + Arrays.toString(dayOfWeeks))
                .map(ConditionEvaluationResult::disabled)
                .orElse(ENABLED);
    }
}
 
@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@ExtendWith(DisabledOnCondition.class)
public @interface DisabledOn {
    DayOfWeek[] value() default {DayOfWeek.FRIDAY};

For example, this test will be skipped during the weekend:

1
2
3
4
5
6
7
8
9
10
11

@Log4j2
@Benchmark
@Special
@DisabledOn({DayOfWeek.FRIDAY, DayOfWeek.SATURDAY, DayOfWeek.SUNDAY})
public class SomeAwkwardTest {
    @Test
    void assertThatResultIsNotEmpty() {
        assertThat("s").isEmpty();
        log.warn("executed");
    }
}

Using Mockito:

Of course, when discussing JUnit, you might also be thinking about Mockito. Without MockitoJUnitRunner, using Mockito might be a bit painful, so why not use an Extension here as well? This Extension will take care of initialising fields annotated with @Mock:

1
2
3
4
5
6

public class MockitoExtension implements TestInstancePostProcessor {
    @Override
    public void postProcessTestInstance(Object testInstance, ExtensionContext context) {
        MockitoAnnotations.initMocks(testInstance);
    }
}

And, a usage example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

@Special
@ExtendWith(MockitoExtension.class)
public class E07MockitoTest {
    @Mock
    private Pair pair;
 
    @BeforeEach
    void setUp() {
        when(pair.getKey()).thenReturn("key");
        when(pair.getValue()).thenReturn("val");
    }
 
    @Test
    void test() {
        assertEquals("key", pair.getKey());
        assertEquals("val", pair.getValue());
    }
}

Using Spring:

If you have ever faced a situation where you needed to implement unit tests for Spring components with JUnit alone, you probably got yourself into quite a bit of trouble as I reckon you had to come up with a very big new extension. Luckily, however, you can use the one already implemented here.

Tagging:

I saved something special for the end. You probably noticed that some tests have been annotated with @Special. JUnit 5 lets you mark tests with various tags and run them by means of filtering by these tags (by including or excluding them).

1
2
3
4

@Target({ TYPE, METHOD, ANNOTATION_TYPE })
@Retention(RetentionPolicy.RUNTIME)
@Tag("special")
public @interface Special {}

The Maven profile that only chooses @Special tests is available in pom.xml and is mentioned in Set up.

Summary

Whole code with examples is accessible here. I hope you enjoyed this article and feel encouraged to try out jUnit 5 in your current and future projects. According to the current version of plans exposed on the project’s github the first general availability release will take place at the beginning of September this year.

JUnit 5 definitely gives us the possibility to write better tests and to do so in an easier manner by means of introducing more generic and reusable solutions. However, the framework is still being developed so we should expect a more stable and advanced version later this year.

I am under the impression that we currently have tools with great potential but not much more than that. JUnit 5 might come with a few ready-made extensions meant to solve popular problems (such as MockitoExtension, SpringExtension) like it was with Runners.
However, both the old and new JUnits can be used at the same time so that the migration process can take place systematically; I would even say it could already be started, if we begin with simpler tests.

Sources:

Piotr Szerszeń

Java developer at PGS Software. Always ready to get to know and – as long as it’s worth it – introduce a new technology or trend. Aside from that, he’s interested in cars, fantasy, & Slavic mythology.

Want to work with our experts?

Contact Us

Latest posts