JUnit test execution order - Programming for fun and profit

How to control JUnit test execution order? It’s not only possible, but also easy in recent versions of the most popular testing library!

For a long time JUnit didn’t have any means to control tests execution order and all tests were executed in non predictable order. This is still default behavior, but JUnit 4.11 shipped with @FixMethodOrder annotation that allow to customize that. The annotation takes org.junit.runners.MethodSorters as its only argument and can change execution order as follows:

DEFAULT: sorts tests in non predictable order
JVM: leaves the ordering for the JVM, which can differ from run to run
NAME_ASCENDING: sorts tests by method name in lexicographical order.

In the following example we’re going to tell JUnit to execute tests in order dictated by tests’ names:

package com.farenda.junit;

import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;

import static org.junit.Assert.assertEquals;

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class ExecutionOrderTest {

    @Test
    public void minNumber() {
        System.out.println("Executing first test.");
        assertEquals(24, Math.min(24, 42));
    }

    @Test
    public void maxNumber() {
        System.out.println("Executing second test.");
        assertEquals(42, Math.max(24, 42));
    }
}

In the output you can see that test ordering works:

Executing second test.
Executing first test.

Simple as that. :-)

How pros do it

They ~~don’t~~ shouldn’t, because tests should be independent.
Anyway, fixing methods order may be helpful to force some execution order to recreate a bug that occurred, when they were executed in a different order.

Let’s see how Spring Framework is using FixMethodOrder to force order of tests that operate on database – DataSourceOnlySqlScriptsTests.java:

package org.springframework.test.context.jdbc;

import javax.sql.DataSource;

import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.MethodSorters;

// imports cut for brevity

import static org.junit.Assert.*;
import static org.springframework.test.transaction.TransactionTestUtils.*;

/**
 * Integration tests for {@link Sql @Sql} support with only a {@link DataSource}
 * present in the context (i.e., no transaction manager).
 *
 * @author Sam Brannen
 * @since 4.1
 */
@RunWith(SpringJUnit4ClassRunner.class)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
@ContextConfiguration
@Sql({ "schema.sql", "data.sql" })
@DirtiesContext
public class DataSourceOnlySqlScriptsTests {

        private JdbcTemplate jdbcTemplate;

        @Autowired
        public void setDataSource(DataSource dataSource) {
                this.jdbcTemplate = new JdbcTemplate(dataSource);
        }

        @Test
        // test##_ prefix is required for @FixMethodOrder.
        public void test01_classLevelScripts() {
                assertInTransaction(false);
                assertNumUsers(1);
        }

        @Test
        @Sql({ "drop-schema.sql", "schema.sql", "data.sql", "data-add-dogbert.sql" })
        // test##_ prefix is required for @FixMethodOrder.
        public void test02_methodLevelScripts() {
                assertInTransaction(false);
                assertNumUsers(2);
        }

        protected void assertNumUsers(int expected) {
                assertEquals("Number of rows in the 'user' table.", expected,
                        JdbcTestUtils.countRowsInTable(jdbcTemplate, "user"));
        }

        @Configuration
        static class Config {

                @Bean
                public DataSource dataSource() {
                        return new EmbeddedDatabaseBuilder()//
                        .setName("empty-sql-scripts-without-tx-mgr-test-db")//
                        .build();
                }
        }

}

Share with the World!