Integration tests with Maven (Part 1): Failsafe Plugin

Everyone knows Maven is great for running unit tests, and it is usually one of the first things that people learn when they are adopting Maven as a technology. Integration tests are another matter, and require a more detailed introduction. In this series of articles, I will explain how to set up integration tests in Maven starting the series by testing a simple jar and then advancing into more complex scenarios.

The application

Let's define a very simple application to be tested. The application will return an exit code equal to the number of command-line parameters. Running the program with no parameters will produce an exit code equal to zero, two parameters exit code two, and so on.

The application:

package org.sonatype.simpleclientapp;

public class Main
{
    public static void main( String[] args )
    {
        System.exit( execute( args ) );
    }

    public static int execute( String[] args )
    {
        if ( args == null )
        {
            return 0;
        }

        return args.length;
    }
}

This class is very simple to test, in the next section you will see how you can write a simple integration test using JUnit.

Creating and running integration tests

Let's create a simple integration test using JUnit:

package org.sonatype.simpleclientapp;

import java.io.File;

import junit.framework.TestCase;

public class MainIT
    extends TestCase
{

    public void testExecute()
        throws Exception
    {
        assertEquals( 0, execute( new String[] {} ) );
        assertEquals( 1, execute( new String[] { "one" } ) );
        assertEquals( 6, execute( new String[] { "one", "two", "three", "four", "five", "six" } ) );
    }

    private int execute( String[] args )
        throws Exception
    {
        File jar = new File( "target/simple-client-app-1.0-SNAPSHOT.jar" );

        String[] execArgs = new String[args.length + 3];
        System.arraycopy( args, 0, execArgs, 3, args.length );
        execArgs[0] = "java";
        execArgs[1] = "-jar";
        execArgs[2] = jar.getCanonicalPath();
        Process p = Runtime.getRuntime().exec( execArgs );
        p.waitFor();
        return p.exitValue();
    }

}

Note that this test needs the packaged jar (simple-client-app-1.0-SNAPSHOT.jar), which isn't created when surefire runs the unit tests.

To run this we need a different approach. By different approach I mean a different plugin: the Failsafe Maven Plugin. The Failsafe Plugin is a fork of the Surefire plugin designed to run integration tests.

The Failsafe plugin goals are designed to run after the package phase, on the integration-test phase.

The Maven lifecycle has four phases for running integration tests:

* pre-integration-test: on this phase we can start any required service or do any action, like starting a database, or starting a webserver, anything... you can think on this as Junit.setUp()
* integration-test: failsafe will run the test on this phase, so after all required services are started.
* post-integration-test: time to shutdown all services... you can think of this as Junit.tearDown()
* verify: failsafe runs another goal that interprets the results of tests here, if any tests didn't pass failsafe will display the results and exit the build.

Configuring Failsafe in the POM:

<plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>failsafe-maven-plugin</artifactId>
        <version>2.4.3-alpha-1</version>
        <executions>
          <execution>
            <goals>
              <goal>integration-test</goal>
              <goal>verify</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

By default, the Surefire plugin executes **/Test\*.java, **/\*Test.java, and **/\*TestCase.java test classes. The Failsafe plugin will look for **/IT\*.java, **/\*IT.java, and **/\*ITCase.java. If you are using both the Surefire and Failsafe plugins, make sure that you use this naming convention to make it easier to identify which tests are being executed by which plugin.

In the next part of this series, I will talk about test coverage and demonstrate a method of including coverage of both Unit Tests and Integration Tests in a report that can be generated for a Maven project.