I’ve been working with TestNG for 2 years now. The team that I work on made a decision to switch to TestNG thanks to one very important feature, @BeforeSuite and @BeforeClass. We are using Gradle as our build tool. Gradle supports TestNG tests execution.

With all the great features that TestNG comes with it also comes with features that could obfuscate test code readability. We also observed that there is no guarantee that the tests in the same class file will be executed together as a set.

With 25 minutes time of my train ride to/from work and determination to build something sweet and simple, I decided to create a Gradle plugin that will run TestNG tests in deterministic order, supporting only a small set of TestNG features.

I’ve made the code available here: https://bitbucket.org/gigu/testngng. The code is still work in progress, however there is already a functional Gradle plugin that can produce same style of XML reports as TestNG itself. There is also option of a Html report with simple and pretty style. I’ve written it in Groovy as I like Groovy. It’s Open and available to anyone.

Features supported by TestNGNG

1. TestNGNG will recursively scan class folder passed in as parameter, in search of possible tests files. It treats this top-level folder a Suite.
2. TestNGNG ENSURES all the tests in a class file are executed as a set of tests.
3. TestNGNG will build a tree of tests and dependencies at the beginning of a run, before it executes a single test. The test tree is build in a form of a: One Test Suite -> Many Test Classes -> Many Tests.
4. TestNGNG supports original TestNG annotations, it doesn’t have it’s own annotations as it is only a runner.
5. TestNGNG support @Test annotation of a method or a class.
6. It supports dependencies between test methods with:

   @Test(dependsOnMethods=”foo”)

7.  It supports @BeforeSuite/Class/Test/Method and @AfterSuite/Class/Test/Method. However, @BeforeTest, @AfterTest and @BeforeMethod, @AfterMethod are treated in the same way and executed before/after each test. Just to avoid (or add to) confusion.
8. It supports disabling of a test by enabled attribute of an annotation:

   @Test(enabled=false)

   I’m not proud of this feature though and am tempted to remove it.

9. It supports exception expectation by expectedException attribute:

   @Test(expectedException=WhatevaException.class)

10. TestNGNG supports data providers, however they have to be declared within the same Test Class file. Data providers could be named, or anonymous. For example:

    @Test(dataProvider = "makeMeSomeData")
      public void testWithProvider(String v1, String v2){
      System.out.println(String.format("%s - %s ", v1, v2));
    }
    @DataProvider
    public Object[][] makeMeSomeData() {
      return new Object[][]{
        {"some1", "Some2"},
        {"some3", "some4"}
       };
    }
    

11. TestNGNG supports setup methods inherited from base classes. For example:

    public abstract class BaseForTestWithTestSetupMethods {
        public String baseValue = "";
        @BeforeMethod
        public void executeBeforeSuite() {
            baseValue = "BeforeMethod";
        }
    }
    

    public class TestClassExtendingFromBaseClass extends BaseForTestWithTestSetupMethods {
        @Test
        public void shouldPass() {
            assertThat(baseValue, is("BeforeMethod"));
        }
    }
    

Gradle plugin

Gradle plugin is very simple to use. It only requires plugin jar on a class path and TestNG as a testCompile time dependency. The plugin adds testngng task to the project. Sample use of a plugin:

buildscript {
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath fileTree(dir: '../gradle-plugin/build/libs/', include: '*.jar')
        classpath fileTree(dir: '../testngng/build/libs/', include: '*.jar')
        classpath group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.8.0'
    }
}
apply plugin: 'java'
apply plugin: 'testngng'
repositories {
    mavenCentral()
}
dependencies {
    testCompile('org.testng:testng:6.5.1')
}

Most of the other TestNG features are not covered as they are useless and very often stand in a way of tests readability and simplicity.

Don’t ask what you test framework can do for you, ask what can you do for your test framework.

How to build the plugin and use it

You would need to checkout the project from a BitBucket and build it with Gradle by typing from command line:

gradle jar

The build jar will be used by test runner Gradle plugin inside inside the testngng_gradle_sample project. You can ran gradle command from the sample project folder:

gradle testngng

I would love to see someone having a go and providing me with some feedback. There is still a whole lot of stuff on my list that needs development in the near future, like a build tests running feedback, more plugin configurations and multiple suites per project/module.

Cheers, Greg