I’ve been working with TestNG for 2 years now. The team that I work on made a decision to switch to TestNG thanks to one very important feature, @BeforeSuite and @BeforeClass. We are using Gradle as our build tool. Gradle supports TestNG tests execution.
With all the great features that TestNG comes with it also comes with features that could obfuscate test code readability. We also observed that there is no guarantee that the tests in the same class file will be executed together as a set.
With 25 minutes time of my train ride to/from work and determination to build something sweet and simple, I decided to create a Gradle plugin that will run TestNG tests in deterministic order, supporting only a small set of TestNG features.
I’ve made the code available here: https://bitbucket.org/gigu/testngng. The code is still work in progress, however there is already a functional Gradle plugin that can produce same style of XML reports as TestNG itself. There is also option of a Html report with simple and pretty style. I’ve written it in Groovy as I like Groovy. It’s Open and available to anyone.
Features supported by TestNGNG
- TestNGNG will recursively scan class folder passed in as parameter, in search of possible tests files. It treats this top-level folder a Suite.
- TestNGNG ENSURES all the tests in a class file are executed as a set of tests.
- TestNGNG will build a tree of tests and dependencies at the beginning of a run, before it executes a single test. The test tree is build in a form of a: One Test Suite -> Many Test Classes -> Many Tests.
- TestNGNG supports original TestNG annotations, it doesn’t have it’s own annotations as it is only a runner.
- TestNGNG support @Test annotation of a method or a class.
- It supports dependencies between test methods with:
@Test(dependsOnMethods=”foo”)
- It supports @BeforeSuite/Class/Test/Method and @AfterSuite/Class/Test/Method. However, @BeforeTest, @AfterTest and @BeforeMethod, @AfterMethod are treated in the same way and executed before/after each test. Just to avoid (or add to) confusion.
- It supports disabling of a test by enabled attribute of an annotation:
@Test(enabled=false)
I’m not proud of this feature though and am tempted to remove it.
- It supports exception expectation by expectedException attribute:
@Test(expectedException=WhatevaException.class)
- TestNGNG supports data providers, however they have to be declared within the same Test Class file. Data providers could be named, or anonymous. For example:
@Test(dataProvider = "makeMeSomeData") public void testWithProvider(String v1, String v2){ System.out.println(String.format("%s - %s ", v1, v2)); } @DataProvider public Object[][] makeMeSomeData() { return new Object[][]{ {"some1", "Some2"}, {"some3", "some4"} }; } - TestNGNG supports setup methods inherited from base classes. For example:
public abstract class BaseForTestWithTestSetupMethods { public String baseValue = ""; @BeforeMethod public void executeBeforeSuite() { baseValue = "BeforeMethod"; } }public class TestClassExtendingFromBaseClass extends BaseForTestWithTestSetupMethods { @Test public void shouldPass() { assertThat(baseValue, is("BeforeMethod")); } }
Gradle plugin
Gradle plugin is very simple to use. It only requires plugin jar on a class path and TestNG as a testCompile time dependency. The plugin adds testngng task to the project. Sample use of a plugin:
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath fileTree(dir: '../gradle-plugin/build/libs/', include: '*.jar')
classpath fileTree(dir: '../testngng/build/libs/', include: '*.jar')
classpath group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.8.0'
}
}
apply plugin: 'java'
apply plugin: 'testngng'
repositories {
mavenCentral()
}
dependencies {
testCompile('org.testng:testng:6.5.1')
}
Most of the other TestNG features are not covered as they are useless and very often stand in a way of tests readability and simplicity.
Don’t ask what you test framework can do for you, ask what can you do for your test framework.
How to build the plugin and use it
You would need to checkout the project from a BitBucket and build it with Gradle by typing from command line:
gradle jar
The build jar will be used by test runner Gradle plugin inside inside the testngng_gradle_sample project. You can ran gradle command from the sample project folder:
gradle testngng
I would love to see someone having a go and providing me with some feedback. There is still a whole lot of stuff on my list that needs development in the near future, like a build tests running feedback, more plugin configurations and multiple suites per project/module.
Cheers, Greg
Not too long ago I had a conversation with one of the senior member of the management team of project I’m working on. I had some ideas on how can we do things faster by improving our testing (not developer testing but end-to-end, QA and regression testing). After few minutes of conversation I was asked a very basic question: “What is the definition of DONE on our project?” I was just going to open my mouth and jump out with an answer like: “Well, it takes us usually 3 days to develop piece of functionality”, but I stopped. I actually wasn’t sure. We spend few more minutes discussing some other issues but when I left I felt that this question is still on the back of my mind, trying to desperately find the answer.
I’ve been writing some general ideas about creativity in the last couple of posts (
Imagine maze and finding a way out of it as a problem. Two people are asked to find exit from the maze, a solution to a problem. One of them is told that at the end of the maze there will be a reward waiting (extrinsic motivation). The other person is a huge maze enthusiast and enjoys a challenge, is also told to take all the time needed to find the most interesting way out of the maze (intrinsic motivation). Two solutions delivered by both maze solvers could be very different as:
Greg Gigon's wEbLOG 