Java – How to disable entire tests (including inherited methods) in TestNG

I’m using the TestNG framework, and many test classes extend the Base abstract Test to provide some additional information.

I want to set @Test (enabled = false) on the entire test class, which is problematic in TestNG because any @Test comment on the method overwrites the first. This means that all methods will still run even if the class is disabled.

I found a workaround provided by the framework author in the pull request #816 – Add listener if defined class has @Test (enabled = false) , modify the @Test comment on the method:

public class TestClassDisabler implements IAnnotationTransformer {

@Override
  public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor,
                        Method testMethod) {
    if (testMethod != null) {
      Test test = testMethod.getDeclaringClass().getAnnotation(Test.class);
      if (test != null && !test.enabled()) {
        annotation.setEnabled(false);
      }
    }
  }
}

It’s like a charm, but only for tests that don’t use inheritance. For this case, testMethod.getDeclaringClass() from the listener returns the original class from which the method was declared, not the class that started the object’s instance of the method.

@Test(enabled = false)
class SpecificTest extends BaseTest {

@Test
    public void testSomething() {}
}

abstract class BaseTest {

@Test
    public void veryGenericTest() {}
}

For this example, only testSomething is disabled and veryGenericTest still runs.

IAnnotationTransformer is not a good choice here. There is no easy way to get the original instance of the calling method, just based on the Method interface. We can only get the base class where the method is declared.

Other solutions are used with IMethodInterceptor, which allows methods to be filtered out and provides access to test class instances before the test suite starts.

public class TestMethodsDisabler implements IMethodInterceptor {

@Override
    public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
        List<IMethodInstance> testsToRun = new ArrayList<>();

for (IMethodInstance method : methods) {
            Test testClass = method.getInstance()
                .getClass()
                .getAnnotation(Test.class);

if (testClass == null || testClass.enabled()) {
                testsToRun.add(method);
            }
        }

return testsToRun;
    }
}

In this way, when we set the @Test(enabled = false) annotation on the class, both standard and inherited methods are disabled.

You need to use this interceptor to create a test suite in testng.xml:

<suite name="ListenersSuite" parallel="false">
    <listeners>
        <listener class-name="your.package.support.TestMethodsDisabler"/>
    </listeners>

<test name="all-test" preserve-order="true" verbose="2">
        <packages>
            <package name="your.package.tests.*"/>
        </packages>
    </test>
</suite>

References

• TestNG documentation – method interceptors