[ Java 文章收集 ] A JUnit Rule to Conditionally Ignore Tests

Source From Here 
Preface 
I always believed that using @Ignore to deactivate tests is a bad idea. Except, maybe as one way to put tests that fail intermittently into quarantine to attend to them later (as Martin Fowler describes it here). This bears the danger that the test suite decays as more and more tests keep getting ignored and forgotten. Therefore you should have a policy in place to ensure that tests aren’t quarantined for too long. Well, so I thought until recently when we ran into this: 

In a project that Frank and I work on, we ran into an SWT issue described here. On non-Windows platforms, asserting whether an SWT widget has got the input focus does not work with automated tests. We decided to ignore focus-related tests on non-Windows platforms for now. Though our build server runs on Linux we found it safe enough as both our development environments run on Windows. 

In JUnit, assumptions are the means to skip tests that aren’t meaningful under the given condition. Expressed that way, our test would look like this: 

view plaincopy to clipboardprint?

1. public void testFocus() {  
2.   assumeTrue( isRunningOnWindows() );  
3.   // ...  
4. }  

But we didn’t want the test code mingled with conditions if it should be executed at all. The code that decides whether the test is ignored should be separated from the test code itself. This led us to creating a ConditionalIgnore annotation and a corresponding rule to hook it into the JUnit runtime. The thing is simple and best explained with an example: 
- SomeTest.java (Test Case) 

view plaincopy to clipboardprint?

1. package test;  
2.   
3. import static org.junit.Assert.*;  
4.   
5. import org.junit.Rule;  
6. import org.junit.Test;  
7.   
8. import test.ConditionalIgnoreRule.ConditionalIgnore;  
9.   
10. public class SomeTest {  
11.     @Rule  
12.     public ConditionalIgnoreRule rule = new ConditionalIgnoreRule();  
13.       
14.     @Test  
15.     @ConditionalIgnore( condition = NotRunningOnWindows.class )   
16.     public void testNotInWindows() {      
17.         System.out.printf("\t[Info] Running testing...\n");  
18.         assertTrue(!System.getProperty( "os.name" ).startsWith( "Windows" ));  
19.     }  
20.   
21.     @Test  
22.     @ConditionalIgnore( condition = RunningOnWindowsOnly.class )      
23.     public void testOnlyInWindows() {     
24.         System.out.printf("\t[Info] Running testing...\n");  
25.         assertTrue(System.getProperty( "os.name" ).startsWith( "Windows" ));  
26.     }  
27. }  

- ConditionalIgnoreRule.java (Conditional Ignore Rule Annotation) 

view plaincopy to clipboardprint?

1. package test;  
2.   
3. import java.lang.annotation.ElementType;  
4. import java.lang.annotation.Retention;  
5. import java.lang.annotation.RetentionPolicy;  
6. import java.lang.annotation.Target;  
7. import java.lang.reflect.Modifier;  
8.   
9. import org.junit.Assume;  
10. import org.junit.rules.MethodRule;  
11. import org.junit.runners.model.FrameworkMethod;  
12. import org.junit.runners.model.Statement;  
13.   
14. public class ConditionalIgnoreRule implements MethodRule {  
15.   
16.     public interface IgnoreCondition {  
17.         boolean isSatisfied();  
18.     }  
19.   
20.     @Retention(RetentionPolicy.RUNTIME)  
21.     @Target({ ElementType.METHOD })  
22.     public @interface ConditionalIgnore {  
23.         Classextends IgnoreCondition> condition();  

    }  

  

    @Override  

    public Statement apply(Statement base, FrameworkMethod method, Object target) {  

        System.out.printf("\t[Info] Apply ignore condition check...\n");  

        Statement result = base;  

        if (hasConditionalIgnoreAnnotation(method)) {  

            IgnoreCondition condition = getIgnoreContition(target, method);  

            if (condition.isSatisfied()) {  

                System.out.printf("\t[Info] Prepare ignore statemnt...\n");  

                result = new IgnoreStatement(condition);  

            }  

        }  

        return result;  

    }  

  

    private static boolean hasConditionalIgnoreAnnotation(FrameworkMethod method) {  

        return method.getAnnotation(ConditionalIgnore.class) != null;  

    }  

  

    private static IgnoreCondition getIgnoreContition(Object target,  

            FrameworkMethod method) {  

        ConditionalIgnore annotation = method  

                .getAnnotation(ConditionalIgnore.class);  

        return new IgnoreConditionCreator(target, annotation).create();  

    }  

  

    private static class IgnoreConditionCreator {  

        private final Object target;  

        private final Classextends IgnoreCondition> conditionType;  

  

        IgnoreConditionCreator(Object target, ConditionalIgnore annotation) {  

            this.target = target;  

            this.conditionType = annotation.condition();  

        }  

  

        IgnoreCondition create() {  

            checkConditionType();  

            try {  

                return createCondition();  

            } catch (RuntimeException re) {  

                throw re;  

            } catch (Exception e) {  

                throw new RuntimeException(e);  

            }  

        }  

  

        private IgnoreCondition createCondition() throws Exception {  

            IgnoreCondition result;  

            if (isConditionTypeStandalone()) {  

                result = conditionType.newInstance();  

            } else {  

                result = conditionType  

                        .getDeclaredConstructor(target.getClass()).newInstance(  

                                target);  

            }  

            return result;  

        }  

  

        private void checkConditionType() {  

            if (!isConditionTypeStandalone()  

                    && !isConditionTypeDeclaredInTarget()) {  

                String msg = "Conditional class '%s' is a member class "  

                        + "but was not declared inside the test case using it.\n"  

                        + "Either make this class a static class, "  

                        + "standalone class (by declaring it in it's own file) "  

                        + "or move it inside the test case using it";  

                throw new IllegalArgumentException(String.format(msg,  

                        conditionType.getName()));  

            }  

        }  

  

        private boolean isConditionTypeStandalone() {  

            return !conditionType.isMemberClass()  

                    || Modifier.isStatic(conditionType.getModifiers());  

        }  

  

        private boolean isConditionTypeDeclaredInTarget() {  

            return target.getClass().isAssignableFrom(  

                    conditionType.getDeclaringClass());  

        }  

    }  

  

    private static class IgnoreStatement extends Statement {  

        private final IgnoreCondition condition;  

  

        IgnoreStatement(IgnoreCondition condition) {  

            this.condition = condition;  

        }  

  

        @Override  

        public void evaluate() {  

            System.out.printf("\t[Info] Ignored by %s...\n", condition.getClass().getSimpleName());  

            Assume.assumeTrue("Ignored by "  

                    + condition.getClass().getSimpleName(), true);  

        }  

    }  

}  

- NotRunningOnWindows.java (Rule1) 

view plaincopy to clipboardprint?

1. package test;  
2.   
3. import test.ConditionalIgnoreRule.IgnoreCondition;  
4.   
5. public class NotRunningOnWindows implements IgnoreCondition{  
6.     @Override  
7.     public boolean isSatisfied() {  
8.         String os = System.getProperty( "os.name" );  
9.         System.out.printf("\t[Debug] OS=%s (Skip test case? %s)\n", os, os.startsWith("Win"));  
10.         return os.startsWith("Win");  
11.     }  
12. }  

- RunningOnWindowsOnly.java (Rule 2) 

view plaincopy to clipboardprint?

1. package test;  
2.   
3. import test.ConditionalIgnoreRule.IgnoreCondition;  
4.   
5. public class RunningOnWindowsOnly implements IgnoreCondition{  
6.     @Override  
7.     public boolean isSatisfied() {  
8.         String os = System.getProperty( "os.name" );  
9.         System.out.printf("\t[Debug] OS=%s (Skip test case? %s)\n", os, !os.startsWith("Win"));  
10.         return !os.startsWith("Win");  
11.     }  
12. }  

If you run test case in Windows, the execution result will be: 

[Info] Apply ignore condition check...
[Debug] OS=Windows 8.1 (Skip test case? false)
[Info] Running testing...
[Info] Apply ignore condition check...
[Debug] OS=Windows 8.1 (Skip test case? true)
[Info] Prepare ignore statemnt...
[Info] Ignored by NotRunningOnWindows...

The ConditionalIgnore annotation requires a ‘condition’ property that points to a class that implements IgnoreContition. At runtime, an instance of theIgnoreContition implementation is created and its isSatisfied() method decides whether the test is ignored (returns true) or not (returns false). Finally there is anIgnoreConditionRule that hooks the annotations into the JUnit runtime. 

If the IgnoreCondition implementation decides to ignore a test case, an AssumptionViolatedException is thrown. Therefore the ConditionalIgnore annotation has the same effect as if an Assume condition would return false. With a slight difference that we consider an advantage: @Before and @After methods are not executed for ignored tests. 

The source code of the rule and its related classes can be found here. 

The remaining issue with Assume is that it affects the test statistics. If an Assume condition is found to be false, it treats the test as having passed even though it hasn’t run. To overcome that, you’d have to provide your own runner that handles AssumptionViolatedException the way you want. 

Even though I just wrote about ignoring tests in length, I am still convinced that tests at best should not be ignored and if so only in exceptional cases. 

Supplement 
* Stackoverflow - Conditionally ignoring tests in JUnit 4