How do I repeat a test case with JUnit 5?

[trancexpress]

Hi all,

as the title goes, how do I repeat a test case with JUnit 5? Not just repeating its single test methods. I.e. this question: https://stackoverflow.com/questions/58005980/how-to-repeat-junit5-tests-on-class-level

In short, I'm trying to reproduce a sporadic integration test failure that depends on internal Eclipse state.

Repeating one of the tests that runs into the sporadic fail led nowhere, so my next step is repeating the entire test case, then the entire test suite.

With JUnit 4 I was able to either use parameterized tests, or define the test multiple times in a test suite. With JUnit 5 repeating tests is based on test method annotations, I find nothing at the class level. And I can't define a test class multiple times in a test suite, since the selector for the suite only selects classes.

Best regards,
Simeon

[marcphilipp]

The recent 5.13.0 release introduced @ParameterizedClass and @ClassTemplate. There's no equivalent to @RepeatedTest on the class level, yet.

Here's a quick implementation:

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.stream.IntStream;
import java.util.stream.Stream;

import org.junit.jupiter.api.extension.ClassTemplateInvocationContext;
import org.junit.jupiter.api.extension.ClassTemplateInvocationContextProvider;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.platform.commons.support.AnnotationSupport;

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@ClassTemplate
@ExtendWith(RepeatedClass.Extension.class)
public @interface RepeatedClass {

	int value();

	class Extension implements ClassTemplateInvocationContextProvider {

		@Override
		public boolean supportsClassTemplate(ExtensionContext context) {
			return AnnotationSupport.isAnnotated(context.getElement(), RepeatedClass.class);
		}

		@Override
		public Stream<? extends ClassTemplateInvocationContext> provideClassTemplateInvocationContexts(
				ExtensionContext context) {
			int totalRepetitions = AnnotationSupport.findAnnotation(context.getElement(), RepeatedClass.class)
					.orElseThrow()
					.value();
			return IntStream.rangeClosed(1, totalRepetitions)
					.mapToObj(repetition -> new ClassTemplateInvocationContext() {
						@Override
						public String getDisplayName(int invocationIndex) {
							return "repetition %d of %d".formatted(repetition, totalRepetitions);
						}
					});
		}
	}
}

Usage:

@RepeatedClass(2)
class SomeTests {
	// ...
}

If you find that useful, please raise an issue so we can discuss making it a core feature.

[trancexpress]

Great, thank you! I'll give this a shot.

We'll document the example above in our internal JUnit 5 documentation, that will be enough for us. A few extra lines of code when needing to repeat a test case is OK.

Does the JUnit 5 documentation have this example, after the introduction of @ClassTemplate? Its a very useful one from my POV.

[marcphilipp]

@trancexpress Did this prove useful for you in the end? If so, would you mind creating an issue for the team to consider making it a core feature?

[trancexpress]

@trancexpress Did this prove useful for you in the end? If so, would you mind creating an issue for the team to consider making it a core feature?

Yes, the snippet is definitely very useful. I've opened this issue:

#4627

Let me know if something is missing there.