Provide access to arguments of parameterized tests in lifecycle callback methods

[larryricker]

Overview

Simple parameter resolution for strings, ints, doubles, longs works in M4, but no longer works in M5 and throws the following exception.

ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable

Was previously using M4, upgraded to M5 and issue began occurring.

Example:

@ParameterizedTest (name = "{index} [{arguments}] User Flags CONDITION NAME")
@ValueSource(strings = { "dev", "sit" })
@DisplayName("{index} [{arguments}] User Flags CONDITION NAME")
public void testUseTestScenario(String testEnvironment, TestInfo info, TestReporter testReporter) throws Exception {
}

Do not get a build exception, get a runtime exception with jUnit5 with gradle 3.5 running in Jenkins.

    => org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable [public void com.orgname.TestSituation.setUpBeforeEach(java.lang.String,org.junit.jupiter.api.TestInfo,org.junit.jupiter.api.TestReporter) throws java.lang.Exception].
  JUnit Jupiter:TestSituation:User Flags Situation A:2 [sit] User Flags Situation A
    MethodSource [className = 'com.orgname.TestSituation', methodName = 'testSituationA', methodParameterTypes = 'java.lang.String, org.junit.jupiter.api.TestInfo, org.junit.jupiter.api.TestReporter']
    => org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg0] in executable [public void com.orgname.TestSituation.setUpBeforeEach(java.lang.String,org.junit.jupiter.api.TestInfo,org.junit.jupiter.api.TestReporter) throws java.lang.Exception].

Related Issues

• ParameterizedTestParameterResolver conflicts with parameterized @BeforeEach #833
• Introduce extension API for accessing arguments of parameterized tests #1139
• Make invocation index for @ParameterizedTest available to test author #1668
• Add ability to access the current repetition of a RepeatedTest from afterEach in a custom Extension #1884
• Add lifecycle methods for parameterized containers #4352

Deliverables

• ...

[sbrannen]

FYI: this is a direct result of the changes performed in conjunction with #833.

[sbrannen]

Assigned to 5.0 M6 in order to make a decision about how to proceed.

[marcphilipp]

The reason is that we no longer resolve parameters from @Source annotations for @BeforeEach etc. methods. Do you need the parameter there?

[larryricker]

Yes. I need the parameter on @beforeeach and @AfterEach for several tests for Selenium and Appium testing. My Appium test examples do not demonstrate 'need', but more like 'nice to have', but are public. https://github.com/larryricker/ProjectOrganizerAppium/
My selenium tests 'need' parameters for @beforeeach and @AfterEach to setup accounts for the correct respective environments.

[sormuras]

@larryricker Look at how Paul is solving the WebDriver injection task: https://github.com/paul-hammant/JUnit5_DependencyInjection_WebDriver

[rendaw]

If I've understood the topic correctly, I need this as well.

I have a dozen or so complex objects that all my tests use that I set up statically. I recently identified a couple default parameters for the object creation that actually need to be tested at multiple values... My only options forward seem to be explicitly forwarding parameters to a setup function in each test case or downgrading to M4.

Also, it would be nice if it somehow worked with BeforeAll so heavy immutable fixtures don't get generated and destroyed at each test method.

[opncow]

Any news on this? Just migrated to JUnit 5 for the improved parameterized tests. "Forwarding parameters to a setup function in each test case" feels more like a step back, though. :-/

[marcphilipp]

IMHO a setup function that gets parameterized test arguments only makes sense when all test methods in a class are parameterized and use at least similar parameters.

Thus, I think it would make sense to support this when the class is parameterized instead of the method (as we plan to do in #878).

[sbrannen]

I agree with @marcphilipp's analysis.

[Tradunsky]

Reproduces on 5.1.1 and on the latest 5.2.0-M1

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter [java.lang.String arg1] in executable 

@ParameterizedTest(name = "meaningful name and argument {3} based on argument {0} and argument {1}")
@CsvSource({
            "6", "10", "40",
            "1", "10", "90"
})
public void meaningfulTestName(String meaningfulArg1, String meaningfulArg2, String meaningfulArg3) {
}

The same with long, int

[sbrannen]

@Tradunsky, your stack trace is incomplete: in executable.

Which executable?

If it's for a @BeforeEach/@AfterEach method, then that's expected since this behavior has not changed since 5.0 GA.

[geo-m]

@Tradunsky
Shouldnt it be "6, 10, 40" instead of "6","10","40" ?

[Tradunsky]

@geo-m Good catch! Works like a charm.
Thanks.

@sbrannen
Thank you as well

[sbrannen]

Indeed, good catch, @geo-m!

[marcphilipp]

It might be worth returning a custom interface so we can add convenience lookup methods to it later rather than adding more methods to ExtensionContext:

interface ExtensionContext {
	// ...
	Optional<ResolvedArguments> getResolvedArguments();
}
interface ResolvedArguments {
	List<?> getRawArguments();
	// we can add lookup methods here later
}

[marcphilipp]

@t1 What was your use case for this enhancement? In which lifecycle methods or extension methods would you want to access the resolved parameters? Resolution happens just before the invocation of the test method so no lifecycle methods or callbacks that are invoked before the invocation would be able to access them.

[marcphilipp]

Team epiphany: There's already a way to access the arguments of test methods. An extension can implement InvocationInterceptor, override interceptTestTemplateMethod, and call the getArguments method on the supplied ReflectiveInvocationContext.

[danielhodder]

The main reason I would want this is to be able to use extensions (like TemporryFolder, or an extension that creates a database) in my @Before setup methods. A really basic example would be something like this:

@ExtendWith(CreateDatabase.class)
class SchemaTest {
    @Before
    public void setup(javax.sql.Datasource db) {
        // Create data here
    }
    
    @Test
    public void test(javax.sql.Datasource db) {
        // Verify data in the database
    }
}

[t1]

@marcphilipp I have moved on to the next project and already completely forgot about this issue and the use-case behind it... my memory is terrible 😳. But I went back and created a reproducer, just to find out that... you are completely right: instead of using a BeforeEachCallback, I can use a TestTemplateInvocationContextProvider. Works like a charm. Many thanks!

[t1]

Just in case anybody is interested, my reproducer/demo is here: https://github.com/t1/junit-testtemplate-invocationinterceptor-demo

[marcphilipp]

@danielhodder That's already possible by implementing a ParameterResolver that provides the same javax.sql.Datasource for both methods and uses ExtensionContext.Store to cache it and automatically close it (via CloseableResource), if necessary. There's an example in the User Guide.

[marcphilipp]

you are completely right: instead of using a BeforeEachCallback, I can use a TestTemplateInvocationContextProvider. Works like a charm. Many thanks!

@t1 You mean InvocationInterceptor, right?

[t1]

@marcphilipp: yes. sorry for the confusion.

[marcphilipp]

To summarize, it's already possible to access the arguments of any test method, including parameterized ones, from an extension by implementing InvocationInterceptor, overriding the appropriate method (for example, interceptTestTemplateMethod), and calling the getArguments method on the supplied ReflectiveInvocationContext.

In general, parameter resolution happens immediately before invoking the method. Therefore, arguments cannot be provided to lifecycle callback methods that are invoked before (for example, @BeforeEach) the (parameterized) test method is called. It would be possible to write an extension that stores them and provides them to lifecycle methods that are invoked after (for example, @AfterEach) the (parameterized) test method.

However, in the specific case of @ParameterizedTest methods, the set of arguments for an invocation is already known prior to invoking lifecycle methods. Thus, it could be made available to them. Since different @ParameterizedTest methods in a test class often have different parameters, re-declaring the parameters directly on lifecycle methods is brittle. We could support injecting ArgumentAccessor but, as with re-declaring the parameters, this would fail if there's a @Test method in the same test class. It seems to me that this use case would be better served by specifying parameters on the class level (which we'll add in #878) which will inject them into the test class constructor or its fields from where lifecycle methods can access them.

[t1]

👍

Minor addition: the same is also true for @TestTemplate tests.