Context
Consider a Junit 5 test suite with a maven-failsafe configuration that is configured to run the tests concurrently in multiple processes using the forkCount
parameter.
Question
Is it possible to programmatically control which forks pick up what test?
Motivation
The performance of the overall test suite can change dramatically based on how tests are distributed across forks. This is especially relevant for Spring-based tests: often one of the slowest parts of a spring-based test is spinning up the application context (AC). Therefore, the Spring TestContext framework caches the AC between tests, but clearly this isn't (can't be) shared between processes. If a parallel test suite uses multiple distinct ACs, then minimizing the distinct number of ACs per process can significantly reduce runtime.
Failed Attempt
One option I've explored is to use test suites. We can configure failsafe to fork based on test suite, and then manually group the tests in appropriate suites. This is however less than ideal, because it relies on manual (thus unreliable) assignment of tests to suites, and because it strictly pins a suite to a fork (which can leave other forks idle when suites aren't approximately the same size).