I have a class that generates Restassured JUnit tests in a specific subfolder of my build/tests/ structure, based on some postman test collections.

I have written a test to take all the *.postman_collection.json files in my src/test/resources/testCollections folder. If I want to verify that the generated files work I have to run the test again. I'd like to be able to run everything with mvn clean test.

Furthermore, I clear out the generated folder every time I run mvn clean, which I would have to undo if I had to run the tests twice.

Code is on github (I had issues picking out a minimalistic snippet to understand it from).

Update: switch to gradle

I tried to follow the suggestions from @chubbsondubs but something is not working as I expect it to.

build.gradle

plugins {
    id 'java-library'
    id 'jacoco'
}

repositories {
    mavenLocal()
    maven {
        url = uri('/')
    }
}

dependencies {
    api 'com.fasterxml.jackson.core:jackson-databind:2.18.3'
    api '.slf4j:slf4j-api:2.0.17'
    api 'ch.qos.logback:logback-classic:1.5.13'

    testImplementation '.junit.jupiter:junit-jupiter:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-engine:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-api:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-params:5.12.1'

    testRuntimeOnly '.junit.platform:junit-platform-launcher'
    
    testImplementation 'io.rest-assured:rest-assured:5.5.1'
    testImplementation 'io.rest-assured:rest-assured-common:5.5.1'
    
    testImplementation '.jacoco:.jacoco.core:0.8.11'
    testImplementation '.jacoco:.jacoco.agent:0.8.11'
    testImplementation '.jacoco:.jacoco.report:0.8.11'

}

group = 'dk.kodeninjaer.testing'
version = '1.0-SNAPSHOT'
description = 'postman-to-restassured'
java {
   sourceCompatibility = '21'
}

test {
    finalizedBy jacocoTestReport // report is always generated after tests run
}

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

tasks.named('test', Test) {
    useJUnitPlatform()
}

jacocoTestReport {
    dependsOn test // tests are required to run before generating the report
    // exclude Main.class from the coverage report
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: [
                "dk/kodeninjaer/testing/converter/Main.class"
            ])
        }))
    }
}

jacoco {
    toolVersion = "0.8.11"
    //reportsDirectory = layout.buildDirectory.dir('customJacocoReportDir')
}


/* generate and compile */

// register a new directory for source files
sourceSets {
    generatedTests {
        java.srcDir 'build/generated/gradle'
        compileClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
        runtimeClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
    }
}

// a task to execute the generatedTests source set
tasks.register("runGeneratedTests",Test) {
    description = 'Runs the generated tests.'
    group = 'verification'
    testClassesDirs = sourceSets.generatedTests.output.classesDirs
    classpath = sourceSets.main.runtimeClasspath + sourceSets.generatedTests.runtimeClasspath
    useJUnitPlatform()

    // run this after you run the test task
    shouldRunAfter test
}

tasks.register("generateUnitTests", JavaExec) {
    description = 'Generates dynamic unit tests from Postman collections'
    group = 'verification'
    classpath = sourceSets.main.runtimeClasspath
    mainClass = "dk.kodeninjaer.testing.converter.Main"
    args = [
            // your args to generate the program
            "src/test/resources/TestCollection.postman_collection.json",
            "dk.kodeninjaer.gradle.testing",
            "PostmanToRestAssuredGenerator"
        ]
    //shouldRunBefore generatedTests
}



// when check task is run we add our generatedTests that way if test task 
// is run it will also run our generatedTests
tasks.named('check') {
   dependsOn(tasks.runGeneratedTests)
}

Error

 Could not create task ':generateUnitTests'.
    > No signature of method: .gradle.api.tasks.JavaExec.mustRunBefore() is applicable for argument types: (.gradle.api.taskspile.JavaCompile_Decorated) values: [task ':compileTestJava']
      Possible solutions: mustRunAfter([Ljava.lang.Object;), mustRunAfter([Ljava.lang.Object;)

Update 2

If I run ./gradlew generateUnitTests and ./gradlew runGeneratedTests or ./gradlew check it executes, but no testreport is generated...

I have a class that generates Restassured JUnit tests in a specific subfolder of my build/tests/ structure, based on some postman test collections.

I have written a test to take all the *.postman_collection.json files in my src/test/resources/testCollections folder. If I want to verify that the generated files work I have to run the test again. I'd like to be able to run everything with mvn clean test.

Furthermore, I clear out the generated folder every time I run mvn clean, which I would have to undo if I had to run the tests twice.

Code is on github (I had issues picking out a minimalistic snippet to understand it from).

Update: switch to gradle

I tried to follow the suggestions from @chubbsondubs but something is not working as I expect it to.

build.gradle

plugins {
    id 'java-library'
    id 'jacoco'
}

repositories {
    mavenLocal()
    maven {
        url = uri('https://repo.maven.apache./maven2/')
    }
}

dependencies {
    api 'com.fasterxml.jackson.core:jackson-databind:2.18.3'
    api '.slf4j:slf4j-api:2.0.17'
    api 'ch.qos.logback:logback-classic:1.5.13'

    testImplementation '.junit.jupiter:junit-jupiter:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-engine:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-api:5.12.1'
    testImplementation '.junit.jupiter:junit-jupiter-params:5.12.1'

    testRuntimeOnly '.junit.platform:junit-platform-launcher'
    
    testImplementation 'io.rest-assured:rest-assured:5.5.1'
    testImplementation 'io.rest-assured:rest-assured-common:5.5.1'
    
    testImplementation '.jacoco:.jacoco.core:0.8.11'
    testImplementation '.jacoco:.jacoco.agent:0.8.11'
    testImplementation '.jacoco:.jacoco.report:0.8.11'

}

group = 'dk.kodeninjaer.testing'
version = '1.0-SNAPSHOT'
description = 'postman-to-restassured'
java {
   sourceCompatibility = '21'
}

test {
    finalizedBy jacocoTestReport // report is always generated after tests run
}

tasks.withType(JavaCompile) {
    options.encoding = 'UTF-8'
}

tasks.withType(Javadoc) {
    options.encoding = 'UTF-8'
}

tasks.named('test', Test) {
    useJUnitPlatform()
}

jacocoTestReport {
    dependsOn test // tests are required to run before generating the report
    // exclude Main.class from the coverage report
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: [
                "dk/kodeninjaer/testing/converter/Main.class"
            ])
        }))
    }
}

jacoco {
    toolVersion = "0.8.11"
    //reportsDirectory = layout.buildDirectory.dir('customJacocoReportDir')
}


/* generate and compile */

// register a new directory for source files
sourceSets {
    generatedTests {
        java.srcDir 'build/generated/gradle'
        compileClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
        runtimeClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
    }
}

// a task to execute the generatedTests source set
tasks.register("runGeneratedTests",Test) {
    description = 'Runs the generated tests.'
    group = 'verification'
    testClassesDirs = sourceSets.generatedTests.output.classesDirs
    classpath = sourceSets.main.runtimeClasspath + sourceSets.generatedTests.runtimeClasspath
    useJUnitPlatform()

    // run this after you run the test task
    shouldRunAfter test
}

tasks.register("generateUnitTests", JavaExec) {
    description = 'Generates dynamic unit tests from Postman collections'
    group = 'verification'
    classpath = sourceSets.main.runtimeClasspath
    mainClass = "dk.kodeninjaer.testing.converter.Main"
    args = [
            // your args to generate the program
            "src/test/resources/TestCollection.postman_collection.json",
            "dk.kodeninjaer.gradle.testing",
            "PostmanToRestAssuredGenerator"
        ]
    //shouldRunBefore generatedTests
}



// when check task is run we add our generatedTests that way if test task 
// is run it will also run our generatedTests
tasks.named('check') {
   dependsOn(tasks.runGeneratedTests)
}

Error

 Could not create task ':generateUnitTests'.
    > No signature of method: .gradle.api.tasks.JavaExec.mustRunBefore() is applicable for argument types: (.gradle.api.taskspile.JavaCompile_Decorated) values: [task ':compileTestJava']
      Possible solutions: mustRunAfter([Ljava.lang.Object;), mustRunAfter([Ljava.lang.Object;)

Update 2

If I run ./gradlew generateUnitTests and ./gradlew runGeneratedTests or ./gradlew check it executes, but no testreport is generated...

• java
• gradle
• code-generation
• rest-assured

• 2 First generated code should not be generated in src/ directories` ... the correct location would be the target directory.. It might be the case that you take a deep look at dynamic tests of JUnit Jupiter junit./junit5/docs/current/user-guide/… If you like to generate those tests and execute them later with Maven you should write a plugin which should be bound to the generate-test-sources .... – khmarbaise Commented Mar 15 at 8:30

1 Answer 1

Since the modality of your tool is to have a developer generate unit tests then run those generated files later. I think you should mirror that way of working in your projects build. So instead of executing your tool within a unit test goal. You should execute it outside of the unit test phase/goal. That would generate the unit tests based on your test Postman collections. Then when the test goal runs the files will already be there ready to run as normal unit tests. Then there is no problem of execute and test from within tests.

As far as I know having another separate build folder with Maven is not possible. But, I'm not much of a maven kinda guy. This is possible with Gradle though.

// register a new directory for source files
sourceSets {
    generatedTests {
        java.srcDir 'src/test/gen'
        compileClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
        runtimeClasspath += sourceSets.main.output + sourceSets.mainpileClasspath + sourceSets.testpileClasspath
    }
}

// a task to execute the generatedTests source set
tasks.register("generatedTests",Test) {
    description = 'Runs the generated tests.'
    group = 'verification'
    testClassesDirs = sourceSets.generatedTests.output.classesDirs
    classpath = sourceSets.main.runtimeClasspath + sourceSets.generatedTests.runtimeClasspath
    useJUnitPlatform()

    // run this after you run the test task
    shouldRunAfter test
}

tasks.register("generateUnitTests", JavaExec) {
    description = 'Generates dynamic unit tests from Postman collections'
    group = 'verification'
    classpath = sourceSets.main.runtimeClasspath
    main = "dk.kodeninjaer.testing.converter.Main"
    args = [
        // your args to generate the program
    ]
}

// when check task is run we add our generatedTests that way if test task 
// is run it will also run our generatedTests
tasks.named('check') {
   dependsOn(tasks.generatedTests)
}

// run this before we compile all tests source sets
tasks.name('compileTestJava') {
   dependsOn(tasks.generateUnitTests)
}