Use a tck-specific standalone.xml

[bstansberry]

This was written with reference to the concurrency runner, but it's likely relevant to others.

Make it easier to run the TCK multiple times (e.g. when debugging problems) when using an existing WildFly installation as the TCK target (i.e. when JBOSS_HOME is set.) Right now it is hard because TCK-specific alterations to the base config (via CLI) fail because a previous run has altered the config from the expected base state.

One way to do this: the arquillian.xml could specify, for example 'standalone-concurrency-tck.xml' as the config to use.

Previous maven goals would ensure this file exists, with content that matches the standard WF config that's under test.

If an already-provisioned installation is being used, the maven-resources-plugin could create standalone-concurrency-tck.xml via file copy. If galleon is provisioning, the Galleon plugin config can control the name of the provisioned config file.

[jamezp]

I think for Arquillian based tests the arquillian.xml is the way to go. In the RESTEasy TCK it works with something like:

mvn test -Dgalleon.skip=true -Dtest=MyFailingTest -Ddebug -Ddebug.port=5005

https://github.com/resteasy/resteasy-tck-runner/blob/fee64ee110072c0f8b170f9d3b1f548d0325ed76/resteasy-tck-adapter/pom.xml#L395-L407

Tests that don't use Arquillian would have to be handled differently. One option would be to simply override the JAVA_OPTS. While this removes WildFly default JVM arguments, it would work.

Personally I would like to see no TCK runners using custom standalone.xml files. The configuration is better done via CLI or provisioning in general.

[bstansberry]

@jamezp To clarify, concurrency isn't using a custom standalone.xml file. Well, at least it's not custom before it runs a CLI script, which AIUI is how you think it should be done.

The problem is once that script runs, the file it ran against is "polluted" and you can't run the script again without first creating a new base file. That defeats the purpose of using an existing installation, instead of having the TCK runner provision a fresh installation each time it runs.

[jamezp]

Ah, okay that makes sense. What I've done elsewhere is change the CLI command to check the resources before it attempts to add new ones. Another option would be to pass -Dwildfly.skip=true to skip the script execution.

[bstansberry]

A more sophisticated CLI script is another valid solution for sure. As is yaml. ;)

Thanks for the -Dwildfly.skip=true tip!

[darranl]

For the security TCKs I was so close to wanting to use ant ;-)

But realistically in that case I am using a shell script so was considering making sure the install and configure step for WildFly can always be skipped if already set up.

[emmartins]

Why not simply copy the standalone.xml to standalone-concurrency-tck-runner.xml (after provisioning if that's used) and then reconfig and use that copy?

[jamezp]

Why not simply copy the standalone.xml to standalone-concurrency-tck-runner.xml (after provisioning if that's used) and then reconfig and use that copy?

For concurrency specifically we're okay now because we check before any adds are done.