Add a new test group called vendor

[sophia-guo]

Currently tests have openjdk, system, functional, external categories (aka test group). Vendor's tests or feature tests can also be specified by JDK_VENDOR and TEST_FLAG combined with playlist tag <feature> or <vendor> .

There are cases where vendors are not able to open-source their tests, or the tests are not suited for inclusion to the main AQAvit set of tests. For those cases, new test targets may be included into a test run via VENDOR_TEST_REPO, VENDOR_TEST_BRANCH, VENDOR_TEST_SHA, VENDOR_TEST_DIR parameters.

However for vendors there are still some special requirements which can't be satisfied by those categories. For example, Temurin reproducible comparing tests on linux platforms requires docker installed. Temurin reproducible comparing tests on mac-x64 requires tests running on mac-aarch64 as Temurin cross compiles for mac-x64. Those are special requirement per vendors.

Current test category doesn't provide the ability to set the platform requirements (or label requirements) based on the vendor or test itself. Maybe we could easily done this by adding one category? So vendor can set special platform requirements.

[smlambert]

I updated the description above to refer to the VENDOR_TEST_REPO, VENDOR_TEST_BRANCH, VENDOR_TEST_SHA, VENDOR_TEST_DIR parameters.

AQAvit documentation refers to [openjdk|system|functional|external] as test 'groups'

Additional benefits of creating a new group called "vendor":

• easier to distinguish between the required AQAvit targets that need to be run for AQAvit verification versus additional/optional tests that vendors may wish also run as part of their release process
• separate test jobs for vendor tests to allow for special handling and platform requirements (such as the reproducibility test example described above
• less concern around accidental sharing of private or proprietary test results if separate jobs (no need to include TAP files from vendor test jobs if intended to be kept private)

[smlambert]

After some more discussions, perhaps consider adding a new level versus a new group, given each group has varying dependencies, and vendor tests may be scattered across functional, perf, system, etc.

[sophia-guo]

Close it for now. Currently sepcial.system works.