Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document test run label support. #69

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Document test run label support. #69

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions content/docs/cli.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,12 @@ Start a KIND cluster for the tests (cannot be used with --start-control-plane).
If set, the specific test case to run.
:::

::: flag --test-run-labels (string)
Optional label set to associate with this test run.
This label set can then be matched against by the `testRunSelector` in `TestFile` objects to optionally exclude selected files.
The syntax is comma-separated list of key=value assignments.
:::

::: flag -v or -vv (int)
Logging verbosity level. 0=normal, 1=verbose, 2=detailed, 3 or more =trace.
:::
Expand Down
25 changes: 25 additions & 0 deletions content/docs/testing/reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,31 @@ timeout | int | Number of seconds that the test is allowed to run for | 30
collectors | list of [collectors](#collectors) | The collectors to be invoked to gather information upon step failure | N/A
commands | list of [commands](#commands) | Commands to run prior to the beginning of the test step. | N/A

## TestFile

A `TestFile` object can be used to provide configuration concerning a single YAML test file that contains it.

```yaml
apiVersion: kuttl.dev/v1beta1
kind: TestFile
testRunSelector:
matchLabels:
flavor: vanilla
```

Supported settings:

| Field | Type | Description | Default |
|-----------------|----------------|---------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
| testRunSelector | label selector | If this selector does not match [labels of this test run](#test-run-labels-and-selectors), the containing file will be ignored. | Empty label selector (matches all possible test label sets). |


### Test Run Labels and Selectors

An invocation of `kuttl test` may specify a label set associated with a test run using a command line flag.
One can then use a `TestFile` object with `testRunSelector` to decide whether a given test YAML file should be included
in a test run or not.

## Collectors

The `Collectors` object is used by the `TestAssert` object as a way to collect certain information about the outcome of an `assert` or `errors` step should it fail. A collector is only invoked in cases where a failure occurs and not if the step succeeds. Collection can occur from Pod logs, Namespace events, or the output of a custom command.
Expand Down