fix(enrichment_tables): Fail when unit testing a config with unused enrichment table

[elobommart]

Summary

When running unit tests for Vector's configuration, enrichment tables are not sanitized the same way transforms are.

This incurs a bug when creating a configuration with an enrichment table that has a transform as an input, and a test that does not use said transform, because said transform is pruned from the transforms list, and Vector then tries to create the enrichment_table but cannot find its input. Moreover, there is no reason for the transforms to be sanitized and not the enrichment tables.

The following configuration currently fails when building tests on master, the reported error being that the input 'cache_builder' does not exist in the configuration. With this fix, this kind of tests build correctly.

Vector configuration

sources:
  src_1:
    type: socket
    address: 0.0.0.0:8001
    mode: udp

enrichment_tables:
  table_1:
    type: memory
    ttl: 600
    inputs:
      - cache_builder

transforms:
  transform_1:
    type: remap
    inputs:
      - src_1
    source: |
      .message = "test"

  cache_builder:
    type: remap
    inputs:
      - src_1
    source: |
      . = set!(value: {}, path: ["k1"], data: "d1")

sinks:
  sink1:
    type: console
    inputs:
      - transform_1
    encoding:
      codec: json
      json:
        pretty: true

tests:
  - name: "test 1"
    inputs:
      - insert_at: transform_1
        type: log
        log_fields:
          message: "test message"
    outputs:
      - extract_from: transform_1
        conditions:
          - type: vrl
            source: |-
              assert_eq!(.message, "test")

How did you test this PR?

via unit tests, running locally

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

[pront]

Hi @elobommart, please sign the CLA and we will take a look.

[elobommart]

I can't sign the CLA. The CLA management app shows the following error :

There is no CLA to sign for vectordotdev/vector
(could not find linked item for owner vectordotdev and repo vector)

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[thomasqueirozb]

Hi @elobommart sorry. We migrated the CLA check to a new workflow. I updated the branch and instructions should be posted here shortly

[elobommart]

Thanks @thomasqueirozb, if the new process is just the comment on the agreement, I think it's ok now ! Should the agreement CI task re-run to validate it ? Can I force this ?

[thomasqueirozb]

@elobommart After reading and reviewing the CLA you just need to post a comment stating "I have read the CLA Document and I hereby sign the CLA". If it doesn't automatically refresh then also comment "recheck" - that's all!

[elobommart]

I have read the CLA Document and I hereby sign the CLA

[elobommart]

recheck

[elobommart]

@thomasqueirozb so I did both, but to no avail :(

[thomasqueirozb]

The issue is that your original commit is not associated with your GitHub account. You should add that email to you GitHub account. You can view the email associated with the commit by running git show 09ebed422ede23b50c7470a39454f2f978c9095f (I don't want to put it out here for anyone/bots to see).

Alternatively I can try to take ownership of that commit or change the commit's owner to your other email and force push to this branch

[elobommart]

recheck

[elobommart]

The issue is that your original commit is not associated with your GitHub account. You should add that email to you GitHub account. You can view the email associated with the commit by running git show 09ebed422ede23b50c7470a39454f2f978c9095f (I don't want to put it out here for anyone/bots to see).

Alternatively I can try to take ownership of that commit or change the commit's owner to your other email and force push to this branch

right, I did not realise that it was still my "old" email, sorry for the noise ! we are good to go now :-)