set-pipeline leaks credentials in diff

[lenianiva]

Summary

When I have a pipeline like this where the resource is provided via a file:

resources:
- name: src
  type: git
  icon: git
  source:
    uri: ((git.root))/user/repo.git
    private_key: ((git.private_key))

jobs:
- name: set-self
  public: true
  build_log_retention:
    builds: 50
  plan:
  - get: src
    trigger: true
  - set_pipeline: self
    file: src/pipeline.yaml

where the credentials are provided by this file:

git:
  root: git.example.com
  private_key: |
    ...

After the set_pipeline step, the credentials show up in plaintext in the Concourse UI.

Steps to Reproduce

1. Install Concourse 8.0.2 (i.e. redaction is by default)
2. Write a pipeline like the above with a random git repo
3. Execute

fly -t main set-pipeline \
  -p random \
  -c pipeline.yaml \
  -l credentials.yaml \
  -y trigger=true

Expected Results

No private keys should show up in the set_pipeline step log.

In the example shown here, it seems like using -l to feed in secrets is permitted.

Actual Results

The set_pipeline step contains credentials in plaintext

resources:
  resource src has changed:
  ��- icon: git
  name: src
  source:
  ��-   private_key: |
  ��-     -----BEGIN OPENSSH PRIVATE KEY-----
  ��-     .....
  ��-     -----END OPENSSH PRIVATE KEY-----
  ��-   uri: git.example.com/user/repo.git
  ��+   private_key: ((git.private_key))
  ��+   uri: ((git.root)/user/repo.git
  type: git

Web Node(s) configuration

No response

Worker(s) configuration

No response

Concourse Version

8.0.2

Browser (if applicable)

No response

Did this use to work?

No response

[taylorsilva]

Not a bug. The problem you're running into is that you're expecting vars to be tracked as secrets by Concourse. Vars are not marked and tracked as secrets though.

There's also another problem here which is after you do fly set-pipeline, the vars are resolved and no longer present in the pipeline. You see the result of this in the diff output from set_pipeline where it tries to replace your resolved var (your private key) with the var names ((git)), which then breaks your pipeline.

The only solution here is to store your secrets in a secrets/credential manager: https://concourse-ci.org/docs/operation/creds/
and have Concourse fetch them during runtime. Vars fetched in this manner are treated as secrets and are redacted.

---

To hopefully help make things clear, here's a breakdown of what's happening.

When you run you fly set-pipeline command with the -l/--load-vars-from flag, all vars in the pipeline that can be resolved from the file are resolved and that is what gets sent to Concourse. Any unresolved vars are left as-is and Concourse would try resolving them during runtime by reaching out to whatever secrets manager it's connected to. Vars fetched during runtime are treated as secrets and are redacted.

So you think Concourse is getting this:

resources:
- name: src
  type: git
  icon: git
  source:
    uri: ((git.root))/user/repo.git
    private_key: ((git.private_key))

jobs:
- name: set-self
  public: true
  build_log_retention:
    builds: 50
  plan:
  - get: src
    trigger: true
  - set_pipeline: self
    file: src/pipeline.yaml

and possibly a copy of the vars file. Concourse does not store a copy of the vars file though. What Concourse actually gets and stores in its db is this:

resources:
- name: src
  type: git
  icon: git
  source:
    uri: git.example.com/user/repo.git
    private_key: |
      -----BEGIN OPENSSH PRIVATE KEY-----
      ...
      -----END OPENSSH PRIVATE KEY-----

jobs:
- name: set-self
  public: true
  build_log_retention:
    builds: 50
  plan:
  - get: src
    trigger: true
  - set_pipeline: self
    file: src/pipeline.yaml

Then in your set_pipeline step, you're only providing the pipeline file, not the file with the vars in it, which results in the diff where your secrets leak, but Concourse doesn't know they're secrets.

Sorry that you're landing in this confusing workflow. I totally see how your assumptions lead you to think it worked differently. I have been having thoughts of maybe including a built-in secrets store for Concourse to simplify smaller deployments like what you're probably running. I don't think that'll happen anytime soon though due to the extra maintenance burden that would bring. I'm trying to be very strict about what new features the project brings in.

[lenianiva]

I see. I'll setup a secret store to handle secrets instead of using variables.