Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Include .tfbackend files in hooks #822

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

thiagowfx
Copy link

@thiagowfx thiagowfx commented Feb 20, 2025

Put an x into the box if that apply:

  • This PR introduces breaking change.
  • This PR fixes a bug.
  • This PR adds new functionality.
  • This PR enhances existing functionality.

Description of your changes

As per the official docs https://developer.hashicorp.com/terraform/language/backend#file:

A backend configuration file has the contents of the backend block as top-level attributes, without the need to wrap it in another terraform or backend block:

address = "demo.consul.io"
path    = "example_app/terraform_state"
scheme  = "https"

*.backendname.tfbackend (e.g. config.consul.tfbackend) is the recommended naming pattern. Terraform will not prevent you from using other names but following this convention will help your editor understand the content and likely provide better editing experience as a result.

How can we test changes

Add foo.tfbackend (a simple HCL file, without blocks, similar to .tfvars), run e.g. pre-commit run --all-files terraform_fmt.

As per the official docs https://developer.hashicorp.com/terraform/language/backend#file:

> *.backendname.tfbackend (e.g. config.consul.tfbackend) is the recommended naming pattern. Terraform will not prevent you from using other names but following this convention will help your editor understand the content and likely provide better editing experience as a result.
Copy link

coderabbitai bot commented Feb 20, 2025

Warning

Rate limit exceeded

@thiagowfx has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 1 minutes and 58 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 0dad069 and 837b9ae.

📒 Files selected for processing (1)
  • .pre-commit-hooks.yaml (6 hunks)
📝 Walkthrough

Summary by CodeRabbit

  • Chores
    • Refined file matching rules to now include the .tfbackend extension, ensuring that configuration validation, formatting, and security checks cover all relevant Terraform files for a smoother development experience.

Walkthrough

This change updates the .pre-commit-hooks.yaml configuration file by modifying the file matching patterns for several Terraform-related hooks. The patterns for infracost_breakdown, terraform_fmt, terraform_validate, terraform_tflint, terraform_tfsec, and terraform_trivy have been adjusted to include the .tfbackend extension alongside the existing .tf and .tfvars extensions. This allows these hooks to properly match files with the new extension.

Changes

File Change Summary
.pre-commit-hooks.yaml - Updated Patterns: infracost_breakdown, terraform_fmt, terraform_validate, terraform_tflint, terraform_tfsec, and terraform_trivy now match files with the .tfbackend extension in addition to .tf and .tfvars.

Suggested reviewers

  • yermulnik

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@thiagowfx thiagowfx changed the title terraform_fmt: include .tfbackend files feat(terraform_fmt): include .tfbackend files Feb 20, 2025
@thiagowfx thiagowfx changed the title feat(terraform_fmt): include .tfbackend files feat(terraform_fmt): Include .tfbackend files Feb 20, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
.pre-commit-hooks.yaml (1)

1-9: Removal of the Infracost Breakdown Hook

It appears that the infracost_breakdown hook has been removed. Please confirm that this hook is no longer required in your workflow. If it was used elsewhere or its removal might affect downstream processes, provide migration guidance or update documentation accordingly.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8a38366 and 1902231.

📒 Files selected for processing (1)
  • .pre-commit-hooks.yaml (1 hunks)
🔇 Additional comments (18)
.pre-commit-hooks.yaml (18)

10-18: Update to Terraform fmt File Pattern

The terraform_fmt hook’s file pattern is now updated to include the .tfbackend file extension:

files: (\.tf|\.tfbackend|\.tfvars)$

This correctly matches files ending with .tf, .tfbackend, or .tfvars. Please double-check that this regex meets the expected naming convention and that there isn’t any unintended overlap with similar file extensions.


19-29: Removal of Terraform Docs Hook

The terraform_docs hook has been removed. Ensure that removing documentation generation via terraform-docs is intentional. If users relied on this functionality, consider providing a notice or migration path.


30-40: Removal of the Terraform Docs (Without Aggregate Defaults) Hook

Similarly, the terraform_docs_without_aggregate_type_defaults hook has been removed. This helps in reducing redundancy if it was duplicative of terraform_docs. Please verify that this removal does not impact users who might have depended on the slight differences in behavior.


41-49: Removal of the Terraform Docs Replace Hook

The hook terraform_docs_replace (which overwrites README.md with terraform-docs output) has been removed. Confirm that this functionality is no longer needed or has been handled elsewhere.


50-58: Removal of the Terraform Validate Hook

The terraform_validate hook has been removed. Verify that any necessary validation for Terraform configuration files is now performed by another tool or hook to ensure configuration integrity.


59-67: Removal of the Terraform Providers Lock Hook

The hook for locking Terraform provider versions (terraform_providers_lock) is removed. Make sure that provider version management is handled appropriately elsewhere or that its removal is a deliberate decision.


68-76: Removal of the Terraform TFLint Hook

The removal of the terraform_tflint hook implies that static analysis with TFLint will no longer be available via this pre-commit configuration. Confirm that this deprecation aligns with your newer static analysis choices.


77-85: Removal of the Terragrunt fmt Hook

The terragrunt_fmt hook has been removed. Please ensure that any format standardization for Terragrunt files is either no longer necessary or is handled by a different mechanism.


86-93: Removal of the Terragrunt Validate Hook

Similarly, the terragrunt_validate hook is removed. If Terragrunt configuration validation is still required, confirm it is provided by another hook or tool.


94-101: Removal of the Terragrunt Validate Inputs Hook

The removal of the terragrunt_validate_inputs hook suggests that validating inputs for Terragrunt is no longer performed in this configuration. Please verify that this is an intended change.


102-110: Removal of the Terragrunt Providers Lock Hook

This segment shows the removal of the terragrunt_providers_lock hook. Ensure that dependency lock management for Terragrunt is either deprecated or superseded by other processes.


111-119: Deprecation of the Terraform tfsec Hook

The terraform_tfsec hook is being deprecated (and removed in this diff) in favor of the new terraform_trivy hook. It’s good practice to remove deprecated hooks once users have been given sufficient notice. Verify that any documentation or upgrade guides have been updated to reflect this change.


120-128: Addition of the Terraform Trivy Hook

A new terraform_trivy hook has been introduced:

id: terraform_trivy
name: Terraform validate with trivy
...
files: \.tf(vars)?$

This looks like an appropriate replacement for tfsec. Please ensure that the regex and configuration match your requirements and that the environment has Trivy installed.


129-139: Deprecation and Removal of the Checkov Hook

The original checkov hook has been removed, which is in line with the new terraform_checkov hook introduction. Verify that users are aware of this migration.


140-149: Addition of the Terraform Checkov Hook

The new terraform_checkov hook has been added to run Checkov on Terraform templates:

id: terraform_checkov
name: Checkov
entry: hooks/terraform_checkov.sh
...
files: \.tf$

This update should provide equivalent functionality as the deprecated hook. Ensure that its integration has been tested and documented.


150-160: Removal of the Terraform Wrapper Module for Each Hook

The hook terraform_wrapper_module_for_each has been removed. Confirm that this functionality is either obsolete or has been integrated into another process.


161-169: Removal of the Terrascan Hook

The terrascan hook has been removed. Please verify that this static analysis tool is no longer part of your workflow or has been replaced.


170-179: Removal of the tfupdate Hook

Finally, the tfupdate hook has also been removed. Double-check that automatic updates for Terraform templates are handled appropriately if needed.

@MaxymVlasov
Copy link
Collaborator

Hi, so it is a new possibility in terraform to define backend. Interesting.

There is a chance that the same changes need to be included in these hooks:

  • terraform_docs
  • terraform_validate
  • terraform_tflint
  • terraform_tfsec
  • terraform_trivy
  • terraform_checkov
  • terrascan

But need to test it. Would much appreciate it if you'll handle it. If not - I'll do it later by myself.
Need to check if addition of that file somehow affect behavior of these hooks and if affects - add .tfbackend to it too

@MaxymVlasov MaxymVlasov added feature New feature or request estimate/2h Need 2 hours to be done labels Feb 20, 2025
@thiagowfx
Copy link
Author

My understanding is that .tfbackend files are supposed to have simple key-value pairs:

a = "1"
b = "2"

...etc.

Formatting them makes sense (terraform_fmt likes to align the equal signs vertically).

I am not completely sure whether the other hooks make sense, would need to look deeper.

I would be happy to send more PRs later on if it turns out they make sense.
For now, I changed only fmt because I am confident it's applicable.

What do you think?

yermulnik
yermulnik previously approved these changes Feb 20, 2025
Copy link
Collaborator

@yermulnik yermulnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • TIL about .tfbackend files 😲
  • LGTM

@yermulnik
Copy link
Collaborator

There is a chance that the same changes need to be included in these hooks:

Do the other hooks take TF backend configuration into account at all? 🤔 If yes, then, yep, we'd better off adding it to those that are aware of this type of files 👍🏻

Co-authored-by: George L. Yermulnik <[email protected]>
@thiagowfx
Copy link
Author

I was thinking about this. .tfbackend isn't too different from .tfvars (https://spacelift.io/blog/terraform-tfvars).

Perhaps it makes sense to add it to all hooks that have .tfvars.

@thiagowfx thiagowfx changed the title feat(terraform_fmt): Include .tfbackend files feat: Include .tfbackend files in hooks Feb 21, 2025
@thiagowfx
Copy link
Author

Updated the file + PR title + PR description.

@yermulnik
Copy link
Collaborator

I was thinking about this. .tfbackend isn't too different from .tfvars (spacelift.io/blog/terraform-tfvars).

Perhaps it makes sense to add it to all hooks that have .tfvars.

Underlying tools infer values for variables from tfvars, whereas I can't say for sure how they will behave encountering "variables" in tfbackend files as these "variables" ain't declared by means of variable block definitions. This is my concern and hesitation.

As @MaxymVlasov already inquired: would you be able to add change to .pre-commit-hooks.yaml locally and test to see whether none tool get confused with the vars in tfbackend file(s)? Thank you in advance 👍🏻

@yermulnik yermulnik dismissed their stale review February 21, 2025 15:18

Take approval back as it was for TF fmt only and we need to be sure that other hooks don't get confused with the tfbackend vars.

@thiagowfx
Copy link
Author

thiagowfx commented Feb 21, 2025

I am only currently using terraform_fmt and terraform_tflint in our projects. I can test these two and report back (well, terraform_fmt is already tested1, so there's just terraform_tflint remaining).

For the other 4 hooks, are there existing samples / tests somewhere I could use for testing?

Footnotes

  1. I made a mistake here, my bad.

@thiagowfx
Copy link
Author

thiagowfx commented Feb 21, 2025

Oh wait, I take it back for terraform_fmt:

$ terraform fmt a.tfbackend
│ Error: Only .tf, .tfvars, and .tftest.hcl files can be processed with terraform fmt

It should be possible to do so.
I'll file an upstream bug.

Setting this back to a Draft for now.

@thiagowfx thiagowfx marked this pull request as draft February 21, 2025 15:30
@thiagowfx
Copy link
Author

thiagowfx commented Feb 21, 2025

Upstream FR / bug: hashicorp/terraform#36564

@yermulnik
Copy link
Collaborator

I am only currently using terraform_fmt and terraform_tflint in our projects. I can test these two and report back (well, terraform_fmt is already tested1, so there's just terraform_tflint remaining).

For the other 4 hooks, are there existing samples / tests somewhere I could use for testing?

It should be as simple as adding each of those hooks to your .pre-commit-config.yaml file, like this:

repos:
- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: 837b9ae2b24688c9cc3727fcf717c8494c421ee6 # commit that you want to test against
  hooks:
    - id: terraform_fmt
    - id: terraform_validate
    - id: terraform_tflint
    - id: terraform_docs
    - id: terraform_tfsec
    - id: terraform_trivy
    #- id: infracost_breakdown # I don't have a API key :shrugging:

So for me none fails (and even terraform_fmt with commit id 0dad06983ea30273c52be9c3c443ad5970de0a90) as they all seem to ignore that type of file and as they operate on a dir rather than on a specific files, hence none throws error.

# running with `0dad06983ea30273c52be9c3c443ad5970de0a90`
> pre-commit run -a
[INFO] Initializing environment for https://github.com/antonbabenko/pre-commit-terraform.
Terraform fmt...........................................................................Passed
Terraform validate......................................................................Passed
Terraform validate with tflint..........................................................Passed
Terraform docs......................................................(no files to check)Skipped
Terraform validate with tfsec (deprecated, use "terraform_trivy").......................Passed
Terraform validate with trivy...........................................................Passed

I'm not sure whether we can enforce each hook to run over specific file(s) instead. @MaxymVlasov can assist perceiving this. Else we should probably be safe with this PR (including TF fmt hook).
I'm deferring the decision to @MaxymVlasov as a main dev here.

@MaxymVlasov
Copy link
Collaborator

About hook examples and how to test it.
That 100% out of scope of this PR, but thats all about tests and their automation. I don't think that unit tests (hello python) or fuzzing tests (hello OSSF, and maybe, python too) will help with that much. Probably we will need add some e2e tests by create separate repo with bunch of code samples and bunch isolated hooks runs.

Good news that we can do it right now (and I'll probably do it anyway, glad you asked, as I tired to test changes somehow locally with hope that I cover everything - that's far from ideal)
Bad news - could be hard to automate it right now, but at least it will be a good starting point.

I can't say ETA when I'll do that, but it will not take more than 1 workday, just need to find that time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
estimate/2h Need 2 hours to be done feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants