Skip to content

Conversation

@Saartank
Copy link
Contributor

Pull Request for Issue #1427: Configuration CLI Tool for Pelican

This PR resolves #1427 by introducing a command-line interface tool to retrieve, query, and get documentation for the configuration parameters in the Pelican system.

Commands and Descriptions

Command Description Example Usage
config View and search for configuration parameters in the Pelican system. pelican config
config dump [flags] The 'dump' command outputs all current configuration parameters and their values to the console. This includes default values that have not been explicitly set. pelican config dump -o json
config get [arguments] The 'get' command retrieves and displays configuration parameters that contain any of the provided argument patterns in their name or value. The search space can be narrowed or expanded using available flags. The matching is case-insensitive. If no arguments are provided, all configuration parameters are retrieved. pelican config get log monitor -m origin -m cache --include-deprecated
config describe [param] The 'describe' command prints detailed documentation for a specified configuration parameter, including its type, default value, description, related components, and whether it is deprecated or hidden. pelican config describe server.webPort
config summary [flags] The 'summary' command outputs configuration parameters whose values differ from their default settings. pelican config summary -o json

Broad Overview of Changes Made in the Code

  1. Embedded docs/parameters.yaml in the Binary
    This was necessary to enable the describe command to output the documentation for specified configuration parameters directly from the binary.

  2. Separated Configuration Initialization Code
    Refactored code in config/config.go to decouple configuration setup from server/client initialization logic within functions like InitServer and InitClient. This change isolates configuration initialization from side effects that occur during server or client startup, so that config values can be printed independently of program execution. This separate initialization code is now invoked when printing configuration, allowing the user to see the configuration values that would be applied if a server or client program were started.

  3. Refactored Configuration-Related Functions to Use a Viper Config Instance
    Modified multiple configuration-related functions to work with a passed viper configuration instance instead of operating directly on the global viper instance. This was essential for calculating default values by passing a new viper instance through these functions.

@Saartank Saartank added enhancement New feature or request client Issue affecting the OSDF client cache Issue relating to the cache component origin Issue relating to the origin component director Issue relating to the director component registry Issue relating to the registry component labels Oct 28, 2024
@Saartank Saartank added this to the v7.12.0 milestone Oct 28, 2024
@Saartank Saartank force-pushed the config-printer-cli-utility branch from a1c7ce4 to 5a080c6 Compare November 1, 2024 15:24
Copy link
Member

@jhiemstrawisc jhiemstrawisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want to say this was a really solid implementation of this tool for your first major PR -- great job! There are lots of comments and change requests, but that's inevitable when the code churn grows this large and when you haven't spent hours and hours in the code base.

A few top-level comments for you to consider (and you'll see some of these repeated throughout the review):

  1. We almost never want to call fmt.Print directly -- unless these are the final result of a top-level CLI command. Most of these should be switched to use our logging package.
  2. In some places you removed a .GetName() call and replaced it with a direct viper.Set("blah") -- I want to see this happening in the opposite direction, because grabbing a name from our param package turns a runtime error into a compilation error, which means we'll catch it before the user does. I don't think there's need to go back and change all of the viper sets to use this, but please look for and revert any cases where you made this switch explicitly.
  3. I'm not in love with the fact that config dump prints values that are unset and empty. For example, config dump generates this snippet for me right now:
Plugin:
    Token: ""

There are some defaults that aren't their value type's assumed "null" value, but in cases where the variable is unset and has no value, we should omit the line entirely. Is that doable?
4. The config sum/summary command generates a nil dereference for me. I included the full debug output for you in my inline comment, but when you fix please consider what kind of unit test would have caught this bug had the test already existed.

If you disagree with any of my comments or requests, always feel free to push back and disagree, but be prepared to back up your opinion -- you are the expert with respect to these code changes, so I may be making incorrect assumptions.

@Saartank Saartank marked this pull request as draft November 5, 2024 22:39
@Saartank Saartank marked this pull request as ready for review November 11, 2024 14:21
Copy link
Member

@jhiemstrawisc jhiemstrawisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting very close, but there are still two things I'd like you to clean up:

  1. In cmd/config_printer/config_printer_test.go, the call to InitConfigDir() in your setupMockConfig() helper function still sets /etc/pelican as the config directory when I run the tests as root. Even though we don't actually read the config there as a part of the test, I'd like you to do a viper.Set("ConfigDir", t.TempDir()) in setupMockConfig to make sure any future additions to the test are less likely to leak into my personal config.
  2. There appears to be an issue with config summary where configurable lists that are empty by default are still being printed:
$ pelican config summary
... A bunch of stuff omitted for brevity ...

Origin:
    ExportVolumes: [] --> this shouldn't be printed
Server:
    Modules: [] --> this shouldn't be printed

One note about these two empty slices -- in parameters.yaml, both have type stringSlice, but Origin.ExportVolumes has a default of none and Server.Modules has a default of []. Whichever of these defaults we consider to be "correct," we should be consistent. If you think cleaning that up is outside the scope of this PR, please open an issue so we can keep track of it.

@Saartank
Copy link
Contributor Author

I have created an issue for the inconsistency in empty stringSlice representation in documentation.

Copy link
Member

@jhiemstrawisc jhiemstrawisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cache Issue relating to the cache component client Issue affecting the OSDF client director Issue relating to the director component enhancement New feature or request origin Issue relating to the origin component registry Issue relating to the registry component

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"condor_config_val" equivalent CLI tool for Pelican

2 participants