Skip to content

Commit 666a6df

Browse files
authored
Merge branch 'dev' into feature/TPT-4209-linode-cli-add-integration-tests-for-nodebalancer-front-end-ip-in-vpc-mtc
2 parents 4cc928c + dc02f0c commit 666a6df

9 files changed

Lines changed: 227 additions & 18 deletions

File tree

.github/workflows/publish-wiki.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,4 +15,4 @@ jobs:
1515
runs-on: ubuntu-latest
1616
steps:
1717
- uses: actions/checkout@v7
18-
- uses: Andrew-Chen-Wang/github-wiki-action@6448478bd55f1f3f752c93af8ac03207eccc3213 # pin@v5.0.3
18+
- uses: Andrew-Chen-Wang/github-wiki-action@1bbb4280446f9630e8e21a18012cbacf3b0f992e # pin@v5.0.6

AGENTS.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Repository Guide
2+
3+
## Setup
4+
- Use `make requirements` for local setup. It installs `.[dev,obj]`; plain `.[dev]` misses `boto3`, which the object-storage plugin and tests use.
5+
- Use a virtualenv if possible; package metadata requires Python `>=3.9`, CI tests Python 3.9 through 3.13, and the Dockerfile builds on Python 3.13.
6+
- On a clean machine, export `LINODE_CLI_TOKEN` before running CLI commands that should not enter interactive `configure()`. Unit tests avoid config prompts with `LINODE_CLI_TEST_MODE=1`.
7+
- `make install` is the closest local equivalent to CI setup, but it is heavyweight: `check-prerequisites -> requirements -> build -> pip3 install --force dist/*.whl`.
8+
- The wiki setup page is mostly useful, but the wiki testing page has stale target names. Trust the `Makefile`: use `make test-unit` and `make test-int`, not `make testunit` or `make testint`.
9+
10+
## Generated Files
11+
- This CLI is spec-driven. Runtime commands load the baked pickle at `linodecli/data-3`; `MANIFEST.in` packages that file into distributions.
12+
- `make build`, `make install`, and `make lint` all run generation through `build`: `clean`, rewrite `linodecli/version.py` from `LINODE_CLI_VERSION` (default `0.0.0.dev`), regenerate root `data-3`, copy it to `linodecli/data-3`, and rebuild `dist/`.
13+
- Prefer `make bake SPEC=/path/to/openapi.json` or `make bake SPEC_VERSION=<tag>`; it resolves the spec via `./resolve_spec_url` when needed, passes `$(BAKE_FLAGS)` (`--debug` by default), and copies root `data-3` to `linodecli/data-3`.
14+
- If you rely on the default `SPEC_VERSION=latest`, set `GITHUB_TOKEN`; `resolve_spec_url` calls the GitHub releases API for `linode/linode-api-openapi` and can hit rate limits without it.
15+
- Manual bake: `python3 -m linodecli bake <spec> --skip-config` writes root `data-3` only; copy it to `linodecli/data-3` yourself or the package keeps the old pickle. `--skip-config` is a hidden sentinel checked in `linodecli/__init__.py` before argparse/config bootstrapping.
16+
- Never hand-edit `linodecli/data-3` or root `data-3`; change bake logic or the source spec/extensions and rebake.
17+
- `CLI._load_openapi_spec()` mutates parsed specs with `_normalize_content_parameters()` before `openapi3.OpenAPI(...)`; this converts OpenAPI Parameter `content` forms to top-level `schema` because the `openapi3` package does not support parameter `content` directly.
18+
19+
## Code Map
20+
- Importing top-level `linodecli` has side effects: `linodecli/__init__.py` constructs a global `CLI`, loads baked ops, and may load/configure user state immediately.
21+
- CLI entrypoints are `linodecli/__init__.py:main` and `linodecli/__main__.py`; console scripts `linode-cli`, `linode`, and `lin` all point to `linodecli:main`.
22+
- `linodecli/cli.py` handles spec loading/baking, baked-op loading, command lookup, custom aliases, and dispatch.
23+
- `linodecli/api_request.py` builds request URLs, request bodies, `X-Filter`, retries, version warnings, and error output.
24+
- `linodecli/output/output_handler.py` handles table, ASCII table, delimited, JSON, and Markdown output. `linodecli/overrides.py` contains command/action/output-mode-specific display overrides.
25+
- `linodecli/configuration/` owns config loading, interactive configuration, OAuth token flow, env token handling, and API URL overrides.
26+
- `linodecli/plugins/` contains hand-written commands outside the generated OpenAPI surface. If you add or change a plugin, read `linodecli/plugins/README.md` for the `call(args, context)` interface and third-party `PLUGIN_NAME` requirement.
27+
- If you touch `linodecli/baked/*.py`, read `linodecli/baked/AGENTS.md` first. Key constraint: baked model state must stay pickle-safe.
28+
29+
## Tests
30+
- `make test` only runs unit tests; it is an alias for `make test-unit`.
31+
- Use `make test-unit` for normal unit verification. It sets `LINODE_CLI_TEST_MODE=1` and `XDG_CONFIG_HOME=/tmp/linode/.config` so imports do not trigger interactive config.
32+
- Focused unit test: `LINODE_CLI_TEST_MODE=1 XDG_CONFIG_HOME=$(mktemp -d) pytest tests/unit/test_cli.py -k '<expr>'`.
33+
- Unit tests for bake/parsing behavior use minimal OpenAPI fixtures in `tests/fixtures/` and helper fixtures in `tests/unit/conftest.py`; add or update a fixture there when changing generated argument/response behavior.
34+
- Integration tests shell out to the installed `linode-cli` binary, not the source tree directly. Re-run `make install` after code changes before trusting integration results.
35+
- Integration and smoke tests hit the real Linode API and create/destroy real resources. Do not run them casually against a personal account.
36+
- `tests/integration/conftest.py` has a session-scoped autouse firewall fixture, so even focused integration runs can create a cloud firewall before the selected test body runs.
37+
- Focused integration run: `make test-int TEST_SUITE=domains TEST_CASE=test_create_a_domain TEST_ARGS='-v'`.
38+
- Integration and smoke tests require `LINODE_CLI_TOKEN`. Long-running cases are skipped unless `RUN_LONG_TESTS=True` exactly. Smoke tests are `make test-smoke`.
39+
40+
## Lint And Format
41+
- `make lint` is not a pure lint pass; it depends on `make build`, so it cleans, rebakes, rebuilds `dist/`, then runs `pylint`, `isort --check-only`, `autoflake --check`, `black --check`, and `twine check dist/*`.
42+
- Because lint rebakes, default `SPEC_VERSION=latest` needs GitHub API access (and often `GITHUB_TOKEN`). Without `SPEC=...` or a token, `make lint` can fail on rate limits during bake, not only on style.
43+
- For quick checks without generation side effects, run style tools directly, for example `black --check linodecli tests`, `isort --check-only linodecli tests`, or `autoflake --check linodecli tests`.
44+
- Formatting uses Black/isort with an 80-character line length. `make format` runs `black`, then `isort`, then `autoflake` and rewrites files in place.
45+
- Keep syntax compatible with Python 3.9 even if developing on a newer interpreter.
46+
47+
## Workflow
48+
- CI enforces PR titles matching `TPT-<number>: <description>` unless the PR is labeled `dependencies`, `hotfix`, `community-contribution`, or `ignore-for-release`.
49+
- `e2e_scripts/` is a git submodule used by CI/e2e workflows, not the core CLI package. Fresh clones leave it empty until `git submodule update --init`.
50+
- If you change behavior, commands, generated-file flow, test/lint setup, or architecture described here, update this `AGENTS.md` in the same change.

README.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,21 @@ Visit the [Wiki](../../wiki) for more information.
1212

1313
## Install
1414

15-
Install via PyPI:
15+
We recommend installing `linode-cli` with `pipx`, which installs each Python CLI tool into its own isolated environment and works on distributions where `pip install` fails because the system Python is marked as externally managed ([PEP 668](https://peps.python.org/pep-0668/)). If pipx isn't installed yet, follow the [pipx installation guide](https://pipx.pypa.io/latest/how-to/install-pipx.html).
16+
17+
To install:
18+
19+
```bash
20+
pipx install linode-cli
21+
```
22+
23+
To upgrade:
24+
1625
```bash
17-
pip3 install linode-cli
26+
pipx upgrade linode-cli
1827
```
1928

20-
Visit the [Wiki](../../wiki/Installation) for more information.
29+
The [Wiki](https://github.com/linode/linode-cli/wiki/Installation) covers other installation methods, including the Docker image, the GitHub Action, and building from source.
2130

2231
## Contributing
2332

linodecli/baked/AGENTS.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
# Baked Module Guide
2+
3+
## Core Constraint
4+
- `linodecli/baked/` converts OpenAPI objects into a pickled runtime command model. The output is `data-3`, copied to `linodecli/data-3` by `make bake` and loaded at CLI startup.
5+
- Constructors and bake helpers may read `openapi3` `Operation`, `MediaType`, `Schema`, and `Parameter` objects, but anything stored on `OpenAPIOperation`, `OpenAPIRequest`, `OpenAPIResponse`, `OpenAPIRequestArg`, `OpenAPIResponseAttr`, or `OpenAPIOperationParameter` must be pickle-safe for runtime loading.
6+
- Do not store raw `openapi3` objects on baked models. Extract strings, numbers, booleans, lists, dicts, `None`, or simple project classes during bake.
7+
- The `openapi3` library exposes spec extensions without the `x-` prefix. Code looks up keys like `linode-cli-use-schema`, `linode-cli-display`, and `linode-filterable`.
8+
- If you change behavior or architecture described in this file, update this `AGENTS.md` in the same change.
9+
10+
## Bake And Runtime Flow
11+
- `linodecli/cli.py:CLI.bake()` iterates spec paths and only `get/post/put/delete`, skips operations with `linode-cli-skip`, resolves command/action, constructs `OpenAPIOperation`, then pickles `self.ops`.
12+
- `CLI._load_openapi_spec()` normalizes OpenAPI Parameter `content` forms into top-level `schema` before calling `openapi3.OpenAPI(...)`; keep this when changing spec loading because `openapi3` does not support parameter `content` directly.
13+
- `OpenAPIOperation.__init__` coordinates bake-time extraction: response model, request/filter model, path parameters excluding `apiVersion`, URL components, docs URL, allowed defaults, action aliases, and CLI code samples.
14+
- Runtime command execution does not parse OpenAPI. `OpenAPIOperation.parse_args()` builds argparse from baked attrs, `linodecli/api_request.py` builds URL/filter/body, and `OpenAPIOperation.process_response_json()` applies output overrides before `response_model.fix_json()` and `OutputHandler.print_response()`.
15+
- `CLI.load_baked()` unpickles `linodecli/data-3`, pops metadata keys (`_base_url`, `_spec_version`, `_spec`) off the ops map, and stores them on the `CLI` instance (`base_url`, `spec_version`, `spec`). Do not assume those keys remain in the runtime command map.
16+
17+
## File Roles
18+
- `operation.py`: main baked operation object, argparse actions, URL/docs resolution, response dispatch. Special user input sentinels live here: `ExplicitNullValue`, `ExplicitEmptyListValue`, `ExplicitJsonValue`.
19+
- `request.py`: turns JSON request schemas into flat CLI args. Arrays of objects get both a parent JSON arg and child dot-path args.
20+
- `response.py`: turns response schemas into output attrs and normalizes API JSON via `fix_json()`.
21+
- `util.py`: composition/property aggregation plus dot-path escaping. Use `escape_arg_segment()` and `get_path_segments()` when schema property names can contain periods.
22+
- `parsing.py`: short help-text extraction and Markdown-to-Rich conversion for baked descriptions.
23+
24+
## OpenAPI Extensions Used Here
25+
- Path/operation routing: `linode-cli-command`, `linode-cli-action`, `linode-cli-skip`.
26+
- RequestBody extension: `linode-cli-allowed-defaults` is read from `operation.requestBody.extensions`, not from the JSON schema or media-type schema.
27+
- Request schema/media-type parsing: `linode-cli-format`, `linode-cli-use-schema`, `linode-cli-skip`.
28+
- Response/output parsing: `linode-cli-display`, `linode-cli-color`, `linode-filterable`, `linode-cli-rows`, `linode-cli-nested-list`, `linode-cli-subtables`, `linode-cli-use-schema`, `linode-cli-skip`.
29+
- Samples: operation extension `code-samples`; only entries whose `lang` lowercases to `cli` are stored.
30+
31+
## Request Parsing Gotchas
32+
- `_aggregate_schema_properties()` merges `oneOf`, `anyOf`, and `allOf`; a field is marked required only if every leaf schema that defines properties requires it. Schemas that only nest composition without their own `properties` do not increment the required-count denominator.
33+
- `OpenAPIRequest.attr_routes` stores per-`oneOf` option args keyed by schema `title`; oneOf entries without titles raise `ValueError`.
34+
- `OpenAPIOperation.arg_routes` returns `self.request.attr_routes` (a dict) when a request exists, but `[]` when there is no request. Treat the empty case as a type inconsistency (list vs dict); callers that always call `.items()` will break on the no-request path.
35+
- Request and response attributes marked `linode-cli-skip` are omitted from generated args/output.
36+
- `linode-cli-use-schema` on `application/json` media types swaps the schema before parsing; this is used for CLI-specific request or display shapes.
37+
- `linode-cli-format: json` stops deeper request parsing and accepts raw JSON. Deeply nested arrays are also treated as JSON.
38+
- For arrays of objects, `_parse_request_model()` adds a parent arg that accepts JSON plus child args for each object property. `ListArgumentAction` groups adjacent child values into list items before `_build_request_body()` expands dot paths.
39+
- Parent list args and child list args are mutually exclusive at runtime; `operation.py` validates conflicts such as `--interfaces` with `--interfaces.purpose`.
40+
- `null` only becomes an explicit JSON null for nullable args. Unspecified `None` values are dropped later by `_traverse_request_body()` in `api_request.py`.
41+
- `OptionalFromFileAction` loads file content only if the value resolves to an existing file; on Windows it also expands glob patterns before checking.
42+
- `PasswordPromptAction` reads an explicit value, then `LINODE_CLI_<DEST>`, then prompts interactively.
43+
44+
## Request Body And Filters
45+
- `api_request._build_request_body()` excludes path params, applies config defaults from `operation.allowed_defaults` when defaults are enabled, expands escaped dot-path keys with `get_path_segments()`, then serializes after `_traverse_request_body()`.
46+
- `--raw-body` is only valid for POST/PUT-style body actions and cannot be combined with generated action args.
47+
- GET filter args are generated only from paginated response attrs marked `linode-filterable`.
48+
- `--order-by` is restricted to filterable attrs and becomes `+order_by` in `X-Filter`; list filters serialize as `+and` entries.
49+
50+
## Response And Output Gotchas
51+
- Bake-time pagination detection is structural: top-level response schema must have exactly `pages`, `page`, `results`, and `data`; only `data.items` becomes display attrs (`is_paginated`).
52+
- `OpenAPIResponse.fix_json()` handles `linode-cli-rows`, then `linode-cli-nested-list`, then unwraps via `"pages" in json` (not `self.is_paginated`), then wraps non-lists. Runtime unwrap is looser than bake-time pagination; do not tighten it to `is_paginated` without checking odd payloads that only include `pages`.
53+
- `linode-cli-rows` paths are extracted from the raw API JSON and concatenated; list values extend the result, scalar/object values are appended.
54+
- `linode-cli-nested-list` flattens configured nested lists and adds `_split` with the final path segment.
55+
- `linode-cli-subtables` is stored on the response model, but table splitting is implemented in `linodecli/output/output_handler.py`.
56+
- `linode-cli-display` controls default columns and ordering; if no columns are selected or displayed, `OutputHandler` falls back to all attrs.
57+
- `linode-cli-color` rendering expects a `default_` color for values not present in the map.
58+
- Response attrs keep full dot-path names for lookup. `OutputHandler` deep-copies attrs before subtable scoping because it mutates attr names and nesting depth while printing.
59+
- JSON output intentionally ignores nested-list depth filtering so nested data can be displayed correctly.
60+
61+
## URL And Docs
62+
- `_get_api_url_components()` uses the operation server or root server, resolves the default API version from server path or the `apiVersion` path parameter, and inserts `/{apiVersion}` when the path lacks it.
63+
- Runtime URL overrides and pagination query params are applied in `api_request._build_request_url()`, not during bake. API version can come from `LINODE_CLI_API_VERSION`, config `api_version`, or the baked default.
64+
- Docs URLs prefer `operation.externalDocs.url`; legacy fallback derives a Linode docs anchor from the first tag and summary.
65+
66+
## Tests
67+
- Unit tests for this module use minimal OpenAPI fixtures under `tests/fixtures/` and fixtures in `tests/unit/conftest.py` that construct real `OpenAPIOperation` objects.
68+
- For bake/request/response changes, add or update a fixture plus focused tests in `tests/unit/test_operation.py`, `test_request.py`, `test_response.py`, `test_api_request.py`, or `test_output.py` as appropriate.
69+
- Focused run: `LINODE_CLI_TEST_MODE=1 XDG_CONFIG_HOME=$(mktemp -d) pytest tests/unit/test_request.py -k '<expr>'`.

linodecli/configuration/auth.py

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -214,11 +214,12 @@ def _get_token_terminal(base_url: str) -> Tuple[str, str]:
214214
:returns: A tuple containing the user's username and token.
215215
:rtype: Tuple[str, str]
216216
"""
217-
print(f"""
218-
First, we need a Personal Access Token. To get one, please visit
219-
{TOKEN_GENERATION_URL} and click
220-
"Create a Personal Access Token". The CLI needs access to everything
221-
on your account to work correctly.""")
217+
print(
218+
"First, we need a Personal Access Token. To get one, please visit\n"
219+
f"{TOKEN_GENERATION_URL} and click\n"
220+
'"Create a Personal Access Token". The CLI needs access to everything\n'
221+
"on your account to work correctly."
222+
)
222223

223224
while True:
224225
token = input("Personal Access Token: ")

linodecli/configuration/helpers.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,9 +97,10 @@ def _check_browsers() -> bool:
9797

9898
# pylint: disable-next=protected-access
9999
if not KNOWN_GOOD_BROWSERS.intersection(webbrowser._tryorder):
100-
print("""
101-
This tool defaults to web-based authentication,
102-
however no known-working browsers were found.""")
100+
print(
101+
"This tool defaults to web-based authentication,\n"
102+
"however no known-working browsers were found."
103+
)
103104
while True:
104105
r = input("Try it anyway? [y/N]: ")
105106
if r.lower() in "yn ":

linodecli/plugins/get-kubeconfig.py

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99
import argparse
1010
import base64
11+
import os
1112
import sys
1213
from pathlib import Path
1314

@@ -19,6 +20,11 @@
1920

2021
PLUGIN_BASE = "linode-cli get-kubeconfig"
2122

23+
# Kubeconfigs contain credentials, so they should only be
24+
# accessible by the user that created them.
25+
KUBECONFIG_FILE_MODE = 0o600
26+
KUBECONFIG_DIR_MODE = 0o700
27+
2228

2329
def call(args, context):
2430
"""
@@ -147,8 +153,27 @@ def _load_config(filepath):
147153

148154
# Dumps data to a yaml file
149155
def _dump_config(filepath, data):
150-
Path.mkdir(filepath.parent, exist_ok=True)
151-
with open(filepath, "w", encoding="utf-8") as file_descriptor:
156+
filepath.parent.mkdir(mode=KUBECONFIG_DIR_MODE, parents=True, exist_ok=True)
157+
158+
# Create the file with restrictive permissions rather than chmod-ing it
159+
# afterwards, so its contents are never briefly readable by other users.
160+
# NOTE: The mode is only applied when the file is created.
161+
def opener(path, flags):
162+
return os.open(path, flags, mode=KUBECONFIG_FILE_MODE)
163+
164+
with open(
165+
filepath, "w", encoding="utf-8", opener=opener
166+
) as file_descriptor:
167+
# Tighten the permissions of pre-existing files that are readable or
168+
# writable by users other than the owner.
169+
# NOTE: os.fchmod is not available on Windows, where POSIX file modes
170+
# are not meaningful anyway.
171+
if (
172+
hasattr(os, "fchmod")
173+
and os.fstat(file_descriptor.fileno()).st_mode & 0o077
174+
):
175+
os.fchmod(file_descriptor.fileno(), KUBECONFIG_FILE_MODE)
176+
152177
yaml.dump(data, file_descriptor)
153178

154179

tests/integration/linodes/test_linode_interfaces.py

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -146,10 +146,6 @@ def test_interface_settings_update(
146146
interface_id,
147147
"--default_route.ipv6_interface_id",
148148
interface_id,
149-
"--default_route.ipv4_eligible_interface_ids",
150-
interface_id,
151-
"--default_route.ipv6_eligible_interface_ids",
152-
interface_id,
153149
"--json",
154150
]
155151
)

0 commit comments

Comments
 (0)