You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This reusable workflow scans agent skills before merge. It uses[SkillSpector](https://github.com/NVIDIA/SkillSpector) static analysis. It does not use an LLM.
3
+
This reusable workflow scans agent skills before merge. It runs[SkillSpector](https://github.com/NVIDIA/SkillSpector) static analysis with `--no-llm`.
4
4
5
-
The default policy reports only new findings with `HIGH` or `CRITICAL` severity and confidence of at least `0.8`. This policy keeps existing findings and lower-confidence results out of pull request comments.
5
+
For pull requests, the default policy reports only new findings that meet both conditions:
6
+
7
+
- The severity is `HIGH` or `CRITICAL`.
8
+
- The confidence is at least `0.8`.
9
+
10
+
The workflow comments on these findings and fails the check by default. Static findings are not confirmed vulnerabilities. Review each finding before merge.
The examples pin the workflow to a full commit SHA. Update this SHA only after you review a newer workflow version.
30
-
31
-
Run the workflow on all pull requests. A change to a script, reference, or dependency file can change a skill's behavior.
32
-
33
-
Run the workflow once. Then, add its status check as a required check in the repository ruleset or branch protection rule. This setting prevents a merge before the scan passes.
34
-
35
-
The calling job must grant `pull-requests: write`. GitHub checks the permission for the comment job before it evaluates the job condition. Therefore, the caller must grant this permission when `comment-on-pr` is `false` too. The scan job does not receive this permission.
36
-
37
-
## Scan scope
38
-
39
-
The workflow finds `SKILL.md` files in hidden and nested skill directories. It scans each nested skill as a separate skill. It removes duplicate results that come from overlapping scans.
34
+
Pin the workflow to a full commit SHA. Review a newer workflow version before you update the SHA.
40
35
41
-
The scan checks the manifest, scripts, references, and dependency files in each skill directory. Skill discovery skips version control, virtual environment, dependency, and cache directories.
36
+
Run the workflow once. Then, add its status check to the repository ruleset or branch protection rule. Keep `merge_group` if the repository uses a merge queue.
42
37
43
-
The workflow fails if SkillSpector cannot inspect required content. For example, it fails for these conditions:
38
+
The caller must grant `contents: read` and `pull-requests: write`. A called workflow cannot increase the permissions that the caller grants.
44
39
45
-
- A symlinked directory in the scan root or a symlinked file in a skill.
46
-
- An unresolved Git LFS pointer in a skill.
47
-
- A Git submodule in the scan root.
48
-
- An unsupported partial file inspection or inspection exception.
49
-
- Any hidden file in a skill or an unexpected excluded directory.
50
-
- Other content that SkillSpector does not inspect.
40
+
The caller must grant both permissions when `comment-on-pr` is `false` too. GitHub checks job permissions before it evaluates step conditions.
51
41
52
-
The workflow permits the `.git` directory because Git creates it during checkout. It also permits a `__pycache__` directory that contains only `.pyc` or `.pyo` files. SkillSpector reports this bytecode with its high-confidence `SC8` rule. The workflow rejects all other scope exclusions.
42
+
## Finding policy
53
43
54
-
SkillSpector does not fully inspect binary images. For pull requests, the workflow compares each partially inspected image with the base tree. It permits the image only if its path and SHA-256 digest match. This policy applies to PNG, JPEG, GIF, and WebP images. The workflow fails for a new, changed, or renamed image. It also fails for a partially inspected image when no base tree is available.
44
+
The workflow creates a temporary SkillSpector baseline when a skill has the same base-tree path. It passes this trusted baseline to the head scan.
55
45
56
-
The reusable workflow rejects every Git submodule in the head and base repositories. This rule also applies to submodules outside skill directories. Remove each submodule before you use the reusable workflow.
46
+
A new or moved skill has no matching base path. The workflow reports its policy findings as new. It never applies a baseline from the pull request head.
57
47
58
-
## Finding policy
48
+
A file change can make a previous finding report again. SkillSpector binds baseline entries to file content and finding evidence.
59
49
60
-
The default policy reports a finding only when all these conditions are true:
50
+
Other events have no base-tree comparison. For these events, the workflow reports all head findings that meet the policy.
61
51
62
-
- The pull request introduces the finding.
63
-
- The severity is `HIGH` or `CRITICAL`.
64
-
- The confidence is at least `0.8`.
52
+
Set `fail-on-findings` to `false` for advisory mode. This mode creates reports and comments, but policy findings do not fail the job. Scanner errors always fail the job.
65
53
66
-
The workflow fails the scan when it reports a policy finding. It also fails when SkillSpector cannot complete a valid scan.
54
+
## Pull request comments and reports
67
55
68
-
Static analysis cannot confirm that a finding is a vulnerability. Review each reported finding before you treat it as a vulnerability.
56
+
The workflow creates one comment when a same-repository pull request has policy findings. A later scan updates the same comment.
69
57
70
-
## Pull request comments
58
+
The workflow removes the comment when no policy findings remain. It does not create a success comment. A failed scan does not change an existing comment.
71
59
72
-
After a complete scan, the workflow adds one comment only when it reports policy findings. It updates the same comment after each later complete scan.
60
+
Fork pull requests do not receive comments. Their workflow runs still create a job summary and an artifact.
73
61
74
-
The workflow removes the comment after the findings are resolved. It does not add a success comment. A failed or incomplete scan does not change an existing comment.
62
+
The comment and job summary list up to 20 findings. The JSON artifact contains all policy findings. Its name is `skillspector-static-report`.
75
63
76
-
Fork pull requests receive annotations and a job summary. The workflow does not comment on fork pull requests.
64
+
The artifact contains rule IDs, severity, confidence, and locations. It omits matched findings, explanations, remediations, code snippets, and file content.
77
65
78
66
## Inputs
79
67
80
68
| Input | Default | Purpose |
81
69
| --- | --- | --- |
82
-
| `runs-on` | `ubuntu-latest` | Select a Linux runner. |
83
-
| `min-confidence` | `0.8` | Set a value from `0.0` through `1.0` for HIGH and CRITICAL findings. |
84
-
| `fail-on-findings` | `true` | Fail the job for policy findings. Scanner errors always fail the job. |
70
+
| `runs-on` | `ubuntu-latest` | Select the Linux runner label. |
71
+
| `min-confidence` | `0.8` | Set the minimum confidence from `0.8` through `1.0`. |
72
+
| `fail-on-findings` | `true` | Fail the job when findings meet the policy. |
85
73
| `comment-on-pr` | `true` | Update one comment on same-repository pull requests. |
86
74
87
-
Set `fail-on-findings` to `false` for advisory mode. Advisory mode uses warning annotations and does not fail the job for policy findings.
| `finding-count` | The number of findings that meet the policy. |
107
-
| `skill-count` | The number of skills in the pull request head. |
80
+
| `skill-count` | The number of skills in the head tree. |
108
81
109
-
## Reports
110
-
111
-
After a complete scan, the workflow creates these results:
112
-
113
-
- Up to 50 annotations for policy findings.
114
-
- A job summary with up to 20 policy findings.
115
-
- A sanitized JSON artifact named `skillspector-static-report`.
116
-
- One pull request comment with up to 20 policy findings when the token can write comments.
117
-
118
-
The JSON artifact contains all policy findings. The comment and artifact omit SkillSpector's matched-evidence and code-snippet fields.
82
+
## Scan scope
119
83
120
-
The job summary states when SkillSpector partially inspected unchanged images. A finding comment contains the same statement.
84
+
The workflow finds regular `SKILL.md` files in hidden and nested directories. It skips `.git`, `.venv`, `__pycache__`, and `node_modules` during discovery.
121
85
122
-
## No-LLM mode and network use
86
+
It runs the SkillSpector CLI once for each directory that contains a manifest. It removes duplicate findings from overlapping nested scans.
123
87
124
-
The workflow always passes `--no-llm`. It does not send skill file content to an LLM. It does not need a model API credential.
88
+
The checkout resolves Git LFS files before the scan. The workflow rejects all symbolic links and Git submodules in the repository.
125
89
126
-
SkillSpector can send declared package names and versions to [OSV.dev](https://osv.dev). This lookup can occur in no-LLM mode.
90
+
The workflow fails if SkillSpector cannot fully inspect a skill. It rejects partial inspections, inspection exceptions, and excluded skill content.
127
91
128
-
The workflow scans local checkouts. It does not ask SkillSpector to clone or download a skill.
92
+
For a repository-root skill, the workflow permits only the `.git` directory exclusion. Skills must not contain binary or hidden content that SkillSpector cannot inspect.
129
93
130
-
The workflow pins [SkillSpector `v2.9.6`](https://github.com/NVIDIA/SkillSpector/releases/tag/v2.9.6) to commit `29b0dc8c39424e8e31ca055fa027adf8ba8f9650`. This version reduces high-severity false positives in OAuth documentation.
94
+
## No-LLM mode and network use
131
95
132
-
## Existing findings and baselines
96
+
The workflow passes `--no-llm` to the baseline and scan commands. It checks the scan metadata to confirm that SkillSpector did not use an LLM.
133
97
134
-
For pull requests, the workflow scans the base and head trees. It reports only findings that the head tree introduces.
98
+
The workflow does not send skill file content to an LLM. It does not need a model API credential.
135
99
136
-
The comparison includes the rule, severity, skill, file, and normalized finding evidence. It ignores a line-number change in the same file. It also ignores an unchanged file move after it verifies the complete file digest. For the `SC8` bytecode rule, a byte change at the same path is a new finding.
100
+
SkillSpector can send declared package names and versions to [OSV.dev](https://osv.dev). This lookup can occur in no-LLM mode.
137
101
138
-
The workflow does not apply a baseline from the head tree. A pull request cannot hide a finding with its own baseline.
102
+
The workflow pins [SkillSpector `v2.9.6`](https://github.com/NVIDIA/SkillSpector/releases/tag/v2.9.6) to commit `29b0dc8c39424e8e31ca055fa027adf8ba8f9650`.
0 commit comments