Skip to content

Commit 211a1f7

Browse files
jasonpcarrollczjaso
jasonpcarroll
and
czjaso
authored
Add common release workflow for core libraries. Update SBOM generation. (#138)
* Add universal release action for core libraries * Remove artifact backup step from universal release action * Rewrite universal release action based on coreMQTT workflow * Add doxygen version number update to universal release action * Remove -DNDEBUG from test build. * Replace FreeRTOS sbom-generator with Anchore SBOM action * Generate SBOM outside repository directory to exclude from ZIP * Use Anchore SBOM action with source directory only for cleaner output * Revert to FreeRTOS sbom-generator * Add SBOM file as release asset * Update SBOM asset name to sbom.spdx * Update sbom-generator to use actual organization name from environment * Update release action to use jasonpcarroll fork of sbom-generator * Add debug step to print source path and directory contents * Add debug prints to scan_dir function for source path scanning * Add unbuffered Python output and debug echo to sbom-generator * Add debug prints to scan_dir.py for path debugging * Fix source path to be relative to repo_path working directory * Remove working-directory from sbom-generator and pass repo-root-path explicitly * Add debug print before file_writer call in scan_dir.py * Add new SBOM generation action and update release action to use it. * Fix SBOM asset upload. * Fix hidden directory removal in release action * Move test validation before file removal and add repository build step * Add rollback functionality to release workflow - Add rollback action in release-rollback folder - Update release action to use rollback on failure - Clean up environment variables by moving to workflow level - Add configurable test and build commands - Remove delete_existing_tag_release functionality * Fix GitHub context variables in composite action - Move GitHub context variables from top-level env to individual steps - GitHub context is not available in composite action env section * Restructure release workflow and simplify rollback - Move checkout to beginning of release workflow - Use working-directory with REPO_DIR environment variable - Validate locally before making any remote changes - Simplify rollback action to only mark releases as broken - Remove tag deletion and commit reset from rollback * Update release workflow. * Fix default build commands. * Have environment variables per step. * Fix GitHub CLI installation for Ubuntu 24.04 * Fix checkout action to use GitHub expressions instead of env vars * Fix working-directory references and add GH_TOKEN * Minor fix. * Fix command execution using eval for complex command strings * Reorder steps to fix git repository access errors * Fix GitHub Actions expressions in with parameters * Use environment variables in with fields * Fix SBOM links. * Fix zip archive. * Fix. * Fix. * Fix tag existence check to use remote tags * Remove environment variables from with fields * Ignore .git folders in SBOM generator * Ignore both .git directories and .git files in SBOM generator * Update release action to update doxygen for release. * Remove release-rollback action. * Update from jasonpcarroll to FreeRTOS org. * Test with included file hashes. * Point to FreeRTOS org instead of fork. --------- Co-authored-by: czjaso <[email protected]>
1 parent 10447b3 commit 211a1f7

File tree

6 files changed

+863
-258
lines changed

6 files changed

+863
-258
lines changed

‎release/action.yml‎

Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
name: 'Core Library Release'
2+
description: 'Universal release action for FreeRTOS core libraries'
3+
inputs:
4+
version_number:
5+
description: 'Release Version Number (e.g., v1.0.0)'
6+
required: true
7+
branch:
8+
description: 'Branch to release from'
9+
required: false
10+
default: 'main'
11+
github_token:
12+
description: 'GitHub token for creating releases'
13+
required: true
14+
default: ${{ github.token }}
15+
run_test_command:
16+
description: 'Command to build and run tests from root of library directory.'
17+
required: false
18+
default: 'sudo apt-get install -y lcov && rm -f ../build && cmake -S ./test -B ../build -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DBUILD_CLONE_SUBMODULES=ON -DCMAKE_C_FLAGS="--coverage -Wall -Wextra -Werror" && make -C ../build all && cd ../build && ctest -E system --output-on-failure'
19+
repo_build_command:
20+
description: 'Command to build from root of library directory.'
21+
required: false
22+
default: 'rm -rf ../build && cmake -S . -B ../build -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && make -C ../build all'
23+
24+
runs:
25+
using: 'composite'
26+
steps:
27+
- name: Validate version number format
28+
shell: bash
29+
env:
30+
VERSION_NUMBER: ${{ inputs.version_number }}
31+
run: |
32+
if [[ ! "$VERSION_NUMBER" =~ ^[vV][0-9]+\.[0-9]+\.[0-9]+$ ]]; then
33+
echo "Error: Version number must be in format vX.Y.Z or VX.Y.Z"
34+
exit 1
35+
fi
36+
37+
- name: Install GitHub CLI
38+
shell: bash
39+
run: |
40+
curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg
41+
sudo chmod go+r /usr/share/keyrings/githubcli-archive-keyring.gpg
42+
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" | sudo tee /etc/apt/sources.list.d/github-cli.list > /dev/null
43+
sudo apt update
44+
sudo apt install gh
45+
46+
- name: Checkout code
47+
uses: actions/checkout@v4
48+
with:
49+
ref: ${{ inputs.branch }}
50+
path: ${{ github.event.repository.name }}-repository
51+
submodules: recursive
52+
53+
- name: Check if tag and release already exist
54+
shell: bash
55+
env:
56+
VERSION_NUMBER: ${{ inputs.version_number }}
57+
GH_TOKEN: ${{ inputs.github_token }}
58+
working-directory: ${{ github.event.repository.name }}-repository
59+
run: |
60+
if git ls-remote --tags origin | grep -q "refs/tags/$VERSION_NUMBER$"; then
61+
echo "Error: Tag $VERSION_NUMBER already exists"
62+
exit 1
63+
fi
64+
65+
if gh release list | grep -q "$VERSION_NUMBER"; then
66+
echo "Error: Release $VERSION_NUMBER already exists"
67+
exit 1
68+
fi
69+
70+
- name: Configure git identity
71+
shell: bash
72+
env:
73+
GITHUB_ACTOR: ${{ github.actor }}
74+
run: |
75+
git config --global user.name $GITHUB_ACTOR
76+
git config --global user.email [email protected]
77+
78+
- name: Update version number in manifest.yml
79+
shell: bash
80+
env:
81+
VERSION_NUMBER: ${{ inputs.version_number }}
82+
working-directory: ${{ github.event.repository.name }}-repository
83+
run: |
84+
if [ -f "./manifest.yml" ]; then
85+
sed -i -b "0,/^version/s/^version.*/version: \"$VERSION_NUMBER\"/g" ./manifest.yml
86+
git add manifest.yml
87+
git commit -m '[AUTO][RELEASE]: Update version number in manifest.yml'
88+
fi
89+
90+
- name: Update version number in doxygen
91+
shell: bash
92+
env:
93+
VERSION_NUMBER: ${{ inputs.version_number }}
94+
working-directory: ${{ github.event.repository.name }}-repository
95+
run: |
96+
if [ -f "./docs/doxygen/config.doxyfile" ]; then
97+
sed -i -b "s/PROJECT_NUMBER *=.*/PROJECT_NUMBER = $VERSION_NUMBER/g" ./docs/doxygen/config.doxyfile
98+
git add docs/doxygen/config.doxyfile
99+
git commit -m '[AUTO][RELEASE]: Update version number in doxygen'
100+
fi
101+
102+
- name: Generate SBOM for Git Repository
103+
uses: FreeRTOS/CI-CD-Github-Actions/sbom-generator@main
104+
with:
105+
directory: ./${{ github.event.repository.name }}-repository
106+
distribution-type: repository
107+
creator: Amazon Web Services, Inc.
108+
download-location: git+https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}.git@${{ inputs.version_number }}
109+
homepage: https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}
110+
namespace-prefix: https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}/releases/download/${{ inputs.version_number }}/
111+
include-file-hashes: true
112+
113+
- name: Run tests for the repository.
114+
shell: bash
115+
env:
116+
RUN_TEST_COMMAND: ${{ inputs.run_test_command }}
117+
working-directory: ${{ github.event.repository.name }}-repository
118+
run: |
119+
eval "$RUN_TEST_COMMAND"
120+
121+
- name: Copy repository folder for archive
122+
shell: bash
123+
env:
124+
REPO_DIR: ${{ github.event.repository.name }}-repository
125+
ARCHIVE_DIR: ${{ github.event.repository.name }}
126+
run: |
127+
cp -r "$REPO_DIR" "$ARCHIVE_DIR"
128+
129+
- name: Remove hidden files and test directory from archive directory
130+
shell: bash
131+
env:
132+
ARCHIVE_DIR: ${{ github.event.repository.name }}
133+
run: |
134+
find $ARCHIVE_DIR -type f -name ".*" -delete
135+
find $ARCHIVE_DIR -type d -name ".*" -exec rm -rf {} + 2>/dev/null || true
136+
rm -rf "$ARCHIVE_DIR/test"
137+
138+
- name: Update README.md to remove testing information from archive directory
139+
shell: bash
140+
env:
141+
ARCHIVE_DIR: ${{ github.event.repository.name }}
142+
run: sed -i '/^## Building Unit Tests$/,/^## Reference examples$/{/^## Reference examples$/!d;} ; /^## CBMC$/,/^## Reference examples$/{/^## Reference examples$/!d;}' $ARCHIVE_DIR/README.md
143+
144+
- name: Generate SBOM for archive.
145+
uses: FreeRTOS/CI-CD-Github-Actions/sbom-generator@main
146+
with:
147+
directory: ./${{ github.event.repository.name }}
148+
distribution-type: archive
149+
creator: Amazon Web Services, Inc.
150+
download-location: https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}/releases/download/${{ inputs.version_number }}/${{ github.event.repository.name }}-${{ inputs.version_number }}.zip
151+
homepage: https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}
152+
namespace-prefix: https://github.com/${{ github.repository_owner }}/${{ github.event.repository.name }}/releases/download/${{ inputs.version_number }}/
153+
include-file-hashes: true
154+
155+
- name: Copy archive SBOMs into archive folder
156+
shell: bash
157+
env:
158+
ARCHIVE_DIR: ${{ github.event.repository.name }}
159+
run: |
160+
cp *archive-SPDX* $ARCHIVE_DIR/
161+
162+
- name: Build archive folder
163+
shell: bash
164+
env:
165+
REPO_BUILD_COMMAND: ${{ inputs.repo_build_command }}
166+
working-directory: ${{ github.event.repository.name }}
167+
run: |
168+
eval "$REPO_BUILD_COMMAND"
169+
170+
- name: Install ZIP tools and create ZIP
171+
shell: bash
172+
env:
173+
REPO_NAME: ${{ github.event.repository.name }}
174+
ARCHIVE_DIR: ${{ github.event.repository.name }}
175+
VERSION_NUMBER: ${{ inputs.version_number }}
176+
run: |
177+
sudo apt-get install zip unzip
178+
zip -r "$REPO_NAME-$VERSION_NUMBER.zip" "$ARCHIVE_DIR"
179+
180+
- name: Push release changes to repository.
181+
shell: bash
182+
env:
183+
BRANCH: ${{ inputs.branch }}
184+
working-directory: ${{ github.event.repository.name }}-repository
185+
run: |
186+
git push origin $BRANCH
187+
188+
- name: Create and push tag for release.
189+
shell: bash
190+
env:
191+
REPO_NAME: ${{ github.event.repository.name }}
192+
VERSION_NUMBER: ${{ inputs.version_number }}
193+
working-directory: ${{ github.event.repository.name }}-repository
194+
run: |
195+
git tag "$VERSION_NUMBER" -a -m "$REPO_NAME Library $VERSION_NUMBER"
196+
git push origin --tags
197+
198+
- name: Create GitHub release.
199+
id: create_release
200+
uses: actions/create-release@v1
201+
env:
202+
GITHUB_TOKEN: ${{ inputs.github_token }}
203+
with:
204+
tag_name: ${{ inputs.version_number }}
205+
release_name: ${{ inputs.version_number }}
206+
body: Release ${{ inputs.version_number }} of the ${{ github.event.repository.name }} Library.
207+
draft: false
208+
prerelease: false
209+
210+
- name: Upload SBOMs to GitHub release.
211+
shell: bash
212+
env:
213+
UPLOAD_URL: ${{ steps.create_release.outputs.upload_url }}
214+
GITHUB_TOKEN: ${{ inputs.github_token }}
215+
run: |
216+
for file in *.spdx*; do
217+
if [ -f "$file" ]; then
218+
if [[ "$file" == *.spdx.json ]]; then
219+
content_type="application/json"
220+
elif [[ "$file" == *.spdx.xml ]]; then
221+
content_type="application/xml"
222+
elif [[ "$file" == *.spdx.yaml ]]; then
223+
content_type="application/yaml"
224+
else
225+
content_type="text/plain"
226+
fi
227+
228+
curl -X POST \
229+
-H "Authorization: token $GITHUB_TOKEN" \
230+
-H "Content-Type: $content_type" \
231+
--data-binary @"$file" \
232+
"${UPLOAD_URL%\{*}?name=$file"
233+
fi
234+
done
235+
236+
- name: Upload library archive to GitHub release.
237+
uses: actions/upload-release-asset@v1
238+
env:
239+
GITHUB_TOKEN: ${{ inputs.github_token }}
240+
with:
241+
upload_url: ${{ steps.create_release.outputs.upload_url }}
242+
asset_path: ${{ github.event.repository.name }}-${{ inputs.version_number }}.zip
243+
asset_name: ${{ github.event.repository.name }}-${{ inputs.version_number }}.zip
244+
asset_content_type: application/zip
245+
246+
- name: Deploy GitHub pages for release.
247+
uses: FreeRTOS/CI-CD-Github-Actions/doxygen-generation@main
248+
with:
249+
ref: ${{ inputs.version_number }}
250+
add_release: "true"
251+

‎sbom-generator/action.yml‎

Lines changed: 68 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,77 @@
1-
name: 'generate-sbom'
2-
description: 'Generate SBOM for FreeRTOS libraries'
1+
name: 'SBOM Generator'
2+
description: 'Generate SPDX 2.3 Software Bill of Materials (SBOM) from directory with manifest.yml'
3+
author: 'AWS'
4+
35
inputs:
4-
repo_path:
5-
description: 'Path to repository folder containing manifest.yml to verify.'
6+
directory:
7+
description: 'Path to directory containing manifest.yml'
8+
required: true
9+
distribution-type:
10+
description: 'Distribution method of package (archive or repository)'
11+
required: true
12+
creator:
13+
description: 'Creator organization name'
14+
required: true
15+
download-location:
16+
description: 'Package download location URL'
17+
required: true
18+
homepage:
19+
description: 'Package homepage URL'
20+
required: true
21+
namespace-prefix:
22+
description: 'Document namespace prefix URI'
23+
required: true
24+
include-file-hashes:
25+
description: 'Include individual file information with hashes'
626
required: false
7-
default: ./
8-
source_path:
9-
description: 'Path to source code'
27+
default: false
28+
exclude:
29+
description: 'Exclude files/directories from verification code (comma-separated)'
1030
required: false
11-
default: ./source
31+
32+
outputs:
33+
sbom-files:
34+
description: 'Generated SBOM file paths'
35+
1236
runs:
13-
using: "composite"
37+
using: 'composite'
1438
steps:
39+
- name: Setup Python
40+
uses: actions/setup-python@v4
41+
with:
42+
python-version: '3.x'
43+
1544
- name: Install dependencies
1645
shell: bash
17-
run: pip install -r $GITHUB_ACTION_PATH/requirements.txt
18-
19-
- name: Run generator script
20-
working-directory: ${{ inputs.repo_path }}
46+
run: |
47+
pip install -r ${{ github.action_path }}/requirements.txt
48+
49+
- name: Generate SBOM
2150
shell: bash
2251
run: |
23-
python3 $GITHUB_ACTION_PATH/scan_dir.py --source-path ${{ inputs.source_path }}
52+
cd ${{ github.workspace }}
53+
54+
# Build command
55+
cmd="python3 ${{ github.action_path }}/sbom_generator.py"
56+
cmd="$cmd ${{ inputs.directory }}"
57+
cmd="$cmd -t ${{ inputs.distribution-type }}"
58+
cmd="$cmd -c '${{ inputs.creator }}'"
59+
cmd="$cmd -d '${{ inputs.download-location }}'"
60+
cmd="$cmd -p '${{ inputs.homepage }}'"
61+
cmd="$cmd -n '${{ inputs.namespace-prefix }}'"
62+
63+
# Add optional flags
64+
if [ "${{ inputs.include-file-hashes }}" = "true" ]; then
65+
cmd="$cmd -f"
66+
fi
67+
68+
# Add exclusions
69+
if [ -n "${{ inputs.exclude }}" ]; then
70+
IFS=',' read -ra EXCLUDES <<< "${{ inputs.exclude }}"
71+
for exclude in "${EXCLUDES[@]}"; do
72+
cmd="$cmd -e '$exclude'"
73+
done
74+
fi
75+
76+
# Execute command
77+
eval $cmd

‎sbom-generator/requirements.txt‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1-
pyyaml
1+
spdx-tools>=0.8.0
2+
PyYAML>=6.0

0 commit comments

Comments
 (0)