Skip to content

Commit 6f6b2c5

Browse files
ci: add conformance test workflow (#36)
* ci: add GitHub Actions workflow for conformance tests Adds a CI workflow that: - Checks out conformance, samples, and SDK repos - Starts the Flower Shop reference server with test data - Runs the full conformance test suite against it - Triggers on PRs, pushes to main, and nightly schedule Results are informational (continue-on-error) so the workflow does not block merges. This surfaces test failures early without gating PRs on a suite that depends on cross-repo compatibility. Addresses #38 * Address review: use python-sdk repo and gate continue-on-error to PR runs only * Disable uv cache prune to avoid post-job lock timeout --------- Co-authored-by: Enric Cusell (Google) <cusell@google.com>
1 parent 52e026a commit 6f6b2c5

1 file changed

Lines changed: 115 additions & 0 deletions

File tree

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
# Copyright 2026 UCP Authors
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
name: Conformance Tests
16+
17+
on:
18+
pull_request:
19+
branches: [main]
20+
push:
21+
branches: [main]
22+
schedule:
23+
# Nightly at 2:17 AM UTC
24+
- cron: "17 2 * * *"
25+
workflow_dispatch:
26+
27+
permissions:
28+
contents: read
29+
30+
env:
31+
MERCHANT_SERVER_PORT: 8182
32+
SIMULATION_SECRET: super-secret-sim-key
33+
DATABASE_PATH: /tmp/ucp_test
34+
35+
jobs:
36+
test:
37+
name: Run Conformance Tests
38+
runs-on: ubuntu-latest
39+
steps:
40+
- name: Check out conformance repo
41+
uses: actions/checkout@v4
42+
with:
43+
path: conformance
44+
45+
- name: Check out samples repo
46+
uses: actions/checkout@v4
47+
with:
48+
repository: Universal-Commerce-Protocol/samples
49+
path: samples
50+
51+
- name: Check out SDK repo
52+
uses: actions/checkout@v4
53+
with:
54+
repository: Universal-Commerce-Protocol/python-sdk
55+
path: python-sdk
56+
57+
- name: Install uv
58+
uses: astral-sh/setup-uv@v6
59+
with:
60+
prune-cache: false
61+
62+
- name: Set up Python
63+
run: uv python install 3.12
64+
65+
- name: Sync dependencies
66+
run: |
67+
uv sync --directory python-sdk/
68+
uv sync --directory samples/rest/python/server/
69+
uv sync --directory conformance/
70+
71+
- name: Initialize test database
72+
run: |
73+
rm -rf ${DATABASE_PATH}
74+
mkdir -p ${DATABASE_PATH}
75+
uv run --directory samples/rest/python/server import_csv.py \
76+
--products_db_path=${DATABASE_PATH}/products.db \
77+
--transactions_db_path=${DATABASE_PATH}/transactions.db \
78+
--data_dir=../../../../conformance/test_data/flower_shop
79+
80+
- name: Start Flower Shop server
81+
run: |
82+
uv run --directory samples/rest/python/server server.py \
83+
--products_db_path=${DATABASE_PATH}/products.db \
84+
--transactions_db_path=${DATABASE_PATH}/transactions.db \
85+
--port=${MERCHANT_SERVER_PORT} \
86+
--simulation_secret=${SIMULATION_SECRET} &
87+
# Wait for the server to be ready
88+
for i in $(seq 1 30); do
89+
if curl -sf http://localhost:${MERCHANT_SERVER_PORT}/.well-known/ucp > /dev/null 2>&1; then
90+
echo "Server is ready"
91+
break
92+
fi
93+
if [ "$i" -eq 30 ]; then
94+
echo "Server failed to start within 30 seconds"
95+
exit 1
96+
fi
97+
sleep 1
98+
done
99+
100+
- name: Run conformance tests
101+
continue-on-error: ${{ github.event_name == 'pull_request' }}
102+
working-directory: conformance
103+
run: |
104+
EXIT_CODE=0
105+
for test_file in *_test.py; do
106+
echo "::group::${test_file}"
107+
if ! uv run "${test_file}" \
108+
--server_url=http://localhost:${MERCHANT_SERVER_PORT} \
109+
--simulation_secret=${SIMULATION_SECRET} \
110+
--conformance_input=test_data/flower_shop/conformance_input.json; then
111+
EXIT_CODE=1
112+
fi
113+
echo "::endgroup::"
114+
done
115+
exit ${EXIT_CODE}

0 commit comments

Comments
 (0)