Skip to content

ci: add conformance test workflow#36

Merged
cusell-google merged 4 commits intoUniversal-Commerce-Protocol:mainfrom
sbeashwar:ci-conformance-tests
Apr 30, 2026
Merged

ci: add conformance test workflow#36
cusell-google merged 4 commits intoUniversal-Commerce-Protocol:mainfrom
sbeashwar:ci-conformance-tests

Conversation

@sbeashwar
Copy link
Copy Markdown
Contributor

@sbeashwar sbeashwar commented Apr 20, 2026

Summary

  • Adds a GitHub Actions workflow that runs the full conformance test suite
  • Checks out conformance, samples, and SDK repos in correct relative paths
  • Starts the Flower Shop reference server with test data
  • Runs all *_test.py files and reports per-file results using GitHub log groups
  • Triggers on: PRs to main, pushes to main, nightly schedule, manual dispatch
  • Results are informational (continue-on-error) - does not block merges

This would have caught the breakage from #28 before merge (see #37).

Addresses #38

Test plan

  • Verify workflow runs on this PR
  • Confirm server health check wait loop works
  • Confirm all test files are discovered and executed

@google-cla
Copy link
Copy Markdown

google-cla Bot commented Apr 20, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Adds a CI workflow that:
- Checks out conformance, samples, and SDK repos
- Starts the Flower Shop reference server with test data
- Runs the full conformance test suite against it
- Triggers on PRs, pushes to main, and nightly schedule

Results are informational (continue-on-error) so the workflow
does not block merges. This surfaces test failures early without
gating PRs on a suite that depends on cross-repo compatibility.

Addresses Universal-Commerce-Protocol#38
@sbeashwar sbeashwar force-pushed the ci-conformance-tests branch from 328f2b5 to 940bcaa Compare April 22, 2026 16:34
@sbeashwar
Copy link
Copy Markdown
Contributor Author

@cusell-google gentle ping - this is a small, focused change ready for review when you have time.

Copy link
Copy Markdown
Contributor

@cusell-google cusell-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @sbeashwar for the nice, small incremental PRs!

Comment thread .github/workflows/conformance-tests.yml Outdated
Comment thread .github/workflows/conformance-tests.yml Outdated
@sbeashwar
Copy link
Copy Markdown
Contributor Author

Verified end-to-end on my fork: https://github.com/sbeashwar/conformance/actions/runs/24915126403

All 13 test files pass (59/59). The red status on the run is from a post-job uv cache prune cleanup timeout (known setup-uv cache lock issue, unrelated to the workflow itself) — the Run conformance tests step succeeded.

@sbeashwar
Copy link
Copy Markdown
Contributor Author

On a second thought, updated to disable post-job uv cache prune (was timing out on a lock and turning the run red even though tests passed). Latest run is fully green: https://github.com/sbeashwar/conformance/actions/runs/24915690807

@sbeashwar
Copy link
Copy Markdown
Contributor Author

sbeashwar commented Apr 26, 2026

@cusell-google addressed the two review comments, switched to python-sdk and gated continue-on-error to PR runs only. Also disabled post-job uv cache prune so the verification run goes fully green: https://github.com/sbeashwar/conformance/actions/runs/24915690807. Ready for re-review.

@sbeashwar
Copy link
Copy Markdown
Contributor Author

@cusell-google addressed the review feedback in the latest commits. Test plan verified on the fork:

  • Workflow runs on this PR
  • Server health check wait loop works
  • All test files are discovered and executed

Ready for re-review when you have a moment.

@sbeashwar sbeashwar requested a review from cusell-google April 29, 2026 15:58
Copy link
Copy Markdown
Contributor

@cusell-google cusell-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@cusell-google cusell-google requested review from damaz91 and removed request for MitkoDeyanovMitev April 30, 2026 09:26
@cusell-google cusell-google merged commit 6f6b2c5 into Universal-Commerce-Protocol:main Apr 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants