Skip to content

Conversation

Copy link

Copilot AI commented Oct 22, 2025

  • Add GitHub Actions summary support
  • Detect if running inside GitHub Actions
  • Use environment variable only (no CLI flag)
  • Use nix log to get build logs for failed packages
  • Add integration test
  • All linting and type checking passes
  • Code formatting matches treefmt style
  • Build result is now a dataclass
  • Feature documented in README
  • Only show configured operations in summary
Original prompt

This section details on the original issue you should resolve

<issue_title>Support Github_Summary</issue_title>
<issue_description>https://github.blog/news-insights/product-news/supercharging-github-actions-with-job-summaries/

would be nice to have. with an argument to define the environment variable to override the environment variables.</issue_description>

<agent_instructions>detect if running inside github actions. Provide sections with logs per failed package as well.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes #222


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI self-assigned this Oct 22, 2025
@coderabbitai
Copy link

coderabbitai bot commented Oct 22, 2025

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.


Comment @coderabbitai help to get the list of available commands and usage tips.

Copilot AI and others added 2 commits October 22, 2025 12:48
- Add --github-summary CLI argument for custom output file
- Auto-detect GitHub Actions environment via GITHUB_ACTIONS env var
- Capture stderr output from failed builds
- Generate markdown summary with failed builds section
- Include build logs in collapsible details sections
- Add comprehensive tests for GitHub summary functionality

Co-authored-by: Mic92 <[email protected]>
- Add new section explaining GitHub Actions job summaries
- Document automatic detection and usage
- Include --github-summary flag in reference section

Co-authored-by: Mic92 <[email protected]>
@Mic92
Copy link
Owner

Mic92 commented Oct 22, 2025

@copilot

Builder: github:Mic92/nix-fast-build#checks.aarch64-darwin.treefmt
Build number: 146
Worker name: eve-012
nix build -L --option keep-going true --max-silent-time 1200 --accept-flake-config --out-link result-aarch64-darwin.treefmt '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv^'
in dir /var/lib/buildbot-worker/worker-012/Mic92_nix-fast-build_nix-build/build (timeout 10800 secs)
watching logfiles {}
argv: [b'nix', b'build', b'-L', b'--option', b'keep-going', b'true', b'--max-silent-time', b'1200', b'--accept-flake-config', b'--out-link', b'result-aarch64-darwin.treefmt', b'/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv^
']
using PTY: False
this derivation will be built:
/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv
building '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv' on 'ssh-ng://customer@mac02'...
copying 1 paths...
copying path '/nix/store/f3z9q39mja6khk647zdyd5ycv00h5zf9-source' to 'ssh-ng://customer@mac02'...
building '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv'...
treefmt-check> treefmt v2.3.1ERRO formatter | ruff-check: failed to apply with options '[check --fix]': exit status 1
treefmt-check>
treefmt-check> warning: The following rules have been removed and ignoring them has no effect:
treefmt-check> - ANN101
treefmt-check>
treefmt-check> PERF401 Use list.extend with an async comprehension to create a transformed list
treefmt-check> --> nix_fast_build/init.py:621:21
treefmt-check> |
treefmt-check> 619 | assert proc.stderr
treefmt-check> 620 | async for line in proc.stderr:
treefmt-check> 621 | stderr_lines.append(line)
treefmt-check> | ^^^^^^^^^^^^^^^^^^^^^^^^^
treefmt-check> 622 |
treefmt-check> 623 | # Start capturing stderr in background
treefmt-check> |
treefmt-check> help: Replace for loop with list.extend
treefmt-check>
treefmt-check> S108 Probable insecure usage of temporary file or directory: "/tmp/summary.md"
treefmt-check> --> tests/test_github_summary.py:42:35
treefmt-check> |
treefmt-check> 40 | def test_get_github_summary_file_explicit() -> None:
treefmt-check> 41 | """Test explicit summary file path."""
treefmt-check> 42 | opts = Options(github_summary="/tmp/summary.md")
treefmt-check> | ^^^^^^^^^^^^^^^^^
treefmt-check> 43 | result = get_github_summary_file(opts)
treefmt-check> 44 | assert result == Path("/tmp/summary.md")
treefmt-check> |
treefmt-check>
treefmt-check> S108 Probable insecure usage of temporary file or directory: "/tmp/summary.md"
treefmt-check> --> tests/test_github_summary.py:44:27
treefmt-check> |
treefmt-check> 42 | opts = Options(github_summary="/tmp/summary.md")
treefmt-check> 43 | result = get_github_summary_file(opts)
treefmt-check> 44 | assert result == Path("/tmp/summary.md")
treefmt-check> | ^^^^^^^^^^^^^^^^^
treefmt-check> |
treefmt-check>
treefmt-check> S108 Probable insecure usage of temporary file or directory: "/tmp/github_summary.md"
treefmt-check> --> tests/test_github_summary.py:54:45
treefmt-check> |
treefmt-check> 52 | try:
treefmt-check> 53 | os.environ["GITHUB_ACTIONS"] = "true"
treefmt-check> 54 | os.environ["GITHUB_STEP_SUMMARY"] = "/tmp/github_summary.md"
treefmt-check> | ^^^^^^^^^^^^^^^^^^^^^^^^
treefmt-check> 55 |
treefmt-check> 56 | opts = Options()
treefmt-check> |
treefmt-check>
treefmt-check> S108 Probable insecure usage of temporary file or directory: "/tmp/github_summary.md"
treefmt-check> --> tests/test_github_summary.py:58:31
treefmt-check> |
treefmt-check> 56 | opts = Options()
treefmt-check> 57 | result = get_github_summary_file(opts)
treefmt-check> 58 | assert result == Path("/tmp/github_summary.md")
treefmt-check> | ^^^^^^^^^^^^^^^^^^^^^^^^
treefmt-check> 59 | finally:
treefmt-check> 60 | # Restore original values
treefmt-check> |
treefmt-check>
treefmt-check> Found 17 errors (12 fixed, 5 remaining).
treefmt-check>
treefmt-check> traversed 25 files
treefmt-check> emitted 18 files for processing
treefmt-check> formatted 9 files (3 changed) in 199ms
treefmt-check> Error: failed to finalise formatting: formatting failures detected
error: build of '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv' on 'ssh-ng://customer@mac02' failed: builder for '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv' failed with exit code 1;
last 25 log lines:
> 52 | try:
> 53 | os.environ["GITHUB_ACTIONS"] = "true"
> 54 | os.environ["GITHUB_STEP_SUMMARY"] = "/tmp/github_summary.md"
> | ^^^^^^^^^^^^^^^^^^^^^^^^
> 55 |
> 56 | opts = Options()
> |
>
> S108 Probable insecure usage of temporary file or directory: "/tmp/github_summary.md"
> --> tests/test_github_summary.py:58:31
> |
> 56 | opts = Options()
> 57 | result = get_github_summary_file(opts)
> 58 | assert result == Path("/tmp/github_summary.md")
> | ^^^^^^^^^^^^^^^^^^^^^^^^
> 59 | finally:
> 60 | # Restore original values
> |
>
> Found 17 errors (12 fixed, 5 remaining).
>
> traversed 25 files
> emitted 18 files for processing
> formatted 9 files (3 changed) in 199ms
> Error: failed to finalise formatting: formatting failures detected
For full logs, run:
nix log /nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv
error: Cannot build '/nix/store/halsjcr540nyihsgrx3hm5lb5kn62xfa-treefmt-check.drv'.
Reason: builder failed with exit code 1.
Output paths:
/nix/store/ga416b5861mw5xj2xpc2jakw0nyl79ji-treefmt-check
program finished with exit code 1
elapsedTime=2.540735

- Test complete workflow with environment variables
- Test explicit flag overriding environment variables
- Test log truncation for very long outputs
- All tests passing with full type checking

Co-authored-by: Mic92 <[email protected]>
help="Override a specific flake input (e.g. `dwarffs/nixpkgs`).",
)
parser.add_argument(
"--github-summary",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot env variable is enough here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed the --github-summary CLI argument. Now uses only GITHUB_STEP_SUMMARY environment variable. (c4b7308)

)

# Capture stderr if available
if proc.stderr:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot use nix log here instead once a build failed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Changed to use nix log command to fetch build logs after build failure instead of capturing stderr during build. (c4b7308)



def test_explicit_github_summary_overrides_env() -> None:
"""Test that explicit --github-summary flag overrides environment variable."""
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot remove.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed test_explicit_github_summary_overrides_env function. (c4b7308)

from pathlib import Path
from tempfile import TemporaryDirectory

from nix_fast_build import (
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot not needed the one integration test is enough.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed tests/test_github_summary.py entirely. Kept only the integration test. (c4b7308)

README.md Outdated
File to write build results to
--result-format {json,junit}
Format of the build result file
--github-summary GITHUB_SUMMARY
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot drop

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed --github-summary flag documentation from reference section. (c4b7308)

README.md Outdated
[--eval-workers EVAL_WORKERS]
[--result-file RESULT_FILE]
[--result-format {json,junit}]
[--github-summary GITHUB_SUMMARY]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot drop

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed "GitHub Actions Job Summaries" section from README. (c4b7308)

Copilot AI changed the title [WIP] Add support for GitHub job summaries Add GitHub Actions job summary support with build logs Oct 22, 2025
Copilot AI requested a review from Mic92 October 22, 2025 12:57
Copilot finished work on behalf of Mic92 October 22, 2025 12:57
- Remove --github-summary CLI argument (use GITHUB_STEP_SUMMARY only)
- Use nix log to fetch build logs after failure instead of capturing stderr
- Remove test_github_summary.py (keep only integration test)
- Remove test_explicit_github_summary_overrides_env
- Revert README documentation about --github-summary flag
- Fix ruff linting issues

Co-authored-by: Mic92 <[email protected]>
Copilot finished work on behalf of Mic92 October 22, 2025 13:10
@Mic92
Copy link
Owner

Mic92 commented Oct 22, 2025

@copilot

Builder: github:Mic92/nix-fast-build#checks.aarch64-darwin.treefmt
Build number: 149
Worker name: eve-002
nix build -L --option keep-going true --max-silent-time 1200 --accept-flake-config --out-link result-aarch64-darwin.treefmt '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv^'
in dir /var/lib/buildbot-worker/worker-002/Mic92_nix-fast-build_nix-build/build (timeout 10800 secs)
watching logfiles {}
argv: [b'nix', b'build', b'-L', b'--option', b'keep-going', b'true', b'--max-silent-time', b'1200', b'--accept-flake-config', b'--out-link', b'result-aarch64-darwin.treefmt', b'/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv^
']
using PTY: False
this derivation will be built:
/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv
building '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv' on 'ssh-ng://customer@mac02'...
copying 1 paths...
copying path '/nix/store/57gswri2g67lfa9hp2zcknl53bjwmn9i-source' to 'ssh-ng://customer@mac02'...
building '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv'...
treefmt-check> treefmt v2.3.1traversed 25 files
treefmt-check> emitted 18 files for processing
treefmt-check> formatted 18 files (2 changed) in 1.185s
treefmt-check> M nix_fast_build/init.py
treefmt-check> M tests/test_github_integration.py
treefmt-check> diff --git a/nix_fast_build/init.py b/nix_fast_build/init.py
treefmt-check> index c2ab95c..189d0bb 100644
treefmt-check> --- a/nix_fast_build/init.py
treefmt-check> +++ b/nix_fast_build/init.py
treefmt-check> @@ -1063,7 +1063,10 @@ def write_github_summary(
treefmt-check> # Truncate very long logs (keep last 100 lines)
treefmt-check> log_lines = result.log_output.strip().split("\n")
treefmt-check> if len(log_lines) > 100:
treefmt-check> - log_lines = ["... (truncated, showing last 100 lines) ...", *log_lines[-100:]]
treefmt-check> + log_lines = [
treefmt-check> + "... (truncated, showing last 100 lines) ...",
treefmt-check> + *log_lines[-100:],
treefmt-check> + ]
treefmt-check> lines.append("\n

")
treefmt-check> lines.append(f"Build Log ({len(log_lines)} lines)\n")
treefmt-check> lines.append("```")
treefmt-check> diff --git a/tests/test_github_integration.py b/tests/test_github_integration.py
treefmt-check> index 81c2c25..ee9ca6c 100644
treefmt-check> --- a/tests/test_github_integration.py
treefmt-check> +++ b/tests/test_github_integration.py
treefmt-check> @@ -1,4 +1,5 @@
treefmt-check> """Integration test for GitHub Actions summary feature."""
treefmt-check> +
treefmt-check> import os
treefmt-check> from pathlib import Path
treefmt-check> from tempfile import TemporaryDirectory
treefmt-check> @@ -26,7 +27,9 @@ def test_github_actions_workflow() -> None:
treefmt-check> os.environ["GITHUB_STEP_SUMMARY"] = str(summary_path)
treefmt-check>
treefmt-check> # Create options
treefmt-check> - opts = Options(flake_url="github:example/repo", flake_fragment="checks.x86_64-linux")
treefmt-check> + opts = Options(
treefmt-check> + flake_url="github:example/repo", flake_fragment="checks.x86_64-linux"
treefmt-check> + )
treefmt-check>
treefmt-check> # Verify it picks up the environment variable
treefmt-check> github_summary_file = get_github_summary_file()
treefmt-check> @@ -145,6 +148,8 @@ def test_long_log_truncation() -> None:
treefmt-check> assert "truncated, showing last 100 lines" in content
treefmt-check>
treefmt-check> # Verify only last 100 lines + truncation message are present
treefmt-check> - assert "log line 50" in content # Should be present (line 50 onwards, first of last 100)
treefmt-check> + assert (
treefmt-check> + "log line 50" in content
treefmt-check> + ) # Should be present (line 50 onwards, first of last 100)
treefmt-check> assert "log line 49" not in content # Should be truncated
treefmt-check> assert "log line 149" in content # Last line should be present
error: build of '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv' on 'ssh-ng://customer@mac02' failed: builder for '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv' failed with exit code 1;
last 25 log lines:
> +
> import os
> from pathlib import Path
> from tempfile import TemporaryDirectory
> @@ -26,7 +27,9 @@ def test_github_actions_workflow() -> None:
> os.environ["GITHUB_STEP_SUMMARY"] = str(summary_path)
>
> # Create options
> - opts = Options(flake_url="github:example/repo", flake_fragment="checks.x86_64-linux")
> + opts = Options(
> + flake_url="github:example/repo", flake_fragment="checks.x86_64-linux"
> + )
>
> # Verify it picks up the environment variable
> github_summary_file = get_github_summary_file()
> @@ -145,6 +148,8 @@ def test_long_log_truncation() -> None:
> assert "truncated, showing last 100 lines" in content
>
> # Verify only last 100 lines + truncation message are present
> - assert "log line 50" in content # Should be present (line 50 onwards, first of last 100)
> + assert (
> + "log line 50" in content
> + ) # Should be present (line 50 onwards, first of last 100)
> assert "log line 49" not in content # Should be truncated
> assert "log line 149" in content # Last line should be present
For full logs, run:
nix log /nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv
error: Cannot build '/nix/store/s4x41s2264647n8xg4vl6r0spy2hz7ry-treefmt-check.drv'.
Reason: builder failed with exit code 1.
Output paths:
/nix/store/yqimdyx0kybx046lvs3qqa4m845mpavl-treefmt-check
program finished with exit code 1
elapsedTime=4.170667

- Add blank line after module docstring
- Format long lines to match formatter expectations
- Split multi-line list comprehension for readability
- Format Options call across multiple lines

Co-authored-by: Mic92 <[email protected]>
async def build(
self, stack: AsyncExitStack, build_output: IO[str], opts: Options
) -> int:
) -> tuple[int, str]:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot make this a dataclass

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Created BuildResult dataclass to replace the tuple return type. (9cb1c16)


def is_github_actions() -> bool:
"""Detect if running inside GitHub Actions."""
return os.environ.get("GITHUB_ACTIONS") == "true"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot still document this behaviour in the readme.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Added "GitHub Actions Job Summaries" section to README documenting the automatic detection and build log behavior. (9cb1c16)

Copilot AI requested a review from Mic92 October 22, 2025 13:26
Copilot finished work on behalf of Mic92 October 22, 2025 13:26
Copy link
Author

Copilot AI commented Oct 22, 2025

@Mic92 Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 8b3b4b0d-d8b6-42f1-957c-a3dd985cf22f

Sorry for the inconvenience!

- Create BuildResult dataclass to replace tuple return type
- Update Build.build() to return BuildResult instead of tuple
- Add GitHub Actions Job Summaries section to README
- Document automatic detection and build log behavior

Co-authored-by: Mic92 <[email protected]>
Copilot finished work on behalf of Mic92 October 22, 2025 13:31
@Mic92
Copy link
Owner

Mic92 commented Oct 22, 2025

@copilot

why do we get counts for stuff that hasn't be done? in our self-test we don't have cachix/attic:

Type Successes Failures
✅ ATTIC 3 0
✅ BUILD 3 0
✅ CACHIX 3 0
✅ DOWNLOAD 3 0
✅ EVAL 14 0
✅ UPLOAD 3 0

- Don't add UPLOAD results when copy_to is not configured
- Don't add CACHIX results when cachix_cache is not configured
- Don't add ATTIC results when attic_cache is not configured
- Don't add DOWNLOAD results when remote or download is disabled
- Also filter out result types with 0 successes and 0 failures from summary table

This prevents showing misleading counts for operations that weren't actually performed.

Co-authored-by: Mic92 <[email protected]>
Copy link
Author

Copilot AI commented Oct 22, 2025

@copilot

why do we get counts for stuff that hasn't be done? in our self-test we don't have cachix/attic:
...

Fixed. Operations that aren't configured (CACHIX, ATTIC, UPLOAD, DOWNLOAD) now skip adding result entries entirely, so they won't appear in the summary table. The summary will only show operation types that actually ran. (e1d3629)

Copilot finished work on behalf of Mic92 October 22, 2025 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Github_Summary

2 participants