Skip to content

dotnet test in binlog-only mode suppresses MTP test result counts #2279

@SwapnilLomate

Description

@SwapnilLomate

When running rtk dotnet test, RTK operates in binlog-only mode and suppresses all stdout. This works correctly for detecting pass/fail, but the Microsoft Testing Platform (MTP) test result summary — total,
succeeded, failed, skipped counts — is written directly to stdout by the test executable, not as an MSBuild event. As a result, RTK reports counts unavailable and the summary is lost entirely.

Current output
ok dotnet test: completed (binlog-only mode, counts unavailable, 0 warnings)

Expected output (or something equivalent)
ok dotnet test: completed (72 passed, 0 failed, 0 skipped, 1s 672ms)

To reproduce

rtk dotnet test --project tests/SomeProject.Test.csproj --filter "SomeFilter"

Root cause

MSBuild binary logs capture build/task events. MTP (.NET 10's default test runner) prints its result summary via the test executable's own stdout, outside the MSBuild event stream. RTK reads the binlog but
never sees the stdout that carries the counts.

Running without RTK confirms the data is there:
Test run summary: Passed!
total: 72
failed: 0
succeeded: 72
skipped: 0
duration: 1s 672ms

Suggested fix

dotnet test failures are detectable from both exit code and stdout, so binlog is not strictly necessary for failure detection on test commands. One of:

  1. Detect dotnet test specifically and switch from binlog-capture to stdout passthrough, filtering to failures + the summary line only — same UX as build, but with counts visible on success.
  2. Tee stdout alongside binlog for test commands and extract the MTP summary line (Test run summary: / total: / succeeded: etc.) to append to RTK's own output.
  3. Parse --report-trx — if RTK injects --report-trx --results-directory automatically for test commands, it could read the TRX XML after the run and extract counts independently of stdout.

Option 1 is the simplest since MTP exit code (0 = all pass, non-zero = failures) is reliable enough to not need binlog for test runs.

Environment

  • .NET 10 / Microsoft Testing Platform (MTP)
  • dotnet test uses --solution or --project flags (MTP mode, not VSTest)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions