Skip to content

Conversation

Gromototo
Copy link

@Gromototo Gromototo commented Oct 9, 2025

Objective

Add a new dune describe tests subcommand to display information about test and cram stanzas defined in the current project.

This information includes metadata such as:

  • name: The test name
  • source_dir: Directory containing the test stanza
  • package: Associated package (if any)
  • enabled: Whether the test is enabled (respects enabled_if conditions)
  • location: Source location of the stanza definition
  • target: Actionable build target (.exe for unit tests, @dir/runtest alias for CRAM tests)

Implementation

Core Architecture

  • Dual stanza support: Handles both Tests.T (unit tests) and Cram_stanza.T (CRAM tests)
  • Refactored logic: Separates collection (collect_test_stanzas) from transformation (describe_stanza) for clarity and maintainability
  • Actionable targets:
    • Unit tests: _build/default/<dir>/<name>.exe
    • CRAM tests: @<source_dir>/runtest (alias target)
  • Crawl pattern: Uses existing Dune crawl infrastructure and Dyn serialization for consistency with other describe subcommands
  • Output formats: Supports both human-readable S-expression and canonical S-expression (--format=csexp)
  • Complete reporting: Includes all tests regardless of enabled status (the boolean value indicates whether the test would run)

Technical Details

  • Exposed Cram_stanza via dune_rules module for type matching
  • Resolved namespace conflicts (Context_name.Map, Import.Main)
  • Scoped Memo.O usage to avoid operator conflicts with Cmdliner.Term

Testing

Added comprehensive blackbox test coverage in test/blackbox-tests/test-cases/describe/describe-tests.t:

Phase 1: Unit Tests (Tests.T)

  • Simple tests with basic metadata
  • Complex tests with full metadata (package, dependencies)
  • Multiple tests in a single stanza
  • Disabled tests (via enabled_if)
  • Tests with dependencies

Phase 2: CRAM Tests (Cram_stanza.T)

  • Simple CRAM tests
  • CRAM tests with package association
  • Disabled CRAM tests

Fixes

Closes #12030

@rgrinberg
Copy link
Member

Looks like a good start. I think you should start with a human readable format before moving on to machine readable formats though.

Following that, the most important extension to this work would be to support cram and ppx_expect tests. Those are at least as important as tests defined using the "test" stanza.

let to_dyn { names; source_dir; package; deps; enabled; action; locks } =
let open Dyn in
record
[ "names", list string names
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the fact that multiple tests can be define in one stanza isn't something that the describe command should necessarily display. A separate test entry for each test in the same stanza seems easier for tools/users to make use of.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with your suggestion. I updated the implementation so that each test in a multi-test stanza is listed individually in the dune describe tests output.

Implement a new subcommand to list all test stanzas (unit tests and CRAM)
with metadata including name, location, target, package, and enabled status.

- Support both Tests.T and Cram_stanza.T stanzas
- Generate actionable targets: `.exe` for unit tests, `@dir/runtest` for CRAM
- Handle multiple tests per stanza as separate entries
- Expose target and location fields for IDE integration
- Add comprehensive test suite in describe-tests.t

Fixes ocaml#12030

Signed-off-by: Rodrigue LEITAO--PEREIRA DIAS <[email protected]>
@Gromototo
Copy link
Author

Looks like a good start. I think you should start with a human readable format before moving on to machine readable formats though.

Following that, the most important extension to this work would be to support cram and ppx_expect tests. Those are at least as important as tests defined using the "test" stanza.

Since implementing Cram stanzas is straightforward, I included Cram support in the new revision.

Supporting ppx_expect tests is less direct, as they are not defined in configuration files. Handling them would likely require parsing the test files themselves ?

@rgrinberg
Copy link
Member

rgrinberg commented Oct 19, 2025

Supporting ppx_expect tests is less direct, as they are not defined in configuration files. Handling them would likely require parsing the test files themselves ?

Right, it's a bit more difficult. I don't think you need to parse the tests themselves (although that would certainly be a good thing too). It's enough to just rely on the rules to discover the test files. That would already be enough granularity.

What do you think about dropping the wrap sexp around the output like $ dune describe workspace? I don't think it buys us much.

PS: CI is pointing out some minor issues

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dune describe doesn't show tests

2 participants