feat(backlog): normalize daily summarize Markdown output (#323)

* feat(backlog): summarize Markdown normalization and TTY/CI rendering

* chore(openspec): drop implementation snapshot from change

* Update title

---------

Co-authored-by: Dominikus Nold <djm81@users.noreply.github.com>

Author: djm81

docs/getting-started/tutorial-daily-standup-sprint-review.md

@@ -27,7 +27,9 @@ Preferred command path is `specfact backlog ceremony standup ...`. The legacy `s
   the adapter supports fetching comments
 - Use **`--summarize`** or **`--summarize-to <path>`** to output a **prompt** (instruction + filter context
   + standup data) for a slash command (e.g. `specfact.daily`) or copy-paste to Copilot to **generate a
-  standup summary**; add **`--comments`**/**`--annotations`** to include comment annotations in the prompt
+  standup summary**; add **`--comments`**/**`--annotations`** to include comment annotations in the prompt.
+  The prompt content is always **normalized to Markdown-only text** (no raw HTML tags or HTML entities) so
+  ADO-style HTML descriptions/comments and GitHub/Markdown content render consistently.
 - Use the **`specfact.backlog-daily`** (or `specfact.daily`) slash prompt for interactive walkthrough with the DevOps team story-by-story (focus, issues, open questions, discussion notes as comments)
 - Filter by **`--assignee`**, **`--sprint`** / **`--iteration`**, **`--search`**, **`--release`**, **`--id`**, **`--first-issues`** / **`--last-issues`**, **`--blockers-first`**, and optional **`--suggest-next`**
 
@@ -142,18 +144,21 @@ the standup table (state, assignee, limit, etc.).
 To get a **prompt** you can paste into Copilot or feed to a slash command (e.g. `specfact.daily`) so an AI can **generate a short standup summary** (e.g. "Today: 3 in progress, 1 blocked, 2 pending commitment"):
 
 ```bash
-# Print prompt to stdout (copy-paste to Copilot)
+# Print prompt to stdout (copy-paste to Copilot). In an interactive terminal, SpecFact renders a
+# Markdown-formatted view; in CI/non-interactive environments the same normalized Markdown is printed
+# without ANSI formatting.
 specfact backlog ceremony standup github --summarize --comments
 
-# Write prompt to a file (e.g. for slash command)
+# Write prompt to a file (e.g. for slash command). The file always contains plain Markdown-only content
+# (no raw HTML, no ANSI control codes), suitable for IDE slash commands or copy/paste into Copilot.
 specfact backlog ceremony standup github --summarize-to ./standup-prompt.md --comments
 ```
 
 The output includes an instruction to generate a standup summary, the applied filter context (adapter,
 state, sprint, assignee, limit), and the same per-item data as `--copilot-export`. With
-`--comments`/`--annotations`, the prompt includes comment annotations when supported. Use it with the
-**`specfact.backlog-daily`** slash prompt for interactive team walkthrough (story-by-story, current focus,
-issues/open questions, discussion notes as comments).
+`--comments`/`--annotations`, the prompt includes normalized descriptions and comment annotations when
+supported. Use it with the **`specfact.backlog-daily`** slash prompt for interactive team walkthrough
+(story-by-story, current focus, issues/open questions, discussion notes as comments).
 
 ---
 

docs/guides/agile-scrum-workflows.md

@@ -91,8 +91,10 @@ SpecFact CLI supports real-world agile/scrum practices through:
   to show suggested next item by value score (business_value / (story_points × priority)).
   **Copilot export**: Use `--copilot-export <path>` to write a summarized Markdown file of each story for
   Copilot. Add `--comments` (alias `--annotations`) to include descriptions and comment annotations in
-  `--copilot-export` and `--summarize` outputs when the adapter supports `get_comments` (GitHub, ADO). Use
-  `--first-comments N` or `--last-comments N` to scope comment volume when needed (default: include all).
+  `--copilot-export` and `--summarize` outputs when the adapter supports `get_comments` (GitHub, ADO). All
+  summarize/copilot-export content is **normalized to Markdown-only text** (no raw HTML tags or entities)
+  so ADO-style HTML fields and Markdown-native fields render consistently. Use `--first-comments N` or
+  `--last-comments N` to scope comment volume when needed (default: include all).
   Use `--first-issues N` or `--last-issues N` (mutually exclusive) to scope daily output to oldest/newest
   items by numeric issue/work-item ID.
   **Kanban**: omit iteration/sprint and use state + limit; unassigned = pullable work. **Scrum/SAFe**: use
@@ -158,8 +160,8 @@ specfact backlog ceremony standup github --interactive   # step-through; detail
 # or
 specfact backlog ceremony standup github --copilot-export ./standup.md --comments --last-comments 5
 # or
-specfact backlog ceremony standup github --summarize --comments     # prompt to stdout for AI to generate standup summary
-specfact backlog ceremony standup github --summarize-to ./standup-prompt.md
+specfact backlog ceremony standup github --summarize --comments     # prompt to stdout for AI to generate standup summary (Markdown-only)
+specfact backlog ceremony standup github --summarize-to ./standup-prompt.md    # plain Markdown file (no HTML/ANSI)
 ```
 
 Use the **`specfact.backlog-daily`** (or `specfact.daily`) slash prompt for interactive walkthrough with the

openspec/changes/backlog-scrum-05-summarize-markdown-output/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-02-27

openspec/changes/backlog-scrum-05-summarize-markdown-output/CHANGE_VALIDATION.md

@@ -0,0 +1,77 @@
+# Change Validation Report: backlog-scrum-05-summarize-markdown-output
+
+**Validation Date**: 2026-02-27T13:01:44+01:00
+**Change Proposal**: [proposal.md](./proposal.md)
+**Validation Method**: Dry-run dependency analysis and OpenSpec strict validation (post-implementation)
+
+## Executive Summary
+
+- **Breaking Changes**: 0 detected
+- **Dependent Files**: 2 affected (implementation and tests; both updated in same change)
+- **Impact Level**: Low
+- **Validation Result**: Pass
+- **User Decision**: Proceed (implementation completed)
+
+## Breaking Changes Detected
+
+None. All changes are additive or internal:
+
+- **`_normalize_markdown_text(text: str) -> str`**: New private helper in `commands.py`; no public API change.
+- **`_is_interactive_tty() -> bool`**: New private helper; no public API change.
+- **`_build_summarize_prompt_content(...)`**: Signature unchanged; behavior change is internal (normalization of body/comment strings before inclusion). All call sites (same module and unit tests) remain compatible.
+
+## Dependencies Affected
+
+### Critical Updates Required
+
+None.
+
+### Recommended Updates
+
+- **`src/specfact_cli/modules/backlog/src/commands.py`**: Already updated (normalization, TTY detection, Rich Markdown rendering).
+- **`tests/unit/commands/test_backlog_daily.py`**: Already updated (HTML normalization tests, existing summarize tests still pass).
+
+### Optional
+
+- **`docs/getting-started/tutorial-daily-standup-sprint-review.md`**: Updated to describe Markdown-only and interactive vs CI behavior.
+- **`docs/guides/agile-scrum-workflows.md`**: Updated to note normalized Markdown-only summarize/copilot-export content.
+
+## Impact Assessment
+
+- **Code Impact**: Single module (`modules/backlog/src/commands.py`); new helpers and wiring inside existing summarize path.
+- **Test Impact**: New unit tests for HTML normalization; existing summarize tests unchanged in contract.
+- **Documentation Impact**: Tutorial and agile guide updated.
+- **Release Impact**: Patch (backward-compatible behavior change; output format improved, not contracted).
+
+## User Decision
+
+**Decision**: Proceed
+**Rationale**: Implementation completed; no breaking changes; OpenSpec validation passed.
+**Next Steps**: Merge via PR from feature worktree to `dev`; optionally run `/opsx:archive` after merge.
+
+## Format Validation
+
+- **proposal.md Format**: Pass
+  - Required sections present: Why, What Changes, Capabilities, Impact
+  - Capabilities section lists new capability and modified daily-standup
+- **tasks.md Format**: Pass
+  - Numbered sections and checkbox task format per config
+  - All tasks completed except 4.3 (now completed by this validation)
+- **specs Format**: Pass
+  - ADDED/MODIFIED requirements with scenarios (When/Then)
+- **design.md Format**: Pass
+  - Context, Goals/Non-Goals, Decisions, Risks documented
+- **Config.yaml Compliance**: Pass
+
+## OpenSpec Validation
+
+- **Status**: Pass
+- **Validation Command**: `openspec validate backlog-scrum-05-summarize-markdown-output --strict`
+- **Issues Found**: 0
+- **Issues Fixed**: 0
+- **Re-validated**: N/A
+
+## Validation Artifacts
+
+- Dependency search: `_normalize_markdown_text`, `_is_interactive_tty`, `_build_summarize_prompt_content` usages confined to `commands.py` and `test_backlog_daily.py`.
+- No temporary workspace created; validation performed in-repo post-implementation.

openspec/changes/backlog-scrum-05-summarize-markdown-output/TDD_EVIDENCE.md

@@ -0,0 +1,39 @@
+## TDD Evidence for backlog-scrum-05-summarize-markdown-output
+
+### Failing-before run (new summarize normalization tests)
+
+- **Command:**
+
+  ```bash
+  hatch test --cover -v tests/unit/commands/test_backlog_daily.py -k "summarize_prompt_normalizes_html"
+  ```
+
+- **Timestamp:** 2026-02-27 (see CI logs / local shell history for exact time)
+
+- **Failure summary:**
+  - `test_summarize_prompt_normalizes_html_description_to_markdown`:
+    - Expected HTML `<p>` / `<br />` and `&amp;` entities to be removed from summarize prompt output.
+    - Actual output still contained raw `<p>Line 1<br />Line 2 &amp; more</p>` in the Description section.
+  - `test_summarize_prompt_normalizes_html_comments_to_markdown`:
+    - Expected HTML `<div>` and `<br>` plus `&amp;` entities in comments to be removed.
+    - Actual output still contained raw `<div>Comment &amp; note<br>next line</div>` in the Comments section.
+
+These failures confirm current behavior violates the new spec delta: summarize prompts include raw HTML and entities from ADO-style bodies and comments instead of normalized Markdown-only content.
+
+### Passing-after run (summarize normalization implemented)
+
+- **Command:**
+
+  ```bash
+  hatch test --cover -v tests/unit/commands/test_backlog_daily.py -k "summarize_prompt_normalizes_html"
+  ```
+
+- **Result:** ✅ 2 passed (normalization tests), remaining tests deselected in this targeted run.
+
+- **Behavior summary:**
+  - `_build_summarize_prompt_content` now:
+    - Normalizes HTML-based `body_markdown` values to Markdown-friendly text (no `<p>`, `<br>` tags or `&amp;` entities).
+    - Normalizes HTML comments before including them under the "Comments (annotations)" section.
+  - New helper `_normalize_markdown_text` (with `@beartype` and `@ensure`) enforces that the returned text does not contain raw HTML tags, satisfying the updated `daily-standup` summarize requirements.
+
+

openspec/changes/backlog-scrum-05-summarize-markdown-output/design.md

@@ -0,0 +1,40 @@
+## Context
+
+`specfact backlog daily` already supports a `--summarize` / `--summarize-to` flow that builds a prompt-ready view of the current standup scope (filters, per-item data, body, comments). When used against ADO, the underlying work item body and comments are often stored as HTML, while GitHub and some ADO comments use Markdown. Today the summarize builder can emit raw HTML fragments and entities into the prompt, which is noisy for both humans and LLMs and inconsistent with Markdown-centric flows elsewhere in SpecFact.
+
+At the same time, SpecFact needs to support both interactive, rich terminal sessions (for humans running standup from a shell) and non-interactive / CI environments where summarize output is consumed by other tools or stored as artifacts.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Normalize all descriptions and comments included in summarize output into clean Markdown text, regardless of provider format.
+- Ensure summarize prompts never contain raw HTML tags or HTML entities.
+- Provide a Markdown-aware, readable view of the summarize content in interactive terminals (e.g. Rich Markdown rendering), while keeping the underlying Markdown text stable and prompt-ready.
+- Preserve existing summarize semantics: same filters, same per-item data fields, same `--summarize` vs `--summarize-to` behavior.
+
+**Non-Goals:**
+
+- Do not change which items are included in standup or summarize scope (filters and selection logic remain as defined in `daily-standup`).
+- Do not change how comments or bodies are stored in providers; normalization is applied only at summarize/export time.
+- Do not introduce a hard dependency on any particular HTML-to-Markdown library that would block offline usage; implementation must remain Python-only and bundle-safe.
+
+## Decisions
+
+- Introduce a small normalization utility (e.g. in the backlog module package) that:
+  - Accepts raw body/comment text and a hint about source format (HTML vs Markdown when known).
+  - Converts HTML to Markdown using a deterministic, testable strategy.
+  - Always returns Markdown-only text suitable for inclusion in prompts.
+- Extend the summarize builder for `backlog daily` so that:
+  - Before assembling the per-item section, it passes body and comment text through the normalization utility.
+  - It treats GitHub/Markdown-native content as Markdown but still routes through the same normalization path for consistency.
+- Add a simple environment/TTY detection layer around summarize output:
+  - If running in an interactive TTY and not explicitly in CI mode, render the normalized Markdown using Rich (or an equivalent Markdown-capable view) for the user.
+  - If output is redirected, piped, or CI mode is detected, emit plain Markdown text without terminal control codes.
+
+## Risks / Trade-offs
+
+- HTML-to-Markdown conversion can be lossy if not carefully tuned; we must verify typical ADO HTML patterns (paragraphs, lists, bold, links) produce acceptable Markdown for standup prompts.
+- Rich or similar libraries must be used in a way that does not leak ANSI control codes into `--summarize-to` files or CI logs; separation between rendered view and underlying text needs to be clear in implementation.
+- Normalization adds a processing step per item/comment; for very large backlogs this can affect performance, so implementation should be efficient and optionally short-circuit when input is already clean Markdown.
+

openspec/changes/backlog-scrum-05-summarize-markdown-output/proposal.md

@@ -0,0 +1,43 @@
+# Change: Normalize daily summarize Markdown output
+
+## Why
+
+
+
+The current `specfact backlog daily --summarize/--summarize-to` output often contains raw HTML fragments and entities from ADO work item comments, mixed with Markdown-formatted text from GitHub and ADO. This makes the standup summary prompt hard to read for humans and noisy for LLMs, even though the underlying data is correct.
+
+## What Changes
+
+
+
+- Normalize backlog comments and descriptions used by `specfact backlog daily --summarize/--summarize-to` so that:
+  - HTML-formatted content is converted into clean Markdown before it is included in the prompt.
+  - Existing Markdown content is preserved as Markdown (no lossy reformatting).
+- For interactive terminal sessions:
+  - Render the summarized standup prompt using a Markdown-aware terminal view (e.g. Rich Markdown rendering) so users see a readable, formatted view instead of raw Markdown or HTML.
+- For non-interactive / CI environments and plain terminals:
+  - Fall back to emitting structured Markdown text directly (never raw HTML), preserving prompt-ready formatting for copy/paste into Copilot or slash commands.
+- Ensure the summarize output logic can distinguish between:
+  - Interactive rich terminal usage (formatted view, still based on the same Markdown text).
+  - Non-interactive/CI usage (plain Markdown text, no color/control codes).
+
+## Capabilities
+### New Capabilities
+- `backlog-daily-markdown-normalization`: Normalize backlog item bodies and comments into Markdown-only text for daily standup summarize prompts, with environment-aware rendering (rich Markdown view in interactive terminals, plain Markdown in CI/non-interactive mode).
+
+### Modified Capabilities
+- `daily-standup`: Clarify that the `--summarize/--summarize-to` scenarios must:
+  - Include only Markdown (no raw HTML fragments or entities) in per-item body/comment fields.
+  - Prefer a Markdown-formatted view in interactive terminals while keeping the underlying output prompt-ready for LLMs.
+
+
+---
+
+## Source Tracking
+
+<!-- source_repo: nold-ai/specfact-cli -->
+- **GitHub Issue**: #324
+- **Issue URL**: <https://github.com/nold-ai/specfact-cli/issues/324>
+- **Last Synced Status**: proposed
+- **Sanitized**: false
+<!-- content_hash: da76bac5d7da3752 -->

openspec/changes/backlog-scrum-05-summarize-markdown-output/specs/backlog-daily-markdown-normalization/spec.md

@@ -0,0 +1,41 @@
+## ADDED Requirements
+
+### Requirement: Normalize HTML and Markdown for summarize output
+
+The system SHALL normalize all backlog item descriptions and comments included in `specfact backlog daily --summarize` and `--summarize-to` output so that the resulting prompt contains **only Markdown-formatted text** (no raw HTML tags or HTML entities), regardless of whether the underlying provider stores content as HTML (e.g. ADO) or Markdown (e.g. GitHub, Markdown-style ADO comments).
+
+#### Scenario: HTML comments from ADO are converted to Markdown
+- **WHEN** `specfact backlog daily --summarize` or `--summarize-to` includes work items whose description or comments are stored as HTML (e.g. ADO discussion/comments)
+- **THEN** the system converts that HTML content into readable Markdown before including it in the summarize prompt
+- **AND** the resulting output does not contain raw HTML tags or un-decoded HTML entities (e.g. `&lt;div&gt;`, `<p>`, `<br />`)
+
+#### Scenario: Existing Markdown comments are preserved as Markdown
+- **WHEN** `specfact backlog daily --summarize` or `--summarize-to` includes items whose description or comments are already stored as Markdown (e.g. GitHub issues, Markdown-formatted ADO comments)
+- **THEN** the system preserves the original Markdown semantics when building the summarize prompt (headings, lists, code fences, emphasis)
+- **AND** the system does not degrade Markdown into a less structured format (e.g. by stripping list markers or collapsing headings)
+
+#### Scenario: Mixed HTML and Markdown sources produce a consistent Markdown prompt
+- **WHEN** the daily summarize command aggregates items from sources that use different underlying formats (HTML and Markdown)
+- **THEN** the combined summarize output is a single, consistent Markdown document suitable for LLM consumption
+- **AND** no raw HTML tags or entities appear anywhere in the per-item body or comments sections
+
+### Requirement: Environment-aware rendering for summarize output
+
+The system SHALL render the same normalized Markdown summarize content differently depending on whether it is running in an interactive terminal session or in a non-interactive / CI environment, while always preserving a prompt-ready Markdown representation that tools can consume.
+
+#### Scenario: Interactive terminal shows rich Markdown view
+- **WHEN** a user runs `specfact backlog daily --summarize` in an interactive terminal that supports rich output (e.g. TTY, not redirected to a file)
+- **THEN** the CLI MAY render the summarize content using a Markdown-aware terminal view (for example, Rich Markdown rendering)
+- **AND** the user sees a readable, formatted standup summary prompt (headings, lists, emphasis) instead of raw Markdown or HTML
+- **AND** the underlying content remains logically the same as the Markdown text used for `--summarize-to` (same sections and text, just rendered differently)
+
+#### Scenario: Non-interactive or CI environments emit plain Markdown
+- **WHEN** `specfact backlog daily --summarize` or `--summarize-to` is run in a non-interactive environment (e.g. CI/CD job, output redirected to a file or piped)
+- **THEN** the system emits plain, prompt-ready Markdown text without ANSI color codes or interactive formatting controls
+- **AND** the output still satisfies the existing summarize requirement to include instruction text, filter context, and per-item data (including normalized body and comments)
+
+#### Scenario: Summarize-to file output is always Markdown-only
+- **WHEN** the user runs `specfact backlog daily --summarize-to <path>`
+- **THEN** the file at `<path>` contains only normalized Markdown content (no raw HTML tags or entities, no terminal control codes)
+- **AND** the file is suitable for direct copy/paste into IDE slash commands or Copilot prompts without additional cleanup
+