Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/release-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ jobs:
run: |
cd docs
make html
make markdown
python3 wrap_run_llm.py
cd _build/html
Expand Down
20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,26 @@ compile:
echo "Total execution time: $${TOTAL_ELAPSED}s" >> logs/timing.log; \
echo "All Notebook execution timings:" && cat logs/timing.log

# Convert Notebook files to Markdown artifacts (no execution)
markdown:
@set -e; \
echo "Exporting notebooks to Markdown..."; \
mkdir -p "$(BUILDDIR)/html/markdown"; \
find $(SOURCEDIR) -path "*/_build/*" -prune -o -name "*.ipynb" -print0 | \
parallel -0 -j3 --halt soon,fail=1 ' \
NB_SRC="{}"; \
REL_DIR=$$(dirname "$$NB_SRC"); \
NB_NAME=$$(basename "$$NB_SRC"); \
NB_BASE=$${NB_NAME%.ipynb}; \
OUT_DIR="$(BUILDDIR)/html/markdown/$$REL_DIR"; \
mkdir -p "$$OUT_DIR"; \
jupyter nbconvert --to markdown "$$NB_SRC" \
--output "$$NB_BASE.md" \
--output-dir "$$OUT_DIR" \
>/dev/null; \
Comment on lines 48 to 57
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script within parallel can be significantly simplified by using GNU Parallel's built-in replacement strings and leveraging jupyter nbconvert's ability to create output directories. This will make the script more concise and idiomatic.

You can replace the current block of shell commands with a single jupyter nbconvert call:

jupyter nbconvert --to markdown "{}" \
    --output "{/.}.md" \
    --output-dir "$(BUILDDIR)/md/{//}" \
    >/dev/null;

Here's what the replacement strings do:

  • {}: The full path to the notebook.
  • {//}: The directory name of the notebook.
  • {/.}: The basename of the notebook without the extension.

The mkdir -p command is also redundant because jupyter nbconvert creates the output directory specified by --output-dir if it doesn't exist.

' || exit 1; \
echo "Markdown artifacts written to: $(BUILDDIR)/html/markdown"

# Serve documentation with auto-build and live reload
serve:
@echo "Starting auto-build server at http://0.0.0.0:$(PORT)"
Expand Down
86 changes: 86 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,89 @@ pre-commit run --all-files
- Reuse the launched server as much as possible to reduce server launch time.
- Do not use absolute links (e.g., `https://docs.sglang.io/get_started/install.html`). Always prefer relative links (e.g., `../get_started/install.md`).
- Follow the existing examples to learn how to launch a server, send a query and other common styles.

## Documentation Build and Markdown Export Workflow

### Overview

The SGLang documentation pipeline is based on **Sphinx** and supports rendering Jupyter notebooks (`.ipynb`) into HTML for web display.
To support downstream documentation systems that cannot ingest HTML, we introduce an additional **Markdown export path** while keeping the existing build logic unchanged.

The key idea is to **reuse the existing build directory (`_build/html`)** and colocate Markdown artifacts alongside the rendered website.

---

### Current Documentation Build Flow

#### 1. Notebook Execution (`make compile`)

The `make compile` target is responsible for executing notebooks before rendering:

* Finds all `.ipynb` files under `docs/` (excluding `_build/`)
* Executes notebooks in parallel using GNU Parallel
* Wraps execution with `retry` to reduce flaky failures
* Executes notebooks via `jupyter nbconvert --execute --inplace`
* Records execution timing in `logs/timing.log`

This step ensures notebooks contain up-to-date outputs before rendering.

---

#### 2. Web Rendering (`make html`)

After compilation, Sphinx builds the website:

* Reads Markdown, reStructuredText, and Jupyter notebooks
* Renders them into HTML pages
* Outputs the website into:

```
docs/_build/html/
```

This directory is the source for online documentation hosting.

---

### Added Capability: Markdown Export (`make markdown`)

To support downstream consumers, we add a **new Makefile target**:

```bash
make markdown
```

This target:

* Does **not modify** `make compile`
* Scans all `.ipynb` files (excluding `_build/`)
* Converts notebooks directly to Markdown using `jupyter nbconvert --to markdown`
* Writes Markdown artifacts into the existing build directory:

```
docs/_build/html/markdown/<relative-path>.md
```

Example:

```
docs/advanced_features/lora.ipynb
→ docs/_build/html/markdown/advanced_features/lora.md
```

---

### Recommended CI Execution Order

In CI, the documentation pipeline follows this order:

```bash
make compile # execute notebooks (ensure outputs are up to date)
make html # build website as usual
make markdown # export markdown artifacts into _build/html/markdown
```

This ensures that:

* The website renders correctly
* Markdown artifacts reflect the same notebook state as the web pages
Loading