Skip to content

Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration #18131

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Kangyan-Zhou merged 10 commits into from
Feb 3, 2026
+89 −1
Merged

Support Markdown/Notebook-Friendly Documentation Export for Downstream Integration #18131

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c1d0fb5
support Markdown/Notebook-Friendly Documentation Export for Downstrea…
klhhhhh Feb 2, 2026
a015e91
support make markdown func
klhhhhh Feb 2, 2026
9449b5c
run make markdown to generate markdown file
klhhhhh Feb 2, 2026
ac2e23f
Support make markdown command
klhhhhh Feb 2, 2026
913894c
Support make markdown command
klhhhhh Feb 2, 2026
8469a77
change markdown path
klhhhhh Feb 3, 2026
dd1936b
change markdown path in comment
klhhhhh Feb 3, 2026
2be1e45
change the command execution step in release doc CI
klhhhhh Feb 3, 2026
6eb2d75
update readme add markdown generate workflow
klhhhhh Feb 3, 2026
07e2a9b
Revise documentation style and build workflow
zhaochenyang20 Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/release-docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ jobs:
run: |
cd docs
make html
make markdown
python3 wrap_run_llm.py
cd _build/html
Expand Down
20 changes: 20 additions & 0 deletions docs/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,26 @@ compile:
echo "Total execution time: $${TOTAL_ELAPSED}s" >> logs/timing.log; \
echo "All Notebook execution timings:" && cat logs/timing.log

# Convert Notebook files to Markdown artifacts (no execution)
markdown:
@set -e; \
echo "Exporting notebooks to Markdown..."; \
mkdir -p "$(BUILDDIR)/html/markdown"; \
find $(SOURCEDIR) -path "*/_build/*" -prune -o -name "*.ipynb" -print0 | \
parallel -0 -j3 --halt soon,fail=1 ' \
NB_SRC="{}"; \
REL_DIR=$$(dirname "$$NB_SRC"); \
NB_NAME=$$(basename "$$NB_SRC"); \
NB_BASE=$${NB_NAME%.ipynb}; \
OUT_DIR="$(BUILDDIR)/html/markdown/$$REL_DIR"; \
mkdir -p "$$OUT_DIR"; \
jupyter nbconvert --to markdown "$$NB_SRC" \
--output "$$NB_BASE.md" \
--output-dir "$$OUT_DIR" \
>/dev/null; \
Comment on lines 48 to 57
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The script within parallel can be significantly simplified by using GNU Parallel's built-in replacement strings and leveraging jupyter nbconvert's ability to create output directories. This will make the script more concise and idiomatic.

You can replace the current block of shell commands with a single jupyter nbconvert call:

jupyter nbconvert --to markdown "{}" \
    --output "{/.}.md" \
    --output-dir "$(BUILDDIR)/md/{//}" \
    >/dev/null;

Here's what the replacement strings do:

  • {}: The full path to the notebook.
  • {//}: The directory name of the notebook.
  • {/.}: The basename of the notebook without the extension.

The mkdir -p command is also redundant because jupyter nbconvert creates the output directory specified by --output-dir if it doesn't exist.

' || exit 1; \
echo "Markdown artifacts written to: $(BUILDDIR)/html/markdown"

# Serve documentation with auto-build and live reload
serve:
@echo "Starting auto-build server at http://0.0.0.0:$(PORT)"
Expand Down
69 changes: 68 additions & 1 deletion docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ find . -name '*.ipynb' -exec nbstripout {} \;
# After these checks pass, push your changes and open a PR on your branch
pre-commit run --all-files
```
---

## Documentation Style Guidelines

Expand All @@ -55,3 +54,71 @@ pre-commit run --all-files
- Reuse the launched server as much as possible to reduce server launch time.
- Do not use absolute links (e.g., `https://docs.sglang.io/get_started/install.html`). Always prefer relative links (e.g., `../get_started/install.md`).
- Follow the existing examples to learn how to launch a server, send a query and other common styles.

## Documentation Build, Deployment, and CI

The SGLang documentation pipeline is based on **Sphinx** and supports rendering Jupyter notebooks (`.ipynb`) into HTML/Markdown for web display. Detailed logits can be found in the [Makefile](./Makefile).

### Notebook Execution (`make compile`)

The `make compile` target is responsible for executing notebooks before rendering:

* Finds all `.ipynb` files under `docs/` (excluding `_build/`)
* Executes notebooks in parallel using GNU Parallel, with a relatively small `--mem-fraction-static`
* Wraps execution with `retry` to reduce flaky failures
* Executes notebooks via `jupyter nbconvert --execute --inplace`
* Records execution timing in `logs/timing.log`

This step ensures notebooks contain up-to-date outputs with each commit in the main branch before rendering.

### Web Rendering (`make html`)

After compilation, Sphinx builds the website:

* Reads Markdown, reStructuredText, and Jupyter notebooks
* Renders them into HTML pages
* Outputs the website into:

```
docs/_build/html/
```

This directory is the source for online documentation hosting.

### Markdown Export (`make markdown`)

To support downstream consumers, we add a **new Makefile target**:

```bash
make markdown
```

This target:

* Does **not modify** `make compile`
* Scans all `.ipynb` files (excluding `_build/`)
* Converts notebooks directly to Markdown using `jupyter nbconvert --to markdown`
* Writes Markdown artifacts into the existing build directory:

```
docs/_build/html/markdown/<relative-path>.md
```

Example:

```
docs/advanced_features/lora.ipynb
→ docs/_build/html/markdown/advanced_features/lora.md
```

### CI Execution

In our [CI](https://github.com/sgl-project/sglang/blob/main/.github/workflows/release-docs.yml), the documentation pipeline first gets all the executed results and renders HTML and Markdown by:

```bash
make compile # execute notebooks (ensure outputs are up to date)
make html # build website as usual
make markdown # export markdown artifacts into _build/html/markdown
```

Then, the compiled results are forced pushed to [sgl-project.io](https://github.com/sgl-project/sgl-project.github.io) for rendering. In other words, sgl-project.io is push-only. All the changes of SGLang docs should be made directly in SGLang main repo, then push to the sgl-project.io.
Loading