[Doc] Comprehensive Guide: Navigating DP, DPA, and Router Best Practices #18092
Description
Motivation
As SGLang expands its support for advanced model architectures (such as DeepSeek), the complexity of choosing the right parallelism strategy increases. We’ve noticed users often get confused between DP (Data Parallel), DPA (Data Parallelism Attention), and the role of the Router.
We need a definitive guide that serves as a "Source of Truth" to help users optimize their deployments for high-throughput, large-scale inference.
Goal
The documentation should cover three main pillars:
-
Understanding DPA (Data Parallelism Attention)
- What is DPA? Define DPA and how it differs from standard data parallelism.
- Target Models: Identify which models (especially MLA-based architectures like DeepSeek) require DPA for optimal efficiency.
- Activation Logic: Explain the conditions under which DPA should be enabled and how SGLang handles it internally.
-
Native DP vs. Router-Based DP
- A strong recommendation to use the SGLang Router for production-grade Data Parallelism instead of the native/built-in DP mode.
- Highlighting the advantages of the Router in terms of load balancing, memory management, and overall system stability.
-
Practical Implementation: DP Routing via Router
- Provide clear instructions on how to set up the Router to handle DP routing.
- Best practices for routing strategies and workload distribution.
- How to verify that the traffic is being routed correctly across instances.
Technical Tasks
- Draft the conceptual section on DPA and its relation to modern LLM architectures.
- Write the comparison between native DP and Router-based DP, emphasizing stability and performance.
- Create a step-by-step "Quick Start" for DP routing.
- Integrate the new guide into the official documentation.
Resource
SGLang v0.4: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs