You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm going to follow https://docs.sglang.ai/references/benchmark_and_profiling.html. When I use DP and serve with two nodes, what's the proper way to do pytorch profiling?
should I set env variable export SGLANG_TORCH_PROFILER_DIR=/root/sglang/profile_log when launching both instances? Should they share a profiler director via an NFS or something else?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi.
I'm going to follow https://docs.sglang.ai/references/benchmark_and_profiling.html. When I use DP and serve with two nodes, what's the proper way to do pytorch profiling?
should I set env variable
export SGLANG_TORCH_PROFILER_DIR=/root/sglang/profile_log
when launching both instances? Should they share a profiler director via an NFS or something else?related issue: #2110
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions