How to make 671B R1 inference in batch?

[Andcircle]

current set up is:
2 nodes of 8 x H100

Launch
python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1 --tp 16 --dist-init-addr 10.0.0.1:5000 --nnodes 2 --node-rank 0 --trust-remote-code

Then use python openai lib to call server get inference result one by one
Tried to call the server simultaneously with 'response = client.chat.completions.create', but the inference speed got much slower.

How can I do batch inference efficiently?
Thanks