My SGLang doesn't print resources usages #3997
Unanswered
someone132s
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
As title, my SGLang runs perfectly, but with one faw that it doesn't print the resources usage info like Throughput.
Here's my log:
root@GPUServer-1:/opt/sglang# docker compose -f docker-compose.dpskr1-qwen32b.yaml up -d
[+] Running 1/1
✔ Container sglang-dpskr1-qwen32b Started 11.0s
root@GPUServer-1:/opt/sglang# docker logs sglang-dpskr1-qwen32b -f
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use
slow_image_processor_class, orfast_image_processor_classinsteadwarnings.warn(
INFO 03-02 11:12:24 init.py:190] Automatically detected platform cuda.
WARNING 03-02 11:12:24 cuda.py:336] Detected different devices in the system:
WARNING 03-02 11:12:24 cuda.py:336] NVIDIA GeForce RTX 2070 SUPER
WARNING 03-02 11:12:24 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:24 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:24 cuda.py:336] Please make sure to set
CUDA_DEVICE_ORDER=PCI_BUS_IDto avoid unexpected behavior.[2025-03-02 11:12:42] server_args=ServerArgs(model_path='tclf90/deepseek-r1-distill-qwen-32b-gptq-int4', tokenizer_path='tclf90/deepseek-r1-distill-qwen-32b-gptq-int4', tokenizer_mode='auto', load_format='auto', trust_remote_code=True, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, quantization=None, context_length=71680, device='cuda', served_model_name='tclf90/deepseek-r1-distill-qwen-32b-gptq-int4', chat_template=None, is_embedding=False, revision=None, skip_tokenizer_init=False, host='0.0.0.0', port=30000, mem_fraction_static=0.85, max_running_requests=None, max_total_tokens=None, chunked_prefill_size=4096, max_prefill_tokens=16384, schedule_policy='lpm', schedule_conservativeness=1.0, cpu_offload_gb=0, prefill_only_one_req=False, tp_size=2, stream_interval=1, stream_output=True, random_seed=1021514162, constrained_json_whitespace_pattern=None, watchdog_timeout=300, download_dir=None, base_gpu_id=0, log_level='info', log_level_http='info', log_requests=True, show_time_cost=True, enable_metrics=True, decode_log_interval=40, api_key=None, file_storage_pth='sglang_storage', enable_cache_report=True, dp_size=1, load_balance_method='round_robin', ep_size=1, dist_init_addr=None, nnodes=1, node_rank=0, json_model_override_args='{}', lora_paths=None, max_loras_per_batch=8, lora_backend='triton', attention_backend='flashinfer', sampling_backend='flashinfer', grammar_backend='outlines', speculative_draft_model_path=None, speculative_algorithm=None, speculative_num_steps=5, speculative_num_draft_tokens=64, speculative_eagle_topk=8, enable_double_sparsity=False, ds_channel_config_path=None, ds_heavy_channel_num=32, ds_heavy_token_num=256, ds_heavy_channel_type='qk', ds_sparse_decode_threshold=4096, disable_radix_cache=True, disable_jump_forward=False, disable_cuda_graph=False, disable_cuda_graph_padding=False, enable_nccl_nvls=False, disable_outlines_disk_cache=False, disable_custom_all_reduce=False, disable_mla=False, disable_overlap_schedule=False, enable_mixed_chunk=True, enable_dp_attention=False, enable_ep_moe=False, enable_torch_compile=True, torch_compile_max_bs=32, cuda_graph_max_bs=8, cuda_graph_bs=None, torchao_config='', enable_nan_detection=False, enable_p2p_check=False, triton_attention_reduce_in_fp32=False, triton_attention_num_kv_splits=8, num_continuous_decode_steps=1, delete_ckpt_after_loading=False, enable_memory_saver=False, allow_auto_truncate=True, return_hidden_states=False, enable_custom_logit_processor=False, tool_call_parser=None, enable_hierarchical_cache=False, enable_flashinfer_mla=False)
2025-03-02 11:12:43,625 - modelscope - WARNING - Model revision not specified, use revision: g128
2025-03-02 11:12:44,374 - modelscope - WARNING - Model revision not specified, use revision: g128
Downloading Model to directory: /root/.cache/modelscope/hub/models/tclf90/deepseek-r1-distill-qwen-32b-gptq-int4
Downloading Model to directory: /root/.cache/modelscope/hub/models/tclf90/deepseek-r1-distill-qwen-32b-gptq-int4
[2025-03-02 11:12:44] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use
slow_image_processor_class, orfast_image_processor_classinsteadwarnings.warn(
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use
slow_image_processor_class, orfast_image_processor_classinsteadwarnings.warn(
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py:590: FutureWarning: The image_processor_class argument is deprecated and will be removed in v4.42. Please use
slow_image_processor_class, orfast_image_processor_classinsteadwarnings.warn(
INFO 03-02 11:12:53 init.py:190] Automatically detected platform cuda.
WARNING 03-02 11:12:53 cuda.py:336] Detected different devices in the system:
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2070 SUPER
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:53 cuda.py:336] Please make sure to set
CUDA_DEVICE_ORDER=PCI_BUS_IDto avoid unexpected behavior.INFO 03-02 11:12:53 init.py:190] Automatically detected platform cuda.
WARNING 03-02 11:12:53 cuda.py:336] Detected different devices in the system:
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2070 SUPER
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:53 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:53 cuda.py:336] Please make sure to set
CUDA_DEVICE_ORDER=PCI_BUS_IDto avoid unexpected behavior.INFO 03-02 11:12:54 init.py:190] Automatically detected platform cuda.
WARNING 03-02 11:12:54 cuda.py:336] Detected different devices in the system:
WARNING 03-02 11:12:54 cuda.py:336] NVIDIA GeForce RTX 2070 SUPER
WARNING 03-02 11:12:54 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:54 cuda.py:336] NVIDIA GeForce RTX 2080 Ti
WARNING 03-02 11:12:54 cuda.py:336] Please make sure to set
CUDA_DEVICE_ORDER=PCI_BUS_IDto avoid unexpected behavior.[2025-03-02 11:13:11 TP1] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
[2025-03-02 11:13:11 TP0] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
[2025-03-02 11:13:12 TP1] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
[2025-03-02 11:13:12 TP1] Init torch distributed begin.
[2025-03-02 11:13:12 TP0] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
[2025-03-02 11:13:12 TP0] Init torch distributed begin.
[2025-03-02 11:13:12 TP0] sglang is using nccl==2.21.5
[2025-03-02 11:13:12 TP1] sglang is using nccl==2.21.5
[2025-03-02 11:13:13 TP0] Load weight begin. avail mem=21.22 GB
[2025-03-02 11:13:13 TP1] Load weight begin. avail mem=21.22 GB
[2025-03-02 11:13:13 TP0] Compute capability below sm80. Use float16 due to lack of bfloat16 support.
[2025-03-02 11:13:14 TP1] Compute capability below sm80. Use float16 due to lack of bfloat16 support.
Loading safetensors checkpoint shards: 0% Completed | 0/4 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 25% Completed | 1/4 [00:03<00:09, 3.01s/it]
Loading safetensors checkpoint shards: 50% Completed | 2/4 [00:05<00:05, 2.84s/it]
Loading safetensors checkpoint shards: 75% Completed | 3/4 [00:07<00:02, 2.41s/it]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:10<00:00, 2.46s/it]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:10<00:00, 2.54s/it]
25%|██▌ | 1/4 [01:14<03:42, 74.12s/it]AUTOTUNE mm(2x5120, 5120x76032)
triton_mm_8 1.3536 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_12 1.3568 ms 99.8% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_1 1.3617 ms 99.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
triton_mm_4 1.3619 ms 99.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
mm 1.3701 ms 98.8%
triton_mm_11 1.9755 ms 68.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_7 1.9820 ms 68.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_16 1.9845 ms 68.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_9 3.8741 ms 34.9% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_3 3.9499 ms 34.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
SingleProcess AUTOTUNE benchmarking takes 2.5025 seconds and 13.1718 seconds precompiling
AUTOTUNE mm(2x5120, 5120x76032)
triton_mm_8 1.4500 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_12 1.4602 ms 99.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_4 1.4602 ms 99.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
triton_mm_1 1.4603 ms 99.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
mm 1.4705 ms 98.6%
triton_mm_11 1.9123 ms 75.8% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_7 1.9185 ms 75.6% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_16 1.9212 ms 75.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_9 3.6953 ms 39.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_3 3.7787 ms 38.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
SingleProcess AUTOTUNE benchmarking takes 2.5036 seconds and 12.7857 seconds precompiling
50%|█████ | 2/4 [02:27<02:26, 73.43s/it]AUTOTUNE mm(4x5120, 5120x76032)
triton_mm_25 1.3589 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_29 1.3640 ms 99.6% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_18 1.3687 ms 99.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
triton_mm_21 1.3687 ms 99.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
mm 1.3794 ms 98.5%
triton_mm_28 1.9835 ms 68.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_24 1.9867 ms 68.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_33 1.9968 ms 68.1% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_26 3.8897 ms 34.9% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_32 3.9680 ms 34.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=8
SingleProcess AUTOTUNE benchmarking takes 2.5108 seconds and 12.2204 seconds precompiling
AUTOTUNE mm(4x5120, 5120x76032)
triton_mm_25 1.4561 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_29 1.4683 ms 99.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_21 1.4690 ms 99.1% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
triton_mm_18 1.4697 ms 99.1% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
mm 1.4807 ms 98.3%
triton_mm_28 1.9204 ms 75.8% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_24 1.9260 ms 75.6% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_33 1.9292 ms 75.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_26 3.7130 ms 39.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_27 3.7862 ms 38.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=8
SingleProcess AUTOTUNE benchmarking takes 2.4975 seconds and 13.5444 seconds precompiling
75%|███████▌ | 3/4 [03:43<01:14, 74.59s/it]AUTOTUNE mm(8x5120, 5120x76032)
triton_mm_42 1.3610 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_46 1.3636 ms 99.8% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_38 1.3784 ms 98.7% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
triton_mm_35 1.3786 ms 98.7% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
mm 1.3828 ms 98.4%
triton_mm_45 1.9866 ms 68.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_41 1.9927 ms 68.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_50 1.9969 ms 68.2% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_43 3.9324 ms 34.6% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_40 3.9873 ms 34.1% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=4
SingleProcess AUTOTUNE benchmarking takes 2.5369 seconds and 12.6361 seconds precompiling
AUTOTUNE mm(8x5120, 5120x76032)
triton_mm_42 1.4564 ms 100.0% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=4
triton_mm_46 1.4645 ms 99.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=4, num_warps=4
triton_mm_35 1.4783 ms 98.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=2, num_warps=2
triton_mm_38 1.4788 ms 98.5% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=128, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
mm 1.4831 ms 98.2%
triton_mm_45 1.9190 ms 75.9% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_41 1.9231 ms 75.7% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=64, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_50 1.9313 ms 75.4% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=64, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=8
triton_mm_43 3.7503 ms 38.8% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=128, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=3, num_warps=4
triton_mm_37 3.8034 ms 38.3% ACC_TYPE='tl.float32', ALLOW_TF32=False, BLOCK_K=32, BLOCK_M=16, BLOCK_N=32, B_PROLOGUE_CAST_TYPE=None, EVEN_K=True, GROUP_M=8, num_stages=5, num_warps=2
SingleProcess AUTOTUNE benchmarking takes 2.5470 seconds and 12.6454 seconds precompiling
100%|██████████| 4/4 [04:57<00:00, 74.44s/it]
[2025-03-02 11:18:29] INFO: Started server process [1]
[2025-03-02 11:18:29] INFO: Waiting for application startup.
[2025-03-02 11:18:29] INFO: Application startup complete.
[2025-03-02 11:18:29] INFO: Uvicorn running on http://0.0.0.0:30000 (Press CTRL+C to quit)
[2025-03-02 11:18:30] INFO: 127.0.0.1:54450 - "GET /get_model_info HTTP/1.1" 200 OK
[2025-03-02 11:18:30] Receive: obj=GenerateReqInput(text='The capital city of France is', input_ids=None, input_embeds=None, image_data=None, sampling_params={'temperature': 0, 'max_new_tokens': 8}, rid='722159a165c94dcea5daf10bb08df73d', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=False, stream=False, log_metrics=True, modalities=None, lora_path=None, session_params=None, custom_logit_processor=None)
[2025-03-02 11:18:39] Finish: obj=GenerateReqInput(text='The capital city of France is', input_ids=None, input_embeds=None, image_data=None, sampling_params={'temperature': 0, 'max_new_tokens': 8}, rid='722159a165c94dcea5daf10bb08df73d', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=False, stream=False, log_metrics=True, modalities=None, lora_path=None, session_params=None, custom_logit_processor=None), out={'text': '!!!!!!!!', 'meta_info': {'id': '722159a165c94dcea5daf10bb08df73d', 'finish_reason': {'type': 'length', 'length': 8}, 'prompt_tokens': 7, 'completion_tokens': 8, 'cached_tokens': 0}}
[2025-03-02 11:18:39] INFO: 127.0.0.1:54462 - "POST /generate HTTP/1.1" 200 OK
[2025-03-02 11:18:39] The server is fired up and ready to roll!
[2025-03-02 11:18:46] INFO: 127.0.0.1:44654 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:19:16] INFO: 127.0.0.1:40438 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:19:46] INFO: 127.0.0.1:38760 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:20:17] INFO: 127.0.0.1:44370 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:20:47] INFO: 127.0.0.1:56310 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:21:17] INFO: 127.0.0.1:50796 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:21:47] INFO: 127.0.0.1:47628 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:22:17] INFO: 127.0.0.1:57110 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:22:47] INFO: 127.0.0.1:59182 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:23:18] INFO: 127.0.0.1:39628 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:23:48] INFO: 127.0.0.1:35914 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:24:18] INFO: 127.0.0.1:52444 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:24:48] INFO: 127.0.0.1:51022 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:24:57] INFO: 10.248.155.225:46858 - "POST /v1/chat/completions HTTP/1.1" 200 OK
[2025-03-02 11:24:57] Receive: obj=GenerateReqInput(text=None, input_ids=[151646, 66365, 369, 1943, 304, 6605, 67914, 66365, 421, 6337, 7389, 323, 6605, 58, 15, 5352, 5778, 660, 961, 364, 8948, 6, 67914, 2979, 364, 8948, 1699, 565, 12329, 1124, 77, 1699, 4008, 25, 220, 56568, 101909, 27369, 100733, 100136, 102556, 9370, 110498, 3837, 107618, 101098, 106185, 20002, 86119, 3837, 99553, 102188, 9370, 101898, 33108, 101086, 3837, 100143, 20002, 18493, 101312, 102122, 15946, 100638, 99912, 86119, 1773, 220, 14880, 100470, 106466, 107494, 109504, 90395, 103944, 102104, 104542, 5373, 54926, 100116, 5373, 101137, 104913, 1773, 81263, 102031, 104552, 57191, 113272, 86119, 37945, 104137, 113272, 90395, 18493, 102618, 104705, 37029, 1565, 151648, 1699, 63, 32181, 249, 22243, 104107, 66898, 77, 6, 8096, 66365, 12330, 67914, 2979, 4605, 488, 1943, 677, 5778, 660, 488, 5196, 77, 6, 488, 1943, 677, 1796, 660, 488, 3355, 488, 5196, 77, 74491, 66365, 835, 1958, 67914, 66365, 421, 912, 64191, 61421, 67914, 2979, 364, 77091, 1699, 6, 8096, 66365, 12330, 1018, 630, 151644, 105043, 100165, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 56007, 2073, 105043, 100165, 11319, 33590, 35946, 49828, 102231, 105839, 99494, 102104, 1773, 101140, 3837, 35946, 99730, 100157, 99283, 3837, 66394, 104198, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 1773, 101889, 3837, 98237, 30534, 53481, 97611, 98380, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 104019, 104496, 35946, 107618, 100638, 100646, 86119, 3837, 100364, 20002, 60548, 88802, 90395, 100136, 100470, 105389, 20002, 104272, 1773, 100161, 3837, 104046, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 3837, 104011, 104552, 57191, 113272, 86119, 13343, 36993, 113177, 104107, 3837, 11622, 46100, 99922, 101987, 1773, 99654, 20002, 104077, 100011, 99794, 97611, 106712, 100772, 34187, 8997, 522, 14871, 1339, 104198, 33464, 39350, 10911, 16, 8125, 632, 12, 24625, 3837, 46944, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 3837, 100006, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 35946, 107618, 100638, 100646, 86119, 90395, 100364, 20002, 60548, 88802, 1773, 105351, 100470, 106466, 101214, 101882, 90395, 103944, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 1773, 62244, 104011, 85106, 104552, 100768, 57191, 102181, 113272, 103936, 3837, 105351, 104137, 104107, 62926, 101987, 97611, 113272, 100178, 1773, 151643, 151644, 107809, 106428, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 99601, 56007, 2073, 107809, 106428, 11319, 55807, 101140, 3837, 35946, 85106, 101128, 20002, 104378, 1773, 99650, 87267, 32664, 97611, 98380, 102342, 99794, 3837, 100631, 99172, 81167, 35946, 64471, 26232, 100364, 99650, 60548, 106168, 88802, 3407, 35946, 99730, 60726, 114116, 97611, 99558, 99788, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 101889, 3837, 100398, 66394, 110913, 60548, 108530, 3837, 101912, 27369, 51154, 5373, 104552, 100768, 5373, 104913, 113272, 5373, 102064, 105395, 5373, 108704, 43959, 49567, 1773, 103991, 99788, 99357, 18947, 103358, 3837, 101912, 104552, 100768, 73670, 49238, 23384, 38507, 3837, 105395, 73670, 100143, 15946, 82847, 99524, 102610, 3837, 108704, 43959, 73670, 61443, 82025, 3407, 104326, 3837, 35946, 85106, 104046, 97611, 100661, 3837, 101912, 111076, 5373, 101991, 33108, 104913, 33071, 1773, 91572, 3837, 66394, 105351, 100345, 20002, 104272, 101921, 102104, 3837, 103944, 101137, 104056, 100354, 3837, 104050, 102031, 104552, 57191, 113272, 13343, 3837, 36993, 104137, 101987, 104107, 100178, 3407, 100161, 3837, 102050, 100158, 110913, 103008, 100364, 101121, 3837, 115987, 99392, 99650, 73670, 105537, 35946, 100638, 100646, 86119, 1773, 99654, 20002, 104077, 101222, 29490, 99794, 97611, 98380, 33108, 99788, 3837, 106581, 105344, 100152, 35946, 36407, 60548, 104056, 88802, 8997, 522, 14871, 1339, 35946, 101909, 105600, 9370, 15469, 110498, 3837, 100006, 100364, 87026, 60548, 101312, 88802, 1773, 114566, 104786, 99604, 100650, 104426, 33108, 98380, 48443, 14374, 3070, 16, 13, 220, 27369, 51154, 57218, 106185, 1019, 12, 3070, 100032, 111436, 334, 5122, 99553, 101284, 100650, 9909, 29524, 99891, 5373, 99361, 5373, 99348, 5373, 100022, 49567, 7552, 9370, 105600, 100032, 8997, 12, 3070, 101349, 107462, 334, 5122, 100364, 48927, 27369, 9370, 111076, 8997, 12, 3070, 104136, 101290, 334, 5122, 104136, 102181, 101290, 57191, 116925, 3407, 14374, 3070, 17, 13, 47685, 47764, 100768, 57218, 104913, 113272, 1019, 12, 3070, 104552, 86119, 334, 5122, 100638, 30540, 8863, 5373, 111867, 5373, 103112, 5373, 100787, 49567, 104552, 86119, 8997, 12, 3070, 104913, 113272, 334, 5122, 71817, 104913, 113272, 33108, 86119, 100638, 8997, 12, 3070, 23384, 38507, 30918, 49238, 334, 5122, 49238, 23384, 38507, 57191, 16530, 49567, 28330, 3837, 29524, 105935, 23384, 38507, 5373, 43268, 33071, 23384, 38507, 40027, 49567, 3407, 14374, 3070, 18, 13, 40666, 248, 102064, 100143, 1019, 12, 3070, 102064, 105395, 334, 5122, 100143, 15946, 82847, 99524, 102610, 107768, 102064, 9909, 29524, 24339, 72881, 5373, 99462, 72881, 5373, 8903, 72881, 49567, 7552, 9370, 105395, 8997, 12, 3070, 102064, 100134, 334, 5122, 99553, 102064, 100134, 101898, 3837, 104136, 117206, 57191, 110376, 3407, 14374, 3070, 19, 13, 53040, 21894, 43959, 57218, 54542, 1019, 12, 3070, 108704, 104223, 334, 5122, 110479, 82025, 5373, 101108, 5373, 100370, 5373, 107604, 49567, 8997, 12, 3070, 106208, 43959, 334, 5122, 102050, 45861, 113773, 57191, 111116, 8997, 12, 3070, 22418, 61443, 103983, 334, 5122, 103983, 108704, 102124, 3837, 102989, 33126, 104542, 57191, 109946, 107616, 3407, 14374, 3070, 20, 13, 53497, 99, 99216, 57218, 99878, 100650, 1019, 12, 3070, 104380, 105293, 334, 5122, 100364, 110479, 104734, 5373, 102140, 100370, 5373, 108067, 99611, 41932, 49567, 8997, 12, 3070, 99878, 101898, 334, 5122, 18493, 110569, 5373, 70500, 5373, 99655, 106483, 99553, 101086, 3407, 14374, 3070, 21, 13, 220, 99605, 57218, 99424, 1019, 12, 3070, 99424, 101898, 334, 5122, 28072, 19564, 31338, 96422, 3837, 29524, 20450, 39352, 5373, 99722, 5373, 102275, 101039, 49567, 8997, 12, 3070, 100565, 101964, 334, 5122, 101914, 107041, 5373, 100378, 5373, 101941, 49567, 3407, 14374, 3070, 22, 13, 220, 46100, 57218, 110569, 1019, 12, 3070, 110569, 100364, 334, 5122, 104136, 110569, 101290, 5373, 104749, 46100, 32100, 5373, 99553, 46100, 19793, 26355, 8997, 12, 3070, 107018, 70500, 334, 5122, 105096, 70500] ... [104561, 100033, 9370, 32100, 1019, 256, 481, 3070, 102767, 115922, 118288, 334, 5122, 101059, 106455, 67338, 2073, 26288, 116037, 54899, 99832, 100044, 854, 100359, 101908, 109994, 100361, 3837, 77288, 101045, 47534, 115645, 104601, 101073, 109404, 100033, 100160, 8997, 256, 481, 3070, 32664, 100625, 9370, 118250, 334, 5122, 101059, 100140, 105125, 94299, 99191, 94299, 99351, 109055, 100625, 3837, 77288, 118250, 34187, 100625, 9370, 100159, 101157, 33108, 104304, 102575, 8997, 256, 481, 3070, 105211, 116477, 334, 5122, 101059, 18493, 109068, 100361, 5373, 58695, 5373, 109033, 105049, 91572, 105685, 3837, 100673, 116477, 33108, 85329, 105211, 3837, 101068, 101096, 101102, 104397, 99558, 102961, 3407, 820, 220, 18, 13, 3070, 99876, 104237, 9370, 107126, 1019, 256, 481, 3070, 94443, 24339, 60686, 99295, 104237, 9370, 101894, 334, 5122, 100625, 5373, 104025, 5373, 108245, 49567, 28404, 36885, 12857, 94443, 24339, 60686, 99295, 104237, 3837, 32664, 101059, 100355, 100011, 108889, 33108, 104561, 102262, 8997, 256, 481, 3070, 99698, 99292, 106567, 104629, 334, 5122, 99698, 99292, 67338, 99936, 109090, 9909, 29524, 114576, 99825, 109090, 5373, 100857, 99825, 109090, 7552, 104137, 113075, 109068, 9370, 101966, 40981, 3407, 820, 220, 19, 13, 3070, 99602, 57218, 104561, 101076, 9370, 104701, 1019, 256, 481, 3070, 104561, 101076, 105983, 334, 5122, 101059, 9370, 104410, 5373, 100960, 100031, 33108, 104729, 18493, 102111, 17447, 99427, 103108, 34204, 100625, 8997, 256, 481, 3070, 105438, 116737, 334, 5122, 101059, 9370, 101276, 100159, 101068, 57218, 100625, 9370, 2073, 101367, 99807, 104285, 854, 100218, 3837, 100673, 105438, 101102, 104052, 99250, 111509, 3407, 820, 220, 20, 13, 3070, 100156, 101950, 57218, 99253, 99180, 109797, 1019, 256, 481, 3070, 100156, 99346, 103985, 334, 5122, 104304, 100673, 101059, 100156, 99346, 109797, 3837, 104657, 112604, 3837, 105672, 99424, 99629, 99746, 8997, 256, 481, 3070, 99253, 99180, 99285, 99297, 334, 5122, 101067, 104304, 9370, 100652, 3837, 101059, 105591, 9370, 99253, 99180, 104052, 109797, 3837, 100156, 100143, 104304, 9370, 115174, 74763, 104052, 101190, 49238, 3407, 820, 220, 21, 13, 3070, 52129, 111529, 57218, 104304, 80565, 1019, 256, 481, 3070, 80942, 99825, 57218, 45861, 114379, 9370, 52129, 111529, 114511, 334, 5122, 16, 24, 19, 20, 7948, 3837, 100625, 32664, 80942, 99825, 33108, 45861, 114379, 79072, 16872, 52129, 111529, 3837, 108730, 104567, 104720, 33108, 99653, 112344, 8997, 256, 481, 3070, 108245, 9370, 73743, 99191, 334, 5122, 16, 24, 19, 20, 7948, 23, 9754, 3837, 108245, 32664, 101059, 104756, 33108, 106449, 109541, 102404, 105695, 3837, 104293, 34187, 101059, 9370, 113206, 3407, 44364, 14374, 3070, 40820, 5373, 101059, 106493, 100091, 18493, 105184, 104304, 15946, 2073, 117807, 19108, 97907, 99917, 56177, 820, 220, 16, 13, 3070, 30858, 69905, 99479, 100676, 104437, 1019, 256, 481, 3070, 104561, 104437, 334, 5122, 30858, 69905, 99479, 16628, 33447, 3837, 101059, 104015, 104415, 104561, 104437, 3837, 104809, 110414, 32108, 105591, 8997, 256, 481, 3070, 100159, 99896, 9370, 105185, 100641, 334, 5122, 101059, 67338, 104231, 104492, 99361, 3837, 105185, 104809, 100159, 99896, 3407, 820, 220, 17, 13, 3070, 99361, 57218, 106814, 100661, 1019, 256, 481, 3070, 104561, 99361, 334, 5122, 18493, 100043, 34324, 104304, 9909, 16, 23, 24, 19, 12, 16, 23, 24, 20, 7948, 7552, 33108, 8903, 100223, 104304, 9909, 16, 24, 15, 19, 12, 16, 24, 15, 20, 7948, 7552, 15946, 3837, 101059, 107359, 101075, 102830, 104561, 107772, 101076, 8997, 256, 481, 3070, 106814, 118280, 334, 5122, 101059, 105591, 18493, 106814, 17447, 107837, 112657, 118280, 33108, 104117, 99788, 3407, 820, 220, 18, 13, 3070, 102304, 9370, 118507, 1019, 256, 481, 3070, 112463, 9370, 105148, 334, 5122, 18493, 100043, 34324, 104304, 15946, 3837, 112463, 99563, 62112, 105148, 33108, 42192, 26232, 68536, 104015, 21125, 48309, 8997, 256, 481, 3070, 118924, 9370, 99427, 67364, 106499, 334, 5122, 18493, 8903, 100223, 104304, 15946, 3837, 118924, 18493, 99427, 67364, 105638, 104561, 102121, 33108, 110233, 101048, 47606, 101120, 86119, 3407, 820, 220, 19, 13, 3070, 99876, 99719, 9370, 113173, 1019, 256, 481, 3070, 31118, 99193, 9370, 99987, 99454, 334, 5122, 18493, 16, 24, 101186, 100072, 33108, 17, 15, 101186, 84607, 3837, 104492, 31118, 99193, 32664, 101059, 9370, 105390, 101070, 103975, 34187, 99987, 99454, 102316, 3837, 100140, 43288, 105767, 102243, 118924, 18493, 99427, 67364, 9370, 102793, 3407, 820, 220, 20, 13, 3070, 104304, 100176, 109413, 1019, 256, 481, 3070, 100361, 33071, 104304, 334, 5122, 100043, 34324, 104304, 33108, 8903, 100223, 104304, 71268, 100409, 100361, 33071, 104304, 3837, 101059, 73670, 101096, 101102, 104397, 106215, 102961, 8997, 256, 481, 3070, 94299, 99191, 94299, 99351, 334, 5122, 101059, 18493, 105184, 104304, 15946, 101910, 94299, 99191, 94299, 99351, 106814, 3837, 104015, 109055, 102304, 3837, 101153, 34187, 101930, 104304, 9370, 104997, 3407, 44364, 14374, 3070, 44991, 5373, 114872, 57218, 105184, 104304, 105386, 102665, 56177, 820, 220, 16, 13, 3070, 99876, 99719, 104896, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 9370, 105390, 101070, 101051, 31118, 99193, 9370, 99987, 99454, 24968, 104746, 114872, 15946, 3837, 101059, 9370, 105390, 115831, 32664, 99876, 104951, 9370, 101120, 105204, 3407, 820, 220, 17, 13, 3070, 104304, 100176, 57218, 102304, 101157, 1019, 256, 481, 220, 114872, 15946, 3837, 101059, 101928, 100146, 100625, 5373, 108245, 49567, 99489, 108388, 3837, 104304, 100176, 33108, 102660, 99427, 71304, 105184, 104304, 3407, 820, 220, 18, 13, 3070, 85329, 57218, 99346, 9370, 104701, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 9370, 85329, 33108, 99346, 99896, 99937, 112809, 104304, 100354, 24968, 104746, 114872, 15946, 3837, 85329, 112604, 33108, 99346, 109797, 99787, 101059, 22045, 105386, 100741, 3407, 820, 220, 19, 13, 3070, 100033, 100160, 9370, 115922, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 104820, 20412, 106304, 105390, 24968, 104746, 114872, 15946, 3837, 101059, 106455, 67338, 2073, 26288, 116037, 54899, 99832, 100044, 854, 100359, 101908, 109994, 100361, 3837, 100160, 102767, 115922, 118288, 3407, 44364, 14374, 3070, 63703, 5373, 102050, 56177, 101059, 106493, 100091, 18493, 114872, 101047, 22045, 20412, 42140, 99522, 100741, 100656, 100154, 105369, 3837, 100630, 85329, 118465, 5373, 100033, 32100, 5373, 99876, 107126, 5373, 100156, 101950, 49567, 1773, 117646, 3837, 101059, 18493, 105184, 104304, 15946, 100006, 2073, 117807, 19108, 33590, 104312, 99519, 41146, 102304, 9370, 118507, 5373, 99876, 99719, 9370, 113173, 101034, 104304, 100176, 9370, 104108, 1773, 103968, 3837, 114872, 9370, 22045, 108653, 101059, 106493, 100091, 9370, 112080, 3837, 109001, 99489, 104923, 110312, 107472, 5122, 114979, 105390, 103941, 44063, 104487, 22045, 3837, 68536, 104939, 57218, 99721, 101258, 103971, 104174, 88991, 101178, 1773, 151643, 151644, 118919, 114872, 101047, 105101, 94432, 151645], input_embeds=None, image_data=None, sampling_params={'temperature': 0.7, 'max_new_tokens': 32768, 'min_new_tokens': 0, 'stop': None, 'stop_token_ids': None, 'top_p': 1.0, 'top_k': -1, 'min_p': 0.0, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'repetition_penalty': 1.0, 'regex': None, 'ebnf': None, 'n': 1, 'no_stop_trim': False, 'ignore_eos': False, 'skip_special_tokens': True}, rid='cbae34093ab54868bafa60bc59c7f1ac', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=True, stream=True, log_metrics=True, modalities=[], lora_path=None, session_params=None, custom_logit_processor=None)
[2025-03-02 11:25:18] INFO: 127.0.0.1:53370 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:25:37] Finish: obj=GenerateReqInput(text=None, input_ids=[151646, 66365, 369, 1943, 304, 6605, 67914, 66365, 421, 6337, 7389, 323, 6605, 58, 15, 5352, 5778, 660, 961, 364, 8948, 6, 67914, 2979, 364, 8948, 1699, 565, 12329, 1124, 77, 1699, 4008, 25, 220, 56568, 101909, 27369, 100733, 100136, 102556, 9370, 110498, 3837, 107618, 101098, 106185, 20002, 86119, 3837, 99553, 102188, 9370, 101898, 33108, 101086, 3837, 100143, 20002, 18493, 101312, 102122, 15946, 100638, 99912, 86119, 1773, 220, 14880, 100470, 106466, 107494, 109504, 90395, 103944, 102104, 104542, 5373, 54926, 100116, 5373, 101137, 104913, 1773, 81263, 102031, 104552, 57191, 113272, 86119, 37945, 104137, 113272, 90395, 18493, 102618, 104705, 37029, 1565, 151648, 1699, 63, 32181, 249, 22243, 104107, 66898, 77, 6, 8096, 66365, 12330, 67914, 2979, 4605, 488, 1943, 677, 5778, 660, 488, 5196, 77, 6, 488, 1943, 677, 1796, 660, 488, 3355, 488, 5196, 77, 74491, 66365, 835, 1958, 67914, 66365, 421, 912, 64191, 61421, 67914, 2979, 364, 77091, 1699, 6, 8096, 66365, 12330, 1018, 630, 151644, 105043, 100165, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 56007, 2073, 105043, 100165, 11319, 33590, 35946, 49828, 102231, 105839, 99494, 102104, 1773, 101140, 3837, 35946, 99730, 100157, 99283, 3837, 66394, 104198, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 1773, 101889, 3837, 98237, 30534, 53481, 97611, 98380, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 104019, 104496, 35946, 107618, 100638, 100646, 86119, 3837, 100364, 20002, 60548, 88802, 90395, 100136, 100470, 105389, 20002, 104272, 1773, 100161, 3837, 104046, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 3837, 104011, 104552, 57191, 113272, 86119, 13343, 36993, 113177, 104107, 3837, 11622, 46100, 99922, 101987, 1773, 99654, 20002, 104077, 100011, 99794, 97611, 106712, 100772, 34187, 8997, 522, 14871, 1339, 104198, 33464, 39350, 10911, 16, 8125, 632, 12, 24625, 3837, 46944, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 3837, 100006, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 35946, 107618, 100638, 100646, 86119, 90395, 100364, 20002, 60548, 88802, 1773, 105351, 100470, 106466, 101214, 101882, 90395, 103944, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 1773, 62244, 104011, 85106, 104552, 100768, 57191, 102181, 113272, 103936, 3837, 105351, 104137, 104107, 62926, 101987, 97611, 113272, 100178, 1773, 151643, 151644, 107809, 106428, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 99601, 56007, 2073, 107809, 106428, 11319, 55807, 101140, 3837, 35946, 85106, 101128, 20002, 104378, 1773, 99650, 87267, 32664, 97611, 98380, 102342, 99794, 3837, 100631, 99172, 81167, 35946, 64471, 26232, 100364, 99650, 60548, 106168, 88802, 3407, 35946, 99730, 60726, 114116, 97611, 99558, 99788, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 101889, 3837, 100398, 66394, 110913, 60548, 108530, 3837, 101912, 27369, 51154, 5373, 104552, 100768, 5373, 104913, 113272, 5373, 102064, 105395, 5373, 108704, 43959, 49567, 1773, 103991, 99788, 99357, 18947, 103358, 3837, 101912, 104552, 100768, 73670, 49238, 23384, 38507, 3837, 105395, 73670, 100143, 15946, 82847, 99524, 102610, 3837, 108704, 43959, 73670, 61443, 82025, 3407, 104326, 3837, 35946, 85106, 104046, 97611, 100661, 3837, 101912, 111076, 5373, 101991, 33108, 104913, 33071, 1773, 91572, 3837, 66394, 105351, 100345, 20002, 104272, 101921, 102104, 3837, 103944, 101137, 104056, 100354, 3837, 104050, 102031, 104552, 57191, 113272, 13343, 3837, 36993, 104137, 101987, 104107, 100178, 3407, 100161, 3837, 102050, 100158, 110913, 103008, 100364, 101121, 3837, 115987, 99392, 99650, 73670, 105537, 35946, 100638, 100646, 86119, 1773, 99654, 20002, 104077, 101222, 29490, 99794, 97611, 98380, 33108, 99788, 3837, 106581, 105344, 100152, 35946, 36407, 60548, 104056, 88802, 8997, 522, 14871, 1339, 35946, 101909, 105600, 9370, 15469, 110498, 3837, 100006, 100364, 87026, 60548, 101312, 88802, 1773, 114566, 104786, 99604, 100650, 104426, 33108, 98380, 48443, 14374, 3070, 16, 13, 220, 27369, 51154, 57218, 106185, 1019, 12, 3070, 100032, 111436, 334, 5122, 99553, 101284, 100650, 9909, 29524, 99891, 5373, 99361, 5373, 99348, 5373, 100022, 49567, 7552, 9370, 105600, 100032, 8997, 12, 3070, 101349, 107462, 334, 5122, 100364, 48927, 27369, 9370, 111076, 8997, 12, 3070, 104136, 101290, 334, 5122, 104136, 102181, 101290, 57191, 116925, 3407, 14374, 3070, 17, 13, 47685, 47764, 100768, 57218, 104913, 113272, 1019, 12, 3070, 104552, 86119, 334, 5122, 100638, 30540, 8863, 5373, 111867, 5373, 103112, 5373, 100787, 49567, 104552, 86119, 8997, 12, 3070, 104913, 113272, 334, 5122, 71817, 104913, 113272, 33108, 86119, 100638, 8997, 12, 3070, 23384, 38507, 30918, 49238, 334, 5122, 49238, 23384, 38507, 57191, 16530, 49567, 28330, 3837, 29524, 105935, 23384, 38507, 5373, 43268, 33071, 23384, 38507, 40027, 49567, 3407, 14374, 3070, 18, 13, 40666, 248, 102064, 100143, 1019, 12, 3070, 102064, 105395, 334, 5122, 100143, 15946, 82847, 99524, 102610, 107768, 102064, 9909, 29524, 24339, 72881, 5373, 99462, 72881, 5373, 8903, 72881, 49567, 7552, 9370, 105395, 8997, 12, 3070, 102064, 100134, 334, 5122, 99553, 102064, 100134, 101898, 3837, 104136, 117206, 57191, 110376, 3407, 14374, 3070, 19, 13, 53040, 21894, 43959, 57218, 54542, 1019, 12, 3070, 108704, 104223, 334, 5122, 110479, 82025, 5373, 101108, 5373, 100370, 5373, 107604, 49567, 8997, 12, 3070, 106208, 43959, 334, 5122, 102050, 45861, 113773, 57191, 111116, 8997, 12, 3070, 22418, 61443, 103983, 334, 5122, 103983, 108704, 102124, 3837, 102989, 33126, 104542, 57191, 109946, 107616, 3407, 14374, 3070, 20, 13, 53497, 99, 99216, 57218, 99878, 100650, 1019, 12, 3070, 104380, 105293, 334, 5122, 100364, 110479, 104734, 5373, 102140, 100370, 5373, 108067, 99611, 41932, 49567, 8997, 12, 3070, 99878, 101898, 334, 5122, 18493, 110569, 5373, 70500, 5373, 99655, 106483, 99553, 101086, 3407, 14374, 3070, 21, 13, 220, 99605, 57218, 99424, 1019, 12, 3070, 99424, 101898, 334, 5122, 28072, 19564, 31338, 96422, 3837, 29524, 20450, 39352, 5373, 99722, 5373, 102275, 101039, 49567, 8997, 12, 3070, 100565, 101964, 334, 5122, 101914, 107041, 5373, 100378, 5373, 101941, 49567, 3407, 14374, 3070, 22, 13, 220, 46100, 57218, 110569, 1019, 12, 3070, 110569, 100364, 334, 5122, 104136, 110569, 101290, 5373, 104749, 46100, 32100, 5373, 99553, 46100, 19793, 26355, 8997, 12, 3070, 107018, 70500, 334, 5122, 105096, 70500] ... [104561, 100033, 9370, 32100, 1019, 256, 481, 3070, 102767, 115922, 118288, 334, 5122, 101059, 106455, 67338, 2073, 26288, 116037, 54899, 99832, 100044, 854, 100359, 101908, 109994, 100361, 3837, 77288, 101045, 47534, 115645, 104601, 101073, 109404, 100033, 100160, 8997, 256, 481, 3070, 32664, 100625, 9370, 118250, 334, 5122, 101059, 100140, 105125, 94299, 99191, 94299, 99351, 109055, 100625, 3837, 77288, 118250, 34187, 100625, 9370, 100159, 101157, 33108, 104304, 102575, 8997, 256, 481, 3070, 105211, 116477, 334, 5122, 101059, 18493, 109068, 100361, 5373, 58695, 5373, 109033, 105049, 91572, 105685, 3837, 100673, 116477, 33108, 85329, 105211, 3837, 101068, 101096, 101102, 104397, 99558, 102961, 3407, 820, 220, 18, 13, 3070, 99876, 104237, 9370, 107126, 1019, 256, 481, 3070, 94443, 24339, 60686, 99295, 104237, 9370, 101894, 334, 5122, 100625, 5373, 104025, 5373, 108245, 49567, 28404, 36885, 12857, 94443, 24339, 60686, 99295, 104237, 3837, 32664, 101059, 100355, 100011, 108889, 33108, 104561, 102262, 8997, 256, 481, 3070, 99698, 99292, 106567, 104629, 334, 5122, 99698, 99292, 67338, 99936, 109090, 9909, 29524, 114576, 99825, 109090, 5373, 100857, 99825, 109090, 7552, 104137, 113075, 109068, 9370, 101966, 40981, 3407, 820, 220, 19, 13, 3070, 99602, 57218, 104561, 101076, 9370, 104701, 1019, 256, 481, 3070, 104561, 101076, 105983, 334, 5122, 101059, 9370, 104410, 5373, 100960, 100031, 33108, 104729, 18493, 102111, 17447, 99427, 103108, 34204, 100625, 8997, 256, 481, 3070, 105438, 116737, 334, 5122, 101059, 9370, 101276, 100159, 101068, 57218, 100625, 9370, 2073, 101367, 99807, 104285, 854, 100218, 3837, 100673, 105438, 101102, 104052, 99250, 111509, 3407, 820, 220, 20, 13, 3070, 100156, 101950, 57218, 99253, 99180, 109797, 1019, 256, 481, 3070, 100156, 99346, 103985, 334, 5122, 104304, 100673, 101059, 100156, 99346, 109797, 3837, 104657, 112604, 3837, 105672, 99424, 99629, 99746, 8997, 256, 481, 3070, 99253, 99180, 99285, 99297, 334, 5122, 101067, 104304, 9370, 100652, 3837, 101059, 105591, 9370, 99253, 99180, 104052, 109797, 3837, 100156, 100143, 104304, 9370, 115174, 74763, 104052, 101190, 49238, 3407, 820, 220, 21, 13, 3070, 52129, 111529, 57218, 104304, 80565, 1019, 256, 481, 3070, 80942, 99825, 57218, 45861, 114379, 9370, 52129, 111529, 114511, 334, 5122, 16, 24, 19, 20, 7948, 3837, 100625, 32664, 80942, 99825, 33108, 45861, 114379, 79072, 16872, 52129, 111529, 3837, 108730, 104567, 104720, 33108, 99653, 112344, 8997, 256, 481, 3070, 108245, 9370, 73743, 99191, 334, 5122, 16, 24, 19, 20, 7948, 23, 9754, 3837, 108245, 32664, 101059, 104756, 33108, 106449, 109541, 102404, 105695, 3837, 104293, 34187, 101059, 9370, 113206, 3407, 44364, 14374, 3070, 40820, 5373, 101059, 106493, 100091, 18493, 105184, 104304, 15946, 2073, 117807, 19108, 97907, 99917, 56177, 820, 220, 16, 13, 3070, 30858, 69905, 99479, 100676, 104437, 1019, 256, 481, 3070, 104561, 104437, 334, 5122, 30858, 69905, 99479, 16628, 33447, 3837, 101059, 104015, 104415, 104561, 104437, 3837, 104809, 110414, 32108, 105591, 8997, 256, 481, 3070, 100159, 99896, 9370, 105185, 100641, 334, 5122, 101059, 67338, 104231, 104492, 99361, 3837, 105185, 104809, 100159, 99896, 3407, 820, 220, 17, 13, 3070, 99361, 57218, 106814, 100661, 1019, 256, 481, 3070, 104561, 99361, 334, 5122, 18493, 100043, 34324, 104304, 9909, 16, 23, 24, 19, 12, 16, 23, 24, 20, 7948, 7552, 33108, 8903, 100223, 104304, 9909, 16, 24, 15, 19, 12, 16, 24, 15, 20, 7948, 7552, 15946, 3837, 101059, 107359, 101075, 102830, 104561, 107772, 101076, 8997, 256, 481, 3070, 106814, 118280, 334, 5122, 101059, 105591, 18493, 106814, 17447, 107837, 112657, 118280, 33108, 104117, 99788, 3407, 820, 220, 18, 13, 3070, 102304, 9370, 118507, 1019, 256, 481, 3070, 112463, 9370, 105148, 334, 5122, 18493, 100043, 34324, 104304, 15946, 3837, 112463, 99563, 62112, 105148, 33108, 42192, 26232, 68536, 104015, 21125, 48309, 8997, 256, 481, 3070, 118924, 9370, 99427, 67364, 106499, 334, 5122, 18493, 8903, 100223, 104304, 15946, 3837, 118924, 18493, 99427, 67364, 105638, 104561, 102121, 33108, 110233, 101048, 47606, 101120, 86119, 3407, 820, 220, 19, 13, 3070, 99876, 99719, 9370, 113173, 1019, 256, 481, 3070, 31118, 99193, 9370, 99987, 99454, 334, 5122, 18493, 16, 24, 101186, 100072, 33108, 17, 15, 101186, 84607, 3837, 104492, 31118, 99193, 32664, 101059, 9370, 105390, 101070, 103975, 34187, 99987, 99454, 102316, 3837, 100140, 43288, 105767, 102243, 118924, 18493, 99427, 67364, 9370, 102793, 3407, 820, 220, 20, 13, 3070, 104304, 100176, 109413, 1019, 256, 481, 3070, 100361, 33071, 104304, 334, 5122, 100043, 34324, 104304, 33108, 8903, 100223, 104304, 71268, 100409, 100361, 33071, 104304, 3837, 101059, 73670, 101096, 101102, 104397, 106215, 102961, 8997, 256, 481, 3070, 94299, 99191, 94299, 99351, 334, 5122, 101059, 18493, 105184, 104304, 15946, 101910, 94299, 99191, 94299, 99351, 106814, 3837, 104015, 109055, 102304, 3837, 101153, 34187, 101930, 104304, 9370, 104997, 3407, 44364, 14374, 3070, 44991, 5373, 114872, 57218, 105184, 104304, 105386, 102665, 56177, 820, 220, 16, 13, 3070, 99876, 99719, 104896, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 9370, 105390, 101070, 101051, 31118, 99193, 9370, 99987, 99454, 24968, 104746, 114872, 15946, 3837, 101059, 9370, 105390, 115831, 32664, 99876, 104951, 9370, 101120, 105204, 3407, 820, 220, 17, 13, 3070, 104304, 100176, 57218, 102304, 101157, 1019, 256, 481, 220, 114872, 15946, 3837, 101059, 101928, 100146, 100625, 5373, 108245, 49567, 99489, 108388, 3837, 104304, 100176, 33108, 102660, 99427, 71304, 105184, 104304, 3407, 820, 220, 18, 13, 3070, 85329, 57218, 99346, 9370, 104701, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 9370, 85329, 33108, 99346, 99896, 99937, 112809, 104304, 100354, 24968, 104746, 114872, 15946, 3837, 85329, 112604, 33108, 99346, 109797, 99787, 101059, 22045, 105386, 100741, 3407, 820, 220, 19, 13, 3070, 100033, 100160, 9370, 115922, 1019, 256, 481, 220, 105184, 104304, 15946, 3837, 101059, 104820, 20412, 106304, 105390, 24968, 104746, 114872, 15946, 3837, 101059, 106455, 67338, 2073, 26288, 116037, 54899, 99832, 100044, 854, 100359, 101908, 109994, 100361, 3837, 100160, 102767, 115922, 118288, 3407, 44364, 14374, 3070, 63703, 5373, 102050, 56177, 101059, 106493, 100091, 18493, 114872, 101047, 22045, 20412, 42140, 99522, 100741, 100656, 100154, 105369, 3837, 100630, 85329, 118465, 5373, 100033, 32100, 5373, 99876, 107126, 5373, 100156, 101950, 49567, 1773, 117646, 3837, 101059, 18493, 105184, 104304, 15946, 100006, 2073, 117807, 19108, 33590, 104312, 99519, 41146, 102304, 9370, 118507, 5373, 99876, 99719, 9370, 113173, 101034, 104304, 100176, 9370, 104108, 1773, 103968, 3837, 114872, 9370, 22045, 108653, 101059, 106493, 100091, 9370, 112080, 3837, 109001, 99489, 104923, 110312, 107472, 5122, 114979, 105390, 103941, 44063, 104487, 22045, 3837, 68536, 104939, 57218, 99721, 101258, 103971, 104174, 88991, 101178, 1773, 151643, 151644, 118919, 114872, 101047, 105101, 94432, 151645], input_embeds=None, image_data=None, sampling_params={'temperature': 0.7, 'max_new_tokens': 32768, 'min_new_tokens': 0, 'stop': None, 'stop_token_ids': None, 'top_p': 1.0, 'top_k': -1, 'min_p': 0.0, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'repetition_penalty': 1.0, 'regex': None, 'ebnf': None, 'n': 1, 'no_stop_trim': False, 'ignore_eos': False, 'skip_special_tokens': True}, rid='cbae34093ab54868bafa60bc59c7f1ac', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=True, stream=True, log_metrics=True, modalities=[], lora_path=None, session_params=None, custom_logit_processor=None), out={'text': '\n嗯，用户现在问“如何看待二战中的意大利？”根据之前的对话，用户对二战时期的法国、德国、日本的历史有了解，现在转向分析意大利在二战中的角色。用户可能在学习二战历史，或者对意大利在战争中的地位和影响感兴趣。\n\n我需要确定用户的需求是什么。他们可能想知道意大利在二战中的作用、失败原因，或者是意大利在轴心国中的地位。考虑到用户之前的互动，他们可能希望得到全面且结构清晰的分析。\n\n接下来，我应该考虑意大利在二战中的关键事件和角色。意大利最初是轴心国的一部分，但在战争后期被盟军击败，国内法西斯政权也被推翻。我需要分析意大利加入轴心国的原因、其在北非和地中海的军事行动、国内抵抗运动，以及墨索里尼的统治等。\n\n我还需要考虑意大利失败的原因，如军事实力不足、国内支持度下降、盟军的胜利等。此外，意大利在二战中的道德和政治影响，如法西斯主义的失败，也是重要的分析点。\n\n用户可能对意大利在二战中的具体战役感兴趣，比如在北非的失败、在地中海的抵抗等。因此，我需要确保内容涵盖这些方面，并提供足够的细节。\n\n最后，我需要确保内容的准确性和客观性，提供中立的分析，帮助用户全面理解意大利在二战中的角色和影响。\n\n总的来说，我需要组织一个结构清晰、内容全面的回答，涵盖意大利加入轴心国的原因、军事行动、失败原因、国内抵抗运动、墨索里尼的统治等多方面的因素，帮助用户深入理解意大利在二战中的地位和影响。\n\n\n### ���战中的意大利：一个复杂的历史角色\n\n意大利在二战中的角色是一个复杂的话题。作为法西斯主义的发源地之一，意大利在二战中既是轴心国的重要成员，又因内部矛盾和外部压力最终走向失败。以下是对二战中意大利的全面分析：\n\n---\n\n### 一、意大利加入轴心国的原因\n\n1. 法西斯主义的扩张野心：\n - 意大利法西斯领导人贝尼托· Mussolini（墨索里尼）深受希特勒的启发，试图通过扩张实现“大意大利”的目标。\n - 意大利希望通过战争恢复其作为欧洲大国的地位，并在国际事务中获得更大的发言权。\n\n2. 经济与政治动机：\n - 意大利在20世纪初经济困难，法西斯政权希望通过战争刺激经济，解决失业问题。\n - 墨索里尼希望通过战争转移国内矛盾，巩固其统治地位。\n\n3. 与德国的联盟：\n - 意大利与德国结盟是基于共同的法西斯意识形态和对英法的敌视。\n - 墨' ... '大利派兵参与东线战场（苏德战场），但表现不佳，最终成为苏联红军的俘虏。\n\n4. 国内抵抗运动：\n - 意大利国内存在强大的抵抗运动，反对法西斯政权和德国的占领。\n - 抗击运动在二战后期发挥了重要作用，最终推翻了法西斯政权。\n\n---\n\n### 三、意大利失败的原因\n\n1. 军事实力不足：\n - 意大利的军事装备和战略水平远逊于德国和盟军，无法支撑长期战争的需求。\n - 意大利的海军和空军在战争中表现平平，无法对盟军构成实质威胁。\n\n2. 国内支持度下降：\n - 意大利国内对战争的支持逐渐瓦解，民众因战争带来的苦难而对法西斯政权失去信心。\n\n3. 盟军的压力：\n - 盟军在北非、地中海和意大利本土的胜利，迫使意大利陷入全面崩溃。\n\n4. 轴心国的失败：\n - 随着德国在东线战场的失败，意大利失去了最重要的盟友，最终走向灭亡。\n\n---\n\n### 四、墨索里尼的统治与失败\n\n1. 法西斯政权的专制：\n - 墨索里尼通过恐怖统治巩固其权力，但这种统治方式无法获得国内的长期支持。\n - 法西斯政权的腐败和低效也成为其失败的重要原因。\n\n2. 墨索里尼的个人失误：\n - 墨索里尼在战略决策上犯下严重错误，例如参战时机和战场选择。\n - 他过于依赖德国，忽视了意大利自身的利益。\n\n3. 法西斯主义的失败：\n - 意大利的失败标志着法西斯主义的彻底失败，也为世界敲响了警钟。\n\n---\n\n### 五、二战后意大利的反思与重建\n\n1. 战后审判与民主化：\n - 意大利在二战后废除了法西斯政权，建立了民主共和国。\n - 意大利社会进行了深刻的反思，试图从法西斯主义的阴影中走出来。\n\n2. 经济重建与欧洲一体化：\n - 意大利在战后通过马歇尔计划等外援实现了经济复苏。\n - 意大利成为欧洲经济共同体的创始成员国，推动了欧洲一体化进程。\n\n---\n\n### 六、总结\n\n意大利在二战中的角色是一个复杂的历史案例。它既是一个法西斯主义的代表，也是一个失败的轴心国成员。意大利的失败不仅是军事和经济的原因，更是其法西斯政权和扩张野心的必然结果。意大利的战后反思和重建也为世界提供了重要的经验教训。通过这段历史，我们可以更深入地理解法西斯主义的危害，以及和平与民主的重要性。', 'meta_info': {'id': 'cbae34093ab54868bafa60bc59c7f1ac', 'finish_reason': {'type': 'stop', 'matched': 151643}, 'prompt_tokens': 9649, 'completion_tokens': 1373, 'cached_tokens': 0}}
[2025-03-02 11:25:48] INFO: 127.0.0.1:58810 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:26:10] INFO: 10.248.155.225:57312 - "POST /v1/chat/completions HTTP/1.1" 200 OK
[2025-03-02 11:26:10] Receive: obj=GenerateReqInput(text=None, input_ids=[151646, 66365, 369, 1943, 304, 6605, 67914, 66365, 421, 6337, 7389, 323, 6605, 58, 15, 5352, 5778, 660, 961, 364, 8948, 6, 67914, 2979, 364, 8948, 1699, 565, 12329, 1124, 77, 1699, 4008, 25, 220, 56568, 101909, 27369, 100733, 100136, 102556, 9370, 110498, 3837, 107618, 101098, 106185, 20002, 86119, 3837, 99553, 102188, 9370, 101898, 33108, 101086, 3837, 100143, 20002, 18493, 101312, 102122, 15946, 100638, 99912, 86119, 1773, 220, 14880, 100470, 106466, 107494, 109504, 90395, 103944, 102104, 104542, 5373, 54926, 100116, 5373, 101137, 104913, 1773, 81263, 102031, 104552, 57191, 113272, 86119, 37945, 104137, 113272, 90395, 18493, 102618, 104705, 37029, 1565, 151648, 1699, 63, 32181, 249, 22243, 104107, 66898, 77, 6, 8096, 66365, 12330, 67914, 2979, 4605, 488, 1943, 677, 5778, 660, 488, 5196, 77, 6, 488, 1943, 677, 1796, 660, 488, 3355, 488, 5196, 77, 74491, 66365, 835, 1958, 67914, 66365, 421, 912, 64191, 61421, 67914, 2979, 364, 77091, 1699, 6, 8096, 66365, 12330, 1018, 630, 151644, 105043, 100165, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 56007, 2073, 105043, 100165, 11319, 33590, 35946, 49828, 102231, 105839, 99494, 102104, 1773, 101140, 3837, 35946, 99730, 100157, 99283, 3837, 66394, 104198, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 1773, 101889, 3837, 98237, 30534, 53481, 97611, 98380, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 104019, 104496, 35946, 107618, 100638, 100646, 86119, 3837, 100364, 20002, 60548, 88802, 90395, 100136, 100470, 105389, 20002, 104272, 1773, 100161, 3837, 104046, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 3837, 104011, 104552, 57191, 113272, 86119, 13343, 36993, 113177, 104107, 3837, 11622, 46100, 99922, 101987, 1773, 99654, 20002, 104077, 100011, 99794, 97611, 106712, 100772, 34187, 8997, 522, 14871, 1339, 104198, 33464, 39350, 10911, 16, 8125, 632, 12, 24625, 3837, 46944, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 3837, 100006, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 35946, 107618, 100638, 100646, 86119, 90395, 100364, 20002, 60548, 88802, 1773, 105351, 100470, 106466, 101214, 101882, 90395, 103944, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 1773, 62244, 104011, 85106, 104552, 100768, 57191, 102181, 113272, 103936, 3837, 105351, 104137, 104107, 62926, 101987, 97611, 113272, 100178, 1773, 151643, 151644, 107809, 106428, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 99601, 56007, 2073, 107809, 106428, 11319, 55807, 101140, 3837, 35946, 85106, 101128, 20002, 104378, 1773, 99650, 87267, 32664, 97611, 98380, 102342, 99794, 3837, 100631, 99172, 81167, 35946, 64471, 26232, 100364, 99650, 60548, 106168, 88802, 3407, 35946, 99730, 60726, 114116, 97611, 99558, 99788, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 101889, 3837, 100398, 66394, 110913, 60548, 108530, 3837, 101912, 27369, 51154, 5373, 104552, 100768, 5373, 104913, 113272, 5373, 102064, 105395, 5373, 108704, 43959, 49567, 1773, 103991, 99788, 99357, 18947, 103358, 3837, 101912, 104552, 100768, 73670, 49238, 23384, 38507, 3837, 105395, 73670, 100143, 15946, 82847, 99524, 102610, 3837, 108704, 43959, 73670, 61443, 82025, 3407, 104326, 3837, 35946, 85106, 104046, 97611, 100661, 3837, 101912, 111076, 5373, 101991, 33108, 104913, 33071, 1773, 91572, 3837, 66394, 105351, 100345, 20002, 104272, 101921, 102104, 3837, 103944, 101137, 104056, 100354, 3837, 104050, 102031, 104552, 57191, 113272, 13343, 3837, 36993, 104137, 101987, 104107, 100178, 3407, 100161, 3837, 102050, 100158, 110913, 103008, 100364, 101121, 3837, 115987, 99392, 99650, 73670, 105537, 35946, 100638, 100646, 86119, 1773, 99654, 20002, 104077, 101222, 29490, 99794, 97611, 98380, 33108, 99788, 3837, 106581, 105344, 100152, 35946, 36407, 60548, 104056, 88802, 8997, 522, 14871, 1339, 35946, 101909, 105600, 9370, 15469, 110498, 3837, 100006, 100364, 87026, 60548, 101312, 88802, 1773, 114566, 104786, 99604, 100650, 104426, 33108, 98380, 48443, 14374, 3070, 16, 13, 220, 27369, 51154, 57218, 106185, 1019, 12, 3070, 100032, 111436, 334, 5122, 99553, 101284, 100650, 9909, 29524, 99891, 5373, 99361, 5373, 99348, 5373, 100022, 49567, 7552, 9370, 105600, 100032, 8997, 12, 3070, 101349, 107462, 334, 5122, 100364, 48927, 27369, 9370, 111076, 8997, 12, 3070, 104136, 101290, 334, 5122, 104136, 102181, 101290, 57191, 116925, 3407, 14374, 3070, 17, 13, 47685, 47764, 100768, 57218, 104913, 113272, 1019, 12, 3070, 104552, 86119, 334, 5122, 100638, 30540, 8863, 5373, 111867, 5373, 103112, 5373, 100787, 49567, 104552, 86119, 8997, 12, 3070, 104913, 113272, 334, 5122, 71817, 104913, 113272, 33108, 86119, 100638, 8997, 12, 3070, 23384, 38507, 30918, 49238, 334, 5122, 49238, 23384, 38507, 57191, 16530, 49567, 28330, 3837, 29524, 105935, 23384, 38507, 5373, 43268, 33071, 23384, 38507, 40027, 49567, 3407, 14374, 3070, 18, 13, 40666, 248, 102064, 100143, 1019, 12, 3070, 102064, 105395, 334, 5122, 100143, 15946, 82847, 99524, 102610, 107768, 102064, 9909, 29524, 24339, 72881, 5373, 99462, 72881, 5373, 8903, 72881, 49567, 7552, 9370, 105395, 8997, 12, 3070, 102064, 100134, 334, 5122, 99553, 102064, 100134, 101898, 3837, 104136, 117206, 57191, 110376, 3407, 14374, 3070, 19, 13, 53040, 21894, 43959, 57218, 54542, 1019, 12, 3070, 108704, 104223, 334, 5122, 110479, 82025, 5373, 101108, 5373, 100370, 5373, 107604, 49567, 8997, 12, 3070, 106208, 43959, 334, 5122, 102050, 45861, 113773, 57191, 111116, 8997, 12, 3070, 22418, 61443, 103983, 334, 5122, 103983, 108704, 102124, 3837, 102989, 33126, 104542, 57191, 109946, 107616, 3407, 14374, 3070, 20, 13, 53497, 99, 99216, 57218, 99878, 100650, 1019, 12, 3070, 104380, 105293, 334, 5122, 100364, 110479, 104734, 5373, 102140, 100370, 5373, 108067, 99611, 41932, 49567, 8997, 12, 3070, 99878, 101898, 334, 5122, 18493, 110569, 5373, 70500, 5373, 99655, 106483, 99553, 101086, 3407, 14374, 3070, 21, 13, 220, 99605, 57218, 99424, 1019, 12, 3070, 99424, 101898, 334, 5122, 28072, 19564, 31338, 96422, 3837, 29524, 20450, 39352, 5373, 99722, 5373, 102275, 101039, 49567, 8997, 12, 3070, 100565, 101964, 334, 5122, 101914, 107041, 5373, 100378, 5373, 101941, 49567, 3407, 14374, 3070, 22, 13, 220, 46100, 57218, 110569, 1019, 12, 3070, 110569, 100364, 334, 5122, 104136, 110569, 101290, 5373, 104749, 46100, 32100, 5373, 99553, 46100, 19793, 26355, 8997, 12, 3070, 107018, 70500, 334, 5122, 105096, 70500] ... [24339, 60686, 99295, 100091, 9370, 28291, 37984, 29490, 100653, 3837, 105101, 18493, 114872, 15946, 107571, 102390, 63109, 28404, 101945, 100450, 3837, 99518, 62112, 101979, 104409, 33108, 105413, 101950, 103941, 104487, 22045, 1773, 87752, 106273, 114872, 15946, 105101, 9370, 100011, 101042, 48443, 44364, 14374, 3070, 14777, 5373, 105101, 101963, 102390, 63109, 28404, 104249, 56177, 16, 13, 3070, 24339, 60686, 99295, 100091, 9370, 105390, 115922, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 24339, 60686, 99295, 107672, 100458, 99685, 99829, 13935, 89714, 94794, 9909, 101241, 50984, 69249, 99685, 7552, 106902, 99658, 65278, 104240, 9370, 110502, 3837, 106455, 67338, 105390, 101884, 2073, 26288, 105101, 97907, 100160, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 111789, 104304, 102005, 41146, 100622, 104123, 107944, 106681, 90395, 18493, 99876, 101436, 15946, 100350, 106155, 102588, 40981, 3407, 17, 13, 3070, 99346, 57218, 101091, 111443, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 17, 15, 101186, 84607, 99346, 103985, 3837, 24339, 60686, 99295, 110394, 111789, 104304, 104313, 99346, 3837, 100638, 108127, 86119, 8997, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 111789, 104304, 104651, 100156, 104409, 3837, 105749, 41146, 107632, 101262, 3407, 18, 13, 3070, 57218, 104114, 9370, 104237, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 57218, 104114, 36885, 99698, 20412, 104210, 100656, 9370, 24339, 60686, 99295, 115884, 33108, 32664, 82847, 24339, 9370, 101271, 57452, 8997, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 106455, 67338, 57218, 104114, 99721, 3837, 100350, 102075, 100033, 85329, 33108, 116275, 3407, 44364, 14374, 3070, 40820, 5373, 105101, 18493, 114872, 101047, 100780, 56177, 16, 13, 3070, 48309, 65676, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 48309, 65676, 9909, 59532, 103451, 5373, 109943, 7552, 102121, 34187, 100722, 105591, 3837, 106455, 100359, 99908, 100208, 99253, 109440, 8997, 256, 481, 220, 16, 24, 19, 16, 7948, 3837, 105101, 105591, 18493, 99829, 101755, 100221, 99316, 99250, 82847, 99292, 109055, 3837, 108653, 41146, 18493, 48309, 65676, 9370, 22045, 3407, 17, 13, 3070, 117564, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 106700, 106455, 100359, 117564, 3837, 77288, 62112, 101076, 105983, 33108, 100033, 108452, 3837, 104183, 99250, 99698, 99292, 109055, 8997, 256, 481, 220, 16, 24, 19, 18, 7948, 3837, 99698, 99292, 109788, 60686, 60686, 69249, 99825, 3837, 100673, 105101, 100156, 108322, 109797, 3407, 18, 13, 3070, 67364, 43268, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 99890, 99807, 100651, 67364, 43268, 102961, 9909, 99908, 99462, 102961, 48272, 77288, 101107, 110871, 3837, 103941, 99787, 108245, 108088, 9370, 116606, 119331, 3407, 19, 13, 3070, 100156, 103439, 101079, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 100156, 47606, 104795, 103439, 101079, 3837, 105018, 24339, 60686, 99295, 110394, 33108, 104114, 9370, 109788, 8997, 256, 481, 96155, 245, 52726, 101079, 18493, 114872, 104694, 111035, 110470, 3837, 103941, 83751, 100712, 34187, 24339, 60686, 99295, 110394, 3407, 44364, 14374, 3070, 44991, 5373, 105101, 22045, 104249, 56177, 16, 13, 3070, 104561, 101157, 102004, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 104561, 101076, 33108, 100033, 100021, 99427, 103108, 34204, 104114, 33108, 99698, 99292, 3837, 101068, 104069, 101930, 104304, 104378, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 106700, 33108, 110100, 18493, 104304, 15946, 101107, 49111, 49111, 3837, 101068, 32664, 99698, 99292, 104384, 103133, 105204, 3407, 17, 13, 3070, 100156, 100143, 26381, 102008, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 100156, 32664, 104304, 106100, 104052, 101190, 49238, 3837, 105672, 62112, 104304, 102220, 115172, 68536, 32664, 24339, 60686, 99295, 110394, 104899, 101381, 3407, 18, 13, 3070, 99698, 99292, 107363, 334, 28311, 256, 481, 74866, 253, 99292, 18493, 48309, 65676, 5373, 117564, 33108, 105101, 105210, 9370, 104629, 3837, 116854, 105101, 102747, 100011, 109797, 3407, 19, 13, 3070, 102390, 63109, 28404, 9370, 22045, 334, 28311, 256, 481, 18137, 248, 237, 99164, 104114, 18493, 67364, 43268, 102961, 9370, 22045, 3837, 105101, 106767, 104509, 99698, 97815, 3837, 103941, 104487, 118973, 3407, 44364, 14374, 3070, 63703, 5373, 101241, 50984, 69249, 99685, 9370, 107632, 57218, 22045, 56177, 16, 13, 3070, 24339, 60686, 99295, 110394, 9370, 95411, 43316, 334, 28311, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 67338, 105439, 107632, 105749, 41146, 105430, 3837, 77288, 100137, 107632, 75768, 101068, 100350, 100156, 9370, 101930, 100143, 8997, 256, 481, 59053, 243, 60686, 99295, 110394, 9370, 105148, 33108, 99285, 59355, 111291, 41146, 22045, 101945, 99917, 3407, 17, 13, 3070, 101241, 50984, 69249, 99685, 9370, 99605, 108452, 334, 28311, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 18493, 100033, 102041, 17447, 100238, 16872, 101120, 32100, 3837, 77557, 73743, 99191, 106261, 33108, 102961, 50404, 8997, 256, 481, 220, 42411, 102767, 105537, 104114, 3837, 107419, 34187, 105101, 105572, 101996, 3407, 18, 13, 3070, 24339, 60686, 99295, 100091, 9370, 22045, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 22045, 108653, 24339, 60686, 99295, 100091, 9370, 104371, 22045, 3837, 109001, 99489, 105353, 99365, 34187, 99511, 75061, 3407, 44364, 14374, 3070, 75108, 5373, 114872, 33447, 105101, 9370, 107639, 57218, 107295, 56177, 16, 13, 3070, 99191, 33447, 107844, 57218, 101367, 32108, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 114872, 33447, 99857, 103931, 24339, 60686, 99295, 110394, 3837, 104809, 101367, 104773, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 99328, 103993, 110312, 107639, 3837, 106455, 45181, 24339, 60686, 99295, 100091, 9370, 111684, 15946, 117667, 3407, 17, 13, 3070, 99346, 107295, 57218, 104123, 104767, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 99191, 33447, 67338, 99313, 106408, 99079, 101039, 49567, 113284, 104415, 99346, 108105, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 99787, 104123, 99346, 109809, 9370, 105405, 116588, 3837, 101890, 34187, 104123, 104767, 101556, 3407, 44364, 14374, 3070, 99566, 5373, 102050, 56177, 105101, 18493, 114872, 101047, 100780, 101909, 102181, 104754, 102206, 1773, 99652, 99929, 101909, 24339, 60686, 99295, 100091, 9370, 99661, 3837, 109381, 22045, 9370, 102390, 63109, 28404, 100450, 1773, 105101, 9370, 22045, 105912, 104561, 33108, 99346, 104249, 3837, 103979, 41146, 24339, 60686, 99295, 110394, 33108, 105390, 115922, 9370, 102309, 59151, 1773, 105101, 9370, 99191, 33447, 107639, 33108, 107295, 109001, 99489, 104257, 100799, 100034, 107472, 1773, 67338, 107083, 100022, 3837, 105773, 33126, 100403, 29490, 101128, 24339, 60686, 99295, 100091, 112614, 3837, 101034, 104939, 57218, 101367, 106508, 1773, 151643, 151644, 118919, 99313, 106408, 99079, 101039, 11319, 99652, 32664, 104123, 103949, 59532, 101803, 111867, 11319, 151645], input_embeds=None, image_data=None, sampling_params={'temperature': 0.7, 'max_new_tokens': 32768, 'min_new_tokens': 0, 'stop': None, 'stop_token_ids': None, 'top_p': 1.0, 'top_k': -1, 'min_p': 0.0, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'repetition_penalty': 1.0, 'regex': None, 'ebnf': None, 'n': 1, 'no_stop_trim': False, 'ignore_eos': False, 'skip_special_tokens': True}, rid='936504d360704ae1b47433847807094e', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=True, stream=True, log_metrics=True, modalities=[], lora_path=None, session_params=None, custom_logit_processor=None)
[2025-03-02 11:26:19] INFO: 127.0.0.1:56442 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:26:49] INFO: 127.0.0.1:38816 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:27:00] Finish: obj=GenerateReqInput(text=None, input_ids=[151646, 66365, 369, 1943, 304, 6605, 67914, 66365, 421, 6337, 7389, 323, 6605, 58, 15, 5352, 5778, 660, 961, 364, 8948, 6, 67914, 2979, 364, 8948, 1699, 565, 12329, 1124, 77, 1699, 4008, 25, 220, 56568, 101909, 27369, 100733, 100136, 102556, 9370, 110498, 3837, 107618, 101098, 106185, 20002, 86119, 3837, 99553, 102188, 9370, 101898, 33108, 101086, 3837, 100143, 20002, 18493, 101312, 102122, 15946, 100638, 99912, 86119, 1773, 220, 14880, 100470, 106466, 107494, 109504, 90395, 103944, 102104, 104542, 5373, 54926, 100116, 5373, 101137, 104913, 1773, 81263, 102031, 104552, 57191, 113272, 86119, 37945, 104137, 113272, 90395, 18493, 102618, 104705, 37029, 1565, 151648, 1699, 63, 32181, 249, 22243, 104107, 66898, 77, 6, 8096, 66365, 12330, 67914, 2979, 4605, 488, 1943, 677, 5778, 660, 488, 5196, 77, 6, 488, 1943, 677, 1796, 660, 488, 3355, 488, 5196, 77, 74491, 66365, 835, 1958, 67914, 66365, 421, 912, 64191, 61421, 67914, 2979, 364, 77091, 1699, 6, 8096, 66365, 12330, 1018, 630, 151644, 105043, 100165, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 56007, 2073, 105043, 100165, 11319, 33590, 35946, 49828, 102231, 105839, 99494, 102104, 1773, 101140, 3837, 35946, 99730, 100157, 99283, 3837, 66394, 104198, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 1773, 101889, 3837, 98237, 30534, 53481, 97611, 98380, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 104019, 104496, 35946, 107618, 100638, 100646, 86119, 3837, 100364, 20002, 60548, 88802, 90395, 100136, 100470, 105389, 20002, 104272, 1773, 100161, 3837, 104046, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 3837, 104011, 104552, 57191, 113272, 86119, 13343, 36993, 113177, 104107, 3837, 11622, 46100, 99922, 101987, 1773, 99654, 20002, 104077, 100011, 99794, 97611, 106712, 100772, 34187, 8997, 522, 14871, 1339, 104198, 33464, 39350, 10911, 16, 8125, 632, 12, 24625, 3837, 46944, 67071, 102217, 30918, 50984, 73218, 100013, 9370, 100168, 110498, 3837, 100006, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 35946, 107618, 100638, 100646, 86119, 90395, 100364, 20002, 60548, 88802, 1773, 105351, 100470, 106466, 101214, 101882, 90395, 103944, 97611, 102104, 104542, 5373, 54926, 100116, 5373, 18830, 104913, 1773, 62244, 104011, 85106, 104552, 100768, 57191, 102181, 113272, 103936, 3837, 105351, 104137, 104107, 62926, 101987, 97611, 113272, 100178, 1773, 151643, 151644, 107809, 106428, 11319, 151645, 27, 14871, 1707, 428, 3423, 25, 11650, 49954, 7889, 25, 671, 69, 23, 69, 23, 69, 23, 35129, 25, 220, 23, 1767, 35779, 17710, 25, 220, 19, 1767, 10892, 1787, 29, 366, 1708, 29, 52289, 1112, 690, 1708, 397, 106287, 3837, 20002, 99601, 56007, 2073, 107809, 106428, 11319, 55807, 101140, 3837, 35946, 85106, 101128, 20002, 104378, 1773, 99650, 87267, 32664, 97611, 98380, 102342, 99794, 3837, 100631, 99172, 81167, 35946, 64471, 26232, 100364, 99650, 60548, 106168, 88802, 3407, 35946, 99730, 60726, 114116, 97611, 99558, 99788, 3837, 101912, 54542, 108704, 5373, 102104, 86119, 5373, 28072, 19564, 31338, 96422, 49567, 1773, 101889, 3837, 100398, 66394, 110913, 60548, 108530, 3837, 101912, 27369, 51154, 5373, 104552, 100768, 5373, 104913, 113272, 5373, 102064, 105395, 5373, 108704, 43959, 49567, 1773, 103991, 99788, 99357, 18947, 103358, 3837, 101912, 104552, 100768, 73670, 49238, 23384, 38507, 3837, 105395, 73670, 100143, 15946, 82847, 99524, 102610, 3837, 108704, 43959, 73670, 61443, 82025, 3407, 104326, 3837, 35946, 85106, 104046, 97611, 100661, 3837, 101912, 111076, 5373, 101991, 33108, 104913, 33071, 1773, 91572, 3837, 66394, 105351, 100345, 20002, 104272, 101921, 102104, 3837, 103944, 101137, 104056, 100354, 3837, 104050, 102031, 104552, 57191, 113272, 13343, 3837, 36993, 104137, 101987, 104107, 100178, 3407, 100161, 3837, 102050, 100158, 110913, 103008, 100364, 101121, 3837, 115987, 99392, 99650, 73670, 105537, 35946, 100638, 100646, 86119, 1773, 99654, 20002, 104077, 101222, 29490, 99794, 97611, 98380, 33108, 99788, 3837, 106581, 105344, 100152, 35946, 36407, 60548, 104056, 88802, 8997, 522, 14871, 1339, 35946, 101909, 105600, 9370, 15469, 110498, 3837, 100006, 100364, 87026, 60548, 101312, 88802, 1773, 114566, 104786, 99604, 100650, 104426, 33108, 98380, 48443, 14374, 3070, 16, 13, 220, 27369, 51154, 57218, 106185, 1019, 12, 3070, 100032, 111436, 334, 5122, 99553, 101284, 100650, 9909, 29524, 99891, 5373, 99361, 5373, 99348, 5373, 100022, 49567, 7552, 9370, 105600, 100032, 8997, 12, 3070, 101349, 107462, 334, 5122, 100364, 48927, 27369, 9370, 111076, 8997, 12, 3070, 104136, 101290, 334, 5122, 104136, 102181, 101290, 57191, 116925, 3407, 14374, 3070, 17, 13, 47685, 47764, 100768, 57218, 104913, 113272, 1019, 12, 3070, 104552, 86119, 334, 5122, 100638, 30540, 8863, 5373, 111867, 5373, 103112, 5373, 100787, 49567, 104552, 86119, 8997, 12, 3070, 104913, 113272, 334, 5122, 71817, 104913, 113272, 33108, 86119, 100638, 8997, 12, 3070, 23384, 38507, 30918, 49238, 334, 5122, 49238, 23384, 38507, 57191, 16530, 49567, 28330, 3837, 29524, 105935, 23384, 38507, 5373, 43268, 33071, 23384, 38507, 40027, 49567, 3407, 14374, 3070, 18, 13, 40666, 248, 102064, 100143, 1019, 12, 3070, 102064, 105395, 334, 5122, 100143, 15946, 82847, 99524, 102610, 107768, 102064, 9909, 29524, 24339, 72881, 5373, 99462, 72881, 5373, 8903, 72881, 49567, 7552, 9370, 105395, 8997, 12, 3070, 102064, 100134, 334, 5122, 99553, 102064, 100134, 101898, 3837, 104136, 117206, 57191, 110376, 3407, 14374, 3070, 19, 13, 53040, 21894, 43959, 57218, 54542, 1019, 12, 3070, 108704, 104223, 334, 5122, 110479, 82025, 5373, 101108, 5373, 100370, 5373, 107604, 49567, 8997, 12, 3070, 106208, 43959, 334, 5122, 102050, 45861, 113773, 57191, 111116, 8997, 12, 3070, 22418, 61443, 103983, 334, 5122, 103983, 108704, 102124, 3837, 102989, 33126, 104542, 57191, 109946, 107616, 3407, 14374, 3070, 20, 13, 53497, 99, 99216, 57218, 99878, 100650, 1019, 12, 3070, 104380, 105293, 334, 5122, 100364, 110479, 104734, 5373, 102140, 100370, 5373, 108067, 99611, 41932, 49567, 8997, 12, 3070, 99878, 101898, 334, 5122, 18493, 110569, 5373, 70500, 5373, 99655, 106483, 99553, 101086, 3407, 14374, 3070, 21, 13, 220, 99605, 57218, 99424, 1019, 12, 3070, 99424, 101898, 334, 5122, 28072, 19564, 31338, 96422, 3837, 29524, 20450, 39352, 5373, 99722, 5373, 102275, 101039, 49567, 8997, 12, 3070, 100565, 101964, 334, 5122, 101914, 107041, 5373, 100378, 5373, 101941, 49567, 3407, 14374, 3070, 22, 13, 220, 46100, 57218, 110569, 1019, 12, 3070, 110569, 100364, 334, 5122, 104136, 110569, 101290, 5373, 104749, 46100, 32100, 5373, 99553, 46100, 19793, 26355, 8997, 12, 3070, 107018, 70500, 334, 5122, 105096, 70500] ... [24339, 60686, 99295, 100091, 9370, 28291, 37984, 29490, 100653, 3837, 105101, 18493, 114872, 15946, 107571, 102390, 63109, 28404, 101945, 100450, 3837, 99518, 62112, 101979, 104409, 33108, 105413, 101950, 103941, 104487, 22045, 1773, 87752, 106273, 114872, 15946, 105101, 9370, 100011, 101042, 48443, 44364, 14374, 3070, 14777, 5373, 105101, 101963, 102390, 63109, 28404, 104249, 56177, 16, 13, 3070, 24339, 60686, 99295, 100091, 9370, 105390, 115922, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 24339, 60686, 99295, 107672, 100458, 99685, 99829, 13935, 89714, 94794, 9909, 101241, 50984, 69249, 99685, 7552, 106902, 99658, 65278, 104240, 9370, 110502, 3837, 106455, 67338, 105390, 101884, 2073, 26288, 105101, 97907, 100160, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 111789, 104304, 102005, 41146, 100622, 104123, 107944, 106681, 90395, 18493, 99876, 101436, 15946, 100350, 106155, 102588, 40981, 3407, 17, 13, 3070, 99346, 57218, 101091, 111443, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 17, 15, 101186, 84607, 99346, 103985, 3837, 24339, 60686, 99295, 110394, 111789, 104304, 104313, 99346, 3837, 100638, 108127, 86119, 8997, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 111789, 104304, 104651, 100156, 104409, 3837, 105749, 41146, 107632, 101262, 3407, 18, 13, 3070, 57218, 104114, 9370, 104237, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 57218, 104114, 36885, 99698, 20412, 104210, 100656, 9370, 24339, 60686, 99295, 115884, 33108, 32664, 82847, 24339, 9370, 101271, 57452, 8997, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 106455, 67338, 57218, 104114, 99721, 3837, 100350, 102075, 100033, 85329, 33108, 116275, 3407, 44364, 14374, 3070, 40820, 5373, 105101, 18493, 114872, 101047, 100780, 56177, 16, 13, 3070, 48309, 65676, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 48309, 65676, 9909, 59532, 103451, 5373, 109943, 7552, 102121, 34187, 100722, 105591, 3837, 106455, 100359, 99908, 100208, 99253, 109440, 8997, 256, 481, 220, 16, 24, 19, 16, 7948, 3837, 105101, 105591, 18493, 99829, 101755, 100221, 99316, 99250, 82847, 99292, 109055, 3837, 108653, 41146, 18493, 48309, 65676, 9370, 22045, 3407, 17, 13, 3070, 117564, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 106700, 106455, 100359, 117564, 3837, 77288, 62112, 101076, 105983, 33108, 100033, 108452, 3837, 104183, 99250, 99698, 99292, 109055, 8997, 256, 481, 220, 16, 24, 19, 18, 7948, 3837, 99698, 99292, 109788, 60686, 60686, 69249, 99825, 3837, 100673, 105101, 100156, 108322, 109797, 3407, 18, 13, 3070, 67364, 43268, 102961, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 99890, 99807, 100651, 67364, 43268, 102961, 9909, 99908, 99462, 102961, 48272, 77288, 101107, 110871, 3837, 103941, 99787, 108245, 108088, 9370, 116606, 119331, 3407, 19, 13, 3070, 100156, 103439, 101079, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 100156, 47606, 104795, 103439, 101079, 3837, 105018, 24339, 60686, 99295, 110394, 33108, 104114, 9370, 109788, 8997, 256, 481, 96155, 245, 52726, 101079, 18493, 114872, 104694, 111035, 110470, 3837, 103941, 83751, 100712, 34187, 24339, 60686, 99295, 110394, 3407, 44364, 14374, 3070, 44991, 5373, 105101, 22045, 104249, 56177, 16, 13, 3070, 104561, 101157, 102004, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 104561, 101076, 33108, 100033, 100021, 99427, 103108, 34204, 104114, 33108, 99698, 99292, 3837, 101068, 104069, 101930, 104304, 104378, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 106700, 33108, 110100, 18493, 104304, 15946, 101107, 49111, 49111, 3837, 101068, 32664, 99698, 99292, 104384, 103133, 105204, 3407, 17, 13, 3070, 100156, 100143, 26381, 102008, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 100156, 32664, 104304, 106100, 104052, 101190, 49238, 3837, 105672, 62112, 104304, 102220, 115172, 68536, 32664, 24339, 60686, 99295, 110394, 104899, 101381, 3407, 18, 13, 3070, 99698, 99292, 107363, 334, 28311, 256, 481, 74866, 253, 99292, 18493, 48309, 65676, 5373, 117564, 33108, 105101, 105210, 9370, 104629, 3837, 116854, 105101, 102747, 100011, 109797, 3407, 19, 13, 3070, 102390, 63109, 28404, 9370, 22045, 334, 28311, 256, 481, 18137, 248, 237, 99164, 104114, 18493, 67364, 43268, 102961, 9370, 22045, 3837, 105101, 106767, 104509, 99698, 97815, 3837, 103941, 104487, 118973, 3407, 44364, 14374, 3070, 63703, 5373, 101241, 50984, 69249, 99685, 9370, 107632, 57218, 22045, 56177, 16, 13, 3070, 24339, 60686, 99295, 110394, 9370, 95411, 43316, 334, 28311, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 67338, 105439, 107632, 105749, 41146, 105430, 3837, 77288, 100137, 107632, 75768, 101068, 100350, 100156, 9370, 101930, 100143, 8997, 256, 481, 59053, 243, 60686, 99295, 110394, 9370, 105148, 33108, 99285, 59355, 111291, 41146, 22045, 101945, 99917, 3407, 17, 13, 3070, 101241, 50984, 69249, 99685, 9370, 99605, 108452, 334, 28311, 256, 481, 4891, 95, 101, 50984, 69249, 99685, 18493, 100033, 102041, 17447, 100238, 16872, 101120, 32100, 3837, 77557, 73743, 99191, 106261, 33108, 102961, 50404, 8997, 256, 481, 220, 42411, 102767, 105537, 104114, 3837, 107419, 34187, 105101, 105572, 101996, 3407, 18, 13, 3070, 24339, 60686, 99295, 100091, 9370, 22045, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 9370, 22045, 108653, 24339, 60686, 99295, 100091, 9370, 104371, 22045, 3837, 109001, 99489, 105353, 99365, 34187, 99511, 75061, 3407, 44364, 14374, 3070, 75108, 5373, 114872, 33447, 105101, 9370, 107639, 57218, 107295, 56177, 16, 13, 3070, 99191, 33447, 107844, 57218, 101367, 32108, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 114872, 33447, 99857, 103931, 24339, 60686, 99295, 110394, 3837, 104809, 101367, 104773, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 99328, 103993, 110312, 107639, 3837, 106455, 45181, 24339, 60686, 99295, 100091, 9370, 111684, 15946, 117667, 3407, 17, 13, 3070, 99346, 107295, 57218, 104123, 104767, 334, 28311, 256, 481, 6567, 226, 237, 26288, 59532, 18493, 99191, 33447, 67338, 99313, 106408, 99079, 101039, 49567, 113284, 104415, 99346, 108105, 8997, 256, 481, 6567, 226, 237, 26288, 59532, 99787, 104123, 99346, 109809, 9370, 105405, 116588, 3837, 101890, 34187, 104123, 104767, 101556, 3407, 44364, 14374, 3070, 99566, 5373, 102050, 56177, 105101, 18493, 114872, 101047, 100780, 101909, 102181, 104754, 102206, 1773, 99652, 99929, 101909, 24339, 60686, 99295, 100091, 9370, 99661, 3837, 109381, 22045, 9370, 102390, 63109, 28404, 100450, 1773, 105101, 9370, 22045, 105912, 104561, 33108, 99346, 104249, 3837, 103979, 41146, 24339, 60686, 99295, 110394, 33108, 105390, 115922, 9370, 102309, 59151, 1773, 105101, 9370, 99191, 33447, 107639, 33108, 107295, 109001, 99489, 104257, 100799, 100034, 107472, 1773, 67338, 107083, 100022, 3837, 105773, 33126, 100403, 29490, 101128, 24339, 60686, 99295, 100091, 112614, 3837, 101034, 104939, 57218, 101367, 106508, 1773, 151643, 151644, 118919, 99313, 106408, 99079, 101039, 11319, 99652, 32664, 104123, 103949, 59532, 101803, 111867, 11319, 151645], input_embeds=None, image_data=None, sampling_params={'temperature': 0.7, 'max_new_tokens': 32768, 'min_new_tokens': 0, 'stop': None, 'stop_token_ids': None, 'top_p': 1.0, 'top_k': -1, 'min_p': 0.0, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'repetition_penalty': 1.0, 'regex': None, 'ebnf': None, 'n': 1, 'no_stop_trim': False, 'ignore_eos': False, 'skip_special_tokens': True}, rid='936504d360704ae1b47433847807094e', return_logprob=False, logprob_start_len=-1, top_logprobs_num=0, return_text_in_logprobs=True, stream=True, log_metrics=True, modalities=[], lora_path=None, session_params=None, custom_logit_processor=None), out={'text': '\n嗯，用户现在问“如何看待马歇尔计划？它对欧洲的发展利弊几何？”。根据之前的对话，用户对二战后的重建和欧洲一体化感兴趣，现在转向分析马歇尔计划的影响。\n\n我需要确定用户的需求是什么。他们可能想知道马歇尔计划的具体内容、实施效果，以及对欧洲发展的利弊。考虑到用户之前的互动，他们可能希望得到全面且结构清晰的分析。\n\n首先，我应该介绍马歇尔计划的基本情况，包括实施时间、主要目标和资金规模。然后，分析其对欧洲的积极影响，如经济发展、政治稳定、欧洲一体化等。接着，讨论其潜在的弊端，如经济依赖、政治干预等。最后，总结马歇尔计划的历史意义和对现代的启示。\n\n我还需要考虑具体的历史事件，如马歇尔计划如何帮助西欧国家恢复经济，如何促进北约的成立等。这些具体事例可以帮助用户更好地理解马歇尔计划的影响。\n\n此外，用户可能对马歇尔计划与冷战的关系感兴趣，比如它如何成为美国遏制苏联扩张的工具。因此，我需要确保内容涵盖这些方面，并提供足够的细节。\n\n最后，我需要确保内容的准确性和客观性，提供中立的分析，帮助用户全面理解马歇尔计划对欧洲发展的利弊。\n\n总的来说，我需要组织一个结构清晰、内容全面的回答，涵盖马歇尔计划的实施、积极影响、潜在弊端、历史意义等多方面的因素，帮助用户深入理解马歇尔计划对欧洲发展的多维度影响。\n\n\n### 马歇尔计划：冷战背景下的欧洲重建\n\n马歇尔计划（Marshall Plan），全称为“欧洲复兴计划”（European Recovery Program，ERP），是二战结束后美国为帮助西欧国家恢复经济而实施的一项大规模援助计划。该计划从1948年实施，到1952年结束，共向西欧16个国家提供了约130亿美元的援助（相当于2023年的约1300亿美元）。马歇尔计划不仅是冷战时期美国对西欧的战略投资，也是促进欧洲经济复苏和政治稳定的重要举措。\n\n以下是对马歇尔计划的全面分析，包括其对欧洲发展的利弊。\n\n---\n\n### 一、马歇尔计划的背景\n\n1. 二战后的欧洲困境：\n - 欧洲在二战中遭受了巨大的破坏，工业设施被摧毁，城市被轰炸，经济陷入崩溃。\n - 食物短缺、燃料不足和失业问题严重，民众生活困苦。\n - 欧洲各国因战争而背负巨额债务，无法独立恢复经济。\n\n2. 冷战的开始：\n - 二战结束后，美国与苏联之间的意识形态和地缘政治矛盾日益加剧。' ... '资本流动方面。\n - 这种依赖在一定程度上限制了西欧国家的经济自主性。\n\n2. 政治干预：\n - 美国通过马歇尔计划对西欧国家的政治和经济政策进行了干预，尤其是在冷战背景下。\n - 这种干预有时引发了西欧国家的不满，认为美国干涉了其内政。\n\n3. 忽视东欧：\n - 马歇尔计划仅针对西欧国家，忽视了东欧国家（如波兰、捷克斯洛伐克）的经济需求。\n - 这种选择性援助加剧了东西欧之间的经济差距。\n\n4. 短期效果与长期问题：\n - 马歇尔计划虽然帮助西欧迅速恢复了经济，但并未解决深层次的经济和社会问题（如贫富差距、社会福利等）。\n - 部分国家在计划结束后面临新的经济挑战。\n\n---\n\n### 五、马歇尔计划的历史意义\n\n1. 冷战的战略工具：\n - 马歇尔计划是美国冷战政策的重要组成部分，通过经济援助巩固了西欧的资本主义阵营。\n - 它为北约的成立奠定了经济和政治基础。\n\n2. 欧洲一体化的起点：\n - 马歇尔计划推动了西欧国家之间的合作，为后来的欧洲经济共同体（EEC）和欧盟的成立奠定了基础。\n\n3. 美国全球影响力的提升：\n - 马歇尔计划不仅帮助西欧恢复经济，也巩固了美国在战后国际秩序中的主导地位。\n\n---\n\n### 六、对当代的启示\n\n1. 国际合作的重要性：\n - 马歇尔计划展示了国际合作在经济复苏和全球治理中的重要作用。\n - 当代国际社会可以借鉴其经验，通过多边合作应对全球性挑战（如气候变化、经济危机等）。\n\n2. 经济援助的政治性：\n - 马歇尔计划提醒我们，经济援助往往伴随着政治目的和条件。\n - 在当代国际援助中，需要平衡援助的经济效果与政治影响。\n\n3. 经济复苏与社会公平：\n - 马歇尔计划虽然促进了经济复苏，但未能解决社会公平问题。\n - 当代经济政策需要更加关注社会公平，避免因经济复苏而加剧社会矛盾。\n\n---\n\n### 七、总结\n\n马歇尔计划是二战后西欧经济复苏的重要推动力，同时也是冷战背景下美国对西欧的战略投资。它在促进经济复苏、推动欧洲一体化、巩固美国影响力等方面取得了显著成就。然而，它也带来了经济依赖、政治干预等弊端。总体而言，马歇尔计划是一个成功的国际援助案例，其经验和教训对当代国际关系和经济政策具有重要的启示意义。', 'meta_info': {'id': '936504d360704ae1b47433847807094e', 'finish_reason': {'type': 'stop', 'matched': 151643}, 'prompt_tokens': 11076, 'completion_tokens': 1735, 'cached_tokens': 0}}
[2025-03-02 11:27:19] INFO: 127.0.0.1:50104 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:27:49] INFO: 127.0.0.1:44364 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:28:19] INFO: 127.0.0.1:45544 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:28:49] INFO: 127.0.0.1:34024 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:29:20] INFO: 127.0.0.1:45826 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:29:50] INFO: 127.0.0.1:59044 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:30:20] INFO: 127.0.0.1:40138 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:30:50] INFO: 127.0.0.1:55878 - "GET /health HTTP/1.1" 200 OK
[2025-03-02 11:31:20] INFO: 127.0.0.1:54434 - "GET /health HTTP/1.1" 200 OK
and here is my configs:
root@GPUServer-1:/opt/sglang# cat docker-compose.dpskr1-qwen32b.yaml
services:
sglang-dpskr1-qwen32b:
image: lmsysorg/sglang:latest
container_name: sglang-dpskr1-qwen32b
volumes:
- ${HOME}/mymodels/huggingface:/root/.cache/huggingface
# If you use modelscope, you need mount this directory
- ${HOME}/mymodels/modelscope:/root/.cache/modelscope
restart: always
network_mode: host # required by RDMA
privileged: true # required by RDMA
# Or you can only publish port 30000
ports:
- 30000:30000
root@GPUServer-1:/opt/sglang#
Beta Was this translation helpful? Give feedback.
All reactions