### Is your feature request related to a problem? _No response_ ### Describe the solution you'd like Sync PR [#104](https://github.com/hsliuustc0106/vllm/pull/104) ### Describe alternatives you've considered _No response_ ### Which component does this relate to? vLLM integration ### Priority Medium ### Use Case _No response_ ### Additional Context _No response_ ### Code of Conduct - [x] I agree to follow this project's Code of Conduct