[Misc] Support more collective_rpc return types

[njhill]

Because of how msgspec is used for the front-end <-> engine IPC message encoding, only "simple" native python data types/structures will correctly roundtrip when returned in the UtilityOutput struct from the call_utility(_async) method.

This change allows arbitrary msgspec-supported types to be returned, as well as tensors/numpy-arrays which will use the existing zero-copy optimizations. It also checks for lists of such types. Such cases will require VLLM_ALLOW_INSECURE_SERIALIZATION to be set since arbitrary types are materialized from strings when decoding.

This in particular is needed for #18465.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request adds support for more complex return types from collective_rpc by introducing a UtilityResult wrapper and custom serialization/deserialization logic. This allows msgspec to handle types it doesn't support natively by encoding type information alongside the data.

The overall approach is sound. I've identified one potential issue in the serialization logic where using cls is not list is too specific and could lead to incorrect behavior with subclasses of list. I've suggested a more robust implementation using isinstance.

[DarkLight1337]

LGTM, thanks

[DarkLight1337]

V1 test passes, let's merge this