Did you consider to apply xDiT for parallel inference

[feifeibear]

Hello Allegro Team,

I hope this message finds you well. I would like to propose the integration of xDiT, a scalable inference engine for Diffusion Transformers (DiTs), into the Allegro ecosystem. xDiT offers several compelling benefits that could significantly enhance the performance and scalability of inference tasks within Allegro.

Key Benefits of xDiT:

1. Ease of Transformation into Sequence Parallel + USP Version:

   • xDiT can be easily adapted to support sequence parallelism combined with Unified Scaling Policy (USP), making it highly flexible and efficient for various inference scenarios.

2. Compatibility with huggingfacee diffusers Ecosystem and ComfyUI:

   • xDiT is designed to be compatible with the diffuser ecosystem and ComfyUI, ensuring seamless integration and interoperability with existing tools and workflows.

3. Scalability with PipeFusion and Hybrid Parallelism:

   • xDiT's support for PipeFusion and hybrid parallelism allows it to scale to very large-scale inference tasks, providing substantial performance improvements.

Additional Highlights:

• Efficient Inference for Video Models:

  • xDiT has demonstrated significant success in reducing inference latency for video models, such as CogVideX-5B and mochi1-10B, achieving notable speedups.

• Performance Gains:

  • The mochi-xdit project has shown a 3.54X reduction in inference latency compared to the official open-source implementation.

References:

• GitHub Repository:

  • xDiT
  • mochi-xDiT

• Medium Articles:

  • Enhancing Parallelism and Speedup for xDiT in Serving the Mochi-1 Video Generation Model
  • Leveraging xDiT to Parallelize the Open-Sourced Video Generation Model CogVideoX

Thank you for considering this proposal. I look forward to your thoughts and feedback.

[maazel-rhymes]

Hi Feifeibear,

Your xDiT looks amazing, and I've starred. However, we're currently really short-handed and do not have enough time for the integration ourselves. If you can help us with the integration, we'd love to make a joint announcement to our developer community! p.s. Allegro will receive an I2V upgrade soon

Best Regards,
Maazel

[feifeibear]

Hi Feifeibear,

Your xDiT looks amazing, and I've starred. However, we're currently really short-handed and do not have enough time for the integration ourselves. If you can help us with the integration, we'd love to make a joint announcement to our developer community! p.s. Allegro will receive an I2V upgrade soon

Best Regards, Maazel

Thanks! Feel free to contact us if you need parallel inference solutions.