update qwen3-235b-a22b-fp8

[yyzxw]

part of #44

[gemini-code-assist]

Summary of Changes

Hello @yyzxw, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the configuration for the "qwen3-235b-a22b-fp8" model. It primarily introduces a new "maxTokens" setting to define the maximum number of tokens the model can handle and refines the GPU resource requirements for its deployments, optimizing the allocation of hardware resources.

Highlights

• Model Configuration Update: Introduced a "maxTokens" setting of "40960" under the "spec.config" section for the "qwen3-235b-a22b-fp8" model.
• Resource Requirements Adjustment: Modified the GPU resource allocation, changing "gpuCount" from "8" to "6" and updating "gpuType" from "nvidia-vgpu" to "vgpu".

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Model Metadata Validation Results

✅ All validations passed!

Changed files:

• models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml

Validation output:

✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid


✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid directory structure


✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid Hugging Face source: Qwen/Qwen3-235B-A22B-FP8
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid ModelScope source: Qwen/Qwen3-235B-A22B-FP8

Please make sure all metadata files conform to the schema defined in docs/model.jsonschema.

[gemini-code-assist]

Code Review

The pull request updates the qwen3-235b-a22b-fp8 model's metadata, specifically adjusting the maxTokens configuration and reducing the required gpuCount from 8 to 6. The maxTokens parameter is added to the config, and the number of GPUs is reduced, potentially affecting the model's performance and resource utilization.

[gemini-code-assist]

The reduction in gpuCount from 8 to 6 could impact the model's performance. It's important to ensure that this change doesn't negatively affect the model's inference speed or ability to handle large batches. Verify the performance after this change.

Consider adding a comment explaining why the gpuCount was reduced. Was it due to resource constraints, cost optimization, or other reasons?

[gemini-code-assist]

Adding maxTokens is good for configuring the model's context window. However, ensure that this value aligns with the model's capabilities and doesn't exceed its maximum supported context length, which could lead to unexpected behavior or errors. It's important to validate that the model functions correctly with this specific maxTokens value.

[github-actions]

Model Metadata Validation Results

✅ All validations passed!

Changed files:

• models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml

Validation output:

✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid


✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid directory structure


✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid Hugging Face source: Qwen/Qwen3-235B-A22B-FP8
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid ModelScope source: Qwen/Qwen3-235B-A22B-FP8

Please make sure all metadata files conform to the schema defined in docs/model.jsonschema.

[kebe7jun]

235b 支持 tp6？

[github-actions]

Model Metadata Validation Results

✅ All validations passed!

Changed files:

• models/alibaba/qwen2.5-7b-instruct/metadata.yaml
• models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml
• models/alibaba/qwq-32b/metadata.yaml
• models/deepseek/deepseek-r1-distill-qwen-14b/metadata.yaml
• models/deepseek/deepseek-r1-distill-qwen-32b/metadata.yaml
• models/deepseek/deepseek-r1/metadata.yaml

Validation output:

✅ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Valid
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid
✅ models/alibaba/qwq-32b/metadata.yaml - Valid
✅ models/deepseek/deepseek-r1-distill-qwen-14b/metadata.yaml - Valid
✅ models/deepseek/deepseek-r1-distill-qwen-32b/metadata.yaml - Valid
✅ models/deepseek/deepseek-r1/metadata.yaml - Valid


✅ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Valid directory structure
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid directory structure
✅ models/alibaba/qwq-32b/metadata.yaml - Valid directory structure
✅ models/deepseek/deepseek-r1-distill-qwen-14b/metadata.yaml - Valid directory structure
✅ models/deepseek/deepseek-r1-distill-qwen-32b/metadata.yaml - Valid directory structure
✅ models/deepseek/deepseek-r1/metadata.yaml - Valid directory structure


✅ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Valid Hugging Face source: Qwen/Qwen2.5-VL-7B-Instruct
✅ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Valid ModelScope source: Qwen/Qwen2.5-VL-7B-Instruct
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid Hugging Face source: Qwen/Qwen3-235B-A22B-FP8
✅ models/alibaba/qwen3-235b-a22b-fp8/metadata.yaml - Valid ModelScope source: Qwen/Qwen3-235B-A22B-FP8
✅ models/alibaba/qwq-32b/metadata.yaml - Valid Hugging Face source: Qwen/QwQ-32B
✅ models/alibaba/qwq-32b/metadata.yaml - Valid ModelScope source: Qwen/QwQ-32B
✅ models/deepseek/deepseek-r1-distill-qwen-14b/metadata.yaml - Valid Hugging Face source: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
✅ models/deepseek/deepseek-r1-distill-qwen-14b/metadata.yaml - Valid ModelScope source: deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
✅ models/deepseek/deepseek-r1-distill-qwen-32b/metadata.yaml - Valid Hugging Face source: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
✅ models/deepseek/deepseek-r1-distill-qwen-32b/metadata.yaml - Valid ModelScope source: deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
✅ models/deepseek/deepseek-r1/metadata.yaml - Valid Hugging Face source: deepseek-ai/DeepSeek-R1
✅ models/deepseek/deepseek-r1/metadata.yaml - Valid ModelScope source: deepseek-ai/DeepSeek-R1


⚠️ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Metadata name (qwen2.5-7b-instruct) doesn't match the normalized Hugging Face model ID (qwen2-5-vl-7b-instruct)
⚠️ models/alibaba/qwen2.5-7b-instruct/metadata.yaml - Display name (Qwen2.5-7B-Instruct) doesn't seem to match the Hugging Face model name (Qwen2.5-VL-7B-Instruct)
⚠️ models/deepseek/deepseek-r1/metadata.yaml - Display name (🔥 DeepSeek-R1-0528) doesn't seem to match the Hugging Face model name (DeepSeek-R1)

Please make sure all metadata files conform to the schema defined in docs/model.jsonschema.