Skip to content

feat(gallery): add Hy-MT2 7B GGUF variants - #12164

Open
localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20260920-run4
Open

localai-org-maint-bot wants to merge 1 commit into
masterfrom
cron/model-gallery-20260920-run4

Conversation

@localai-org-maint-bot

Copy link
Copy Markdown
Collaborator

Description

Add Hy-MT2-7B Q4_K_M, Q6_K, and Q8_0 builds to the model gallery, linked as variants. This fills the gap beside the existing 1.8B translation model. Include installation commands and a translation prompt example.

Notes for Reviewers

  • Official weights and original model; Apache-2.0.
  • All three downloads are pinned to ab8472660ac61fac25f1af43fac2599d52a8a775. Each SHA256 matches both HF LFS metadata and the pinned download's x-linked-etag header. Preserve the publisher's different Hy/HY filename casing.
  • HF metadata identifies hunyuan-dense, supported by LocalAI's pinned llama.cpp revision 50631b3d2c569ad8e5c112090cd28570b1268ee0. These are standard K-quants/Q8_0, not the separate STQ formats.
  • Use native Jinja chat formatting, publisher sampling defaults, and an 8K initial context. Users can increase context within the model's 262K limit.
  • go test ./core/gallery -count=1 -ginkgo.focus='gallery/index.yaml|gallery variant lint helpers' passes. A semantic YAML comparison confirms all 1,890 existing entries remain unchanged. All new model paths and hashes match pinned HF metadata. git diff --check passes.
  • Full go test ./core/gallery -count=1: 392 passed, 3 failed, 9 skipped, identically on unchanged master and this branch. Failures are Forbidden responses from external GitHub/gist fixtures at request_test.go:22, backends_test.go:208, and backends_test.go:213.
  • Inference was not run locally.

Signed commits

  • Yes, I signed my commits. Human DCO sign-off is required; repository policy prohibits AI sign-off.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

Offer the official Q4_K_M, Q6_K, and Q8_0 builds for translation.
Pin the downloads and document installation and translation prompts.

Assisted-by: Codex:gpt-6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant