Add TeleChat - #50477

Summary

Adds native support for TeleChat (v1, TeleChat2 and TeleChat2.5, model_type: "telechat"). vLLM's native backend loads it into Llama's implementation, and this lets vLLM serve it through its Transformers backend without remote code. InternLM3 (#50578), IQuestCoder (#50579) and TeleChat3 (#50580) were split out of this PR.

One model covers every checkpoint with model_type: "telechat", except TeleChat-52B. That is a different architecture, and its config is rejected with a pointer to trust_remote_code. Differences from Llama:

• Weights: only o_proj and down_proj have biases, so the attention and MLP __init__s differ from Llama's. Their forwards are Llama's.
• Conversion on load: the original BLOOM-style names (transformer.h.N.self_attention.{query,dense}, word_embeddings, ln_f) are renamed. The fused self_attention.key_value is laid out per head ([k_0, v_0, k_1, v_1, …]). A TeleChat-only TeleChatInterleaveKeyValue op regroups it so that Chunk can split it into k_proj and v_proj. save_pretrained writes the original layout back, and base-model-only checkpoints (Tele-AI/telechat-7B) also load. vLLM's Transformers backend needs [Transformers backend] Apply Transformers' WeightConverters when loading weights vllm-project/vllm#66811 to apply the converter.
• Config: the original keys (n_layer, n_head, ffn_hidden_size, layer_norm_epsilon, seq_length) are mapped explicitly in __post_init__. rope_scaling is ignored, because the original code never reads it.
• RoPE: training_seqlen and base_seqlen move into rope_parameters:

-training_seqlen: 8192, base_seqlen: 8192, seq_length: 32768
+max_position_embeddings: 32768
+rope_parameters: {rope_type: "telechat_ntk", rope_theta, original_max_position_embeddings: 8192, base_seqlen: 8192}

Once the sequence outgrows original_max_position_embeddings, the original code applies a stepwise NTK scaling. TeleChatRotaryEmbedding reproduces it:

L = max(seq_len, original_max_position_embeddings)
ntk_alpha = 2 ** ceil(log2(L / base_seqlen) + 1) - 1     # base_seqlen defaults to 4096, as in TeleChat v1
base = rope_theta * ntk_alpha ** (dim / (dim - 2))
cos, sin *= 0.1 * ln(L / original_max_position_embeddings) + 1

The frequencies are cached in the inv_freq buffer, like dynamic_rope_update:

forward(x, position_ids)
  L = max(position_ids.max() + 1, original_max_position_embeddings)
  if L != cached L
    inv_freq, scaling = original (L at the training length) or recomputed for L
  # under torch.compile the cache check can't be traced, so inv_freq and scaling are recomputed instead

Unlike dynamic_rope_update, this rescales for the exact current length, as the original does, not the longest length seen so far.

Tokenizers

The checkpoints' tokenizer_config.json names Telechat2Tokenizer, which only exists in their remote code. telechat is added to MODELS_WITH_INCORRECT_HUB_TOKENIZER_CLASS, so the tokenizer loads as TokenizersBackend from the bare tokenizer.model without trust_remote_code. That conversion only matches the original tokenizer with #50555. Merge #50555 first.

Supersedes the stale v4-based #35694.

Evidence

Random tiny models built with each of the 5 versions of the original code. With training_seqlen=64, a 200-token input puts every position past it (NTK alpha 15, mscale 1.11), and the cached greedy decode crosses it:

All of the remaining difference is fp16 rounding. The original rounds cos/sin to fp16 before and after applying mscale, which the native implementation deliberately doesn't do. *The original v1 code crashes once cached decoding passes training_seqlen, because its rotary embedding uses the chunk length rather than the total length. So that comparison covers the 5 steps it completes.

Real weights:

*The eager path of these versions of the original code only works in fp16.

• The cached rotary embedding gives the same cos/sin as recomputing for every length, for sequences that grow, shrink and cross the training length.
• TeleChat2-3B's Hub config loads as max_position_embeddings=32768 and rope_parameters={"rope_type": "telechat_ntk", "rope_theta": 10000.0, "original_max_position_embeddings": 8192, "base_seqlen": 8192}.
• RUN_SLOW=1 integration tests (logits and generation for TeleChat2-3B, default dtype="auto", which loads the bf16 weights): 2 passed on B200, after the RoPE caching change.
• Fast tests: the common suite, plus the NTK formula checked against the original at boundary lengths. TeleChatInterleaveKeyValue has a split-and-merge round-trip unit test.

Merge Danger

Door: two-way

A new model type, a new telechat entry in conversion_mapping and a new TeleChat-only conversion op. Depends on #50555 for tokenization. For vLLM, TeleChat also needs vllm-project/vllm#66811.

Blast Radius: new model

Loading these checkpoints without trust_remote_code now resolves to the native config, model and tokenizer, where it previously required remote code. With trust_remote_code=True, auto_map still takes precedence. For TeleChat-52B (same model_type), the config now fails with an explicit error instead of a remote-code prompt, unless trust_remote_code=True is passed.

🤖 Generated with Claude Code