Add LFM2.5 (Liquid Foundation Model 2.5) support

[Jacqkues]

Summary

• Add candle-transformers/src/models/lfm2.rs: model implementation for LiquidAI's LFM2.5 hybrid architecture (attention + short convolution layers)
• Add candle-examples/examples/lfm2/: example for text generation with LFM2.5-1.2B and LFM2.5-1.2B-Thinking variants
• Register lfm2 module in candle-transformers/src/models/mod.rs

Model details

LFM2.5 is a hybrid architecture from LiquidAI that interleaves full attention layers with short convolution layers. Key features:

• Per-head QK normalization with RoPE
• Depthwise short convolution with B/C/X gating
• SwiGLU MLP with computed intermediate size
• KV cache (attention) + conv state cache (convolution) for efficient generation

Test plan

• Compiles with cargo build --example lfm2 --release
• Tested inference with LFM2.5-1.2B-Thinking on CPU (Apple Silicon) — generates coherent text at ~5.9 tokens/s

Supported models

• LFM2.5-1.2B
• LFM2.5-1.2B-Thinking

[AYastrebov]

Any progress on this?

[EricLBuehler]

Hey @Jacqkues! Thanks for the PR.

This looks great, and I added a few small comments.

[EricLBuehler]

This caches masks only by t and builds a square (seq_len, seq_len) mask. But after a prefix cache, attention has shape (batch, heads, seq_len, index_pos + seq_len).

This will either fail to broadcast or apply the wrong mask for chunked continuation/prefix-cache use. As such, this should mirror existing Llama/quantized LFM2 behavior where we key by (seq_len, kv_len) and use crate::utils::build_causal_mask(seq_len, index_pos, device), with cache.mask(seq_len, index_pos) at the call site.

[EricLBuehler]

I dont see this model on HF Hub, but I did see LiquidAI/LFM2.5-1.2B-Instruct. Perhaps this is a typo?

[Jacqkues]

Ah yes it was a typo !

[EricLBuehler]

Thank you @Jacqkues!