Expand model test matrices to cover LoHa, LoKr, and LN Tuning

[emredeveloper]

Motivation

The shared model test matrices explicitly note that LoHa, LoKr, and LN Tuning are still missing:

• tests/test_decoder_models.py
• tests/test_encoder_decoder_models.py
• tests/test_feature_extraction_models.py

Adding these methods would exercise their behavior across causal LM, seq2seq, and feature-extraction architectures through the existing common tests, including training, save/load, merge/unmerge where supported, adapter deletion/unload, disabling adapters, and gradient checkpointing.

I searched the current issues and open pull requests for this test-matrix work and did not find an overlapping contribution.

Proposed scope

If maintainers agree that this coverage is useful, I would like to:

1. Add LoHaConfig and LoKrConfig to the applicable ALL_CONFIGS matrices.
2. Add LNTuningConfig where the tested model architecture has a supported/default target-module mapping.
3. Run the focused matrix cases on CPU first and address only compatibility failures directly exposed by this coverage.
4. Keep the work in one PR across the three task families, rather than splitting small changes by method or file.

Questions before coding

1. Would you welcome this test coverage as one PR?
2. For LN Tuning, the current default mapping covers decoder and T5/BART-style architectures, but not BERT, RoBERTa, or DeBERTa used by the feature-extraction matrix. Would you prefer:
   • extending the default LN Tuning mappings where the architecture has appropriate normalization layers,
   • passing explicit target_modules only in the tests, or
   • limiting the initial coverage to architectures already present in the mapping?
3. Is adding all three methods to the existing broad matrices acceptable for CI runtime, or would a smaller representative subset be preferable?

I will wait for maintainer approval and scope guidance before making code changes or opening a PR.

AI assistance disclosure

AI assistance was used to inspect the repository, search for overlapping issues/PRs, and draft this coordination issue. I reviewed the proposed scope and will be responsible for understanding any eventual changes and running the relevant tests.

[BenjaminBossan]

Thanks for offering to extend the test matrices. Yes, adding LoKr, LoHa, and LN-Tuning to these three files would be appreciated. test_seq_classifier.py is also missing them. There is no need to add LN-Tuning if the used models don't have layers that are targeted by default, adding a comment to state this is enough. The runtime overhead should be acceptable. Please ensure that the newly added tests all pass (ideally on CPU and GPU, but if the latter is not available, CPU only is okay).