[EvanZhuang]

What does this PR do?

BERT-like models used as decoders (is_decoder=True, e.g. BertLMHeadModel, RobertaForCausalLM) build their self-attention mask in _create_attention_masks without position_ids, so create_causal_mask can't detect packed (padding-free) inputs. With eager and sdpa, each document in a packed row then attends to the documents before it. FA2 is not affected, since it gets position_ids directly.

Reproducer (CPU, float32):

import torch

from transformers import AutoTokenizer, BertLMHeadModel

tok = AutoTokenizer.from_pretrained("google-bert/bert-base-uncased")
docs = ["The cat sat on the mat.", "Paris is the capital of France and a large city."]
ids = [tok(d)["input_ids"] for d in docs]

for attn in ("eager", "sdpa"):
    model = BertLMHeadModel.from_pretrained("google-bert/bert-base-uncased", is_decoder=True, attn_implementation=attn)
    model.eval()
    with torch.no_grad():
        # each document on its own
        separate = torch.cat([model(input_ids=torch.tensor([x])).logits[0] for x in ids])
        # the same documents packed in one row, padding-free: position_ids restart at each document
        packed_ids = torch.tensor([sum(ids, [])])
        position_ids = torch.cat([torch.arange(len(x)) for x in ids]).unsqueeze(0)
        packed = model(input_ids=packed_ids, position_ids=position_ids, use_cache=False).logits[0]
    print(f"{attn}: max |separate - packed| = {(separate - packed).abs().max().item():.2e}")

On main this prints 1.34e+01 for eager and sdpa; with this PR, 6.82e-05.

This PR passes position_ids from forward through _create_attention_masks (new optional argument) to create_causal_mask. The mask only changes for packed inputs (position_ids given, no attention_mask, no cache), as in the other decoder models.

Models: the change is in BertModel. utils/check_copies.py --fix_and_overwrite updated the copies of BertModel._create_attention_masks (BertGeneration, BridgeTower, Electra, Esm, RobertaPreLayerNorm, RoCBert, Xmod), and I passed position_ids at their call sites. utils/modular_model_converter.py regenerated Roberta and XLMRobertaXL (from BertModel), Camembert, Data2VecText and XLMRoberta (from RobertaModel), and Ernie (call site in modular_ernie.py).

Test: attention_mask_padding_matches_padding_free_with_position_ids should cover this, but it builds the model from the common tester config, where these models have is_decoder=False. Their LM heads then ran with a bidirectional mask, which has no packed-sequence support, and the test only passed because of its loose tolerance. The test now sets config.is_decoder = True, like the other generation tests (encoder-decoders are already skipped). The XLM-RoBERTa-XL eager/sdpa variants, skipped with "doesn't work for some reason, FIXME", now pass, so I removed the skips.

This is needed for the follow-up to the review of #50450 (a separate PR that scales this test's tolerance to the logits): with that tolerance, these tests fail without this PR.

Measurements

CPU, with the testers' own configs and inputs.

• Padded vs padding-free logits, float32, as decoders, 5 seeds, max|diff| / max|logits|: 0.9-2.3% on main for Bert, BertGeneration, Camembert, Data2VecText, Electra, Ernie, Esm, Roberta, RobertaPreLayerNorm, RoCBert, XLMRoberta, XLMRobertaXL and Xmod (BridgeTower text model: 15%); at most 2.6e-7 with this PR.
• Eager/sdpa padding-free tests of the 9 affected test classes, 10 runs each: with the follow-up's tolerance and is_decoder=True, 160 of 180 fail with the current modeling code and 0 of 180 with this PR. Without is_decoder=True they fail in about 30% of runs whether or not the modeling code is fixed, because the bidirectional mask ignores packing. With the current tolerance they pass (180/180 with this PR).
• The full test files of the 14 models pass as on main (1570 passed vs 1568; the two extra are XLM-RoBERTa-XL's). The eager/sdpa padding-free tests of all of tests/models show no new failures, and the FA2 variants of the 9 test classes pass on an H100.

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: bert, bert_generation, bridgetower, camembert, data2vec, electra, ernie, esm, roberta, roberta_prelayernorm, roc_bert, xlm_roberta, xlm_roberta_xl, xmod