[Windows][E2E] Several models got fail_accuracy on BMG/LNL.

[libohao1201]

🐛 Describe the bug

E2E (hf & timm) models got fail_accuracy

python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype bfloat16 --only crossvit_9_240  
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype bfloat16 --only jx_nest_base
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype float16 --only crossvit_9_240  
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype float16 --only jx_nest_base 
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --bfloat16  --only convmixer_768_32 
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --bfloat16  --only convnext_base 
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --bfloat16  --only rexnet_100
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --bfloat16  --only sebotnet33ts_256 
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --float16  --only convnext_base 

:: On BMG
python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype bfloat16 --only gernet_l

:: On BMG
python benchmarks\dynamo\huggingface.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --float16   --only DebertaV2ForQuestionAnswering

Versions

stock pytorch	pip install torch --index-url https://download.pytorch.org/whl/test/xpu git clone https://github.com/pytorch/pytorch.git git checkout b1940b5867e40e40ebdce4db76f76d3d0b71d3f4
torch-xpu-ops	Commit (pin): 026b2c8
Driver	32.0.101.6647
Conda	python 3.10

[Stonepia]

Update:

python benchmarks\dynamo\huggingface.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --float16   --only DebertaV2ForQuestionAnswering

@libohao1201 , could you check with a newer version of transformers? This has encountered before: #1216

[libohao1201]

Verified on BMG with latest driver (6793) + Pytorch 2.8 (3560b8) + transformer(243e186e - pinned), and the following cases still have "fail_accuracy" error.

python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype bfloat16 --only jx_nest_base

python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --inference --amp --amp-dtype float16 --only jx_nest_base

python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --bfloat16  --only convnext_base

python benchmarks\dynamo\timm_models.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --float16  --only convnext_base  

python benchmarks\dynamo\huggingface.py --accuracy -d xpu -n10 --backend=inductor --cold-start-latency --training  --float16   --only DebertaV2ForQuestionAnswering