TypeError: Accelerator.__init__() got an unexpected keyword argument 'dispatch_batches'

[SiyuWu528]

System Info

transformers==4.37.2 python 3

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

#huggingface trainer, to train the model
from transformers import Trainer, TrainingArguments

model.to(device)
model_name = f"{model_ckpt}-finetuned"
batch_size = 2
training_args = TrainingArguments(
output_dir= model_name,
save_safetensors = False,
per_device_train_batch_size=batch_size,
per_device_eval_batch_size=batch_size,
evaluation_strategy='epoch',
logging_strategy='epoch',
learning_rate=1e-5,
num_train_epochs=10,
weight_decay=0.01,
gradient_accumulation_steps=2,
max_grad_norm=1.0,
optim='adamw_torch'
)

trainer = Trainer(
model= model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
compute_metrics=compute_metrics

)

trainer.train()

Expected behavior

I was run perfectly fun before Nov 12th midnight, and stop working on Nov 13th.....

[PrettyBoyHelios]

same here, trying to figure out what hapened

[andresmijares]

same...

I can confirm that downgrading to 0.28.0 still works

[LysandreJik]

cc @muellerzr, seems high priority given the number of users impacted

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[Rocketknight1]

gentle ping @muellerzr @SunMarc

[SunMarc]

Do you still have the issue with the latest transformers @andresmijares @PrettyBoyHelios @SiyuWu528 ? A minimal reproducer will be very helpful

[rsanchezpizani]

Yes, as per this morning still the issue persists.

accelerate 1.2.1 pypi_0 pypi
transformers 4.33.3 pypi_0 pypi

TypeError: Accelerator.init() got an unexpected keyword argument 'dispatch_batches'

[SunMarc]

Could you share a minimal reproducer @rsanchezpizani ?

[zyandtom]

Yes, as per this morning still the issue persists.

accelerate 1.2.1 pypi_0 pypi transformers 4.33.3 pypi_0 pypi

TypeError: Accelerator.init() got an unexpected keyword argument 'dispatch_batches'

degrading accelerate to 0.34.2 worked

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[j-adamczyk]

@LysandreJik @muellerz we're also getting this on production. Downgrading accelerate is not really a solution.

[SunMarc]

Please share a traceback or a minimal reproducer @j-adamczyk . We can't do much without enough information.

[j-adamczyk]

Relevant stacktrace part:

Traceback (most recent call last):
  File "/opt/ml/code/train.py", line 121, in <module>
    gmae = train_model(kpi, channel, config, dataset)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/ml/code/train.py", line 74, in train_model
    trainer = HuberLossTrainer(
              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/trainer.py", line 347, in __init__
    self.create_accelerator_and_postprocess()
  File "/usr/local/lib/python3.11/site-packages/transformers/trainer.py", line 3979, in create_accelerator_and_postprocess
    self.accelerator = Accelerator(

TypeError: Accelerator.__init__() got an unexpected keyword argument 'dispatch_batches'

Relevant training code:

default_training_args = dict(
    optim="adafactor",
    learning_rate=5e-5,
    num_train_epochs=3,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=16,
    evaluation_strategy="steps",
    save_steps=100,
    eval_steps=100,
    logging_strategy="no",
    disable_tqdm=True,
    save_total_limit=1,
    load_best_model_at_end=True,
    seed=0,
    data_seed=0,
    dataloader_num_workers=1,
    bf16=True,
    tf32=True,
)

class HuberLossTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False):
        labels = inputs["labels"]
        outputs = model(**inputs)
        logits = outputs["logits"].flatten()
        loss = huber_loss(logits, labels)
        return (loss, outputs) if return_outputs else loss

trainer = HuberLossTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    eval_dataset=dataset["valid"],
    tokenizer=tokenizer,
    compute_metrics=compute_gmae,
)
trainer.train()

This is a part of automated pipeline. It worked last on 05.01.2025, stopped working with the error above on 26.01.2025. Right now I'm trying downgrading accelerate version by version to pinpoint this.

Linux OS, relevant software versions:

Package                                  Version
---------------------------------------- --------------
accelerate                               1.3.0
torch                                    2.2.2+cu121
transformers                             4.37.2

Accelerate version was chosen by Poetry from:

torch = {version = "2.2.*", source = "pytorch-gpu", markers = "extra!='pytorch-cpu'"}
transformers = {version = "4.37.*", extras = ["torch"]}

[tool.poetry.extras]
pytorch-cpu = []

[[tool.poetry.source]]
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

[[tool.poetry.source]]
name = "pytorch-gpu"
url = "https://download.pytorch.org/whl/cu121"
priority = "explicit"

[SunMarc]

Nice, I will try to find the issue !

[SunMarc]

You need to update your version of transformers to the latest one ! When we switched to accelerate v1, we deprecated a few arguments in Accelerator class. If you don't want to upgrade transformers, you can also downgrade accelerate.

[j-adamczyk]

Why isn't this covered by package requirements then? I mean, if a breaking change is introduced, you can limit the upper allowed version of transformers. Then Poetry wouldn't even allow me to run this setup, because it would fail on dependency resolution

[SunMarc]

accelerate don't depend on transformers

[infnetdanpro]

I found the solution:

pip install transformers==4.39.1
pip install accelerate==0.27.2

blame the developers who don't specify the correct dependencies

[marthos1]

모두 감사합니다. 제 포를 찾아주세요..

[FHYQ-Dong]

accelerate don't depend on transformers

if is_accelerate_available():
    from accelerate import Accelerator, skip_first_batches
    from accelerate import __version__ as accelerate_version
    from accelerate.utils import DistributedDataParallelKwargs, GradientAccumulationPlugin

    if version.parse(accelerate_version) > version.parse("0.20.3"):
        from accelerate.utils import (
            load_fsdp_model,
            load_fsdp_optimizer,
            save_fsdp_model,
            save_fsdp_optimizer,
        )

But transformers depends on accelerate (at least to some extend). Covered by package requirements may be a better choice.