Decision: batch the prompts of a request with slice_logits_to_keep - #50548

Decision: batch the prompts of a request with slice_logits_to_keep#50548

ArthurZucker wants to merge 7 commits into

mainhuggingface/transformers:mainfrom

decision-batchinghuggingface/transformers:decision-batchingCopy head branch name to clipboard