Skip to content

Decision: batch the prompts of a request with slice_logits_to_keep - #50548

Open
ArthurZucker wants to merge 7 commits into
mainfrom
decision-batching
Open

Decision: batch the prompts of a request with slice_logits_to_keep#50548
ArthurZucker wants to merge 7 commits into
mainfrom
decision-batching

Integration tests: device_map="auto" so 27-35B checkpoints fit the 24…

99c0040
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar