Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use llama-8b-instruct tokenizer.json and tokenizer_config.json for `l…
…lama3_8b_fp16` dataset (#953) The tokenizers specified for this dataset are for `llama3_8b_fp16`, while the model is `llama3_8b_fp16_instruct`. The `eos_token` for `8b` and `8b-instruct` are different: ```text 8b: <|begin_of_text|> {generated_text} <|end_of_text|> <|end_of_text|> - 128001 8b-Instruct: <|begin_of_text|> {generated_text} <|eot_id|> <|eot_id|> - 128009 ``` Using the wrong config causes Llama to output text forever. Our model generated `128009`s, but the server doesn't recognize it as the proper stop token and keeps calling for generations. More details [here](#934 (comment))
- Loading branch information