自己代码生成的: are?CC(=O)OC(C)(C)C 生成的很混乱，并且自己代码写出来的模型生成文本还会带上输入给模型的prompt

[yzlbig]

Reminder

• I have read the above rules and searched the existing issues.

System Info

Reproduction

`def main(file_path):
    with open(file_path, 'r') as file:
        data = json.load(file)

    data = data[:5]  
    
    prompts = [item["prompt"] for item in data]
    targets = [item["target_new"] for item in data] 

 
    model_path = "export_llama2-7b-chat"
    tokenizer_path = "export_llama2-7b-chat"
    
    device = 'cuda:0'
    tok = AutoTokenizer.from_pretrained(model_path)
    tok.padding_side = 'right'
    model = ModelParallelLlama.from_pretrained(tokenizer_path, device_map='auto', torch_dtype=torch.float16,
        output_hidden_states=True
        )
    max_prompt_len = max([len(tok.encode(_)) for _ in prompts]) + 1

    max_target_len = max([len(tok.encode(_)) for _ in targets]) + 1


    print(model)
    for prompt, target_new in zip(prompts, targets):
    # prompt = "The reactants in the single retrosynthesis of CC(C)(C)OC(=O)Nc1nc2c(-c3cccc([N+](=O)[O-])c3)cc(C#N)cc2s1 are"
    # target_new = "CC(C)(C)OC(=O)OC(=O)OC(C)(C)C.N#Cc1cc(-c2cccc([N+](=O)[O-])c2)c2nc(N)sc2c1"
    
        prompt_tok = tok(
            prompt, 
            padding="max_length", 
            max_length=max_prompt_len, 
            return_tensors="pt"
        ).to(device)
        tok.padding_side = 'right'
        
        target_new_tokens = tok.encode(target_new, add_special_tokens=False)

        token_ids = prompt_tok['input_ids'].to(device)

        gen_token = model.generate(
                input_ids=token_ids,
                attention_mask=prompt_tok['attention_mask'],
                max_new_tokens=len(target_new_tokens),
                pad_token_id=tok.eos_token_id,
                use_cache=False,
            )

        generated_text = tok.decode(gen_token[0], skip_special_tokens=True)
        target_new_text = tok.decode(target_new_tokens, skip_special_tokens=True)
        print("target_new_text：", target_new_text)
        # 打印生成的文本
        print("Generated Text:", generated_text)

if __name__ == "__main__":
    
    file_path = "data/uspto50k_canonical_smiles_test.json"
    main(file_path)`

Others

自己生成的（prompt和webui中chat模块一样The reactants in the single retrosynthesis of O=C(NCc1ccccc1S(=O)(=O)C1CC1)C(F)(F)F are?）：
target_new_text： NCc1ccccc1S(=O)(=O)C1CC1.O=C(OC(=O)C(F)(F)F)C(F)(F)F
Generated Text: are?CC(=O)OC(C)(C)C

webui中chat模块生成的（System Info上贴了对应的图）：
问：The reactants in the single retrosynthesis of O=C(NCc1ccccc1S(=O)(=O)C1CC1)C(F)(F)F are?
答：NCc1ccccc1S(=O)(=O)C1CC1.O=C(OC(=O)C(F)(F)F)C(F)(F)F

我想问问是我代码中的generate方法出问题了吗，还是哪个方法出问题了，我实在是找不出来。想不通的是，我自己代码写出来的模型生成文本还会带上输入给模型的prompt，如下面的Generated Text所示。

target_new_text： CC(C)(C)OC(=O)OC(=O)OC(C)(C)C.N#Cc1cc(-c2cccc(N+[O-])c2)c2nc(N)sc2c1
Generated Text: The reactants in the single retrosynthesis of CC(C)(C)OC(=O)Nc1nc2c(-c3cccc(N+[O-])c3)cc(C#N)cc2s1 are?[NH4+]

懂得问题出在哪的朋友能帮忙看看吗，万分感谢。

[yzlbig]

总的疑惑就是，同样的prompt（比如说The reactants in the single retrosynthesis of O=C(NCc1ccccc1S(=O)(=O)C1CC1)C(F)(F)F are?）webui中的chat模块给的是合理的，我自己写的代码得到的结果就很混乱，懂得问题出在哪的朋友能帮忙看看吗，实在是感谢哇！

[yzlbig]

输入给模型的prompt是：The reactants in the single retrosynthesis of O=C(NCc1ccccc1S(=O)(=O)C1CC1)C(F)(F)F are?
自己代码生成的: are?CC(=O)OC(C)(C)C 生成的很混乱，并且自己代码写出来的模型生成文本还会带上输入给模型的prompt
webui中chat模块生成的：NCc1ccccc1S(=O)(=O)C1CC1.O=C(OC(=O)C(F)(F)F)C(F)(F)F 生成的比较合理
这是标准答案：NCc1ccccc1S(=O)(=O)C1CC1.O=C(OC(=O)C(F)(F)F)C(F)(F)F

我想问问自己写的有什么问题？