-
Notifications
You must be signed in to change notification settings - Fork 504
Open
Labels
questionFurther information is requestedFurther information is requested
Description
Required prerequisites
- I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
- I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
- Consider asking first in a Discussion.
Questions
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("baichuan-inc/Baichuan-7B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("baichuan-inc/Baichuan-7B", device_map="auto", trust_remote_code=True)
inputs = tokenizer('犹有渔人数家住', return_tensors='pt')
inputs = inputs.to('cuda:0')
pred = model.generate(**inputs, max_new_tokens=128,repetition_penalty=1.1,
do_sample=True,
top_k=30,
top_p=0.92,
#temperature=0.001,
num_return_sequences=5)
#print(pred.cpu())
sample_outputs = pred.cpu()
for i, sample_output in enumerate(sample_outputs):
print("{}: {}".format(i, tokenizer.decode(sample_output, skip_special_tokens=True)))
#print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
Checklist
- I have provided all relevant and necessary information above.
- I have chosen a suitable title for this issue.
Metadata
Metadata
Assignees
Labels
questionFurther information is requestedFurther information is requested