-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to reproduce the performance on the ReDial dataset? #8
Comments
I also encountered the same problem as you. Since the author provided the missing files, I trained according to the guidance. However the metric results also similar to yours. Here are the detail of redial dataset metrics: conv:
rec:
Here is my config of conversational task:
(1)@dandyxxxx, you can see that I set 'n_prefix_conv=50,' but the results don't match the paper. Could you share your configuration details? Maybe we can work together to solve the problem. Thank you very much! |
Thanks for your replying! This help me a lot. |
@linshan-79 I have the same problem. Did you finally solve it? Thank you so much. |
I trained according to the code provided on GitHub, but since the dataset link you provided cannot be opened, I used mapping based objects_ Lang=en_ 202112.ttl dataset. The final results of my training are as follows:
conv:
'test/dist@2': 0.310709750246931, 'test/dist@3': 0.49851841399746016, 'test/dist@4': 0.6383519119514605
rec:
'test/recall@1': 0.029324894514767934, 'test/recall@10': 0.16729957805907172, 'test/recall@50': 0.37953586497890296
(1)These results differ greatly from the results presented in the paper. Can you give me some guidance? I hope to reproduce results similar to yours. Thank you very much.
(2)According to your paper, do I need to set --n_prefix_conv 50 in the train_conv.py and --use_resp in the train_rec. py?
The text was updated successfully, but these errors were encountered: