Skip to content

Changes to support MS MARCO#68

Closed
ryan-clancy wants to merge 6 commits into
masterfrom
marco
Closed

Changes to support MS MARCO#68
ryan-clancy wants to merge 6 commits into
masterfrom
marco

Conversation

@ryan-clancy

Copy link
Copy Markdown
  • Add the qrels file
  • Make the measures parameter for trec_eval configurable

@ryan-clancy ryan-clancy requested review from charW and lintool May 10, 2019 15:06
Comment thread searcher.py Outdated
@ryan-clancy

Copy link
Copy Markdown
Author

@lintool How should topics be handled for MS MARCO? Currently, topic is a required parameter for the search hook - should we extract the topics file (queries.dev.small.tsv ?) and include it in jig, or make topic an optional parameter (with some collection == "msmarco" check) so the image generates the topics (like here?)

@lintool

lintool commented May 11, 2019

Copy link
Copy Markdown
Member

msmarco formatting should define a generic topic type, and the topics themselves in the tsv format should be passed in from outside the jig, i.e., in the same we pass in TREC topic files.

@ryan-clancy ryan-clancy closed this Jun 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants