When will the ground truth answers be made public?

[yuanliping]

When will the ground truth answers be made public?

[keshik6]

We won't be releasing the ground truth answers (both test and dev sets). You can submit your results to the evaluation server (will be available soon) to benchmark your methods on HourVideo. We provide ground truth annotations for 2 sample videos here.

Thanks.

[helloYwen123]

We won't be releasing the ground truth answers (both test and dev sets). You can submit your results to the evaluation server (will be available soon) to benchmark your methods on HourVideo. We provide ground truth annotations for 2 sample videos here.

Thanks.

:) I understand the need for anonymity in the test set, but why is the dev set also not released? Two sample videos are far from sufficient for proper development and tuning. Without the dev ground truth, the dataset is nearly unusable for meaningful experimentation. Could you clarify the reasoning behind keeping both dev and test sets anonymous?

[Wiselnn570]

We won't be releasing the ground truth answers (both test and dev sets). You can submit your results to the evaluation server (will be available soon) to benchmark your methods on HourVideo. We provide ground truth annotations for 2 sample videos here.
Thanks.

:) I understand the need for anonymity in the test set, but why is the dev set also not released? Two sample videos are far from sufficient for proper development and tuning. Without the dev ground truth, the dataset is nearly unusable for meaningful experimentation. Could you clarify the reasoning behind keeping both dev and test sets anonymous?

same concern