lose RAMS dataset file?

[caddy-liu]

Hi.

I tried to follow the code on dataset RAMS.And I changed the argument called 'test-file' in convert_gen_to_output.py to 'data/RAMS_1.0/data/test.jsonlines' when evaluating head score but only received low performance as follow:

Precision: 33.5640 Recall: 36.2133 F1: 34.8384.

I did't find the file called 'test_head_coref.jsonlines' and 'test_head.jsonlines'. Are they include in RAMS or created by yourself?

It puzzled me. Did I downloaded wrong version? My version: RAMS_1.0c.tar.gz [current as of March 2023, 4.5MB].

Thanks for reply.

[raspberryice]

The two files are processed by us. See the end of the README.md for instructions on how to download these files. Note that only Span F1 (should be the original exact match score) and Head F1 (match of head words) are reported in our paper.

We reported these metrics to match a previous paper "A two-step approach for implicit event argument detection.", not the original RAMS paper because the evaluation setting was different.

[caddy-liu]

Thank you very much for your answer！