1.依赖

python == 3.8.0

allennlp == 2.4.0

pip install allennlp -i https://pypi.tuna.tsinghua.edu.cn/simple

2.训练

2.1 使用脚本训练

train

python train.py

eval

python evaluate.py

predict

python predict.py

2.2 使用 allennlp train 训练 embedding+bag_of_embedding

train

allennlp train scripts/my_text_classifier.jsonnet --serialization-dir checkpoint --include-package my_text_classifier

eval

allennlp evaluate checkpoint/model.tar.gz data/movie_review/test.tsv --include-package my_text_classifier

predict

allennlp predict checkpoint/model.tar.gz data/movie_review/test.jsonl --include-package my_text_classifier --predictor sentence_classifier

2.3 使用 allennlp train 训练 bert embedding+bert pool

这里需要下载bert预训练模型放到``bert_pretrain`文件夹下

train

allennlp train scripts/my_text_classifier_bert.jsonnet --serialization-dir checkpoint_bert --include-package my_text_classifier

eval

allennlp evaluate checkpoint_bert/model.tar.gz data/movie_review/test_small.tsv --include-package my_text_classifier

predict

allennlp predict checkpoint_bert/model.tar.gz data/movie_review/test.jsonl --include-package my_text_classifier --predictor sentence_classifier