Why are my test results far better than the paper results?

[hexiangdong2020]

I used Husformer to test it on the WESAD dataset and got the final result (Acc: 91.02%, F1: 87.38%). The results of this test are much higher than those tested on the WESAD dataset in the paper (Acc: 78.68%, F1: 79.51%). Why is this happening? Is it because the paper uses a different metric than the open source code? @zdz0086 @SMARTlab-Purdue @wonsu0513 @R7-Robot

[xnes]

Hello @hexiangdong2020 , how did you make to run the code? I am trying to use cogload but I'm lost with the bugs.

Did you have to do a lot of changes?

[Marfbin]

Hi, @hexiangdong2020 , I'm having trouble replicating the husformer code using the DEAP dataset. The definition of output_dim=1 in the main.py file causes the final output shape of the model to look like this (1024, 1), which causes problems in focalloss for calculating log_softmax for the first dimension, which results in logpt values being all zeros. How did you solve this problem？

[Shahtaj-Khalid]

I used Husformer to test it on the WESAD dataset and got the final result (Acc: 91.02%, F1: 87.38%). The results of this test are much higher than those tested on the WESAD dataset in the paper (Acc: 78.68%, F1: 79.51%). Why is this happening? Is it because the paper uses a different metric than the open source code? @zdz0086 @SMARTlab-Purdue @wonsu0513 @R7-Robot

Hi, is it possible for you to upload your test code? I'm trying run Husformer for WESAD too but facing issues.

[Bluemax666]

I used Husformer to test it on the WESAD dataset and got the final result (Acc: 91.02%, F1: 87.38%). The results of this test are much higher than those tested on the WESAD dataset in the paper (Acc: 78.68%, F1: 79.51%). Why is this happening? Is it because the paper uses a different metric than the open source code?

In the paper they say that the metrics used are average multi-class accuracy (Acc) and multiclass average F1-score (F1). But in the code its just the accuracy and a weird f1 score that outputs a different value than sklearn.metrics.f1_score. The basic accuracy is higher than the average multiclass accuracy because the number of samples of each class is not balanced, and the less represented classes count less in the accuracy than in the average multiclass accuracy.

[kuanyshbakytuly]

Hi, @hexiangdong2020 , I'm having trouble replicating the husformer code using the DEAP dataset. The definition of output_dim=1 in the main.py file causes the final output shape of the model to look like this (1024, 1), which causes problems in focalloss for calculating log_softmax for the first dimension, which results in logpt values being all zeros. How did you solve this problem？

hi, could you share with remake_label() function?

[zhangning105]

Hi，@kuanyshbakytuly，do you have the remake_label() function yet? Could you share it, thanks a lot!

[tgnrv]

@Shahtaj-Khalid hello, did you handle the issues with your code? If yes, could share, please? I am also on testing WESAD dataset.

[tgnrv]

@kuanyshbakytuly , salam, did you test it properly? Can you share with your code?

[Texasnum]

@tgnrv Hello, I'm also running the WESAD dataset and have encountered the problem of remake_label(). If you have solved it, could you share your solution? Thank you!