How to test Ablation study results? #14
Description
Hello, I was trying to understand the importance of the modules in your framework and accordingly I was following the instructions given in the ablation study.
This is how I changed the code as per the instructions to test it on the Epilepsy dataset:
In the TimeDart.py file
class ClsModel(nn.Module):
def pretrain(self, x):
x_out = self.encoder(
x_embedding_bias,
is_mask=False,#change made for w/o AR
)
Similarly for the w/o DIFF section, I commented the diffusion code and simply changed the dimensions of x_out:
# Noising Diffusion
# noise_x_patch, noise, t = self.diffusion(
# x
# ) # [batch_size, seq_len, patch_len]
# noise_x_embedding = self.enc_embedding(
# noise_x_patch
# ) # [batch_size, seq_len, d_model]
# noise_x_embedding = self.positional_encoding(noise_x_embedding)
# # For Denoising Patch Decoder
# predict_x = self.denoising_patch_decoder(
# query=noise_x_embedding,
# key=x_out,
# value=x_out,
# is_tgt_mask=True,
# is_src_mask=True,
# ) # [batch_size, seq_len, d_model]
# For Decoder
predict_x = self.projection(x_out) # [batch_size, input_len, num_features]
After making these changes, the results we obtained were significantly different from those in the paper; more specifically, the results actually improved after removing these modules. Can you help us figure out what went wrong?