bio_interface inference returns 0.0-only scores — is script update needed?

[SarahNozawa]

Hi, I’m using deeprank-gnn-esm via the pip-installed CLI.

1. I installed the package with pip and confirmed that fnat prediction works with the provided CLI (deeprank-gnn-esm-predict).
2. I then switched the target to bio_interface (and used a checkpoint whose target is bio_interface) and confirmed that inference runs without crashing.
3. However, the output score I get is always 0.0 (or 0/1 class label) and seems stuck at 0, so I can’t obtain the probability-like continuous score I expected.
   In the output HDF5, outputs is an integer label and raw_outputs is a 2D vector (2-class). It’s unclear what post-processing is expected to get probabilities.
4. Question: Is a fundamental change needed in the prediction/CSV export script to support bio_interface properly, or is there an intended way to get probabilities with the current CLI?

If you can point me to the recommended inference path/options for bio_interface (probability output), that would help a lot.

Thanks!

[ntxxt]

Hi there,

I have investigated it. Indeed, the support for bio_interface inference needs to be updated.
But to solve your problem, if you use the script to convert the output hdf5 into csv output,
src/deeprank_gnn/tools/hdf5_to_csv.py
you will ger the continuous output as follows;

epoch	set	model	targets	prediction	raw_prediction_0	raw_prediction_1
epoch_0000	test	1ATN_1w_model_0	n	0	0.7302980423	0.2697019875

We will update the CI but I hope it helps for now.

Best,

[SarahNozawa]

Thanks for the reply!

I tried hdf5_to_csv.py (after a slight fix at my end) and can see raw_prediction_0/1, but the outputs don’t look like useful probabilities like yours: even when I change the input structure including your example (1ATN_1w), the model seems saturated and returns [1.0, 1.576..e-11(a very tiny number)] every time. So I can’t get any meaningful score...

Any idea what could cause this (e.g., expected post-processing or a mismatch in the inference setup)?

[ntxxt]

Hi there,

One reason I can think of is the possible mis-match between deeprank-gnn-esm versions. Currently, we are maintaining the code at https://github.com/haddocking/deeprank-gnn-esm intead of this one.
So maybe update all the code based on that branch and use the checkpoint there under
paper_pretrained_models/biological_vs_crystal_interfaces/gnn_esm

This is the version I used to do the testing above. You should get the same result!