Dear Matbench Discovery team,
Thank you for building and maintaining an excellent benchmarking resource for materials discovery.
I would like to propose the addition of a simple ensemble entry to the leaderboard – constructed from a subset of the currently published models.
Ensemble learning is well known to improve predictive performance when the base models exhibit some degree of diversity. This condition appears to be satisfied in Matbench Discovery, which includes models with substantially different architectures and inductive biases. Compared to previously included ensembles such as CGCNN+P – which combines ten models of the same kind – a heterogeneous ensemble could yield even greater predictive gains.
Including such an ensemble entry (e.g. through simple averaging or a weighted ensemble) could offer several benefits:
- Provide a useful reference point for the upper bound of current model performance.
- Demonstrate the value of diversity among models in a real-world materials discovery setting.
- Encourage users to think critically about combining existing models.
If this suggestion aligns with your goals, I’d be happy to assist in preparing a minimal working example or contributing a PR. Please let me know if this would be a welcome addition, or if there are any specific guidelines to follow when submitting ensemble-based results.
Dear Matbench Discovery team,
Thank you for building and maintaining an excellent benchmarking resource for materials discovery.
I would like to propose the addition of a simple ensemble entry to the leaderboard – constructed from a subset of the currently published models.
Ensemble learning is well known to improve predictive performance when the base models exhibit some degree of diversity. This condition appears to be satisfied in Matbench Discovery, which includes models with substantially different architectures and inductive biases. Compared to previously included ensembles such as CGCNN+P – which combines ten models of the same kind – a heterogeneous ensemble could yield even greater predictive gains.
Including such an ensemble entry (e.g. through simple averaging or a weighted ensemble) could offer several benefits:
If this suggestion aligns with your goals, I’d be happy to assist in preparing a minimal working example or contributing a PR. Please let me know if this would be a welcome addition, or if there are any specific guidelines to follow when submitting ensemble-based results.