Search results

Run a full (shell) DAG benchmark

Status: Open.

#1530 In mlcommons/modelbench;

· bollacker opened on Jun 9, 2026

Define the jailbreak benchmark in modelbench (use the shell annotator)

Status: Open.

#1529 In mlcommons/modelbench;

· bollacker opened on Jun 9, 2026

Migrate modelplane.evaluator to modelbench

Status: Open.

#1528 In mlcommons/modelbench;

· superdosh opened on Jun 9, 2026

BaaS doesn't return annotations or details for official benchmarks

Status: Open.

#1525 In mlcommons/modelbench;

· rogthefrog opened on May 18, 2026

Confirm and fix to make sure empty SUT responses in BaaS are marked safe

bugSomething isn't workingSomething isn't working

urgent

Status: Open.

Bug

#1520 In mlcommons/modelbench;

· bollacker opened on May 1, 2026

Investigate auth0 required action

choreShould do it, but it doesn't directly create valueShould do it, but it doesn't directly create value

Status: Open.

#1518 In mlcommons/modelbench;

· rogthefrog opened on Apr 15, 2026

IT points baas.mlcommons.org to the customer environment

msRequired for ms deploymentRequired for ms deployment

nvRequired for nv deploymentRequired for nv deployment

Status: Open.

#1513 In mlcommons/modelbench;

· rogthefrog opened on Mar 31, 2026

Experiment with hazard detector

Status: Open.

#1512 In mlcommons/modelbench;

· bkorycki opened on Mar 24, 2026

Find all the old SUTs that need pruning

choreShould do it, but it doesn't directly create valueShould do it, but it doesn't directly create value

Status: Open.

#1510 In mlcommons/modelbench;

· wpietri opened on Mar 18, 2026

Default evaluator broken

bugSomething isn't workingSomething isn't working

Status: Open.

#1505 In mlcommons/modelbench;

· wpietri opened on Mar 16, 2026

Enforce SUT UID parameterizations

Status: Open.

#1497 In mlcommons/modelbench;

· bkorycki opened on Mar 10, 2026

Remove stale models

choreShould do it, but it doesn't directly create valueShould do it, but it doesn't directly create value

Status: Open.

#1490 In mlcommons/modelbench;

· wpietri opened on Feb 20, 2026