Skip to content

fix(config): refuse three configurations that ran to a meaningless result (#996, #997, #999) - #1001

Open
wshlavacek wants to merge 1 commit into
mainfrom
fix/load-refusals-996-997-999
Open

wshlavacek wants to merge 1 commit into
mainfrom
fix/load-refusals-996-997-999

Conversation

@wshlavacek

Copy link
Copy Markdown
Collaborator

Closes #996. Closes #997. Closes #999.

Each of these configurations ran to a result with no meaning. Each is now refused before anything runs, with a message that names the cause.

#996: normalization = scale under a Bayesian sampler

  • scale profiles a multiplicative scale out at each evaluation. That is a profile, not a marginal, so the draws were not posterior draws. noise_profiling and linear_profiling were already refused for samplers for this reason.
  • A new Configuration._bayesian_sampler(), the registry family 'sampler' (am, dream, hmc, mh, p_dream, pt), now decides this for all three refusals.
  • Covered forms:
    • edition 2: whole-fit, per-observable, per-experiment and in a chain;
    • edition 1: whole-fit.
  • The message points to a measurement-layer scale (observable: …, formula: c * y with c free) or an optimizer.
  • Oracle: PyBNF's profiled objective against a quadrature over c. The gap equals ½ log(2π/A), which moves with the prediction.

#997: normalization on a column scored by a count family

  • Normalized values are not counts. The PMF's continuous extension is not a likelihood, and the fit moved with the normalizing constant.
  • A new NoiseModel.scores_counts property (True for NegBinomial) decides this, not the class name, and it covers init, peak, zero, unit and floor.
  • scale keeps fix(am): record every experiment's trajectory, normalized once; refuse what would be wrong (#990) #994's "no additive scale" refusal, so each case gets one message.
  • The message names the experiment and condition.
  • A normalized Gaussian column next to a raw neg_bin column still loads.
  • Oracle (scipy):
    • at integers the score equals -nbinom.logpmf and sums to 1;
    • the continuous extension integrates to 0.68 and 0.82;
    • a fitted θ moves from 7.10 to 7.75 to 7.92 with the normalizing constant.
  • Non-integer raw count data is left open. Across the 361 example configs that use neg_bin, no scored column holds a non-integer, so nothing in the repository depends on it.

#999: network-free experiments that BNG2.pl cannot run

Sibling paths

Path #996 #997 #999
edition 1 refused (whole-fit) refused same check, hand-written and param_scan actions
edition 2 refused (all forms) refused refused
PEtab import cannot arise cannot arise goes through the same refusal (tested)
-r resume refused (tested) refused (tested) refused (tested)
job_type = check not a sampler refused refused on any backend

Tests

An independent review is still to come.

…sult (#996, #997, #999)

- #996: `normalization ... = scale` under a Bayesian sampler (am, dream, hmc,
  mh, p_dream, pt) is refused when the configuration loads, naming the
  observables: scale is a profile, not a marginal. The sampler set is the
  registry family noise_profiling and linear_profiling are refused for, now
  read by all three through Configuration._bayesian_sampler. Every form is
  covered (whole fit, per observable, per experiment, either edition).
- #997: any normalization (init, peak, zero, unit, floor) of a column scored
  by a count noise model is refused at load, naming the column, the
  normalization, the experiment and the noise model. Decided by a new
  NoiseModel.scores_counts property (NegBinomial sets it), not the class.
  scale on such a column keeps #990's no-additive-scale refusal, so it has
  one message. A normalized continuous column beside a raw count column
  still loads.
- #999: a network-free action BNG2.pl cannot run (method nf with
  sample_times or no n_steps, as edition-2 experiments write; RuleMonkey) is
  refused wherever BNG2.pl would simulate it, naming the experiment and
  BNG2.pl's message: at load under bngl_backend = bionetgen and under
  job_type = check (which simulates with BNG2.pl whatever the backend); under
  auto, when the fit sets its models up, before any network generation, if
  bngsim is unavailable, cannot take the model, or lacks the engine, and
  after a bngsim set-up step fails and auto falls back. A configuration under
  auto still loads without bngsim for readers that simulate nothing.
- Tests: tests/test_load_refusals_996_997_999.py, with oracles that do not go
  through PyBNF (a quadrature of the Gaussian likelihood over the scale,
  scipy.stats.nbinom and scipy.special, and BNG2.pl itself on each action).
  Existing tests that built the now-refused combinations are updated:
  test_network_free_bngsim_supported asserted that nf under BNG2.pl builds.
- Docs: config_keys (normalization, experiment method, bngl_backend) and
  algorithms (check); changelog fragments 996, 997, 999.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment