Repository navigation
fix(config): refuse three configurations that ran to a meaningless result (#996, #997, #999) - #1001
Open
wshlavacek wants to merge 1 commit into
Open
wshlavacek wants to merge 1 commit into
wshlavacek wants to merge 1 commit into
Conversation
…sult (#996, #997, #999) - #996: `normalization ... = scale` under a Bayesian sampler (am, dream, hmc, mh, p_dream, pt) is refused when the configuration loads, naming the observables: scale is a profile, not a marginal. The sampler set is the registry family noise_profiling and linear_profiling are refused for, now read by all three through Configuration._bayesian_sampler. Every form is covered (whole fit, per observable, per experiment, either edition). - #997: any normalization (init, peak, zero, unit, floor) of a column scored by a count noise model is refused at load, naming the column, the normalization, the experiment and the noise model. Decided by a new NoiseModel.scores_counts property (NegBinomial sets it), not the class. scale on such a column keeps #990's no-additive-scale refusal, so it has one message. A normalized continuous column beside a raw count column still loads. - #999: a network-free action BNG2.pl cannot run (method nf with sample_times or no n_steps, as edition-2 experiments write; RuleMonkey) is refused wherever BNG2.pl would simulate it, naming the experiment and BNG2.pl's message: at load under bngl_backend = bionetgen and under job_type = check (which simulates with BNG2.pl whatever the backend); under auto, when the fit sets its models up, before any network generation, if bngsim is unavailable, cannot take the model, or lacks the engine, and after a bngsim set-up step fails and auto falls back. A configuration under auto still loads without bngsim for readers that simulate nothing. - Tests: tests/test_load_refusals_996_997_999.py, with oracles that do not go through PyBNF (a quadrature of the Gaussian likelihood over the scale, scipy.stats.nbinom and scipy.special, and BNG2.pl itself on each action). Existing tests that built the now-refused combinations are updated: test_network_free_bngsim_supported asserted that nf under BNG2.pl builds. - Docs: config_keys (normalization, experiment method, bngl_backend) and algorithms (check); changelog fragments 996, 997, 999.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #996. Closes #997. Closes #999.
Each of these configurations ran to a result with no meaning. Each is now refused before anything runs, with a message that names the cause.
#996:
normalization = scaleunder a Bayesian samplerscaleprofiles a multiplicative scale out at each evaluation. That is a profile, not a marginal, so the draws were not posterior draws.noise_profilingandlinear_profilingwere already refused for samplers for this reason.Configuration._bayesian_sampler(), the registry family'sampler'(am, dream, hmc, mh, p_dream, pt), now decides this for all three refusals.observable: …, formula: c * ywithcfree) or an optimizer.c. The gap equals ½ log(2π/A), which moves with the prediction.#997:
normalizationon a column scored by a count familyNoiseModel.scores_countsproperty (True forNegBinomial) decides this, not the class name, and it coversinit,peak,zero,unitandfloor.scalekeeps fix(am): record every experiment's trajectory, normalized once; refuse what would be wrong (#990) #994's "no additive scale" refusal, so each case gets one message.neg_bincolumn still loads.-nbinom.logpmfand sums to 1;neg_bin, no scored column holds a non-integer, so nothing in the repository depends on it.#999: network-free experiments that BNG2.pl cannot run
grid.bng2pl_network_free_refusals, checked against BNG2.pl 2.9.3 itself):method nfwithoutn_steps, which covers the edition-2 time course (sample_times) and dose-response;nf, i.e. RuleMonkey. That is the BNG2.pl half of Nine loud failures that block a valid job or give the wrong cause: RuleMonkey experiments, one-time-point data, PEtab import and export cases, error messages that hide their reason, and a crash on deeply nested formulas #962 item 1.bngl_backend = bionetgenand underjob_type = check, which always runs BNG2.pl;autowhen the model would land on BNG2.pl: bngsim unavailable, bridge can't take it, missing engine, or a failed set-up that fell back.autois not refused at load. Whether it uses bngsim depends on the machine, and readers such as PEtab export simulate nothing.Sibling paths
param_scanactions-rresumejob_type = checkTests
tests/test_load_refusals_996_997_999.py, 55 tests.test_network_free_bngsim_supported::test_nf_under_bng2pl_buildsasserted the combination An edition-2 network-free experiment (method: nf) under bngl_backend = bionetgen fails every simulation, because BNG2.pl's NFsim refuses the sample_times PyBNF writes, and nothing refuses it at load #999 refuses, and its comment wrongly said BNG2.pl's NFsim honours the output points. It now asserts the refusal.neg_bin+ normalization oram+scale. They now assert the load-time refusal, andam's own checks are exercised on Gaussian columns.bng_command, and space-separated data files. Those will be filed separately.mainbuild.ruffandtools/changelog.py checkare clean.An independent review is still to come.