IBL behavior notebook loads training-task data into the "full task" set (stray 50%-contrast trials)
Summary
In the IBL behavior data notebook, the aggregate all_trials table is never filtered by task protocol, so the "full task" analysis set silently includes trainingChoiceWorld (basic-task) sessions. The tell-tale symptom is 50%-contrast trials appearing in the full-task data — the full task (biasedChoiceWorld) contains 0% at 50% contrast; those trials only exist in the basic task.
Root cause
The protocol filter is applied only to the single-subject demo, not to the aggregate table used for analysis. In the notebook:
code cell #8 — this is ONLY for the single-subject demo
sess = subject_trials[subject_trials['task_protocol'].str.contains('biasedChoiceWorld')].index.unique()[0]
trials = subject_trials[subject_trials.index == sess]
This filters subject_trials (one mouse) to pick a demo session. When all_trials is later built across all subjects, no equivalent protocol filter is applied, so basic-task sessions leak in.
Fix
Filter the full task by protocol on all_trials. Replace the body of code cell #24:
code cell #24 — full-task filter, applied to all_trials
proficient_trials = all_trials[
all_trials['task_protocol'].str.contains('biasedChoiceWorld', na=False)
].copy()
to include recording sessions too:
...str.contains('biased|ephys', case=False, na=False)
Verification
_abs = proficient_trials['contrastLeft'].fillna(0) + proficient_trials['contrastRight'].fillna(0)
assert (_abs.round(3) == 0.5).sum() == 0 # no 50%-contrast trials
print(sorted(proficient_trials['probabilityLeft'].round(2).unique())) # -> [0.2, 0.5, 0.8]
After the fix, 50%-contrast trials are gone and probabilityLeft takes only {0.2, 0.5, 0.8}, confirming the data is fully inside the biased full task.
Why it matters: any trial-history / previous-trial analysis on the unfiltered set is confounded, because the basic task uses counterbiasing repeat-on-error trials that don't exist in the full task (based on the paper).
Students brought up this issue first by noticing inconsistency between data and the paper, then we started working on it. That is what we came up (by using Claude AI), but this might require verification by someone who are familiar with this dataset.
IBL behavior notebook loads training-task data into the "full task" set (stray 50%-contrast trials)
Summary
In the IBL behavior data notebook, the aggregate all_trials table is never filtered by task protocol, so the "full task" analysis set silently includes trainingChoiceWorld (basic-task) sessions. The tell-tale symptom is 50%-contrast trials appearing in the full-task data — the full task (biasedChoiceWorld) contains 0% at 50% contrast; those trials only exist in the basic task.
Root cause
The protocol filter is applied only to the single-subject demo, not to the aggregate table used for analysis. In the notebook:
code cell #8 — this is ONLY for the single-subject demo
sess = subject_trials[subject_trials['task_protocol'].str.contains('biasedChoiceWorld')].index.unique()[0]
trials = subject_trials[subject_trials.index == sess]
This filters subject_trials (one mouse) to pick a demo session. When all_trials is later built across all subjects, no equivalent protocol filter is applied, so basic-task sessions leak in.
Fix
Filter the full task by protocol on all_trials. Replace the body of code cell #24:
code cell #24 — full-task filter, applied to all_trials
proficient_trials = all_trials[
all_trials['task_protocol'].str.contains('biasedChoiceWorld', na=False)
].copy()
to include recording sessions too:
...str.contains('biased|ephys', case=False, na=False)
Verification
_abs = proficient_trials['contrastLeft'].fillna(0) + proficient_trials['contrastRight'].fillna(0)
assert (_abs.round(3) == 0.5).sum() == 0 # no 50%-contrast trials
print(sorted(proficient_trials['probabilityLeft'].round(2).unique())) # -> [0.2, 0.5, 0.8]
After the fix, 50%-contrast trials are gone and probabilityLeft takes only {0.2, 0.5, 0.8}, confirming the data is fully inside the biased full task.
Why it matters: any trial-history / previous-trial analysis on the unfiltered set is confounded, because the basic task uses counterbiasing repeat-on-error trials that don't exist in the full task (based on the paper).
Students brought up this issue first by noticing inconsistency between data and the paper, then we started working on it. That is what we came up (by using Claude AI), but this might require verification by someone who are familiar with this dataset.