Dandiset 000409

Reproducing the IBL brain-wide map (2025) from dandiset 000409 25

Notebook reproduce_ibl_2025.ipynb

• ✕Not reproducedResults, Behavioural task; Fig. 1d legendverifier disagreed at first

  On 0% contrast trials mice were rewarded on 58.7 +/- 0.4% (mean +/- s.e.m. across mice) of trials, better than chance (t138 = 20.18, P = 5.2e-43).

  Over all trials the mean over 139 mice is 58.26% (s.e.m. 0.42), t138 = 19.68, P = 6.9e-42. The t statistic is within tolerance and the mean is 0.44 points below 58.7, just outside the 0.4 tolerance. Over analysed trials the mean is 59.29% and t138 = 16.21, both outside tolerance. On the 454 canonical sessions, which the Fig. 1d legend names, the values are 58.27% with t138 = 19.68 (all trials) and 59.31% with 16.23 (analysed trials). No reading puts both values within tolerance, and the criterion defines no partial case. Mice are clearly above chance, so the effect in the claim's statement holds.

• ✕Not reproducedFig. 4d,l,m; Results, Representation of visual stimulus

  The latency of the stimulus trajectory separation ordered the visual regions as LGd < VISp, which is about equal to LP, < VISpm < VISam, at about 34, 42, 42, 57 and 78 ms.

  Latencies (Methods' definition) LGd 31.7, VISp 46.5, LP 48.6, VISpm 61.3, VISam 59.2 ms. LGd is first and VISp and LP are within 10 ms, but VISam is not later than VISpm, and VISp, LP and VISam are outside 10% of 42, 42 and 78 ms. The code's significance condition and its 0 to 100 ms window change the values by less than 1 ms. The paper's results table (meta/Results/stim.pkl) gives 34.0, 44.7, 46.8, 59.6 and 66.0 ms, closer to these values than the text is for VISp, LP, VISpm and VISam, and also has VISam later than VISpm. All five distances are significant (p = 0.001). With the 16 region-sessions of invalid blocks included the five latencies are unchanged. The code's canonical-window variant now keeps the full window's baseline, as the code does, and gives 31.9, 46.8, 48.9, 61.7 and 59.6 ms.

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out. Here 16 region-sessions.
  • Trials follow the mask here, without the saturation intervals the code uses.
  • The Methods define latency as the first crossing of 70% of the range; the code adds point-wise significance. The Methods' definition decides; the code's is a variant and gives the same values here.
• ✕Not reproducedResults, Representation of visual stimulus; Fig. 4c

  The stimulus trajectory distance was significant in 104 regions.

  Tested after the first verification round. Of 201 canonical regions, 195 have 20 or more units in sessions with valid blocks. The distance is significant in 46 regions with Benjamini-Hochberg at q = 0.01 (reading a) and 62 at P < 0.01 (reading b), against 104 +/- 10.4; 42 regions reach the smallest attainable p of 0.001. With the 64 sessions of invalid blocks 51 and 70; with the code's permutation classes 47 and 67, and 53 and 76 with all sessions. The code's Benjamini-Yekutieli correction at 0.05 gives the same counts as reading (a). Only an uncorrected P < 0.05 comes near (91 to 98). Significance agrees with stim_euc_sig of region_info.csv in 133 of 201 regions (41 in both, 5 here only, 63 in the paper only). The cause of the difference is not found here; the trial mask lacks the saturation intervals the code uses.

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out; CL, LING, MH, P5, PC5 and SOC then have fewer than 20 units and are not tested.
  • The Methods permute stimulus sides within each block and choice; the code merges the unbiased block with the 0.2 blocks. The Methods decide; the code's classes are a variant.
  • One set of permutations per session, shared by its regions; the code draws them per insertion.
  • Trials follow the mask here, without the saturation intervals the code uses.
• ◐Partially reproducedMethods, Animals

  On the day of recording the mice were aged 13 to 178 weeks (mean 44.96, median 27.0) and weighed 16.1 to 35.7 g (mean 23.9, median 23.84).

  Weight holds under every reading (reading a gives 16.1 to 36.2 g, mean 24.19, median 24.00). Age does not under any reading. At the first session the mice were 13.1 to 63.1 weeks old, mean 28.09, median 26.00; over all sessions at most 65.9 weeks, against the paper's 178 and mean 44.96. The files give each mouse one weight for all its sessions. The paper's code takes age and reference weight from the IBL database, not from the recording day, which may explain the age difference.

  How the test differed from the paper’s method

  • Age is session start minus date of birth in the files, which the Methods describe; the paper's code uses the database's age field. Measured values are reading (a).
• ◐Partially reproducedFig. 6a,i; Results, Representation of feedback

  Whether the trial was correct could be decoded from the 200 ms after feedback onset in nearly all recorded regions, almost perfectly in many.

  7 of 10 tested regions are significant, including IRN (27 sessions, effect 0.407, paper table 0.42); HATA (Fisher p = 0.0255), PA (0.1455) and TR (0.0227) are not. The paper's table marks 8 of these 10 significant (HATA yes, PA and TR no). With BH at q = 0.05, 9 are significant. 23 of 111 region-sessions have a balanced accuracy above 0.9; median accuracy 0.913 in GRN and 0.905 in IRN. Effects correlate with the paper's (Spearman 0.96). The paper's own table has 8 of these 10 regions significant, so this region set could not reach the criterion's 9 with the paper's own results.

  How the test differed from the paper’s method

  • The tested regions are VISp, GRN, IRN and seven drawn at random (ACAv, CL, HATA, LDT, PA, PVT, TR), each with all its canonical sessions, not all 201 regions; the Benjamini-Hochberg correction runs over these 10 regions.
  • One run of nested cross-validation (paper 10) and 100 null sessions (paper 200); each null reuses the C chosen for the real labels in each outer fold. On three example sessions the full nested null has a higher 95th percentile in 6 of 9 comparisons (0.555 to 0.622 against 0.531 to 0.598), so per-session p-values here can be smaller than the paper's procedure would give.
  • Synthetic choices and feedback come from an action-kernel model fitted per session with a logistic choice policy, written here; the paper uses the companion paper's model package.
  • The trial mask follows the Methods and the code's truncation, without the saturation intervals of the IBL aggregate trials table, which are not in the dandiset.
• ◐Partially reproducedResults, Representation of feedback; Fig. 6j-l; Extended Data Fig. 12

  During reward delivery, activity showed coherent oscillations of about 10 Hz that were missing after negative feedback and were closely related to licking.

  In 27 IRN sessions the population PSTH after correct feedback peaks at a median 7.08 Hz, outside the 9 to 11 Hz of the criterion. Power at the peak is higher after correct than incorrect feedback in 20 of 27 sessions (p = 0.0042), and the lick PSTH peaks at a median 7.23 Hz, within 1 Hz of the neural peak. In 4 sessions the neural peak is at 4.00 Hz, the edge of the band, and in one at 4.25 Hz. The power difference holds when each unit's PSTH is z-scored (18 of 27 sessions, p = 0.028, median peak 7.28 Hz) and when power at the peak is taken as a fraction of the 4 to 20 Hz power (26 of 27, p = 1e-07); both measure at the correct-trial peak and keep the evoked difference. With the band taken within 1 Hz of each session's lick peak, which does not use the neural correct-trial PSTH, power is higher after correct feedback in 15 of 26 sessions (p = 0.038), and in 15 of 26 (p = 0.031) after subtracting each condition's slow evoked response (Gaussian smoothing, sigma 60 ms; faster components of the evoked response remain). Each condition at its own peak after the subtraction gives 16 of 27 (p = 0.077). In the 9 to 11 Hz band there is no difference (12 of 27, p = 0.7 raw, 0.66 after the subtraction). Condition (ii) holds when the band follows the licks.

  How the test differed from the paper’s method

  • Tested in IRN only, the paper's example region; the paper's analysis is a population trajectory over regions.
  • The spectral measure (Hann-windowed spectrum of the session PSTH, 4 to 20 Hz) is chosen here; the paper describes the oscillation without a measure.
• ✓ReproducedAbstract; Methods, Sessions and insertions

  The data come from 459 sessions of 139 mice recorded in 12 laboratories.

  The pinned version holds 459 sessions of 139 mice from 12 labs, one processed and one raw file per session.

• ✓ReproducedAbstract; Results, Recordings; Methods

  699 Neuropixels insertions yielded 621,733 units, an average of 889 per probe.

  699 insertions and 621,733 units, 889.5 per probe.

• ✓ReproducedResults, Recordings; Methods, Neurons and brain regions

  Stringent quality metrics identified 75,708 well-isolated neurons, an average of 108 per probe.

  75,708 units have ibl_quality_score 1, 108.3 per probe. Recomputing the three metrics gives 75,708 with a noise cutoff below 5 and 95,430 with the Methods' 20. The criterion's side check, that the score is the fraction of the three metrics passed with the Methods' cutoff of 20, holds for 540,399 of 621,733 units; with a cutoff of 5 it holds for every unit.

  How the test differed from the paper’s method

  • The files hold spike sorting revision 2025-05-06; the paper's code loads revision 2024-05-06. The counts are identical.
• ✓ReproducedResults, Neural analyses; Methods, Neurons and brain regions

  The main analyses use a canonical dataset of 201 regions with at least 2 sessions of 5 well-isolated neurons and at least 20 neurons in total, 62,857 neurons.

  Every inclusion step gives the counts of the code's inclusion table: 75,708 well-isolated units in 268 regions, 65,336 in grey matter, 63,357 with 5 per session (245 regions), 62,990 with 2 sessions (210), and 62,857 neurons in 201 regions over 454 sessions. The region table region_info.csv marks 210 regions canonical, those left before the 20-neuron step.

• ✓ReproducedAbstract

  The probes covered 279 brain areas.

  Units fall in 279 Beryl regions other than void and root (266 with a well-isolated unit).

• ✓ReproducedResults, Behavioural task; Methods, Animals

  The 139 mice were 94 male and 45 female.

  94 male and 45 female mice; each mouse has one sex in all its files.

• ✓ReproducedMethods, Materials and apparatus

  The insertions used Neuropixels 1.0 probes of versions 3A (109) and 3B2 (586), and Neuropixels 2.4 probes (4).

  586 insertions use PRB_1_4_0480_1 (403) or PRB_1_4_0480_1_C (183), 109 NP1010 and 4 NP2010.

  How the test differed from the paper’s method

  • The files name imec models, not versions; NP1010 is read as 3A (its devices have no PXI port or slot) and NP2010 as the 2.x probes. The counts match under this reading.
• ✓ReproducedResults, Behavioural task

  Recorded sessions lasted on average 645 trials (median 602, range 401 to 1,525).

  Mean 645.1, median 601, range 401 to 1,525 trials per session.

• ✓ReproducedResults, Behavioural task

  After training, mice made correct choices on 81.4 +/- 0.4% (mean +/- s.d.) of trials.

  Reading (c), the mean over 459 sessions, gives 81.58% correct, within 81.4 +/- 0.4. The mean over mice is 80.76% (all trials) and 84.62% (analysed trials); all trials pooled give 81.94%. Across mice the s.d. is 5.12 and the s.e.m. 0.43. On the 454 sessions of the canonical set the mean over sessions is 81.59% and over mice 80.77%.

  How the test differed from the paper’s method

  • The paper does not say how the mean is taken; four readings are shown and only the mean over sessions is within tolerance.
• ✓ReproducedFig. 1c legend

  22.8% of first wheel-movement times occurred under 80 ms after stimulus onset.

  22.75% of all 296,090 trials (reading a); 22.99% of trials with both times (b); 22.49% as a mean over sessions (c). 12.06% of trials with both times have the first movement before stimulus onset.

• ✓ReproducedFig. 1d; Results, Behavioural task

  Mice performed better and faster on trials with high visual contrast.

  Median over mice of percent correct rises at every step, 59.2% at 0% to 96.8% at 100% (higher in 139 of 139 mice, p = 1.5e-24); median first movement time falls at every step, 196 to 137 ms (shorter in 133 of 139 mice, p = 5.9e-24).

• ✓ReproducedFig. 1e,f; Results, Behavioural task

  Towards the end of the sessions, performance decreased and first wheel-movement times increased.

  Last 50 trials against trials 51 to 300 from the end, median over mice of percent correct 80.0 against 85.0 (lower in 105 of 139 mice, p = 7.9e-13), median first movement time 279 against 165 ms (longer in 131 of 139 mice, p = 2.6e-22).

• ✓ReproducedFig. 1g; Results, Behavioural task

  After a block switch, mice took around 5 to 10 trials to adjust their behaviour, as shown by the fraction of correct choices on 0% contrast trials after the switch.

  Over 395 sessions with valid blocks and 16,447 zero-contrast trials, the fraction correct starts at 0.338 after the switch. Fitted tau is 3.86 trials, outside reading (a); 90% of the change is covered at 9.9 trials, within reading (b), on the point estimate. Resampling mice (1000 resamples) gives a 95% interval of 2.79 to 5.30 trials for tau and 7.4 to 13.2 trials for reading (b), with 75.9% of resamples within 4.5 to 11. Per-mouse tau has a median of 3.88 (interquartile 1.86 to 11.30, 125 mice).

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out.
  • The paper gives no measure of the adjustment time; two readings were fixed before the analysis and the claim holds under reading (b) only.
• ✓ReproducedFig. 1h; Results, Behavioural task

  For all contrast values mice chose the side of the block's likely stimulus more often, responding right more often in right blocks than in left blocks.

  In 395 sessions with valid blocks (125 mice) the right-block fraction of right choices is higher at all nine signed contrasts, median differences 0.013 to 0.333, every p below 1e-8. With all sessions the differences are smaller at every contrast (0.012 to 0.314).

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out.
• ✓ReproducedFig. 4a,i; Results, Representation of visual stimulus; Discussion

  The visual stimulus side, decoded from the first 100 ms after stimulus onset, was represented in a narrower set of regions than choice or feedback, with strong signatures in visual cortex such as VISp.

  4 of 10 tested regions are significant for stimulus (CL, GRN, IRN, VISp), against 5 for choice and 7 for feedback. VISp is significant (35 sessions, Fisher p = 1.3e-08, effect 0.039, paper table 0.02). Significance agrees with region_info.csv in 10 of 10 regions and the effects correlate with the paper's (Spearman 0.91, p = 0.00025). With BH at q = 0.05 the counts are 4, 7 and 9. The paper reports 40 of 201 regions for stimulus. With the full nested search for each null (30 nulls in each of the 35 VISp sessions) VISp stays significant, Fisher p 1.8e-05 against 2.8e-05 with the reused C on the same nulls, 13 sessions below 0.05 under each, and significant with BH at q = 0.01 among the 10 regions. The decoder fits an L1 penalty (scikit-learn 1.9.1; at C = 0.01 no coefficient of 13 is nonzero in one VISp session, against 12 with L2).

  How the test differed from the paper’s method

  • The tested regions are VISp, GRN, IRN and seven drawn at random (ACAv, CL, HATA, LDT, PA, PVT, TR), each with all its canonical sessions, not all 201 regions; the Benjamini-Hochberg correction runs over these 10 regions.
  • One run of nested cross-validation (paper 10) and 100 null sessions (paper 200); each null reuses the C chosen for the real labels in each outer fold. On three example sessions the full nested null has a higher 95th percentile in 6 of 9 comparisons (0.555 to 0.622 against 0.531 to 0.598), so per-session p-values here can be smaller than the paper's procedure would give.
  • The trial mask follows the Methods and the code's truncation, without the saturation intervals of the IBL aggregate trials table, which are not in the dandiset.
• ✓ReproducedFig. 5a,i; Results, Representation of choice

  Choice, decoded from the 100 ms before the first wheel movement, was represented in a larger number of regions than the visual stimulus, with strong effects in hindbrain regions such as GRN.

  GRN is significant for choice (14 sessions, Fisher p = 6e-15, effect 0.277, paper table 0.28); 5 tested regions are significant for choice against 4 for stimulus, a margin of one region. Significance agrees with region_info.csv in 9 of 10 regions (LDT, 2 sessions, p = 0.0154, is significant in the paper). Effects correlate with the paper's (Spearman 0.98). The paper reports 172 of 201 regions. The margin over stimulus rests on one region; CL's choice result (2 sessions, Fisher p = 0.0010) sits at the 2-session floor, and the full nested null was run only for VISp stimulus. With 100 nulls the per-session p cannot go below 1/101, so a region of 2 sessions cannot have a Fisher p below about 0.0010; LDT's result may be partly this limit.

  How the test differed from the paper’s method

  • The tested regions are VISp, GRN, IRN and seven drawn at random (ACAv, CL, HATA, LDT, PA, PVT, TR), each with all its canonical sessions, not all 201 regions; the Benjamini-Hochberg correction runs over these 10 regions.
  • One run of nested cross-validation (paper 10) and 100 null sessions (paper 200); each null reuses the C chosen for the real labels in each outer fold. On three example sessions the full nested null has a higher 95th percentile in 6 of 9 comparisons (0.555 to 0.622 against 0.531 to 0.598), so per-session p-values here can be smaller than the paper's procedure would give.
  • Synthetic choices and feedback come from an action-kernel model fitted per session with a logistic choice policy, written here; the paper uses the companion paper's model package.
  • The trial mask follows the Methods and the code's truncation, without the saturation intervals of the IBL aggregate trials table, which are not in the dandiset.
• ✓ReproducedFig. 4d,l,m; Results, Representation of visual stimulus

  The early visual wave was followed by later stimulus encoding in the MRN, SCm, PRNr, IRN and GRN, with latencies of about 100 to 120 ms.

  IRN 118.3 ms and GRN 109.9 ms, both within 90 to 132 ms, both distances significant (p = 0.001). MRN, SCm and PRNr were not tested; the paper's results table gives 89.4 ms for IRN and GRN in its 0 to 100 ms window, in which IRN has no latency here and GRN 100.0 ms. With the window extended to 0 to 200 ms the latencies are 117.9 and 109.5 ms; the IRN distance is still rising at 190 ms. The outcome rests on the criterion's 0 to 150 ms window.

  How the test differed from the paper’s method

  • Only IRN and GRN of the five regions named are tested.
  • The 64 sessions whose probability_left alternates on every trial are left out.
• ⊘BlockedFig. 4b; Results, Representation of visual stimulus· Methods underspecified

  Single-cell tests controlling for choice and block found few stimulus-selective neurons (0.5% of all neurons); even in the regions with the largest fractions, such as visual cortex, the fraction did not exceed about 10%.

  The test ran on 5,228 units in 200 region-sessions of 14 regions. The null run (pseudo-session sides and blocks) selects 14 units (2.7e-3, 95% interval 1.5e-3 to 4.5e-3), far above the paper's 5e-5, so by the criterion the procedures differ. Under the null the simple test alone passes 7.1e-3 and the combined test 6.9% of units. The regional fractions agree with region_info.csv (14 of 14 within 0.02; largest 5.9% in CL with 17 units; visual regions 2.1% to 3.8%), and the pooled 1.82% is above the paper's 0.41% because the tested regions over-represent visual areas. Were it not for the false-positive check the claim would be reproduced. With the stimulus side shuffled over the tested trials (one permutation per region-session) the test selects 4 units (7.7e-4, interval 2.1e-4 to 2.0e-3), while the product of the two tests' single rates is 4.7e-5. So the dependence of the two tests alone puts the rate above the paper's, and the pseudo-session null is about 3.5 times higher again.

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out. Here 19 region-sessions and 381 units.
  • Regions are the 14 loaded for the neural claims, not all canonical regions.
  • The paper does not state which trials enter the simple Mann-Whitney test; the same trials as the combined test are used.
• ⊘BlockedFig. 5b; Results, Representation of choice· Methods underspecified

  More neurons responded to choice than to the visual stimulus (4% of all neurons), with the largest fractions in hindbrain, midbrain and thalamic regions.

  The null run with synthetic choices selects 13 of 5,228 units (2.5e-3, interval 1.3e-3 to 4.2e-3), far above 5e-5. Otherwise the pattern holds: pooled 5.64% choice-selective against 1.82% for stimulus (270 against 70 units selective for one only, p = 3.8e-29); largest fractions CL 35%, GRN 22%, IRN 14%; regional fractions correlate with the paper's (Spearman 0.84, p = 0.00016). With the choices shuffled over the tested trials the test selects 3 units (5.7e-4, interval 1.2e-4 to 1.7e-3), against a product of the two tests' single rates of 3.3e-5.

  How the test differed from the paper’s method

  • The 64 sessions whose probability_left alternates on every trial are left out.
  • Regions are the 14 loaded for the neural claims.
  • Synthetic choices and feedback come from an action-kernel model fitted per session with a logistic choice policy, written here; the paper uses the companion paper's model package.