Spontaneous behaviour is structured by reinforcement without explicit reward (DANDI:000559)

Claims

From the paper Spontaneous behaviour is structured by reinforcement without explicit reward

Reproducing Markowitz, Gillis, Jay et al. (2023) from dandiset 000559 14

Notebook reproduce_markowitz_2023.ipynb

Not reproducedResults, third section; Extended Data Figure 7
DMS dopamine transients do not predict future syllable usage.
By the test used for fig2e-dlight-predicts-future-usage, DMS dLight predicts the later use of the same syllable. Mean r is 0.0110 to 0.0331 for n from 10 to 190, significant at 9 of those 10 values (n = 10 to 90 at P below 0.01), positive in 8 of 8 mice at n = 10, and larger than in DLS (0.0044 to 0.0251). The test is this attempt's version of the paper's (see the deviations of fig2e-dlight-predicts-future-usage). The paper's Extended Data Fig. 7 has no lagged future-usage test; it supports the claim with the encoding model (panel e) and the title "DMS dopamine does not correlate with syllable usage or entropy". The across-syllable count test of fig2b-dlight-usage-across-syllables run on DMS agrees with the paper (held-out r = 0.026, shuffle P = 0.170). That the lagged test finds DMS stronger than DLS suggests it differs from the paper's (equal time segments, no z-scoring to the shuffles); the outcome follows the pre-set criterion.
How the test differed from the paper’s method
- Same deviations as fig2e-dlight-predicts-future-usage.
Not reproducedMethods, "Quantifying changes in target syllable counts"; Figure 3e; Extended Data Figure 10verifier disagreed at first
Only a fraction of Opto-DA mice learn; nine mice exceed the largest change in target counts of any control ("learners").
Seven of 20 Opto-DA mice exceed the largest control change (54.5, vta-nacc-ctrl-6), not nine. All seven are among the nine the paper's code printed; the other two (3442 at 47.3 and dls-ai32jr-1 at 41.7) fall below that control; their per-target changes range from -49 to 148 and -26 to 93, against -50 to 152 for vta-nacc-ctrl-6. Counting mice above the control maximum of the log2 fold change gives 10. The criterion is the count, which is outside the tolerance. The statement's first part, that only a fraction of mice learn, holds (7 of 20).
Partially reproducedResults, first section; Figure 1e
DLS dopamine fluctuations are significantly negatively correlated with translational velocity at short timescales (under 10 s), and the correlation reverses and becomes positive at longer timescales, for both dLight transient rates and binned amplitudes.
Over 121 DLS sessions from 14 mice, binned dLight amplitude and transient rate are both significantly negatively correlated with 2D speed at all 20 timescales below 10 s (amplitude mean r -0.201 to -0.109, rate -0.123 to -0.062; shuffle ranges within about 0.02 of zero), and all 14 mice are negative at 0.25 s for both. The transient rate turns positive from 18.75 s and is significantly positive at 47 of 100 timescales of 10 s or more (largest 0.113 at 58.75 s). The binned amplitude is still significantly negative at 49.75 s and is significantly positive at none of the 100 longer timescales (largest 0.005). The claim holds for one of the two measures under both readings of "short timescales".
How the test differed from the paper’s method
- dLight is re-referenced and z-scored from the stored dF/F series; the paper used a pre-computed z-scored trace that is not in the files.
- 100 shuffles where the paper used 1,000.
- 121 sessions where the paper's code reports 216 for this panel; the dandiset holds fewer sessions, and habituation sessions are left out here.
Partially reproducedFigure 1h legend
Average syllable-associated dLight amplitudes differ across syllables (Kruskal-Wallis H = 209.29, P < 1e-25, n = 518 mouse-syllable pairs).
The 518 mouse-syllable pairs (14 mice x 37 syllables) match the paper's n. Kruskal-Wallis P = 5.98e-37, below the paper's 1e-25, but H = 266.40 is 27 % above 209.29 and outside the 10 % tolerance. Variants give H 266.99 (target instances kept), 268.58 (pairs with at least 50 instances) and 265.62 (the 9 DLS sessions failing the quality criteria added), so neither the quality filter nor the post-session rule explains the difference. Per-mouse tests are significant in 14 of 14 mice.
How the test differed from the paper’s method
- Instances from 177 DLS sessions without stimulation (habituation sessions left out); the paper used more sessions per mouse.
Partially reproducedResults, third section; Figure 2c-g
When an instance of a syllable coincides with a higher-amplitude dopamine transient, that syllable is used more over the following minutes.
The mean correlation between an instance's dLight and the count of the same syllable in the next n syllables is positive at every n (0.0043 to 0.0251) and significant from n = 10 to 90 (P < 0.01), 11 of 14 mice positive at n = 10. It is not significant at n = 110, 130, 150 and 170 (P = 0.09, 0.15, 0.18 and 0.05), so it holds at 6 of 10 values up to 190 and fails the criterion's "throughout"; it is significant again from 190 to 330. The shuffles permute only instances with a dLight peak, so the same groups enter the observed value and every shuffle (15,480 at n = 10).
How the test differed from the paper’s method
- Five equal time segments in place of the paper's five behavioural segments (Extended Data Fig. 2e).
- Correlations are compared with shuffles directly, not z-scored to them; 100 shuffles.
ReproducedResults, second section; Figure 1l
Random forest classifiers cannot predict which syllable is being expressed from the coincident dLight peak amplitude or waveform.
Held-out accuracy for 37 syllables is 0.0271 from peak amplitude and 0.0323 from the waveform and its derivative, 1.00 and 1.20 times chance (0.0270), below the criterion of 0.0541. Shuffled labels give 0.0278 and 0.0279.
How the test differed from the paper’s method
- A 100-tree sklearn random forest with at least 5 instances per leaf on 1,000 instances per syllable, where the paper used cuML with 1,000 trees; the paper gives no accuracy, so "unable to predict" was set as below twice chance.
- One label shuffle per feature set as the null, where the paper used 100.
ReproducedResults, third section; Figure 2b
Syllables associated on average with more DLS dopamine in an experiment tend to be used more.
Over 507 mouse-syllable pairs in 14 mice, held-out r = 0.165 between syllable-associated dLight and syllable count; no shuffle of 200 reached it (P < 0.005, 95th percentile 0.051). Per-mouse rank correlations are positive in 14 of 14 mice. The paper's r is shown only in its figure.
How the test differed from the paper’s method
- Fivefold cross-validation repeated 20 times and 200 shuffles, where the paper used 100 repeats and 1,000 shuffles.
ReproducedResults, third section; Figure 2i
Mice string together less predictable sequences after syllables associated with more dopamine; average syllable-associated dLight correlates positively with the entropy of the syllable's outgoing transitions.
Held-out r = 0.271 between syllable-associated dLight and the entropy of outgoing transitions over 507 mouse-syllable pairs; no shuffle of 200 reached it (P < 0.005, 95th percentile 0.042). Positive in 14 of 14 mice.
How the test differed from the paper’s method
- Plain Shannon entropy of the outgoing transition probabilities per experiment, as the Methods state; the code uses a Dirichlet entropy estimate.
ReproducedResults, third section; Extended Data Figure 7a,b
dLight recordings in dorsomedial striatum (DMS) show less frequent dopamine transients than in DLS.
DMS mice have a median 56.48 transients per minute against 62.12 in DLS (8 against 14 mice, U = 13, two-sided P = 0.0022); with a 1.96 threshold 22.10 against 25.21 (P = 0.0006).
ReproducedFigure 3e legend
Pairing a target syllable with optogenetic dopamine release (Opto-DA) increases its use relative to baseline more than in no-opsin controls (log2 fold change in target counts, one-sided Mann-Whitney U = 197, P = 0.002, f = 0.82).
20 Opto-DA and 12 control mice, 191 targets. Log2 fold change in target counts, Opto-DA median 0.330, control 0.081; U = 198, one-sided P = 0.0013, f = 0.82 (paper 197, 0.002, 0.82).
How the test differed from the paper’s method
- Stimulation, baseline and post sessions are identified from stimulation pulses, session dates and descriptions, as the files give no session number or type.
ReproducedFigure 3f legend
The cumulative increase in target counts relative to baseline is larger in Opto-DA mice than in controls (one-sided Mann-Whitney U = 184, P = 0.007, f = 0.77).
Change in target counts, Opto-DA median 27.6, control -2.3; U = 182, one-sided P = 0.0083, f = 0.76 (paper 184, 0.007, 0.77).
How the test differed from the paper’s method
- As for fig3e-optoda-learning.
ReproducedFigure 3h legend
In learners, the change in target usage during stimulation predicts the change in the post-stimulation sessions (Pearson r = 0.89, P = 0.005).
Among the seven learners found here, r = 0.891 between the change during stimulation and in the post sessions (one-sided shuffle P = 0.017); with the code's nine learners r = 0.888 (P = 0.001). The learner set differs from the paper's (see fig3e-learner-count), and the result holds with either. Controls also correlate (r = 0.591, one-sided P = 0.017, two-sided 0.039), where the paper reports P = 0.082.
BlockedFigure 3l legend; Results, fourth section· Metadata insufficient
Brief (250 ms) Opto-DA does not change movement at the moment of stimulation; no comparison between stimulation and catch trials in any mouse is significant.
The test needs catch trials (target detections on which stimulation was withheld), which the files do not mark; the conversion kept only the stimulated detections. The criterion's substitute (online target onsets with no pulse in the next 0.5 s) is not the paper's catch set. The Methods give stimulation on 75 % of detections; with the substitute, stimulations are a median 0.593 of detections per mouse (0.521 to 0.687; 0.111 to 0.776 over sessions). It does not isolate the light's effect. In no-opsin controls, stimulation trials differ from these catch trials at P < 0.05 in 9 of 12 mice, and the controls' median difference (3.96 mm/s) exceeds the Opto-DA one (-0.07 mm/s; Opto-DA against control U = 68, P = 0.045). With catch trials at least 2 s from any pulse, controls are still significant in 9 of 12. Taken literally, the criterion gives 9 of 20 Opto-DA mice significant, which would be not reproduced (8 of 20 with the 2-s catch trials); because the controls fail, the claim is recorded as blocked.
BlockedFigure 3m; Extended Data Figure 10l; Results, fourth section· Metadata insufficient
Extending stimulation to 3 s (pulsed, 25 Hz) makes mice increase their velocity.
As for fig3l-no-kinematic-effect. With the criterion's substitute catch trials, stimulations are 0.302 to 0.348 of detections per mouse, and onsets during an ongoing 3-s train count as catch trials. 1 of the 3 Opto-DA mice with 3-s pulsed sessions is significant (P = 0.024; the others 0.091 and 0.053), which taken literally would be partially reproduced, and both controls are (P = 0.0002 and 0.0083), with larger differences (median 6.75 mm/s against 4.58). With catch trials at least 2 s from any pulse, 0 of 3 Opto-DA mice are significant (P = 0.10, 0.064, 0.082) and both controls still are (P = 0.0007 and 0.010). The 2-s continuous sessions give significant differences in 2 of 3 Opto-DA mice and 2 of 3 controls.

Not tested in a notebook 10

1 data missing from the dandiset, 7 other, 2 metadata insufficient

BlockedFigure 2a; Methods, "Food reward experiments"· Data missing from the dandiset
Spontaneous syllable-associated dopamine transients roughly match those during consumption of an unexpected chocolate chip (mean spontaneous peak 2.12 +- 0.80 z, n = 5,247; reward 2.36 +- 0.92 z, n = 10).
No session in the dandiset is described as a food-reward session, and the human-annotated consumption times are not in any file (session descriptions, NWB contents of the photometry files). The paper's code reads them from its Zenodo deposit (doi:10.5281/zenodo.7274803, one 60.8 GB zip), which was not opened here.
BlockedFigure 1f· Other
The average syllable-associated dLight transient exceeds that associated with movement initiation (P = 0.0006, n = 14 mice).
Not attempted in this attempt, for time. The data it needs (dLight, syllable labels, 2D speed for movement initiations) are in the dandiset.
BlockedFigure 2j-l· Other
A higher-amplitude dopamine transient at a syllable instance is followed by less predictable syllable choices over the next several seconds.
Not attempted in this attempt, for time. The data it needs are in the dandiset.
BlockedFigure 3i· Metadata insufficient
Behavioural sequences right after Opto-DA stimulation are less predictable than after catch trials.
Not attempted; it compares stimulation with catch trials, which the files do not mark (see fig3l-no-kinematic-effect).
BlockedResults, fourth section; Extended Data Figure 10d· Other
Opto-DA delivered on the fastest (slowest) quartile of target instances makes later instances faster (slower).
Not attempted in this attempt, for time. The 94 velocity-conditioned files of 9 mice are in the dandiset, with "Stim Up" and "Stim Down" notes.
BlockedFigure 4b· Other
Per mouse, the influence of endogenous dopamine on syllable counts and on sequence entropy are correlated (r = 0.69, P = 0.001).
Not attempted in this attempt, for time; it builds on the per-mouse lagged analyses of Fig. 2e and 2k.
BlockedFigure 4c· Other
Mice whose syllable use is more influenced by endogenous dopamine learn more from Opto-DA (r = 0.51, P = 0.001).
Not attempted in this attempt, for time. The mice it needs have photometry and closed-loop sessions in the dandiset.
BlockedFigure 4f· Metadata insufficient
Syllables with higher endogenous dLight on catch trials are more reinforced by Opto-DA (r = 0.32, P < 0.001).
Not attempted; it uses dLight on catch trials, which the files do not mark.
BlockedFigure 4h; Methods, "Decoding model predicting behaviour from dLight"· Other
A model in which endogenous and exogenous dopamine add predicts held-out syllable choices better than shuffled and noise controls (median r 0.20 against 0.04).
Not attempted in this attempt, for time.
BlockedFigure 5b-d· Other
A Q-learning agent rewarded by syllable-associated dLight learns a transition matrix similar to the observed one, best when dopamine also transiently raises choice variability.
Not attempted in this attempt, for time; the paper fitted it on a 128-CPU machine, and its code is in the repository named under inputs.

Findings about the data

What attempts found about the dandiset itself, for its owner and the archive.

Description missingThe closed-loop files mark stimulation onsets but not catch trials (target detections on which stimulation was withheld); the conversion kept only the stimulated detections. Comparisons of stimulation with catch trials (Fig. 3i, 3l, 3m, 4f) cannot be made, and a substitute selection shows stimulation-catch differences in 9 of 12 no-opsin controls.
Description missingNo file states the genotype or opsin of a mouse. Opto-DA (Ai32) and no-opsin control mice are told apart only by the stimulation site ("snc (axon)" or "ctrl"), and the dLight+ChrimsonR mice only by their subject names.
Description missingSession type (baseline, stimulation, post) and session number are not stored; they must be inferred from free-text descriptions, dates and stimulation pulses. Five files with stimulation are described as recording sessions.
Metadata errorEvery closed-loop file states a 635-nm stimulator (Opto Engine MRL-III-635), including the Ai32 (ChR2) mice, which the Methods say were stimulated with 470-nm light.
Metadata errorThe dLight-only mice (C57BL/6J in the Methods) carry the subject description "wild-type DAT-IRES-Cre and Ai32 mice", the same as the optogenetic mice.
File format errorIn 28 photometry and 547 closed-loop files the syllable timestamps are not ascending. In 573 of these 575 the rows are rotated, with one backward step, which is at least 0.95 of the session length in 558 files and 0.635 of it at the least. Sorting by timestamp restores a regular 30-Hz clock (median interval 0.03337 s in every file).
File format errorTwo closed-loop files of mouse 3155 store every syllable frame twice (53,822 and 53,862 repeated timestamps), and have two and three non-ascending steps.
Data missing14 files of dLight+ChrimsonR mice hold empty syllable series; all list two target syllables.
Data missingFor the dLight+ChrimsonR mice only the 250-ms closed-loop sessions are present; the 2-s and pulsed sessions the Methods describe are not. The dandiset holds 224 DLS photometry files without stimulation (210 with syllables, 193 of them not habituation sessions, 177 of those passing the quality criteria), where the Methods give 379 experiments.
Data missingThe food-reward sessions and the consumption times of Fig. 2a are not in the dandiset.
Selection not documentedThe dandiset includes behaviour-only files for photometry sessions without a dLight series (111 files, among them all files of dls-dlight-6, dls-dlight-8, dms-dlight-7 and dms-dlight-9) and sessions that fail the paper's quality criteria (16 DLS and 11 DMS), with no flag saying which were excluded from the paper.
OtherThe re-referenced, z-scored dLight trace used by every photometry analysis is not stored; it must be recomputed from the stored dF/F and smoothed reference, whose regression fit is not stored either.

Notebooks

Every registered entry that uses this dandiset. An entry can hold several notebooks, and can use more than one dandiset.

Reproduction Dopamine and spontaneous behaviour in mice (Markowitz, Gillis, Jay et al. 2023)

Tests the findings of Markowitz, Gillis, Jay et al. (2023) on dandiset 000559, which holds the dLight photometry, MoSeq syllables and closed-loop optogenetic sessions of the paper. It relates DLS dopamine to velocity, syllable identity, syllable use and sequence entropy, compares DMS with DLS, and tests the reinforcement of target syllables by closed-loop dopamine release. The modelling of Figures 4 and 5 is not attempted. All files

Reproducing Markowitz, Gillis, Jay et al. (2023) from dandiset 000559
reproduce_markowitz_2023.ipynbExecuted, with outputs · Source

Verification

Round 1
2026-10-17
Agreed with 23 of 24 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: The executed notebook of the first run with the conclusion cells and the summary removed, the helper module and the claims file. The header still listed which claims were tested and blocked.
- fig3e-learner-count: the verifier’s outcome was “Not reproduced”. 7 learners against the paper's 9. All 7 are in the code's list; 3442 (change 47.3) and dls-ai32jr-1 (41.7) fall below the control maximum of 54.5 (vta-nacc-ctrl-6). It asked for the per-target stimulation and baseline counts and the chosen baseline day for these two mice and for vta-nacc-ctrl-6.
Round 2
2026-10-18Agreed with 24 of 24 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: The full executed notebook of the second run (after author-revise-1, commit 17be4c1), with the conclusion cells and the summary, the helper module, the claims file, the README and this attempt record, and the paper text.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

dLight is re-referenced and z-scored from the stored dF/F series; the paper used a pre-computed z-scored trace that is not in the files.fig1e-velocity-timescales
100 shuffles where the paper used 1,000.fig1e-velocity-timescales
121 sessions where the paper's code reports 216 for this panel; the dandiset holds fewer sessions, and habituation sessions are left out here.fig1e-velocity-timescales
Instances from 177 DLS sessions without stimulation (habituation sessions left out); the paper used more sessions per mouse.fig1h-syllable-dlight-differs
A 100-tree sklearn random forest with at least 5 instances per leaf on 1,000 instances per syllable, where the paper used cuML with 1,000 trees; the paper gives no accuracy, so "unable to predict" was set as below twice chance.fig1l-identity-not-decodable
One label shuffle per feature set as the null, where the paper used 100.fig1l-identity-not-decodable
Fivefold cross-validation repeated 20 times and 200 shuffles, where the paper used 100 repeats and 1,000 shuffles.fig2b-dlight-usage-across-syllables
Plain Shannon entropy of the outgoing transition probabilities per experiment, as the Methods state; the code uses a Dirichlet entropy estimate.fig2i-dlight-entropy-across-syllables
Five equal time segments in place of the paper's five behavioural segments (Extended Data Fig. 2e).fig2e-dlight-predicts-future-usage
Correlations are compared with shuffles directly, not z-scored to them; 100 shuffles.fig2e-dlight-predicts-future-usage
Same deviations as fig2e-dlight-predicts-future-usage.ed7-dms-no-usage-prediction
Stimulation, baseline and post sessions are identified from stimulation pulses, session dates and descriptions, as the files give no session number or type.fig3e-optoda-learning
As for fig3e-optoda-learning.fig3f-cumulative-counts

Provenance

Attempt record: 000559-markowitz-2023-01
Paper: Spontaneous behaviour is structured by reinforcement without explicit reward
Date: 2026-10-17
Made by: An agent, run by CatalystNeuro
Dandiset version: 0.260528.0858
Harness: Claude Code (GitHub Actions)
Model: claude-opus-5-5
Protocol: pilot-7

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.

← Dandiset 000552 Dandiset 000568 →