{
 "dandiset": "001172",
 "page": "dandisets/001172/",
 "generated": "2026-10-19T14:45:08+00:00",
 "caution": "A failure to reproduce is not a finding that the paper is wrong. It may be a failure of the attempt.",
 "outcome_counts": {
  "reproduced": 13,
  "partially_reproduced": 7,
  "not_reproduced": 1,
  "blocked": 5
 },
 "outcome_counts_with_outside_data": {
  "reproduced": 0,
  "partially_reproduced": 0,
  "not_reproduced": 0,
  "blocked": 0
 },
 "outcome_counts_elsewhere": {
  "reproduced": 0,
  "partially_reproduced": 0,
  "not_reproduced": 0,
  "blocked": 0
 },
 "attempts": [
  {
   "id": "001172-lohani-2022-01",
   "dandiset": "001172",
   "dandiset_version": "0.260129.0829",
   "paper": "10.1038/s41593-022-01202-6",
   "paper_title": "Dual color mesoscopic imaging reveals spatiotemporally heterogeneous coordination of cholinergic and neocortical activity",
   "date": "2026-10-18",
   "kind": "agent",
   "accountable": "CatalystNeuro",
   "agent": {
    "harness": "Claude Code (GitHub Actions)",
    "model": "claude-opus-5-5",
    "protocol": "pilot-7"
   },
   "tier": "runs",
   "entry": "001172-lohani-2022",
   "entry_title": "Cholinergic and calcium dynamics across behavioural states (Lohani, Moberly et al. 2022)",
   "description": "Tests the findings of Lohani, Moberly et al. (2022) on dandiset 001172, which holds dual-colour mesoscopic imaging of ACh3.0 and jRCaMP1b with face video, pupil and wheel data from 4 of the paper's 6 mice. It tests the regression of both signals on behaviour, transients at face movement and locomotion onsets, amplitudes in sustained states, and the state dependence of correlations between areas and between the two signals, using the Allen parcel masks of the paper's code. ECoG, pharmacology, control mice and cholinergic axon imaging are not in the dandiset.",
   "notebooks": [
    {
     "path": "reproduce_lohani_2022.ipynb",
     "title": "Reproducing Lohani, Moberly et al. (2022) from dandiset 001172",
     "executed": "notebooks/001172-lohani-2022/reproduce_lohani_2022.html"
    }
   ],
   "outcomes": [
    {
     "claim": "allen-parcel-link",
     "statement": "The paper's Allen parcel masks, from its code, can be laid on the dandiset's \u0394F/F pixels, so that the areas the paper names can be found in the data.",
     "source": "Methods, Preprocessing of imaging data and Post-processing analysis; parcells_updated121519.mat and brainMask.mat in the paper's code at commit c337686",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Per session (pixels non-zero in either \u0394F/F movie) the non-zero pixels equal the code's brainMask.mat rotated by 180 degrees in all 9 sessions (agreement 1.0; best other orientation 0.98). Per movie, 25 of the 26 movies of the 13 sessions (the 9 state sessions and the 4 visual stimulus sessions) agree at 1.0; the calcium movie of grabAM08 11232019 spont agrees at 0.996, as 263 pixels inside the mask, all in the right AUD parcel, are zero in every frame and none outside is non-zero. With 100% contrast gratings in one visual stimulus session per mouse (47 to 49 trials), the largest calcium response of the 46 parcels is in a left visual parcel in all 4 mice and left VISp exceeds right VISp in all 4 (0.108 to 0.216 against -0.024 to 0.065 percent \u0394F/F), but left VISp ranks 1, 1, 3 and 2, so first in 2 of 4 mice where the criterion asks 3 for reproduced. The dependent claims are tested; the partial result bears on parcel borders, not on which areas are where.",
     "deviations": [
      "The criterion's \"pixels with non-zero \u0394F/F in the blue and green movies\" can be read per movie or per session; the per-session reading was chosen after seeing that one movie fails the 99.9% bar under the per-movie reading. Under the per-movie reading the mask part fails on that movie and the claim would be not reproduced by the letter of the criterion. The failing pixels are all in the right AUD parcel, which the left-hemisphere analyses do not use."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig2a-r2-differs-across-areas",
     "statement": "The variance of ACh3.0 and jRCaMP1b signals explained by a ridge regression on FaceMap PCs 1 to 25, locomotion speed and pupil area differs across cortical areas (Friedman's ANOVA p < 0.001 for both, n = 6 mice).",
     "source": "Results, Calcium and cholinergic signals encode behavioral state; Figure 2a,b",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Friedman across the 23 left parcels with 4 mice, ACh chi2 = 67.98, p = 1.4e-06, calcium chi2 = 63.51, p = 6.7e-06. Full-model R2 ranges 0.030 to 0.397 (ACh) and 0.084 to 0.387 (calcium). Spontaneous sessions alone give p = 6e-07 and 0.005. With |speed| as the wheel predictor p = 1.3e-06 and 7.7e-06. For the three mice that match rows of the paper's own R2 file (grabAM05, grabAM07, grabAM08), mean full-model R2 is 0.182, 0.126 and 0.140 (ACh) and 0.207, 0.216 and 0.227 (calcium) against the file's 0.166, 0.126 and 0.142 and 0.193, 0.229 and 0.254.",
     "deviations": [
      "4 of 6 mice; spontaneous and air-puff sessions (9) averaged per mouse, as the Methods do not say which sessions were used.",
      "The ridge regression code is not public; the penalty is chosen per parcel by marginal likelihood over a grid of 91 values, as ridgeMML does by optimisation.",
      "The wheel predictor is the files' WheelSpeed, which is signed, as the Methods (\"a differential of distance\") and the code's wheel_changepoints.m (db1SpeedMpS) define speed. The first revision had switched to |speed|; the second revision returned to the signed series, and |speed| is a variant.",
      "Predictors are scaled with the training folds' statistics; the first run z-scored them over the whole session."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig2b-ap-gradient-inverted",
     "statement": "The coupling of behaviour to fluorescence has anterior-posterior gradients that are inverted between the signals, with ACh most strongly linked to behaviour in frontal areas and calcium in posterior areas.",
     "source": "Results, Calcium and cholinergic signals encode behavioral state; Figure 2b",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Spearman of mean R2 with anterior-posterior rank (1 = MOs) gives ACh rho = -0.761 (p = 2.5e-05) and calcium rho = 0.555 (p = 0.0059). Per mouse ACh rho -0.44 to -0.87; calcium 0.57 to 0.70 in three mice and -0.01 in grabAM05. Spontaneous sessions alone give -0.897 and 0.655. With |speed| -0.758 and 0.558.",
     "deviations": [
      "4 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig2c-face-best-predictor-ach",
     "statement": "For ACh3.0, facial movement (FaceMap PC1) predicts activity better than locomotion or pupil, in single-variable models (Friedman p = 0.006, face against locomotion and against pupil p = 0.031) and as unique contribution (Friedman p = 0.003).",
     "source": "Results, Calcium and cholinergic signals encode behavioral state; Figure 2c",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Single-variable R2 averaged over left parcels, face PC1 0.081, wheel 0.026, pupil 0.009; face highest in all 4 mice, Friedman p = 0.018 (exact p = 0.0046). Unique contributions face 0.081, pupil 0.016, wheel 0.004, face largest in all 4 mice, Friedman p = 0.039 (exact p = 0.042). The same holds with |speed|. Signed-rank tests give p = 0.125, the smallest possible with 4 mice, so the paper's p = 0.031 cannot be reached.",
     "deviations": [
      "The paper's post hoc signed-rank tests are replaced by the direction in every mouse.",
      "Unique contribution of face is from shuffling PC1 only; the Methods do not say which face PCs were shuffled."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig2c-face-best-predictor-calcium",
     "statement": "For jRCaMP1b, facial movement (FaceMap PC1) is the most robust predictor of activity (single-variable Friedman p = 0.006; face against locomotion and pupil p = 0.031).",
     "source": "Results, Calcium and cholinergic signals encode behavioral state; Figure 2c",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Changed to reproduced in the third verification round, from the partially_reproduced of the first revision, which rested on |speed| as the wheel predictor. With the signed speed of the Methods and the code, now the main analysis, single-variable R2 is face PC1 0.112, wheel 0.084, pupil 0.010, face best in all 4 mice by 0.016 to 0.053 over wheel speed, Friedman p = 0.018 (exact p = 0.0046). Unique contributions face 0.058, pupil 0.025, wheel 0.014, face largest in all 4 mice (in grabAM05 by 0.002 over pupil), Friedman p = 0.039. With |speed| wheel speed is higher in grabAM06 (0.147 against 0.142), face is best in 3 of 4 mice and the claim would be partial. In the paper's own per-mouse values (Fig2RawData.mat) face PC1 is the best single variable in all 6 mice, and in grabAM05 (row 1, matched by full-model R2 across parcels, Spearman 0.94 to 0.98) the wheel R2 is 0.0246 for calcium against 0.0253 here with signed speed and 0.0483 with |speed|. The paper found face against pupil not significant for the unique contribution here (p = 0.063).",
     "deviations": [
      "As for fig2c-face-best-predictor-ach.",
      "The regression code is not public; signed speed follows the Methods and wheel_changepoints.m."
     ],
     "disagreed": true,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-face-onset-transients",
     "statement": "Facial movement onset without locomotion is associated with significant whole-cortex transients in ACh (0.64 \u00b1 0.09 \u0394F/F, p = 0.001) and calcium (0.69 \u00b1 0.15 \u0394F/F, p = 0.006).",
     "source": "Results, paragraph on state transitions; Figure S7",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "ACh 0.728 \u00b1 0.119 percent \u0394F/F (paper 0.55 to 0.73), calcium 0.920 \u00b1 0.131 (paper 0.54 to 0.84), positive in all 4 mice, one-sample t-test p = 0.0089 and 0.0060. 12 to 31 onsets per mouse. The ACh mean is inside tolerance by 0.002. The same peak at random times in sustained quiescence (200 draws) has per mouse means 0.049 to 0.088 (ACh) and 0.113 to 0.177 (calcium); every mouse's transient is above the null's 95th percentile. Keeping the quiescent period after the last locomotion epoch (which the code's onset function drops) gives 0.728 and 0.871.",
     "deviations": [
      "4 of 6 mice; face onsets from the code's procedure, which the Methods describe only in part."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-locomotion-onset-transients",
     "statement": "Locomotion onset is associated with transient increases in ACh (0.68 \u00b1 0.17 \u0394F/F, p = 0.010) and calcium (1.50 \u00b1 0.11 \u0394F/F, p < 0.001).",
     "source": "Results, paragraph on state transitions; Figure S7",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "ACh 0.761 \u00b1 0.272 percent \u0394F/F (paper 0.51 to 0.85, p = 0.068 here), calcium 1.985 \u00b1 0.489 (paper 1.39 to 1.61, p = 0.027), positive in all 4 mice; grabAM08 has the largest values (ACh 1.555, calcium 3.436). 17 to 39 onsets per mouse. Null peaks at random quiescent times have per mouse means 0.044 to 0.082 (ACh) and 0.111 to 0.212 (calcium); every mouse's transient is above the null's 95th percentile.",
     "deviations": [
      "4 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-ach-transients-same-across-states",
     "statement": "ACh transients vary across cortical areas (p < 0.001) but do not differ between locomotion and facial motion onsets (p = 0.734, two-way repeated measures ANOVA).",
     "source": "Results, paragraph on state transitions; Figure S7",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Two-way repeated measures ANOVA (4 mice), onset type F = 0.03, p = 0.88; area F = 9.39, p < 0.0001. Whole-cortex locomotion minus face per mouse -0.285 to 0.625 (three negative, one positive), mean 0.034 percent \u0394F/F. An absence result with 3 denominator degrees of freedom, which carries little weight.",
     "deviations": [
      "4 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-calcium-transients-largest-at-locomotion",
     "statement": "Calcium transients vary across cortical areas (p < 0.001) and states (p = 0.005), with locomotion onset associated with the largest signal.",
     "source": "Results, paragraph on state transitions; Figure S7",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Locomotion transient above face transient in all 4 mice (by 0.447 to 2.245, mean 1.065 percent \u0394F/F), onset type F = 6.94, p = 0.078; area F = 31.6, p < 0.0001.",
     "deviations": [
      "4 of 6 mice; the ANOVA has 3 denominator degrees of freedom for onset type."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-ap-gradient-transients",
     "statement": "Transient magnitudes show a significant anterior-posterior gradient for ACh but not for calcium.",
     "source": "Results, paragraph on state transitions; Figure S7",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "ACh rho = -0.605 (p = 0.0022) at face onset and -0.752 (p = 3.5e-05) at locomotion onset, larger anteriorly; calcium rho = 0.014 (p = 0.95) and 0.106 (p = 0.63).",
     "deviations": [
      "The paper does not say which onsets the gradient refers to; both are tested."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3b-ecog-desynchronization",
     "statement": "Sustained facial movement and locomotion are associated with progressively increasing 30 to 100 Hz ECoG power.",
     "source": "Results; Figure 3b",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "The files hold no ECoG; the conversion code reads the Spike2 LFP channel but does not write it. The paper offers its data on request only.",
     "deviations": [],
     "disagreed": false,
     "data": "dandiset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3c-ach-sustained-face",
     "statement": "Sustained facial motion elevates whole-cortex ACh (low face -0.13 \u00b1 0.04, high face 0.18 \u00b1 0.08 \u0394F/F; low against high p = 0.046, post hoc LSD), with a main effect of state (p = 0.015).",
     "source": "Results, paragraph on sustained states; Figure 3c",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "With 3 mice (grabAM07 has no sustained high face epoch), low face -0.129 \u00b1 0.031 and high face 0.158 \u00b1 0.047 percent \u0394F/F, both inside the paper's ranges; high above low in all 3 mice (0.138 to 0.384), LSD p = 0.027; state effect F = 11.1, p = 0.023. Matched epochs per mouse 10, 2 and 13 (71.7, 14.6 and 93.7 s per state). Without grabAM06, high exceeds low in both other mice but LSD p = 0.207.",
     "deviations": [
      "3 of the paper's 6 mice; grabAM06 contributes 2 epochs per state."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3c-ach-locomotion-no-further-increase",
     "statement": "Locomotion does not further increase whole-cortex ACh beyond high facial motion (locomotion 0.16 \u00b1 0.06 \u0394F/F; locomotion against high face p = 0.717).",
     "source": "Results, paragraph on sustained states; Figure 3c",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Locomotion minus high face LSD t = 1.09, p = 0.34. The locomotion mean, 0.250 percent \u0394F/F, is above the paper's 0.10 to 0.22, and exceeds high face in all 3 mice (by 0.027 to 0.179), so the data point against the claim; the outcome rests on an absence test with 3 mice, which carries little weight.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3c-calcium-sustained",
     "statement": "Sustained facial movement and locomotion both increase whole-cortex calcium (low face -0.02 \u00b1 0.02, high face 0.17 \u00b1 0.04, locomotion 0.63 \u00b1 0.19 \u0394F/F; low against high p = 0.020, locomotion against high p = 0.044), with a main effect of state (p = 0.005).",
     "source": "Results, paragraph on sustained states; Figure 3c",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Low face -0.002, high face 0.164 (inside the paper's ranges), locomotion 1.106 percent \u0394F/F (paper 0.44 to 0.82). High above low in all 3 mice (0.124 to 0.222) but LSD p = 0.24; locomotion above high in all 3 (0.656 to 1.2), LSD p = 0.001; state effect F = 48.3, p = 0.0016. Without grabAM06 high minus low has p = 0.266 and locomotion minus high p = 0.019.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s7-ap-gradient-sustained",
     "statement": "Differences across sustained states have significant anterior-posterior gradients, for ACh comparing high and low facial movement, with larger increases in anterior areas, and for calcium comparing locomotion and high facial movement.",
     "source": "Results, paragraph on sustained states; Discussion; Figure S7",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "ACh high minus low face rho = -0.801, p = 4.3e-06 (larger anteriorly); calcium locomotion minus high face rho = 0.329, p = 0.125. Without grabAM06 (2 epochs per state) ACh rho = -0.874 (p = 5.3e-08) and calcium rho = 0.437 (p = 0.037), so the calcium half would be met with 2 mice; the criterion fixes the 3 mice.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3d-ach-frontal-above-visual",
     "statement": "During high facial motion ACh3.0 amplitude is higher in frontal-motor areas than in visual areas (paired t-test, p < 0.05).",
     "source": "Results; Figure 3d",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "High face ACh, frontal-motor 0.166 to 0.427 against visual 0.032 to 0.108 percent \u0394F/F, higher in all 3 mice, paired t = 4.34, p = 0.049. Without grabAM06 (2 epochs per state) the difference holds in both other mice but p = 0.247.",
     "deviations": [
      "3 of 6 mice; p is just below 0.05."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig3e-scopolamine-motor-not-visual",
     "statement": "The muscarinic blocker scopolamine decreases cortical activity in motor but not visual areas, and the nicotinic blocker mecamylamine has no significant effect.",
     "source": "Results; Figure 3e",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "No drug or vehicle sessions are in the dandiset (its sessions are spontaneous, air-puff and visual stimulus); the paper offers its data on request only.",
     "deviations": [],
     "disagreed": false,
     "data": "dandiset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4a-ach-correlations-up-with-face",
     "statement": "Increased facial movement is associated with significant increases in between-area correlations of ACh in anterior somatosensory and motor areas.",
     "source": "Results, State-dependent modulation of cortical network architecture; Figure 4a",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "High minus low face, 64 of 253 pairs increase significantly (BY q = 0.01) and none decreases; 39 of 45 motor-somatosensory pairs (0.867) against 25 of 208 other pairs (0.120). Mean change in z per mouse 0.095, 0.314 and 0.362. At q = 0.05, 98 increases.",
     "deviations": [
      "3 of 6 mice; the permutation test permutes epochs within mice, so its power comes from epochs (25 per state in all) more than mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4a-ach-correlations-down-with-locomotion",
     "statement": "Locomotion is associated with a reduction in between-area correlations of ACh that is significant across the cortex.",
     "source": "Results, State-dependent modulation of cortical network architecture; Figure 4a",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Locomotion minus high face, 243 of 253 pairs (0.960) decrease significantly, none increases, all six area groups involved; mean r 0.539 against 0.791; per mouse mean change in z -0.337 to -0.648.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4b-calcium-correlations-up-with-face",
     "statement": "Increased facial movement is associated with broad, significant increases in between-area calcium correlations.",
     "source": "Results, State-dependent modulation of cortical network architecture; Figure 4b",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "High minus low face, 161 of 253 pairs (0.636) increase significantly, none decreases, all six area groups involved; per mouse mean change in z 0.227 to 0.567.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig4b-calcium-correlations-down-posterior",
     "statement": "Locomotion occurs with modest decreases in calcium correlations that are significant primarily across posterior cortical areas.",
     "source": "Results, State-dependent modulation of cortical network architecture; Figure 4b",
     "outcome": "reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "Locomotion minus high face, 100 pairs decrease significantly and none increases; 55 of 66 posterior pairs (0.833) against 45 of 187 other pairs (0.241). Per mouse mean change in z -0.183 to -0.513.",
     "deviations": [
      "3 of 6 mice; posterior is defined here as ranks 12 to 23 of the Methods' order."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig5-coupling-up-with-face",
     "statement": "Sustained facial movement is associated with a significant increase in the correlation between ACh and calcium signals broadly across the cortex.",
     "source": "Results; Figure 5",
     "outcome": "partially_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "ACh-calcium coupling increases in all 23 left parcels (z 0.173 to 0.737) and significantly in 10 (none decreases), short of the 12 the criterion asks; 22 have uncorrected p < 0.05 and 20 pass BY at q = 0.05. Mean r 0.573 against 0.185; per mouse mean change 0.182 to 0.694.",
     "deviations": [
      "3 of 6 mice."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "fig5-coupling-down-frontal-with-locomotion",
     "statement": "Sustained locomotion is associated with a significant reduction in the correlation between ACh and calcium that is restricted to the frontal cortex.",
     "source": "Results; Figure 5",
     "outcome": "not_reproduced",
     "blocked_by": null,
     "notebook": "reproduce_lohani_2022.ipynb",
     "notes": "No parcel decreases significantly at q = 0.01 (2 at q = 0.05). Coupling is lower with locomotion in all 23 parcels (z -0.370 to -0.037) and in each mouse (-0.138 to -0.269); the largest decreases are in PTLp and SSp-tr (p = 0.0007 and 0.0006), and MOs and MOp have p = 0.072 and 0.076. The decrease is broad here, not restricted to frontal cortex.",
     "deviations": [
      "3 of 6 mice, with 25 matched epochs per state in all."
     ],
     "disagreed": false,
     "data": "with_labels",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s3-hemodynamic-correction",
     "statement": "The spatial regression on 395 nm signals removes stimulus-evoked and behaviour-related negative hemodynamic transients better than pixel-wise regression, shown in GFP and mCherry control mice and in ACh3.0 mice.",
     "source": "Results, first section; Methods, Validation of hemodynamics correction; Figures S3, S4",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "The comparisons rest on GFP, mCherry and backscatter data from control mice, which are not in the dandiset. The ACh3.0 part (visual stimulus negativity in V1 before and after correction) would need the uncorrected blue \u0394F/F, which is not in the files. It could in principle be recomputed from the raw blue and violet movies of the visual stimulus sessions (about 50 GB each), but that needs a reimplementation of the spatial regression and the atlas areas in native space, because the code's registration transforms are not in the files; this attempt did not do it. A check on one session (data check) shows the archived blue \u0394F/F shares almost no fluctuation with the violet \u0394F/F (r = 0.070) where the raw movies share much (r = 0.576), consistent with a correction having been applied.",
     "deviations": [],
     "disagreed": false,
     "data": "dandiset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s2-ach-sensor-validation",
     "statement": "ACh3.0 specifically reports cholinergic signalling, ex vivo (carbachol, blocked by scopolamine) and in vivo (basal forebrain stimulation).",
     "source": "Results, first section; Figure S2",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "The slice imaging with carbachol and scopolamine and the basal forebrain stimulation sessions are not in the dandiset.",
     "deviations": [],
     "disagreed": false,
     "data": "dandiset",
     "where": null,
     "proposed": null
    },
    {
     "claim": "s8-cholinergic-axons",
     "statement": "The relationship between behavioural state and cholinergic signalling is also seen with mesoscopic imaging of GCaMP6s in cholinergic axons.",
     "source": "Results, paragraph on sustained states; Figure S8",
     "outcome": "blocked",
     "blocked_by": "data_missing",
     "notebook": null,
     "notes": "No ChAT-Cre Ai162 (GCaMP6s) mice are in the dandiset.",
     "deviations": [],
     "disagreed": false,
     "data": "dandiset",
     "where": null,
     "proposed": null
    }
   ],
   "findings": [
    {
     "kind": "metadata_error",
     "statement": "The \u0394F/F series (DFFOnePhotonSeriesBlue, Green and Violet) are given the starting time of the first LED pulse, the time of raw frame 0, but their frame 0 is raw frame 499, as the detrending drops the first frames. Their times are 49.9 s early in every session; both the raw movies and the alignment of calcium with wheel speed show it."
    },
    {
     "kind": "metadata_error",
     "statement": "The FaceMap, pupil, eye and video timestamps are spaced 111 ms (9 Hz) where the camera pulses are 100 ms apart, span about 10% longer than the session, and end with two values equal to the first camera pulse. With these timestamps face movement and wheel speed are misaligned by 23 to 60 s; with frame i at camera pulse i, as the paper's code does, they align."
    },
    {
     "kind": "metadata_error",
     "statement": "RoiResponseSeriesParcellated holds the calcium (green excitation) \u0394F/F averaged over its ROIs, but its segmentation is linked to the ACh imaging plane. Its image masks are transposed relative to the \u0394F/F images, and its starting time is that of the first green pulse, 64.9 s before its first frame (150 frames after the \u0394F/F's first frame)."
    },
    {
     "kind": "metadata_insufficient",
     "statement": "The \u0394F/F descriptions mention detrending only; the blue and green \u0394F/F are also hemodynamically corrected and registered to the Allen atlas, rotated by 180 degrees, as the paper's code shows (the data are consistent with the correction, in one session checked). The Allen parcels the paper analyses are not in the files; they come from the paper's code. The parcellation in the files is a different, functional one with 89 to 107 ROIs per session and no area names."
    },
    {
     "kind": "metadata_error",
     "statement": "In the VisualStimulus tables the columns from stimulus_size on are shifted by one; stimulus_size holds the duration (2) and the first screen coordinate holds the size (20 degrees)."
    },
    {
     "kind": "data_missing",
     "statement": "The air-puff session grabAM07 11262019 has no Airpuff table, so its puff times are unknown."
    },
    {
     "kind": "selection_undocumented",
     "statement": "No air puff in the Airpuff tables is followed by a locomotion onset within 5 s (29 puffs), which fits the paper's code keeping only puffs without running. The tables then hold a selection of the puffs given, which the files do not say."
    },
    {
     "kind": "metadata_insufficient",
     "statement": "WheelSpeed and pupil_area have the unit \"unknown\"; the paper's code treats wheel speed as m/s, and its range fits that. WheelSpeed is signed (minimum -0.07 to -0.14 per session), which its description does not say."
    },
    {
     "kind": "other",
     "statement": "In the calcium \u0394F/F of grabAM08 11232019 spont, 263 pixels inside the brain mask, all in the right AUD parcel, are zero in every frame; in the other 25 \u0394F/F movies every pixel of the mask is non-zero."
    },
    {
     "kind": "metadata_insufficient",
     "statement": "The VisualStimulus tables give screen coordinates but not where the monitor stood relative to the mouse, so the hemisphere expected to respond cannot be read from the files."
    },
    {
     "kind": "data_missing",
     "statement": "The dandiset holds 4 of the 6 ACh3.0 and jRCaMP1b mice of the paper's code (grabAM05 to grabAM08, not grabAM09 and grabAM10), and no ECoG, drug sessions, control mice or ChAT-GCaMP6s mice. Five of the paper's findings cannot be tested."
    }
   ],
   "verifications": [
    {
     "date": "2026-10-18",
     "kind": "agent",
     "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook)",
     "reviewed": "Round 1. The first executed notebook with the conclusion cells and the summary removed, the helper module and the claims file, with the paper (PMC10661869). The verifier had no attempt record and took its counts from the claims file. It gave its own outcome for each claim; all 26 agree with the record as it then stood.",
     "agreed": 26,
     "of": 26,
     "disagreements": []
    },
    {
     "date": "2026-10-18",
     "kind": "agent",
     "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook)",
     "reviewed": "Round 2. The full executed notebook of the first revision, with its conclusions and summary, the helper module, the claims file, the attempt record and the paper (PMC10661869). The verifier agreed with 25 of the 26 outcomes and did not accept the reason for fig2c-face-best-predictor-calcium. It also asked for corrections to the robustness text without grabAM06, the runtime and some minor statements.",
     "agreed": 25,
     "of": 26,
     "disagreements": [
      {
       "claim": "fig2c-face-best-predictor-calcium",
       "outcome": "reproduced",
       "reason": "The arithmetic is not disputed; with |speed| wheel beats face in grabAM06 (0.147 vs 0.142), so the result is partial. The reason for switching to |speed| is disputed. The Methods define speed as \"a differential of distance\" after unwrapping, which keeps its sign; the paper's code computes it signed (wheel_changepoints.m line 50, db1SpeedMpS) and takes abs only for the threshold on mean epoch speed (line 135); the file's WheelSpeed is very likely that same series. The code repository contains Figure2Regression/Fig2RawData.mat, the paper's own per-mouse R2 (6 x 52); for calcium its single-variable face R2 is above wheel in all 6 mice; if its rows 1 to 4 are grabAM05 to grabAM08 (not established), its wheel R2 for mouse 1 is 0.0246 (calcium) and 0.0014 (ACh), against 0.0253 and 0.0017 here with signed speed and 0.0483 and 0.0031 with |speed|; the other mice are mixed. On this evidence signed speed is the better-supported reading, and under it the claim is reproduced. The outcome should either go back to reproduced with |speed| as the variant, or stay partial with a reason that matches the sources. Compare the per-mouse values with Fig2RawData.mat after confirming its row-to-mouse mapping; this would also give the first external check of the ridgeMML port."
      }
     ]
    },
    {
     "date": "2026-10-18",
     "kind": "agent",
     "by": "Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook)",
     "reviewed": "Round 3. The one proposed change of outcome, fig2c-face-best-predictor-calcium from partially_reproduced to reproduced, with the notebook cells behind it (cells 15, 26, 27, 33, 36 and 38 to 43), the attempt record and the paper (PMC10661869).",
     "agreed": 1,
     "of": 1,
     "disagreements": []
    }
   ]
  }
 ],
 "entries": [
  {
   "name": "001172-lohani-2022",
   "type": "reproduction",
   "title": "Cholinergic and calcium dynamics across behavioural states (Lohani, Moberly et al. 2022)",
   "description": "Tests the findings of Lohani, Moberly et al. (2022) on dandiset 001172, which holds dual-colour mesoscopic imaging of ACh3.0 and jRCaMP1b with face video, pupil and wheel data from 4 of the paper's 6 mice. It tests the regression of both signals on behaviour, transients at face movement and locomotion onsets, amplitudes in sustained states, and the state dependence of correlations between areas and between the two signals, using the Allen parcel masks of the paper's code. ECoG, pharmacology, control mice and cholinergic axon imaging are not in the dandiset.",
   "dandisets": [
    "001172"
   ],
   "papers": [
    "10.1038/s41593-022-01202-6"
   ],
   "derived_from": [],
   "notebooks": [
    {
     "path": "reproduce_lohani_2022.ipynb",
     "title": "Reproducing Lohani, Moberly et al. (2022) from dandiset 001172",
     "executed": "notebooks/001172-lohani-2022/reproduce_lohani_2022.html"
    }
   ]
  }
 ]
}