Neurons detect cognitive boundaries to structure episodic memories in humans (DANDI:000207)

Claims

From the paper Neurons detect cognitive boundaries to structure episodic memories in humans

Reproducing Zheng et al. (2022) from dandiset 000207 9

Notebook reproduce_zheng_2022.ipynb

Partially reproducedFigure 5a-d
The state of the population of all medial temporal lobe neurons changed abruptly in the first 300 ms after soft and hard boundaries, and only slowly in clips with no boundary, measured as the distance in principal-component space from the state at the boundary.
After a hard boundary the distance from the state at the boundary is larger than in clips with no boundary (9.03 against 7.42, p = 0.0001) and larger than before the boundary (after minus before 0.84 against 0.04, p = 0.004); 12 of 18 subjects show it alone. After a soft boundary the distance is larger than without a boundary (8.15, p = 0.003) but equally large before the boundary (after minus before 0.00, p = 0.55). That the shift completes within 300 ms was not shown.
How the test differed from the paper’s method
- 549 units from 18 sessions against the paper's 580 from 19.
- Pseudo-trials are matched across subjects by clip name; the paper does not say how.
- Principal components are fitted on the three condition averages from 1 s before to 1.5 s after the boundary, giving 22 components for 99% of the variance against the paper's 55. The averages are of the same trials that are then compared.
- A trial is z-scored from 1 s before to 8 s after clip onset; the paper says "the entire trial".
- Clips with soft boundaries are aligned to their first soft boundary, one pseudo-trial per clip. The authors' published code does not cover this analysis.
- The comparison with the second before the boundary is this attempt's; the paper shows curves.
- Two SB clips have their first soft boundary less than 1 s after onset. Leaving them out gives the same result (SB after minus before -0.03, p = 0.59; HB 0.84, p = 0.003).
ReproducedFigure 2a; Results, paragraph 3
In the time discrimination task, subjects chose the frame that was shown first in 73% of trials when the two frames were separated by no boundary or by a soft boundary, and in 53% of trials, significantly fewer, when they were separated by a hard boundary.
Time discrimination accuracy is 0.528 across a hard boundary, 0.728 with no boundary and 0.720 with a soft boundary (paper 0.53, 0.73, 0.73). It is lower across a hard boundary in 18 of 19 subjects (paired p < 1e-6 against both).
How the test differed from the paper’s method
- 19 of the paper's 20 subjects; the twentieth is not in the dandiset.
- Trials without a response are kept as coded in the file; leaving them out moves no mean by more than 0.01.
- Paired tests over subjects are added to the paper's one-way ANOVA.
ReproducedFigure 2b; Results, paragraph 3
Reaction times in the time discrimination task were longer when the two frames were separated by a hard boundary (2.10 s) than by no boundary (1.62 s) or a soft boundary (1.59 s).
Mean reaction time is 2.05 s across a hard boundary, 1.62 s with no boundary and 1.54 s with a soft boundary (paper 2.10, 1.62, 1.59 s), longer across a hard boundary in 18 of 19 subjects. The file's RT column is used, which runs from the disappearance of the frames.
How the test differed from the paper’s method
- 19 of the paper's 20 subjects.
- Trials without a response have an RT of 0 in the file and are kept; leaving them out gives 2.08, 1.64 and 1.56 s.
ReproducedFigure 2c; Results, paragraph 3
Confidence ratings in the time discrimination task were lower when the two frames were separated by a hard boundary (1.95 on a scale of 1 to 3) than by no boundary (2.52) or a soft boundary (2.59).
Mean confidence is 1.98 across a hard boundary, 2.45 with no boundary and 2.51 with a soft boundary (paper 1.95, 2.52, 2.59), lower across a hard boundary in 18 of 19 subjects.
How the test differed from the paper’s method
- 19 of the paper's 20 subjects.
- Trials without a response have a confidence of 0 in the file and are kept; leaving them out gives 2.00, 2.48 and 2.55.
ReproducedExtended Data Figure 2a; Results, paragraph 4
Scene recognition accuracy did not differ between frames from clips with no boundary (76%), a soft boundary (75%) and a hard boundary (75%).
Scene recognition accuracy over targets and foils is 0.738, 0.739 and 0.718 for clips with no, a soft and a hard boundary (paper 0.76, 0.75, 0.75), with no difference in a one-way ANOVA (F(2, 54) = 0.38, p = 0.69). Accuracy for hard-boundary clips is the lowest in every variant, by 0.01 to 0.05, without reaching significance.
How the test differed from the paper’s method
- 19 of the paper's 20 subjects.
- The paper does not say whether foils are counted. Targets and foils are; targets alone give 0.737, 0.718 and 0.688 (p = 0.59).
ReproducedExtended Data Figure 5; Results, section 2
76 of 580 medial temporal lobe neurons (13.1%) changed their firing at clip onsets and 4 of 580 (0.7%) at clip offsets, and none of them was a boundary cell or an event cell.
98 of 549 units (17.9%) fire more after clip onset, against 13.1% in the paper; the gap of 4.8 points is just inside the tolerance of 5, and 98 is more than a proportion of 13.1% would plausibly give (p = 9e-4). 26 units (4.7%) fire more after clip offset, the chance level, against 0.7% in the paper. Seven sessions hold 82 of the 98 onset-responsive units; the other eleven have 16 of 359 (4.5%), about chance, so the overall proportion averages over sessions that differ widely. The spike clock is shown to be aligned for ten of those eleven; for subject 3 (1 of 31 units responsive) it could not be established. None of the 8 boundary and event cells found responds to onsets or offsets, which is too few to judge that part of the claim.
How the test differed from the paper’s method
- 549 units from 18 sessions against the paper's 580 from 19.
- The paper's one-tailed test does not state its direction; an increase is tested. A decrease is found in 3.1% of units at onset.
- The permutation test is a sign-flip test with 1000 permutations.
BlockedFigure 3a-c; Results, section 2; Methods, Boundary cell· Methods underspecifiedverifier disagreed at first
42 of 580 medial temporal lobe neurons (7.24%) were boundary cells, which changed their firing in the second after soft and hard boundaries but not in the middle of clips with no boundary; 2.11% were expected by chance.
The analysis was run, but the procedure behind the paper's count could not be determined. 1 of 549 units (0.18%) meets the Methods' definition of a boundary cell, against 42 of 580 (7.24%) in the paper. The paper's chance level is 2.11%; the literal definition gives 0.03% here, alpha = 0.10 gives 0.10% and a looser pooled reading 0.55%, and the verifier found in simulation that the five-test definition stays at or below about 1% for every response profile it tried. No reading tried here produces the paper's chance level, so the paper's count came from a different procedure. Under the pooled reading there are 15 boundary cells (2.73%), more than in any of 300 shuffles. The authors' published code contains no cell selection. Recorded as not_reproduced before verification.
How the test differed from the paper’s method
- 549 units from 18 sessions against the paper's 580 from 19. Subject 19 is left out because its spike times are not on the clock of the trial tables, and the dandiset's units differ from the paper's in number and in their split by region.
- The paper's "permutation t-test" is implemented as a sign-flip test for before against after and a one-sided label-permutation test between conditions, with 1000 permutations.
- Soft-boundary trials are every soft boundary of a session, in SB clips and in HB clips (75 per session). The Methods do not say; this follows the authors' published plotting code (B01_raster_psth_encoding_BAligned_BSep_NWB.m, lines 29 to 37). The first soft boundary of each SB clip, used before that code was read, is kept in the notebook as an alternative.
- Each condition is tested on its own, five tests per class, which is the literal reading of the Methods. The authors' published code has no cell selection to check this against.
- 300 label shuffles for the chance level instead of 1000.
BlockedFigure 3e-g; Results, section 2; Methods, Event cell· Methods underspecifiedverifier disagreed at first
36 of 580 medial temporal lobe neurons (6.20%) were event cells, which changed their firing in the second after hard boundaries but not after soft boundaries or in the middle of clips with no boundary; 2.26% were expected by chance.
As fig3c-boundary-cells. 7 of 549 units (1.28%) meet the Methods' definition of an event cell, more than in any of 300 label shuffles (mean 1.22, maximum 6), but about a fifth of the paper's 36 of 580 (6.20%). At alpha = 0.10 there are 12 (2.19%, above all 300 shuffles) and under the pooled reading 10 (1.82%, 4 of 300 shuffles as many). The chance levels of these readings are 0.22%, 0.72% and 0.65%, against the paper's 2.26%. Recorded as partially_reproduced before verification.
How the test differed from the paper’s method
- The same as for fig3c-boundary-cells.
BlockedResults, section 2; Supplementary Table 4· Methods underspecifiedverifier disagreed at first
Boundary cells were most common in the parahippocampal gyrus (18 of 68 neurons, 26%, against 7% in the amygdala and 3.5% in the hippocampus), and event cells in the hippocampus (27 of 343, 7.9%, against 4.3% in the amygdala and 2.9% in the parahippocampal gyrus).
Depends on the cell selections of fig3c-boundary-cells and fig3g-event-cells, whose procedure could not be determined. With the literal definition there is no boundary cell among 53 parahippocampal units (paper 18 of 68), and event cells are 2 of 267 in the hippocampus, 3 of 229 in the amygdala and 2 of 53 in the parahippocampal gyrus. The dandiset's split of units by region also differs from the paper's. Recorded as not_reproduced before verification.
How the test differed from the paper’s method
- The same as for fig3c-boundary-cells.

Not tested in a notebook 11

8 metadata insufficient, 3 data missing from the dandiset

BlockedFigure 2d-f· Metadata insufficient
Recognition accuracy for a target frame fell with the time between the frame and the boundary before it, for soft boundaries (r = -0.61) and hard boundaries (r = -0.44), but not for clips with no boundary (r = 0.085).
The dandiset does not give the position of a tested frame within its clip. Its name says frameA or frameB, which by the paper's Methods (one frame from each half) is probably the first or the second half; no column states it. For clips with soft or hard boundaries the frame names also do not identify the clip.
BlockedFigure 3d,h; Extended Data Figures 3 and 4· Metadata insufficient
Boundary cells and event cells did not change their firing at clip onsets or offsets during encoding, nor at image onsets or offsets in the two memory tests.
The dandiset does not say which units the paper classified as boundary or event cells, and the Methods' definitions select 1 and 7 units here, fewer than the 10 of each that the claim's criterion requires. The authors' published code does not select or list the cells. The notebook reports that none of the 8 responds to clip onsets or offsets.
BlockedFigure 3i-j· Metadata insufficient
Boundary cells responded to hard boundaries about 100 ms earlier than event cells; their firing peaked 197 ms after the boundary, against 301 ms for event cells.
Needs the paper's boundary and event cells, which the dandiset does not label; 1 and 7 are found here, fewer than the 10 of each that the claim's criterion requires.
BlockedResults, section 2; Supplementary Tables 2 and 4· Data missing from the dandiset
Of 405 neurons recorded outside the medial temporal lobe, 8 were boundary cells and 9 were event cells, so boundary responses are largely confined to the medial temporal lobe.
All 565 units in the dandiset are in the hippocampus, amygdala or parahippocampal gyrus. The paper's 405 units from other areas are not included.
BlockedFigure 4a-c; Extended Data Figures 6 and 7· Metadata insufficient
Boundary cells fired more after soft and hard boundaries during encoding when the frame that followed the boundary was later recognised than when it was forgotten; there was no such difference for clips with no boundary, for the time discrimination task, or for event cells.
Scene recognition trials with soft or hard boundaries cannot be linked to the clip they test. Their frame names carry 10 distinct numbers, with up to 12 trials per number, where the clips are numbered 1 to 30. The authors' published code links nothing either; it groups test trials only by boundary type. The paper's boundary cells are not labelled.
BlockedFigure 4e-h· Data missing from the dandiset
Event cells' spikes were locked more strongly to the phase of the theta-band local field potential after soft and hard boundaries when the temporal order of the clip was later remembered than when it was not; boundary cells showed no such effect.
The dandiset holds spike times only. There are no local field potentials from which to compute theta phase.
BlockedFigure 5i-j· Metadata insufficient
The distance the population state travelled in the second after a boundary correlated positively with later recognition accuracy for soft (r = 0.65) and hard (r = 0.57) boundaries, and negatively with time discrimination accuracy for hard boundaries (r = -0.68).
Needs the memory outcome of each clip with a soft or hard boundary. Test trials of those conditions cannot be linked to clips, in either memory test, and the authors' published code contains no mapping.
BlockedFigure 6a-h· Metadata insufficient
During scene recognition, population activity correlated with the activity during encoding shortly after the soft or hard boundary of the tested clip, for correctly recognised targets and for foils wrongly called old, but not for forgotten targets or correctly rejected foils.
Needs each scene recognition trial paired with the encoding of its clip. That is possible only for clips with no boundary, which are the control condition of the claim.
BlockedFigure 6i· Metadata insufficient
The reinstated state was most similar to the state about 1.2 s after the boundary that preceded the tested frame, and 1.5 to 1.7 s before the tested frame itself was shown.
As fig6-reinstatement. The time of the tested frame within its clip is not given either.
BlockedFigure 6j-m· Metadata insufficient
Reinstatement after soft and hard boundaries was stronger for old decisions made with high confidence than with low confidence.
As fig6-reinstatement.
BlockedFigure 6n-o· Data missing from the dandiset
For foils wrongly called old, reinstatement was stronger when the foil was highly similar to its target; for correctly rejected foils it did not vary with similarity.
The similarity ratings of targets and foils, collected from a separate group of raters, are not in the dandiset. The trials could not be linked to clips either.

Findings about the data

What attempts found about the dandiset itself, for its owner and the archive.

Metadata errorIn the most recent published version (0.230530.1822) the spike times of subjects 13, 17 and 18 are shifted by 191.5, -16.8 and 9.3 s against the encoding trials, which did not move, so spikes and clips no longer line up. The version the paper cites (0.220216.0323) does not have this shift.
Metadata errorThe spike times of subject 19 are near -7.02e12 s while its events run from 0 to 2892 s, in both versions read. Its 16 units cannot be aligned to the task.
Identifier missingScene recognition and time discrimination trials with soft or hard boundaries cannot be linked to the clips they test. Their frame names are numbered differently from the clips (10 distinct numbers in recognition, 1 to 60 in time discrimination, against clips 1 to 30). The position of a tested frame within its clip is not given.
Identifier missingThe dandiset does not say which units the paper classified as boundary cells or event cells, so the paper's selection cannot be compared with a new one cell by cell.
Data missingThe dandiset has no local field potentials, no units outside the medial temporal lobe (the paper has 405), no similarity ratings and no stimuli, and it lacks the behaviour of one of the paper's 20 subjects.
Selection not documentedThe dandiset has 565 medial temporal lobe units where the paper analyses 580, with a different split by region (hippocampus 279 against 343, amygdala 233 against 169, parahippocampal gyrus 53 against 68). Nothing says how the two sets relate.
Metadata errorSeveral column descriptions do not match the values. stimCategory holds 0, 1, 2 and is described as 1, 2, 3. A missing second or third boundary is stored as the clip's start time and described as 0. The accuracy code 0 for "no response" is never used; trials without a response are coded incorrect, or, in 20 time discrimination trials, correct. resp_value is 1 for those trials and described as -1. RT is measured from the disappearance of the frames, which its description does not say, and is negative in 151 time discrimination trials.
Metadata errorThe events series holds 43 entries with a code 0 that its description does not list, with timestamps as low as -7.0e12 s. The spike times have no stated unit.
Methods not availableThe paper's Methods do not specify the permutation test, which soft boundaries are used, or how trials are matched across subjects for the pseudopopulation. The definitions of boundary and event cells, read literally, give a chance level about seventy times lower than the paper reports.
Methods not availableThe code the dandiset links to (github.com/rutishauserlab/cogboundary-zheng, commit 60f7ece) is four scripts that plot rasters from the NWB files. It contains none of the paper's analyses, no cell selection, chance level, statistics, state-space or reinstatement analysis, and nothing that links a tested frame to its clip.
Metadata errorThe authors' plotting code drops missing boundaries by testing for a stored time above 0. In version 0.220216.0323 a missing boundary is stored as the clip's start time, so the code would keep 150 soft-boundary rows where 75 exist and its fixed row ranges for the three conditions would be wrong. In version 0.230530.1822 missing boundaries are NaN and the code works, but that version has the shifted spike times.
OtherMethods and published code disagree on the firing-rate traces. The Methods give a 200 ms window moved in 2 ms steps; the code uses 10 ms bins smoothed with a Gaussian window of 20 bins. The code shows 0.5 s before and 1 s after a boundary. It also treats every soft boundary, including those inside hard-boundary clips, as a soft-boundary trial (75 per session), which the Methods do not state.
OtherFour hard-boundary clips (HB_7, HB_11, HB_14, HB_15) drive strong population activity in the half second before the hard boundary; all contain a soft boundary between 2.75 and 3.40 s. This raises the baseline of a before-against-after test of the hard boundary.

Notebooks

Every registered entry that uses this dandiset. An entry can hold several notebooks, and can use more than one dandiset.

Reproduction Cognitive boundaries and human medial temporal lobe neurons (Zheng et al. 2022)

Tests the findings of Zheng et al. (2022) that can be tested on dandiset 000207, which holds spike times of medial temporal lobe neurons and the behaviour of 19 patients in a task with video clips that contain no boundary, a soft boundary or a hard boundary. It compares memory for temporal order and for scenes across the three kinds of clip, looks for boundary cells and event cells with the paper's definitions, counts neurons that respond to clip onsets, and measures how far the population state moves after a boundary. Eleven of the paper's twenty findings cannot be tested, because the dandiset has no local field potentials, no neurons outside the medial temporal lobe, no labels for the paper's cell classes, and no way to link test trials with soft or hard boundaries to the clips they came from. All files

Reproducing Zheng et al. (2022) from dandiset 000207
reproduce_zheng_2022.ipynbExecuted, with outputs · Source

Verification

Round 1
2026-10-13
Agreed with 17 of 20 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: Round 1, the notebook at commit 8284b56 executed, with the author's conclusions and summary withheld, the helper module and the claims file. Outcomes below are compared with those recorded before verification.
- fig3c-boundary-cells: the verifier’s outcome was “Blocked”. Blocked by methods_underspecified. In simulation the five-test conjunction under label shuffling stayed at or below about 1% chance for every response profile tried, so the paper's 2.11% cannot come from the procedure as written, and failing to match 42 cells does not show the effect is absent.
- fig3g-event-cells: the verifier’s outcome was “Blocked”. As fig3c-boundary-cells.
- fig3-cell-regions: the verifier’s outcome was “Blocked”. Follows from fig3c-boundary-cells and fig3g-event-cells.
Round 2
2026-10-13Agreed with 20 of 20 outcomes. The same independent agent as round 1. Reviewed: Round 2, the notebook executed after the round-1 changes, with the conclusions and summary, the helper module, the entry README and this record. Its other findings were a header that still counted the outcomes from before verification, a summary sentence on chance levels that covered only four readings, and round-1 reasons recorded in words it had not used. All were corrected.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

19 of the paper's 20 subjects.fig2b-order-reaction-time fig2c-order-confidence edfig2a-recognition-similar
The same as for fig3c-boundary-cells.fig3g-event-cells fig3-cell-regions
549 units from 18 sessions against the paper's 580 from 19.edfig5-clip-onset-cells fig5d-state-shift
19 of the paper's 20 subjects; the twentieth is not in the dandiset.fig2a-order-accuracy
Trials without a response are kept as coded in the file; leaving them out moves no mean by more than 0.01.fig2a-order-accuracy
Paired tests over subjects are added to the paper's one-way ANOVA.fig2a-order-accuracy
Trials without a response have an RT of 0 in the file and are kept; leaving them out gives 2.08, 1.64 and 1.56 s.fig2b-order-reaction-time
Trials without a response have a confidence of 0 in the file and are kept; leaving them out gives 2.00, 2.48 and 2.55.fig2c-order-confidence
The paper does not say whether foils are counted. Targets and foils are; targets alone give 0.737, 0.718 and 0.688 (p = 0.59).edfig2a-recognition-similar
549 units from 18 sessions against the paper's 580 from 19. Subject 19 is left out because its spike times are not on the clock of the trial tables, and the dandiset's units differ from the paper's in number and in their split by region.fig3c-boundary-cells
The paper's "permutation t-test" is implemented as a sign-flip test for before against after and a one-sided label-permutation test between conditions, with 1000 permutations.fig3c-boundary-cells
Soft-boundary trials are every soft boundary of a session, in SB clips and in HB clips (75 per session). The Methods do not say; this follows the authors' published plotting code (B01_raster_psth_encoding_BAligned_BSep_NWB.m, lines 29 to 37). The first soft boundary of each SB clip, used before that code was read, is kept in the notebook as an alternative.fig3c-boundary-cells
Each condition is tested on its own, five tests per class, which is the literal reading of the Methods. The authors' published code has no cell selection to check this against.fig3c-boundary-cells
300 label shuffles for the chance level instead of 1000.fig3c-boundary-cells
The paper's one-tailed test does not state its direction; an increase is tested. A decrease is found in 3.1% of units at onset.edfig5-clip-onset-cells
The permutation test is a sign-flip test with 1000 permutations.edfig5-clip-onset-cells
Pseudo-trials are matched across subjects by clip name; the paper does not say how.fig5d-state-shift
Principal components are fitted on the three condition averages from 1 s before to 1.5 s after the boundary, giving 22 components for 99% of the variance against the paper's 55. The averages are of the same trials that are then compared.fig5d-state-shift
A trial is z-scored from 1 s before to 8 s after clip onset; the paper says "the entire trial".fig5d-state-shift
Clips with soft boundaries are aligned to their first soft boundary, one pseudo-trial per clip. The authors' published code does not cover this analysis.fig5d-state-shift
The comparison with the second before the boundary is this attempt's; the paper shows curves.fig5d-state-shift
Two SB clips have their first soft boundary less than 1 s after onset. Leaving them out gives the same result (SB after minus before -0.03, p = 0.59; HB 0.84, p = 0.003).fig5d-state-shift

Provenance

Attempt record: 000207-zheng-2022-01
Paper: Neurons detect cognitive boundaries to structure episodic memories in humans
Date: 2026-10-13
Made by: An agent, run by CatalystNeuro
Dandiset version: 0.220216.0323
Harness: Claude Code
Model: not recorded
Protocol: pilot-1

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.

verifier-1-and-2.md

← Dandiset 000147 Dandiset 000213 →