DANDI:000044

Reproducing Grosmark and Buzsaki (2016) from dandiset 000044 20

Notebook reproduce_grosmark_2016.ipynb

• ✕Not reproducedResults, paragraph 2

  The Bayesian decoder was built from 491 place cells over the eight sessions.

  a spatial Bayesian decoder (2), constructed from the firing-rate vectors of place cells (n = 491 cells) during track running (eight novel exploration sessions)

  Results, paragraph 2

  417 of the 562 pyramidal units have a smoothed rate-map peak of 1 Hz or more on running, below 491 by more than the tolerance of 49. At 0.5 Hz the count is 480 (within tolerance), at 2 Hz 352, so the count depends strongly on the threshold; the criterion fixes 1 Hz and the other thresholds are variants that do not decide the outcome. By subject, Achilles 181, Buddy 30, Cicero 127, Gatsby 79.

  How the test differed from the paper’s method

  • The place-cell definition is a common choice (peak of 1 Hz or more on running above 5 cm/s, 2 cm bins, 4 cm smoothing); the paper's definition is in the supplementary Methods, which were not available.
• ✕Not reproducedFigure 1C

  Sequence scores (rZ) of candidate events are higher in POST sleep than in PRE sleep (replay).

  Cumulative distribution of rZ for PRE, MAZE, and POSTevents, and 95% confidence intervals. ... Inset, mean ± SE of sequence scores in each condition. *P < 0.05; **P < 0.005; ***P < 0.0005 (Kruskal-Wallis test, followed by post hoc Tukey-Kramer tests).

  Figure 1C

  Pooled over sessions, as the criterion fixes the test, POST does not exceed PRE (mean rZ 0.763 against 0.770, Tukey on ranks after Kruskal-Wallis p = 0.412). Within sessions it does; POST is higher in 8 of 8 sessions (sign test p = 0.0039; one-sided rank-sum p < 0.05 in 6), and with each session's PRE mean subtracted POST is higher by 0.106 (p = 2.3e-14). The pooled test is dominated by the three circular-track sessions, which score lower in every epoch (PRE 0.161 and POST 0.260 against 1.112 and 1.207 on the linear tracks, where the event takes the better of two templates) and supply 11161 of 23801 POST events but 5912 of 16448 PRE events. The same test run on each kind of track finds POST above PRE on the five linear-track sessions (Tukey on ranks p = 5.1e-07) and on the three circular-track sessions (p = 1.1e-05). By subject, POST minus PRE is +0.089 (Achilles), +0.185 (Buddy), +0.021 (Cicero) and -0.173 (Gatsby) when pooled within subject.

  How the test differed from the paper’s method

  • Candidate events are population bursts of all units holding a ripple (150 to 250 Hz envelope above 3 SD on one LFP channel per session), outside REM in PRE and POST and below 5 cm/s in MAZE; the paper's detection is in the unavailable Methods.
  • The rZ null is 200 column-cycle shuffles of the posterior; the paper's shuffle is not stated in the main text.
  • On the linear tracks each event takes the better of the two direction templates, which raises their scores over those of the single-template circular track.
  • The Tukey comparison is scipy's Tukey HSD on the pooled ranks, not MATLAB's Tukey-Kramer after kruskalwallis.
  • Immobility for MAZE candidate events is taken from the 2D head speed, while running for the rate maps is taken from the speed of the linearized position, both at 5 cm/s.
• ✕Not reproducedResults, paragraph 5; Figure 4C

  Slow-firing neurons showed the strongest increases in coactivation from PRE to POST sleep.

  In contrast, slow-firing neurons showed the strongest increases in coactivation from PRE to POST sleep (Fig. 4B).

  Results, paragraph 5; Figure 4C

  The POST minus PRE change does not fall with firing rate; Spearman correlation -0.068 on the averaged curve (p = 0.55), negative in 5 of 8 sessions. The averaged change is 0.0019 in the slowest window and -0.0007 in the fastest, with a maximum of 0.0090.

  How the test differed from the paper’s method

  • Coactivation over whole epochs, all states.
• ✕Not reproducedFigure 4C legend

  Slow-firing cells showed the greatest replay, the partial correlation between RUN and POST coactivation accounting for PRE.

  Moreover, it was the slow-firing cells that showed the greatest replay (partial correlation across coactivation values between RUN and POST, accounting for PRE).

  Figure 4C legend

  The partial correlation of MAZE running and POST coactivation given PRE is 0.186 in the slowest window and 0.115 in the fastest, but the Spearman correlation over windows is 0.095 and negative in only 4 of 8 sessions. By subject it is -0.588 (Achilles), -0.646 (Buddy), -0.070 (Cicero) and 0.671 (Gatsby).

  How the test differed from the paper’s method

  • MAZE coactivation is taken over running bins (above 5 cm/s on the track).
• ✕Not reproducedResults, paragraph 6; Figure 4D

  Slow-firing, but not fast-firing, neurons increased their contribution to sequences from PRE to POST sleep, in a decoding restricted to each group.

  Slow-firing, but not fast-firing, neurons increased their contribution to neuronal sequences from PRE to POST sleep (Fig. 4D).

  Results, paragraph 6; Figure 4D

  On the pooled test neither half shows replay (low-rate POST minus PRE -0.024, p = 0.964; high-rate -0.049, p = 0.9999), as for all cells (-0.008, p = 0.916). In the within-session variant both halves show replay, larger for low-rate cells (+0.116 against +0.058, p of at most 0.0001 for each), which would be the criterion's partial pattern. On the linear-track sessions alone the pooled test also finds both, larger for low-rate cells (+0.077, p = 0.0046, against +0.044, p = 0.018).

  How the test differed from the paper’s method

  • Variant decodes use 50 column-cycle shuffles per event in place of 200.
• ✕Not reproducedResults, paragraph 6; Figure 4E

  PRE to POST increases in sequence content were limited to cells with high ripple-specific recruitment (gain).

  Conversely, PRE to POST increases in sequence content were limited to cells that showed a high degree of ripple-specific recruitment (gain) (Fig. 4E and figs. S20 to S22)

  Results, paragraph 6; Figure 4E

  On the pooled test neither half shows replay (high-gain -0.001, p = 0.71; low-gain -0.111, p = 1.0). In the within-session variant high-gain cells show replay (+0.111, p < 0.0001) and low-gain cells do not (+0.016, p = 0.14), the paper's pattern. On the linear-track sessions alone the pooled test finds replay in neither half (+0.023, p = 0.25; +0.010, p = 0.21) and the within-session variant finds it in both (+0.045, p = 0.029; +0.050, p = 0.0028).

  How the test differed from the paper’s method

  • Gain is the rate within PRE and POST candidate events over the PRE and POST rate, split at the session median.
  • Variant decodes use 50 column-cycle shuffles per event in place of 200.
• ◐Partially reproducedResults, paragraph 3; Figure 2Dverifier disagreed at first

  The PRE to POST increase in sequence score is eliminated when the place fields of the 10 to 20% of cells with the highest dPCC are shuffled, and remains after shuffling the bottom 75%.

  Replay (POST > PRE) was eliminated after shuffling or removal of the top 10 to 20% of cells with the highest ΔPCC scores, whereas it remained after shuffling or removal of the bottom 75%

  Results, paragraph 3; Figure 2D

  On the pooled test of the criterion replay is absent after shuffling the top 10% (POST minus PRE -0.100, p = 1.0) and the top 20% (-0.127, p = 1.0), but also after shuffling the bottom 75% (+0.022, p = 0.21), so one of the two conditions holds. The pooled test finds no replay before shuffling either (-0.008, p = 0.916), so on all sessions it cannot show a removal. On the five linear-track sessions replay is present before shuffling (+0.101, p < 0.0001), absent after shuffling the top 10% (+0.015, p = 0.23) or 20% (-0.039, p = 0.995) and present after the bottom 75% (+0.098, p < 0.0001); shuffling a random 10% or 20% leaves it (+0.099, +0.088). dPCC is measured on the events that are decoded again. Ranking cells on the even-numbered events and measuring replay on the odd-numbered ones, on the linear tracks replay falls from +0.116 to +0.046 (p = 0.024) after the top 10% and +0.042 (p = 0.038) after the top 20%, so out of sample it is reduced but not removed.

  How the test differed from the paper’s method

  • Variant decodes use 50 column-cycle shuffles per event in place of 200.
  • dPCC is the difference of the criterion's PCC, whose significance test is not the paper's (fig2b-strong-contributors); the ranking does not use that test.
• ◐Partially reproducedResults, paragraph 6

  Place cells divided into equal groups by sleep firing rate have medians of 0.39 Hz (low-rate) and 1.12 Hz (high-rate).

  we divided place cells into equal subgroups by either “off-line” sleep-firing rate or ripple-rate gain ... Low-rate neurons (median, 0.39 Hz) ... than high-rate cells (median, 1.12 Hz)

  Results, paragraph 6

  Split by non-REM sleep rate, the low-rate median is 0.435 Hz under both readings, just outside 0.39 +/- 0.039, and the high-rate median 1.091 Hz (a, within sessions) or 1.092 (b, pooled), within 1.12 +/- 0.112. With the rate over all of PRE and POST (a variant) both are within tolerance (0.395 and 1.041 under reading a). The split is of 417 place cells, not 491.

  How the test differed from the paper’s method

  • The paper does not say whether the split is within sessions or pooled; both readings are shown and give the same medians.
  • Off-line sleep rate is taken as the non-REM rate in PRE and POST; the rate over all of PRE and POST is a variant.
• ◐Partially reproducedResults, paragraph 6; fig. S16 as cited

  Low-rate neurons carried more spatial information per spike on the maze, and increased their within-ripple firing-rate gain from PRE to POST more than high-rate cells.

  Low-rate neurons (median, 0.39 Hz) contributed more spatial information per spike on the maze and displayed increased within-ripple firing-rate gains from PRE to POST sleep than high-rate cells

  Results, paragraph 6; fig. S16 as cited

  Under reading (a), which the criterion selects as neither reading of rate-split-medians matched, low-rate cells carry more spatial information (median 0.975 against 0.622 bits per spike, p = 4.1e-08), but their ripple-gain change from PRE to POST is not larger (0.251 against 0.155, p = 0.948). Reading (b) gives the same.

  How the test differed from the paper’s method

  • Ripple gain is the rate within candidate events over the epoch rate; the paper's definition is in fig. S16, which was not available.
• ✓ReproducedResults, paragraph 2; Figure S1 (as cited in the text)

  Recordings were made in four rats, in eight novel exploration sessions, each a PRE sleep, a MAZE epoch on one of two linear tracks or a circular track, and a POST sleep.

  Simultaneous recordings of well-isolated CA1 pyramidal single units were performed in four rats ... exploration of one of two linear or a circular track (MAZE) ... preexperience sleep (PRE) and postexperience sleep (POST) ... (eight novel exploration sessions)

  Results, paragraph 2; Figure S1 (as cited in the text)

  4 subjects and 8 sessions, each with PRE, MAZE and POST epochs in that order (8 of 8), on a 1.6 m linear track (4 sessions), a 2 m linear track (1) and a circular track (3). PRE lasts 2.98 to 5.37 h, MAZE 34.5 to 92.7 min and POST 2.22 to 4.51 h. The rooms the paper names are not in the files.

• ✓ReproducedFigure 1C; Figure 2C legend

  Sequence content is higher in MAZE immobility events than in PRE sleep events.

  the marked increase in sequence content observed in the MAZE and POSTepochs is driven by the recruitment of plastic cells

  Figure 1C; Figure 2C legend

  MAZE events score higher than PRE events (mean rZ 1.270 against 0.770, n = 2460 and 16448; Tukey on ranks p printed as 0), and MAZE is higher in all 8 sessions (one-sided rank-sum p from below 0.0001 to 0.0405). It holds on the linear-track sessions (p printed as 0) and on the circular-track sessions (p = 0.0007) separately. MAZE has 76 to 953 scored events per session.

  How the test differed from the paper’s method

  • Events and scores as for fig1c-post-gt-pre.
• ✓ReproducedResults, paragraph 5; Figure 4C

  In PRE sleep, coactivation was dominated by fast-firing neurons.

  In PRE sleep, coactivity was dominated by fast-firing neurons, and these correlations remained unchanged into POST sleep.

  Results, paragraph 5; Figure 4C

  PRE coactivation rises with firing rate, from 0.0208 in the slowest window to 0.0486 in the fastest; Spearman correlation 0.861 on the averaged curve, positive in 6 of 8 sessions. The POST minus PRE change of the fastest window does not differ from 0 over sessions (sign-rank p = 0.383), which fits the statement that these correlations remain into POST.

  How the test differed from the paper’s method

  • Coactivation over whole epochs, all states; the paper's restriction, if any, is not stated in the main text.
• ⊘BlockedResults, paragraph 3; Figure 2B· Methods underspecified

  216 neurons showed significant PCC scores in PRE or POST and were considered to contribute strongly to sequences.

  Neurons that showed significant PCC scores in either PRE or POST were considered to be strongly contributing to sequence formation (n = 216 neurons (Fig. 2B)

  Results, paragraph 3; Figure 2B

  The test fixed in the criterion (one-sided signed-rank test of a cell's contributions over its events, p < 0.05) finds 303 strongly contributing cells (72.7% of 417 place cells), against 216. With every rate map shifted it calls 56.1% of cells significant (0.843 on the 1.6 m tracks, 0.905 on the 2 m track, 0.140 on the circular track), so about 234 would be expected by chance. The paper calls these PCC scores significant, which implies a 5% false-positive rate; this test is far from it, so it is not the paper's procedure. The excess is on the linear tracks, where the contribution is taken on the template the event was selected for. A best-template variant of the contribution has a false-positive rate of 0.206 (0.132 in PRE, 0.118 in POST) and finds 188 cells, outside the tolerance; at alpha 0.001 its false-positive rate is 0.031 and it finds 95. The false-positive rates are measured on every fourth PRE and POST event, so on all events they would likely be higher. The best-template numbers are not identical across machines (the verifier's executed copy printed 190 cells and 0.129 in PRE); no conclusion rests on the difference. The PCC definition and its test are in the unavailable Methods.

  How the test differed from the paper’s method

  • The PCC and its significance test are choices made here, as the Methods were not available.
  • The 20 shifts of a cell's map are drawn afresh for every event.
• ⊘BlockedResults, paragraph 3; Figure 2C as cited· Methods underspecified

  Most neurons strongly contributing to PRE sequences (73%) also contributed to POST sequences.

  Importantly, the majority of neurons strongly contributing to PRE (73%) also contributed to POST sequences (Fig. 2C and fig. S19)

  Results, paragraph 3; Figure 2C as cited

  Blocked as it depends on fig2b-strong-contributors. Of 248 PRE strongly contributing cells, 235 (94.8%) also contribute in POST, against a chance level of 69.5%. With the best-template variant 72 of 102 (70.6%), within 73 +/- 7.3, against a chance level of 37.9%.

• ⊘BlockedFigure 2E· Methods underspecifiedverifier disagreed at first

  The PRE to POST increase in sequence content is attributable to a small number of plastic cells; removing them, unlike removing rigid cells, removes the increase.

  The PRE to POST increase in sequence content is attributable to only a small number of plastic cells.

  Figure 2E

  Blocked as it depends on fig2b-strong-contributors. On the pooled test, removing plastic cells gives POST minus PRE -0.168 (p = 1.0) and removing rigid cells +0.038 (p = 0.019); removing as many random strongly contributing cells gives -0.023 (p = 0.99). On the linear-track sessions -0.060, +0.136 and +0.094. With classes from the even-numbered events and replay on the odd-numbered ones, the linear tracks give -0.009 (p = 0.54) and +0.090 (p = 0.0027). With the best-template classes, pooled, -0.125 and +0.020 (p = 0.24).

• ⊘BlockedResults, paragraph 3; fig. S2 as cited· Methods underspecifiedverifier disagreed at first

  Rigid and plastic neurons were similarly distributed along the track.

  Rigid and plastic neurons were similarly distributed along the track (fig. S2).

  Results, paragraph 3; fig. S2 as cited

  Blocked as it depends on fig2b-strong-contributors. Rate-map peaks of rigid and plastic cells do not differ by the KS test (medians 0.419 and 0.486 of the track, D = 0.145, p = 0.072); with the split over all place cells p = 0.079, with the best-template classes p = 0.12.

• ⊘BlockedResults, paragraph 4; Figure 3A· Methods underspecifiedverifier disagreed at first

  Plastic neurons had higher place-specific indices and fewer place fields than rigid cells.

  Plastic neurons had higher place-specific indices and fewer place fields compared with rigid cells (Fig. 3A)

  Results, paragraph 4; Figure 3A

  Blocked as it depends on fig2b-strong-contributors. Plastic cells carry more spatial information (median 0.904 against 0.594 bits per spike, p = 1.9e-05; best-template classes p = 0.0035), but the number of fields does not differ (means 1.484 and 1.685, p = 0.073).

• ⊘BlockedResults, paragraph 4; Figure 3B· Methods underspecifiedverifier disagreed at first

  Plastic but not rigid neurons increased their spatial specificity (in-field versus out-of-field firing ratio) steadily during the MAZE session.

  plastic but not rigid neurons increased their spatial specificity steadily during learning (Fig. 3B and fig. S13)

  Results, paragraph 4; Figure 3B

  Blocked as it depends on fig2b-strong-contributors. Both groups raise their in-field ratio over the session (change of log10 ratio, plastic median 0.155, p = 0.0079; rigid 0.129, p = 0.030; between groups p = 0.41). With the best-template classes plastic p = 0.020 and rigid p = 0.28.

• ⊘BlockedResults, paragraph 5; Figure 4A· Methods underspecifiedverifier disagreed at first

  Session-wide firing rates of rigid neurons were higher than those of plastic neurons.

  Session-wide firing rates of rigid neurons were significantly higher compared with plastic neurons (Fig. 4, A and B, and figs. S14 and S15).

  Results, paragraph 5; Figure 4A

  Blocked as it depends on fig2b-strong-contributors. Rigid cells fire somewhat faster (median 0.700 against 0.591 Hz) without significance (p = 0.062; split over all place cells p = 0.053; best-template classes p = 0.74).

• ⊘BlockedResults, paragraph 5; Figure 4A· Methods underspecifiedverifier disagreed at first

  Ripple-related spiking, bursting and pairwise coactivation were higher in plastic than in rigid neurons.

  Ripple-related spiking, bursting, and pair-wise coactivation were higher in plastic versus rigid neurons (Fig. 4A and figs. S14 to S16).

  Results, paragraph 5; Figure 4A

  Blocked as it depends on fig2b-strong-contributors. Ripple gain (4.489 against 3.939, p = 0.015) and bursting (0.180 against 0.147, p = 0.028) are higher in plastic cells; coactivation is not (0.028 against 0.027, p = 0.90). With the best-template classes none reaches p < 0.05 (gain p = 0.15, bursting p = 0.11).