Dandiset 001754

Reproducing Knierim et al. (2000) from dandiset 001754 4

Notebook reproduce_knierim_2000.ipynb

• ✕Not reproducedAbstract; dandiset description ("Place cells eventually demonstrated normal spatially specific firing")

  Hippocampal neurons recorded on the space shuttle form a robust spatial code on the three-dimensional Escher staircase track in weightlessness.

  On flight day 9, 10 of 25 Rat1 units (0.40) have spatial information above the 95th percentile of 500 circular-shift shuffles on the Escher staircase (binomial p = 1.6e-7 against 0.05), and 2 of 28 Rat2 units (0.071, p = 0.41). By the criterion Rat1 is above chance but below 0.5 and Rat2 is not above chance, so neither the reproduced nor the partially reproduced rule is met. Repeats with 2000 shuffles and five other seeds give 10 or 11 of 25 for Rat1 and 2 of 28 for Rat2, and the run of 200 shuffles in the data check gives 10 and 2; 5 Rat1 units have p between 0.01 and 0.1. The outcome is the same as in 001754-knierim-2000-01, which recorded 12 of 25 for Rat1 (11 to 13 over seven runs); on the earlier version's files the notebook gives 11 or 12 of 25 for Rat1 and 2 or 3 of 28 for Rat2 over five seeds. Every real spike has a sample at its own time in both versions; the draft's trace adds samples at the other rat's spike times in each window, so shifted spikes find a sample more often and the shuffles change. Shifted spikes are now dropped about as often in Rat2's FD9 ES epoch (median 0.058) as in Rat1's (0.060), against 0.306 to 0.307 and 0.074 to 0.075 on the earlier version's files, and a count-matched variant gives 12 of 25 (0.48) for Rat1 and 2 of 28 for Rat2. With each unit's spikes replaced by a Poisson train of the same mean count, 9 of 122 flight ES units are significant (0.074, binomial p = 0.16 against 0.05), and 12 of 125 (0.096, p = 0.023) when each spike is moved to its nearest position sample, which reproduces the real drop rates; per session the false-positive fraction is 0 to 0.21, and in the judged sessions 0.160 and 0.154 for Rat1 and 0.037 and 0.214 for Rat2. Against these fractions Rat1's 10 of 25 is still above chance (p = 0.0035 and 0.0026) and Rat2's 2 of 28 is not (p = 0.28 and 0.99), so the outcome is the same, though the criterion's 0.05 is below the false-positive fraction of some sessions. With each unit's real train reversed in time within its epoch, which keeps its bursts and changes in rate and breaks its relation to the position, 2 of 122 flight ES units are significant (1 of 25 in Rat1's FD9 session, 0 of 27 in Rat2's), and Rat3's ES1 trains against the ES2 trace and the reverse give 0 of 16 and 0 of 19, so with real temporal structure the test is conservative. A unit's 95th percentile of the shuffles is nearly the same for its reversed train (Spearman rho 0.995), 1.7 to 2.3 times that of a Poisson train of the same count per session, and higher the fewer 10 s periods hold its spikes (rho -0.874 with the effective fraction of 10 s bins filled) and the fewer spikes it has (rho -0.601); the two explain 80% of its variance on a log scale. Rat2's FD9 units are the most clumped (median 0.145 of the bins, against 0.190 to 0.358 in the other sessions) and have the highest median 95th percentile (3.572). The test has less power for such units, and its power was not measured, so a low fraction, Rat2's especially, is weak evidence against a spatial code; the criterion's rule decides the outcome. The trace moves as a tracked animal would in Rat2's windows (median step 0.50 pixels and path length 33.0 pixels per second in its FD9 ES window), and units of neither rat are selective in any MC window (0 to 2 of 11 to 32), so neither settles whose position it is. Rat2's units are not selective with the trace in any window (1 of 16 and 2 of 28 in their own ES epochs, 0 of 15 and 0 of 41 in Rat1's), and Rat1's units are selective in Rat2's FD4 ES epoch (10 of 15), so the trace in Rat2's epochs may not be Rat2's position; the files say it is taken to be, and that the source does not name the animal tracked. On flight day 4, 11 of 20 Rat1 units, 1 of 16 Rat2 units, and 11 of 16 and 1 of 19 Rat3 units in its ES1 and ES2 epochs are significant. In MC epochs 0 to 2 units per session are significant. The "three orthogonal surfaces" part of the abstract is tested apart, and blocked.

  How the test differed from the paper’s method

  • The paper's Methods were not available; the selection (50 spikes at a known position), the 32 x 32 binning, the shuffle and the thresholds are common choices fixed in the claims file.
  • Occupancy is the original analysis's occupancy map for the epoch, and shuffled spikes take the nearest position sample within 0.25 s, as the dandiset gives position only at spike times. A spike whose nearest sample is the lost position is dropped.
  • Robust is read as at least half of the units significant in each flight day 9 session.
  • Rat2's flight day 9 session is judged although whose position the trace holds in it could not be established.
• ✓ReproducedDandiset description (not checked against the paper's text)

  In flight, spatial information content of place cells is comparable to preflight levels.

  Flight day 9 ES rate maps (64, Rats 1 and 2) have a median spatial information of 1.385 bits/spike against 1.448 for preflight ES maps (53); two-sided Mann-Whitney p = 0.45, difference -0.064 bits/spike, ratio 0.96. The main test's maps and numbers are the same as in 001754-knierim-2000-01, and so is the outcome. Per rat (p = 0.25 and 0.88), maps with archived names containing CEL (0.43), maps with a peak of 1 Hz or more (0.89), MC maps (0.92), all flight maps (0.12), bins with 5 or more occupancy counts (0.44) and information adjusted for rate x occupancy (0.37) all give p of 0.12 or more. Rate x occupancy is proportional to the spike count in the flight maps (about 22 per spike) and is assumed to be so in the preflight maps, where it cannot be checked. Information falls with the spike count (Spearman rho -0.226 over all ES maps). Rat1's compared sessions have similar counts and Rat2's do not (median unit spikes 359 and 352 preflight, 437 and 205 on FD9; median rate x occupancy 7862 and 7628 against 9105 and 5226), so Rat2's FD9 maps rest on fewer spikes, which by sampling bias raises their spatial information. The rate maps are smoothed with an unrecorded kernel, so no shuffle-corrected value could be computed. The dandiset description says preflight was recorded on a flat rectangular track, so this may compare different apparatus, and Rat2's preflight file shares no tetrode with its flight files. Rat2's FD9 ES maps have a median of 1.343 bits/spike although only 2 of its 28 units are significant against shuffles, so the rate maps' spatial information does not show that Rat2's cells were selective. Over the 124 tested flight ES units, the rate maps of units significant against shuffles have a higher median than the others (1.855 against 1.076, p = 9.0e-7), but Rat2's 26 non-significant FD9 units have a median of 1.283, close to the preflight 1.448. The bootstrap 95% interval of the FD9 minus preflight median difference is -0.425 to 0.312 bits/spike (-29% to +22% of the preflight median); for Rat1 alone it is -0.750 to 0.543 (p = 0.25). The test has no equivalence margin, so a drop of up to 29% of the preflight median (40% for Rat1 alone) is not excluded, and the result shows only that no difference was detected.

  How the test differed from the paper’s method

  • The source is the dandiset's description of the paper's findings, not the paper's text.
  • The paper's test is unknown; a two-sided Mann-Whitney U test at 0.05 was fixed in advance.
  • Spatial information comes from the original rate maps, without correction for the number of spikes, as the preflight files hold no position.
  • The unvisited bins (NaN) and the lost-position bin [63, 0] are left out of each map.
  • The preflight epochs labelled ES may have been recorded on a flat rectangular track, as the dandiset description says.
  • The assignment of the preflight files to rats is taken from the files, though the conversion notes list it as an open question.
• ✓ReproducedDandiset description (not checked against the paper's text)

  On the first in-flight exposures (flight day 4) spatial selectivity was abnormal.

  Flight day 4 ES rate maps (104, Rats 1 to 3) have a median of 1.177 bits/spike against 1.441 preflight (98); one-sided Mann-Whitney p = 0.024, below the criterion's 0.05 (ratio 0.82). This differs from 001754-knierim-2000-01 (not_reproduced, 107 maps, p = 0.055) because the draft leaves out every copy in the groups of identical maps, and three Rat2 FD4 ES maps that the first attempt counted once each (7.583 bits/spike each, mean rate below 0.001 Hz) are no longer in the dandiset. On the earlier version's maps the test gives p = 0.055 with all 107 distinct maps and p = 0.024 with the 104 that are in the draft, so the outcome rests on whether those three maps are counted. The FD4 median is lower in each rat (ratios 0.65 to 0.88), significantly only for Rat1 (p = 0.032). The test passes with maps of peak rate 1 Hz or more (p = 0.0095), over bins with 5 or more occupancy counts (p = 0.023) and with information adjusted for rate x occupancy (p = 0.0011), and not with maps whose archived names contain CEL (p = 0.073) or with MC maps (p = 0.060). Rat1's and Rat2's FD4 maps have fewer spikes than preflight, so sampling bias works against the claim. FD4 against FD9 for Rats 1 and 2 gives 1.081 against 1.385 (p = 0.095); spike-based information minus the shuffle mean is 0.025 against 0.211 for Rats 1 and 2 (p = 0.13) and 0.232 against 0.211 with Rat3 (p = 0.31). The spike-based fraction of significant units is 24 of 71 on FD4 and 12 of 53 on FD9 (p = 0.94), so the spike-based measures do not show lower selectivity on flight day 4. Without Rat2, whose position trace could not be attributed, Rats 1 and 3 give 1.242 against 1.510 (p = 0.035), on the draft and on the earlier version alike, as the three left-out maps are Rat2's.

  How the test differed from the paper’s method

  • The source is the dandiset's description of the paper's findings, not the paper's text.
  • Abnormal is read as lower spatial information; the description does not say what was abnormal.
  • The paper's test is unknown; a one-sided Mann-Whitney U test at 0.05 was fixed in advance.
  • The preflight epochs labelled ES may have been recorded on a flat rectangular track, as the dandiset description says.
  • The draft leaves out all copies of identical maps; three Rat2 FD4 ES maps counted in the first attempt are not in the test.
• ✓ReproducedFigure 3 (title only; which sessions the figure compares is not known)

  Place fields are stable across sessions.

  In Rat3's two flight day 4 ES epochs, the 15 units with 50 or more spikes in each have a median rate-map correlation of 0.537 (range -0.107 to 0.849), above the 95th percentile of 1000 re-paired medians (0.102, p = 0.001) and above 0.5; 8 of 15 exceed 0.5. These numbers and the outcome are the same as in 001754-knierim-2000-01. With smoothing of 0.5 to 2 bins and 50 or 100 spikes the median is 0.315 to 0.707, always above its null (p = 0.001); at 0.5 bins it is below 0.5. The dandiset's ES1 and ES2 rate maps, now linked to the same unit, give a median of 0.340 over 18 units (0.157 in the first attempt, when maps were paired by short names). A unit is the same sorted cluster in both epochs, as the units' description now states; mean rates are related across the epochs (Spearman rho 0.598, re-paired 95th percentile 0.339). In the first epoch 11 of 16 units are significant against shuffles and in the second 1 of 19, with similar occupancy and shuffle drops and more spikes per tested unit in the second (median 455 against 301). For several stable units the 95th percentile of the shuffles rises more than their information in the second epoch (TT4 cluster 1, information against 95th percentile, 2.449 against 1.326 and then 3.592 against 4.305; TT5 cluster 1, 0.910 against 0.504 and then 1.472 against 2.393); the epochs use 348 and 299 bins with 5 or more occupancy counts. Poisson trains give 0 to 2 false positives in either epoch. These units fire fewer spikes in fewer 10 s periods in the second epoch (TT4 cluster 1, 1582 spikes filling an effective 0.267 of the 10 s bins and then 423 filling 0.096; TT5 cluster 1, 8844 at 0.575 and then 2221 at 0.307), which raises the shuffles' 95th percentile (see flight-spatial-code); the stability measure does not use the shuffle test.

  How the test differed from the paper’s method

  • Only the title of Figure 3 was available; which sessions it compares is unknown. The test uses the only pair of same-track sessions with position on one day.
  • Takes the same sorted cluster in both epochs to be the same neuron; the files hold no waveforms to check it.