DANDI:000947

Reproducing Schwab et al. (2020) from dandiset 000947 18

Notebook reproduce_schwab_2020.ipynb

• ✕Not reproducedResults, Basic approach, database, and activity at rest, paragraph 5; Table 1

  209 single units met the criteria to be included as GPi neurons (105 in NHP G, 104 in NHP I) and 185 as VLa neurons (66 and 119).

  a total of 209 single units met the criteria to be included as GPi neurons and 218 as VLa neurons (Table 1).

  Results, Basic approach, database, and activity at rest, paragraph 5; Table 1

  The dandiset rule gives 268 GPi units (Gaia 137, Isis 131) and 279 VLa units (78 and 201), against 209 and 185 (218 in the text). Counting the same channel and sort letter on one day once gives 132 and 186; the paper's records found in the dandiset are 199 and 165. That merge joins some units the paper kept apart (47 of the 117 merged units spanning several files hold more than one of the paper's records). 518 of the 547 units are in files the paper used. Taking the rate over the length of the raw recording instead of the span of the spike train would leave out 2 more GPi units. The dandiset holds more units than the paper analysed and does not say which were kept or which units in consecutive files are one neuron.

  How the test differed from the paper’s method

  • Units are selected by the dandiset's location labels and the Methods' rate and trial criteria; the paper's anatomical criteria (within 0.5 mm of an M1-responsive site, the VLa border) cannot be applied from the files.
  • Each file's unit is counted once; the paper pools a neuron recorded over consecutive files.
• ✕Not reproducedResults, Movement-related increases in firing are common in both GPi and VLa, paragraph 2; Table 1

  Monophasic changes are more common in VLa than in GPi (88.9% against 66.7% of responses; chi-squared p = 2.8e-12), and polyphasic responses more common in GPi.

  Monophasic changes in firing ... were the most common change detected in both GPi and VLa, amounting to 66.7% and 88.9% of all responses detected, respectively. Monophasic responses were more common in VLa than in GPi

  Results, Movement-related increases in firing are common in both GPi and VLa, paragraph 2; Table 1

  What the paper reports

  • chi-squared, p = 2.8 × 10⁻¹²

  Monophasic responses are 71.0% in GPi and 76.7% in VLa, chi-squared p = 0.051, and VLa is outside tolerance. With the paper's records 69.0% and 81.2% (p = 0.00036), both within tolerance, so the outcome depends on the selection of units. With one record per presumed neuron 62.6% and 74.5% (p = 0.0023), VLa outside tolerance.

  How the test differed from the paper’s method

  • Unit selection by the dandiset rule (see table1-unit-counts).
• ✕Not reproducedResults, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3A

  Peri-movement changes begin later in GPi than in VLa (median onsets -47 and -90 ms; rank sum p = 2.4e-5).

  on average, GPi changes in discharge began later than those in VLa (median onset times: -47 and -90 ms for GPi and VLa populations, respectively

  Results, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3A

  What the paper reports

  • rank sum, z = 3.31, p = 2.4 × 10⁻⁵

  Median onsets of all changes are -48 ms in GPi (641) and -67 ms in VLa (613); GPi is later by 19 ms without significance (rank sum p = 0.079, one-sided p = 0.04), and the VLa median is outside tolerance. In Gaia the medians are equal (-48.5 and -48 ms). With one record per presumed neuron -73 and -72 ms (p = 0.95), with one value per presumed neuron -47 and -53 ms (p = 0.44). With the paper's records -49 and -85 ms (p = 0.004), which meets the criterion, so the outcome depends on the selection of units. At alpha 0.0001 -42.5 and -63 ms (p = 0.052).

  How the test differed from the paper’s method

  • Unit selection by the dandiset rule (see table1-unit-counts).
• ✕Not reproducedResults, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3C

  VLa decreases precede GPi increases (median onsets -108 and -62 ms; rank sum p = 0.01).

  VLa decreases did not follow but rather preceded GPi increases (median onset times: -62 and -108 ms for GPi and VLa populations, respectively; z = 2.58, p = 0.01, rank sum test; Fig 3C).

  Results, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3C

  What the paper reports

  • rank sum, z = 2.58, p = 0.01

  VLa decreases begin at a median of -19 ms (244) and GPi increases at -69 ms (377); VLa does not lead (rank sum p = 0.17). In Gaia VLa decreases come later (12.5 against -75 ms, p = 0.0028). With the paper's records VLa leads by 13 ms without significance (-86 against -73 ms, p = 0.2). The paper itself gives -62 ms for GPi increases here and -76 ms in Table 1. With one record per presumed neuron VLa decreases come later (-8 against -92 ms, p = 0.0082), and with one value per presumed neuron too (20.5 against -45 ms, p = 0.021). With random alignment the detector finds a decrease in 33.7% of VLa unit-directions (44.2% in the data), with a median onset of -8 ms, so many VLa decreases here may be chance detections, which would pull their median late. At alpha 0.0001 the medians are -21.5 (VLa) and -57.5 ms (GPi), p = 0.3, and chance still gives a decrease in 30.8% of VLa unit-directions (42.0% in the data), so the stricter threshold does not remove the chance decreases.

  How the test differed from the paper’s method

  • The ISI-based detection finds more decreases than the paper's on the same units (330 against 286 on the linked units), which weighs on this comparison.
• ✕Not reproducedResults, Correlated activity in GPi-VLa cell pairs is rare and unbiased, paragraph 1; Fig 4A-F; Table 2

  Short-time-scale spike-to-spike correlations between simultaneously recorded GPi and VLa units are rare (3.5% of 427 pairs over whole recordings, 2.2% of 183 at rest, 5.2% of 172 during movement) and their peaks are not larger than those of jittered controls (p = 0.975, 0.675 and 0.998).

  only 3.5% of the CCFs showed any statistically significant modulation (15 of 427 cell pairs; ... the overall distribution of the peak absolute correlation values was not greater than the distribution of control peak absolute correlations

  Results, Correlated activity in GPi-VLa cell pairs is rare and unbiased, paragraph 1; Fig 4A-F; Table 2

  What the paper reports

  • peak absolute correlations against jittered controls, whole recordings: one-sided permutation test, p = 0.975 (n = 427 pairs)
  • peak absolute correlations against jittered controls, rest: one-sided permutation test, p = 0.675 (n = 183 pairs)
  • peak absolute correlations against jittered controls, movement: one-sided permutation test, p = 0.998 (n = 172 pairs)

  By the KS permutation test the peak CCFs exceed those of jittered controls over whole recordings (338 pairs, p = 0.003) and during movement (306 pairs, p = 0.045), not at rest (298 pairs, p = 0.705). Significant pairs are 5.6%, 3.4% and 6.5%, none above 5% by a binomial test, and mostly positive (15 of 19 whole, 15 of 20 movement). The shift is small (median peak 0.042 against a control median of 0.038). The part of the statement that correlations are rare holds; the part that peaks are not larger than controls does not. Keeping only pairs with an approximate c0 of at most 0.3 leaves 332, 121 and 172 pairs (paper 427, 183 and 172) with KS p = 0.003, 0.333 and 0.025; one file per pair of presumed neurons gives 0.003, 0.627 and 0.037. The significant peaks fall at lags spread over 0 to 10 ms, and GPi and VLa spikes share a sample as often as independent trains would (11,376 against 11,435 expected), so shared artefacts do not explain the excess. Spike times are at the sample resolution (24414.0625 Hz) in every file. With the paper's records, paired when a GPi and a VLa record share a file (329, 298 and 301 pairs), the peaks exceed the controls over whole recordings (p = 0.005) and at rest (p = 0.033), not during movement (p = 0.203), so the outcome would be the same. Neither pair set matches the paper's 427 whole-recording pairs (338 on the dandiset rule, 329 on the paper's records); this attempt did not find out how the paper counted its pairs.

  How the test differed from the paper’s method

  • The paper's exclusion of pairs with a minimum detectable correlation c0 above 0.3 is not applied in the main test, which the criterion fixed before the analysis; the paper's simulation is not in its deposit. It is shown as a variant, with c0 approximated by a simulation on the CCF counts (independent Poisson counts at the pair's expected count, plus an injected peak).
  • Only VLa spikes are jittered; the Methods do not say which train.
• ✕Not reproducedResults, Correlated activity in GPi-VLa cell pairs is rare and unbiased, paragraph 5; Fig 4G-H; Table 2

  Trial-to-trial noise correlations between GPi and VLa are rare (5.4% of 332 pairs at rest, 7.2% of 372 during movement), not different from shuffled controls (p = 0.07 and 0.388), and equally often positive and negative.

  We found that the overall distribution of noise correlations was not significantly different from the control distribution (Fig 4G and 4H; p = 0.07 during rest, p = 0.388 during movement; permutation test).

  Results, Correlated activity in GPi-VLa cell pairs is rare and unbiased, paragraph 5; Fig 4G-H; Table 2

  What the paper reports

  • distribution of noise correlations against controls, rest: permutation test, p = 0.07 (n = 332 pairs)
  • distribution of noise correlations against controls, movement: permutation test, p = 0.388 (n = 372 pairs)
  • positive against negative significant correlations, rest: chi-squared, p = 0.64 (n = 18 pairs)
  • positive against negative significant correlations, movement: chi-squared, p = 0.34 (n = 27 pairs)

  Absolute noise correlations exceed trial-shuffled controls at rest and during movement (KS permutation p < 0.001 in both). 10.1% and 10.8% of pair-targets (17.5% and 18.6% of pairs) are significant against 5.4% and 7.2% in the paper. Signs are balanced (38 and 29 at rest, p = 0.33; 39 and 33 during movement, p = 0.56), as the paper reports. Spike counts drift with trial time in many units (VLa 36.1% and 38.3% of pair-targets, Spearman p < 0.05), but with counts linearly detrended the excess remains (9.3% and 9.7% significant, KS p < 0.001), and so it does against each pair-target's own shuffles (10.8% and 14.2%), with one file per pair of presumed neurons (KS p < 0.001) and with the paper's records (9.1% and 7.4% significant, KS p < 0.001 in both).

  How the test differed from the paper’s method

  • The paper's test statistic for noise correlations is not printed in its code; the KS permutation test on absolute correlations was fixed in the criterion before the analysis.
  • Significance uses the 2.5 to 97.5 percentiles of the shuffles pooled over pairs, as the paper's code does per time point.
• ✕Not reproducedResults, VLa activity decreases with bursts and pauses in GPi activity, paragraphs 1 and 2; Fig 5A-C; Table 2

  At rest, GPi burst onsets, burst offsets and pause onsets are each followed by a small, slow decrease of the population VLa firing rate.

  Small but significant transient decreases in VLa population spike rate were evident following both the onsets and offsets of bursts. ... a deceleration in VLa activity was also associated with pauses in GPi activity (Fig 5C).

  Results, VLa activity decreases with bursts and pauses in GPi activity, paragraphs 1 and 2; Fig 5A-C; Table 2

  What the paper reports

  • decreases against increases among pairs with a significant change, burst onsets: chi-squared, p = 0.04 (n = 19 pairs)
  • decreases against increases among pairs with a significant change, burst offsets: chi-squared, p = 0.05 (n = 13 pairs)
  • decreases against increases among pairs with a significant change, pause onsets: chi-squared, p = 0.16 (n = 13 pairs)

  The population VLa rate does not fall below the control band within 300 ms of GPi burst onsets (150 pairs, lowest point 2.5% below baseline), burst offsets (150, 1.9%) or pause onsets (110, 1.9%). Bursts here last 67.1 ms on average and pauses 145.3 ms, against 24 and 195 ms in the paper, and 50.7% of pairs show a change after burst onsets against 11.6% in the paper. The same per-pair test on the shifted controls gives 47.5%, 47.8% and 47.0%, so the test as the Methods describe it is at chance here, and the paper's per-pair test must differ. Computed as Fig5_S14.m does (each pair in percent of its baseline, band shifted over 0 to 800 ms), the population rate does not fall below the band either (lowest 98.1%, 98.4% and 96.9%). The Methods' rule on noisy baselines leaves out more than half of the pairs with at least 5 events (157 of 307 for burst onsets and offsets, 143 of 253 for pause onsets). With the paper's records (131, 132 and 86 pairs kept) the rate does not fall below the band either. Because the burst detector and the per-pair test differ from the paper's, this outcome says little about the paper's finding.

  How the test differed from the paper’s method

  • Burst and pause detection is a reimplementation of the Legendy and Salcman surprise method; the paper's implementation is not in its deposit (Fig5_S14.m plots stored averages and builds the control band; it holds no burst detector and no per-pair test).
  • A first run used log10 in the surprise and gave bursts far longer than the paper's; the natural log, as in the Methods' formula in the paper's MATLAB, was then used. The notebook shows the durations with log10 (bursts 254.0 ms, pauses 237.0 ms) as a variant.
  • The control shifts all events of a pair by one random offset of 2 to 20 s and uses VLa rate in rest samples only; the paper does not describe its control for Fig 5.
• ◐Partially reproducedResults, Basic approach, database, and activity at rest, paragraph 6; Fig 2A

  The firing rate of most units ramps during the start-position hold period (97% of GPi and 98% of VLa units with a significant linear trend); slopes are distributed symmetrically around zero, without a difference between GPi and VLa (rank sum p = 0.29).

  The mean firing rate of most single units showed small but significant ramps during the start-position hold period ... 97% and 98% of GPi and VLa cells, respectively). ... For both GPi and VLa unit populations, the observed slopes were distributed symmetrically around zero

  Results, Basic approach, database, and activity at rest, paragraph 6; Fig 2A

  What the paper reports

  • rank sum, p = 0.29

  98.5% of GPi and 98.6% of VLa units have a significant control-period slope (97.0% and 97.5% at alpha 0.01), and 49.9% of significant slopes are positive. GPi and VLa slopes differ (mean -0.598 against 0.233 spikes/s2, rank sum p = 0.042), where the paper finds no difference (p = 0.29); they also differ with one record per presumed neuron (p = 0.013) and with one mean slope per presumed neuron (p = 0.033). With the paper's records the fractions are 99.5% and 98.8% and the slopes do not differ (p = 0.187), which would meet the criterion, so the outcome depends on the selection of units. With every trial aligned to a random time, the same slope test finds a significant slope in 98.4% of GPi and 99.0% of VLa units, so the fractions do not show ramps beyond what noise in a smoothed function gives; the paper's code uses the same test.

  How the test differed from the paper’s method

  • The criterion did not define a partial outcome; the outcome is partial because the first part (ramps are common) holds and the second (no difference between areas) does not.
• ◐Partially reproducedResults, Movement-related increases in firing are common in both GPi and VLa, paragraph 3; Table 1; Fig 2B

  Monophasic increases are the most common form of response in both GPi and VLa (48.7% and 65.3% of responses), and increases make up most detected changes (61.5% and 68.8%, biphasic responses counted twice), more than decreases (chi-squared p = 1.2e-17).

  Monophasic increases were the most common form of response both in GPi and in VLa (48.7% and 65.3% of responses detected, respectively; Table 1). ... increases in firing were also more common than decreases both in GPi and in VLa (61.5% and 68.8% of changes, respectively

  Results, Movement-related increases in firing are common in both GPi and VLa, paragraph 3; Table 1; Fig 2B

  What the paper reports

  • chi-squared, p = 1.2 × 10⁻¹⁷

  Monophasic increase is the most common form in GPi (46.9%) and VLa (50.9%), and increases are 58.8% and 60.2% of changes (746 of 1254 overall, binomial p = 1.9e-11). The GPi values are within tolerance; the VLa values are below 65.3% and 68.8% by more than 10%, also with the paper's records (54.1% and 61.4%), with one record per presumed neuron (50.1% and 60.3%) and at alpha 0.0001 (51.9% and 60.1%). With random alignment the detector finds a decrease in 25.2% (GPi) and 33.7% (VLa) of unit-directions and an increase in 14.8% and 16.2%, so chance detections add mostly decreases.

  How the test differed from the paper’s method

  • Detrending of the control period and the ISI function are implemented here from the Methods; the paper's own implementation of these steps is not in its deposit.
• ◐Partially reproducedResults, Movement-related increases in firing are common in both GPi and VLa, paragraph 4; Table 1

  The mean integrated change in firing over the peri-movement epoch is positive in both GPi and VLa (3.89 and 1.75 spikes), significantly different from zero.

  This inclination toward increases was confirmed quantitatively by integrating changes in firing rate from baseline across the peri-movement epoch individually for each neuron. The mean of this integrated change was positive both for GPi and VLa

  Results, Movement-related increases in firing are common in both GPi and VLa, paragraph 4; Table 1

  What the paper reports

  • GPi: rank sum, z = 7.37, p = 1.7 × 10⁻¹³
  • VLa: rank sum, z = 6.59, p = 4.7 × 10⁻¹¹

  The mean integrated change is positive in GPi (3.07 spikes, signed rank p = 1.1e-10) and VLa (1.50 spikes, p = 3.7e-18), both more than 10% below 3.89 and 1.75. With the paper's records 3.42 and 1.97, with one record per presumed neuron 2.13 and 1.34.

  How the test differed from the paper’s method

  • Table 1 gives the unit as spikes x 10^3; the code's computation gives spikes, the reading used.
• ◐Partially reproducedResults, Movement-related increases in firing are common in both GPi and VLa, paragraph 5; Materials and methods, Spike sorting and detection of peri-movement discharge, paragraph 8

  A majority of GPi neurons (109 of 206) and some VLa neurons (56 of 179) respond significantly more strongly to one movement direction.

  the majority of GPi neurons (109 out of 206) and some VLa neurons (56 out of 179) showed significantly stronger responses to one of the movement directions

  Results, Movement-related increases in firing are common in both GPi and VLa, paragraph 5; Materials and methods, Spike sorting and detection of peri-movement discharge, paragraph 8

  122 of 264 responsive GPi units (46.2%) and 66 of 272 VLa units (24.3%) are directional, chi-squared p = 1.7e-07; both fractions are more than 10% below 52.9% and 31.3%. With the paper's records 48.2% and 26.8%, with one record per presumed neuron 53.8% and 21.9% (p = 9.8e-09).

• ✓ReproducedData Availability; Dryad doi:10.5061/dryad.0cfxpnvxm, Gaia_Analy.mat and Isis_Analy.matchecks the link to the paper’s data

  The units the paper analysed can be found in the dandiset, and the dandiset's spike times and trials reproduce the per-unit peri-movement activity the paper's code stored.

  Data and code can be found at https://doi.org/10.5061/dryad.0cfxpnvxm.

  Data Availability; Dryad doi:10.5061/dryad.0cfxpnvxm, Gaia_Analy.mat and Isis_Analy.mat

  Records are matched by file, channel id and sort letter. 364 of the 394 records the paper's code keeps (92.4%) are found in every file they span (Isis GPi 104 of 104, Gaia GPi 95 of 105, Gaia VLa 60 of 66, Isis VLa 105 of 119). For 724 matched unit-directions the SDF from the dandiset correlates with the stored SDF at a median r of 0.9992, above 0.9 for 94.6%. Valid trial counts equal the paper's for 62.6%. Detection agrees on the presence of an increase for 90.9% and of a decrease for 80.7% of unit-directions. 25 matched units the paper calls VLa are labelled "Thalamus" in the dandiset, and 97 records span 2 to 5 files. On the side of the outside data, the paper's list gives 209 GPi and 185 VLa units, as Table 1 does.

  How the test differed from the paper’s method

  • Trial exclusion follows the Methods (6 median absolute differences from the mean); the paper's stored trial counts differ for 37% of unit-directions, by a median of 0.
• ✓ReproducedResults, Basic approach, database, and activity at rest, paragraph 5; Table 1

  The resting firing rate of GPi neurons (mean 70.1 spikes/s) is significantly higher than that of VLa neurons (13.5 spikes/s), and the rates are consistent between the two animals.

  the resting firing rate of GPi neurons was significantly higher than that of VLa neurons ... and those rates and the differences between neural populations were highly consistent for the two animals

  Results, Basic approach, database, and activity at rest, paragraph 5; Table 1

  What the paper reports

  • GPi against VLa resting rates: rank sum, p = 10⁻⁸¹
  • rates and differences between the two animals: rank sum, p = 1

  Mean rest rate over unit-directions 75.65 spikes/s in GPi (531) and 12.78 in VLa (552), GPi higher with rank sum p = 1.2e-171. The animals differ slightly in GPi (73.17 and 78.25, p = 0.041), where the paper reports no difference; not in VLa (p = 0.53). With one record per presumed neuron the animals do not differ in GPi (74.23 and 74.21, p = 0.69). With the paper's records 64.26 and 13.53, with one record per presumed neuron 74.22 and 12.77.

  How the test differed from the paper’s method

  • Unit selection by the dandiset rule (see table1-unit-counts).
• ✓ReproducedResults, Basic approach, database, and activity at rest, paragraph 5; Table 1

  Action potentials of GPi neurons are shorter than those of VLa neurons (trough to peak 0.24 against 0.50 ms; rank sum p = 1e-33).

  Also, as expected, the action potentials of GPi neurons were short in duration as compared with those of VLa neurons

  Results, Basic approach, database, and activity at rest, paragraph 5; Table 1

  What the paper reports

  • rank sum, p = 10⁻³³

  Mean trough-to-peak width 0.247 ms in GPi and 0.509 ms in VLa, 532 of 547 units measured, rank sum p = 2.9e-37. One width is 0 and 9 reach the end of the 1.2 ms window; without them the means are 0.249 and 0.499 ms (p = 1.9e-35). No file gives one channel id to two electrodes, so each unit's channel names one electrode.

  How the test differed from the paper’s method

  • Widths come from the dandiset's 200 Hz high-pass filtered traces (the paper filtered at 300 Hz) and from the spikes in one stored chunk (about 3.2 s, median 273 spikes) per file and series.
• ✓ReproducedResults, Movement-related increases in firing are common in both GPi and VLa, paragraph 1; Table 1

  Nearly all units change their discharge around movement for at least one direction (GPi 99%, 206 of 209; VLa 88.1%, 163 of 185 in Table 1, 94% in the text).

  Nearly all single units showed a significant peri-movement change in discharge for at least one direction of movement, with slightly fewer VLa units responding (99% of GPi neurons and 94% of VLa neurons; p = 0.02, chi-squared test).

  Results, Movement-related increases in firing are common in both GPi and VLa, paragraph 1; Table 1

  What the paper reports

  • chi-squared, p = 0.02

  264 of 268 GPi (98.5%) and 272 of 279 VLa units (97.5%) respond in at least one direction. VLa matches the text's 94% but not Table 1's 88.1%; the criterion accepts either reading. With alpha 0.0001 the fractions with an increase alone are 81.0% and 77.4%. With every trial aligned to a random time 62.3% of GPi and 70.8% of VLa units are responsive, so the data are well above chance; with one record per presumed neuron the fractions are 100.0% and 98.4%.

  How the test differed from the paper’s method

  • The paper gives 88.1% (Table 1) and 94% (text) for VLa; both readings were listed and only the text's matches.
• ✓ReproducedResults, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3B

  GPi decreases lag behind VLa increases (median onsets -27 and -88 ms; rank sum p = 4.9e-4), against the gating prediction.

  the distribution of GPi decreases lagged in time behind that of VLa increases (median onset times: -27 and -88 ms for GPi and VLa populations, respectively

  Results, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 1; Fig 3B

  What the paper reports

  • rank sum, z = 3.19, p = 4.9 × 10⁻⁴

  GPi decreases begin at a median of -29.5 ms (264) and VLa increases at -84 ms (369), rank sum p = 0.00021. The GPi median is 0.2 ms inside its tolerance (-29.7 to -24.3 ms). Gaia 0 against -95 ms (p = 0.0014), Isis -44 against -81 ms (p = 0.055). The lag also holds with the paper's records (-36 against -85 ms, p = 0.00069), with one record per presumed neuron (-46.5 against -92 ms, p = 0.011), with one value per presumed neuron (-1.5 against -71.2 ms, p = 0.00074) and at alpha 0.0001 (-30 against -81.5 ms, p = 0.00054), but in all four the GPi median is outside its tolerance. With the paper's records the outcome would be partially reproduced, so it depends on the selection of units, and the match of the medians is fragile.

• ✓ReproducedResults, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 2

  Comparing every GPi response with every VLa response, 96% of comparisons conflict with gating (178,846 of 187,154); only 4% have a GPi onset 0 to 50 ms before the VLa onset with the opposite sign.

  A large preponderance of these comparisons (96%, n = 178,846 of 187,154 comparisons) directly conflicted with the gating hypothesis because of the relative timing or signs of the responses

  Results, Response onset latencies in GPi and VLa are incompatible with gating and rebound, paragraph 2

  95.4% of 392,933 comparisons conflict with gating (95.6% with the paper's records, 95.1% with one record per presumed neuron). With onsets shuffled across unit-directions the fraction is 95.1% (95.0 to 95.2), so the number does not distinguish the data from random onsets; it follows the gating window (98.0% for 0 to 20 ms, 86.6% for 0 to 200 ms).

• ✓ReproducedResults, Task performance, paragraph 1; S2 Fig

  Reaction times do not differ between the animals or reach directions; movement durations are longer for reaches to the right target, and NHP G moves more slowly than NHP I.

  Reaction times did not differ significantly between the two animals ... or between the two reach directions ... Movement durations were longer for reaches to the more-distant, right target than to the left ... NHP G moved more slowly overall compared with NHP I

  Results, Task performance, paragraph 1; S2 Fig

  What the paper reports

  • reaction time, animal: ANOVA, F[1,452] = 0, p = 0.97
  • reaction time, direction: ANOVA, F[1,452] = 0.2, p = 0.68
  • reaction time, animal × direction interaction: ANOVA, F[1,452] = 9.6, p = 0.002
  • movement duration, direction: ANOVA, F[1,453] = 416, p = 4 × 10⁻⁶⁶
  • movement duration, animal: ANOVA, F[1,453] = 1,419, p = 10⁻¹⁴¹
  • movement duration, animal × direction interaction: ANOVA, F[1,452] = 36.3, p = 10⁻⁹

  In the 165 files that hold the units analysed, reaction times differ neither between the animals (F = 0.02, p = 0.89) nor between directions (F = 0.69, p = 0.41), and movement durations are longer to the right target (Gaia 0.273 against 0.216 s, Isis 0.195 against 0.174 s; p = 5.8e-38) and longer in Gaia (p = 7.3e-56). Over all 250 pre-MPTP files with trials reaction times differ between the animals (F = 15.19, p = 0.00011); over the files of the paper's records they do not (F = 3.79, p = 0.053). The paper also reports a significant animal by direction interaction for reaction times (p = 0.002), which is not part of the claim; here p = 0.12.

  How the test differed from the paper’s method

  • The first version used all 250 pre-MPTP files with trials, including files with motor cortex, SMA or VLp units only, and gave a reaction-time difference between the animals. After round 1 the test was restricted to the 165 files with units of the dandiset rule, the files of the paper's data as the dandiset identifies them; the outcome changed from partially reproduced to reproduced. All files and the files of the paper's records are shown as variants.
  • Movement durations exist for 20 of the 91 Gaia files used.
  • The observations are file and direction medians (330 for RT); the paper's ANOVA has 452 error degrees of freedom and does not say what its observations are. No file set here gives 452 (326 on the files with units, 496 on all files, 320 on the files of the paper's records), and the reaction-time animal effect depends on the files used.