Datasets / DANDI:000121

DANDI:000121

✓ Verified by an independent agent

Structure and variability of delay activity in premotor cortex

Converted by CatalystNeuro 0.0 MB 0 files
  • 6 reproduced
  • 7 partially reproduced
  • 6 not reproduced
  • 1 blocked

View on DANDI Paper JSON How to read outcomes

Claims

From the paper Structure and variability of delay activity in premotor cortex

Reproducing Even-Chen et al. (2019) from dandiset 000121 19

Notebook reproduce_even_chen_2019.ipynb

  • Not reproducedMaterials and methods, Dimensionality reduction, paragraph 2verifier disagreed at first

    The top six PCs of the condition-averaged delay activity explain cumulatively 58, 82, 88, 92, 94 and 95% of the variance for J (120 conditions) and 81, 89, 92, 94, 95 and 96% for R (132 conditions).

    The top 6 PCs explained the following cumulative percentages of the variance: 58% 82% 88% 92% 94% 95% (J); 81% 89% 92% 94% 95% 96% (R)

    Materials and methods, Dimensionality reduction, paragraph 2

    R has the paper's 132 conditions. Twelve of thirteen values are within tolerance; R's PC1 is 72.8% against 81 +- 8.1%, outside by 0.14 points. The criterion's partial case needs all R values within tolerance, so by its letter the result is not reproduced. With the day's mean taken over the analysed delay windows R's PC1 is 80.5%, and without day-mean subtraction 80.6%. Spikes exist only within trials (99.93 to 99.95% of PMd spikes inside a trial in every session), so the main day mean is the rate over trial time. J's values are within tolerance on 96 of its 120 conditions.

    How the test differed from the paper’s method

    • The day's average rate is the session's rate over its recorded time; the paper does not say how it was computed. The other choice is shown.
    • J is 5 of its 16 sessions and 96 of its 120 conditions.
  • Not reproducedResults, Structure of neural activity during delay period, paragraph 2; Figure 2a

    The spatial plane captures a large part of the variance of the condition-averaged activity for the ring, horizontal, vertical and 3-rings tasks; 66.8, 64.4, 47.7 and 68.2% for J and 33.7, 30.7, 31.0 and 30.1% for R (p < 1e-4).

    The spatial plane captures a large percentage of the overall variance of the trial-averaged neural responses for the ring, horizontal, vertical, and 3-rings tasks: 66.8%, 64.4%, 47.7%, and 68.2% (monkey J), 33.7%, 30.7%, 31.0%, 30.1% (monkey R)

    Results, Structure of neural activity during delay period, paragraph 2; Figure 2a

    What the paper reports

    • p < 10⁻⁴ (n = both monkeys)

    Measured values are reading (a) with tasks by the Methods' dates (R ring, horizontal, vertical, 3-rings; J ring, vertical, 3-rings). No pairing of readings puts more than three of the seven values within tolerance. R's shares (29.5 to 36.9%) lie near the paper's (30.1 to 33.7%); J's are lower (40.2 to 54.4% against 47.7 to 68.2%) on 5 of 16 sessions without the horizontal task. The shares exceed those of planes fit to targets shuffled across conditions with p = 0.005 to 0.16 (shuffle medians 11.5 to 24.8%), so the stated p < 1e-4 does not hold against this null; the paper's null is not stated. 1000 shuffles cannot give p below 1.0e-03, but the observed p values lie well above that.

    How the test differed from the paper’s method

    • J's plane is fit on 96 of its 120 conditions and its horizontal task is absent.
    • Two definitions of the share and two namings of R's horizontal and vertical tasks were tested.
  • Not reproducedResults, Structure of neural activity during delay period, paragraph 4

    Projected onto the top PCs, the distance dimension is about half as large as the speed dimension.

    when projecting the speed and distance dimensions onto the top PCs, the distance projection is approximately half compared to the projection of the speed dimension

    Results, Structure of neural activity during delay period, paragraph 4

    The distance axis projects onto the top PCs as much as the speed axis for J (ratio 1.01) and more for R (1.68). With the plane projected out of the distance axis the ratios are 0.73 and 1.14.

    How the test differed from the paper’s method

    • The paper does not define the distance dimension or the projection; both are defined here.
  • Not reproducedResults, Structure of neural activity during delay period, paragraph 4

    Across all single trials the speed dimension explains 2% (J) and 1% (R) of the variance of the neural state, and the spatial plane 20% (J) and 10% (R).

    This dimension explains J: 2%, R: 1% of the variance in the neural state across all trials; for reference, the spatial plane explains J: 20%, R: 10% for single trials

    Results, Structure of neural activity during delay period, paragraph 4

    Only R's speed value (0.86% against 1%) is within tolerance. All four values are below the paper's and keep its order, J above R. Within single sessions the plane explains 11.2 to 20.7% (J) and 6.5 to 14.2% (R).

    How the test differed from the paper’s method

    • J is 5 of its 16 sessions.
  • Not reproducedResults, Accuracy of endpoint prediction from delay activity, paragraph 2; Figure 4a and 4b

    Direction classification of the ring targets reaches 32.2% for J (24 targets) and 16% for R (36 targets); distance classification over 12 targets reaches 18.9% (J) and 21.4% (R).

    direction was J:32.2% ± 0.6% accuracy for 24 targets (R:16% ±.5% for 36 targets) while distance classification for only 12 targets was lower for J (18.9% ±.1%) and comparable for R (21.4% ±.2% for R)

    Results, Accuracy of endpoint prediction from delay activity, paragraph 2; Figure 4a and 4b

    All four accuracies are below the paper's (32.2, 16, 18.9, 21.4%) and well above chance (4.2, 2.8, 8.3%). With hyperparameters chosen on the 3-rings trials they are 24.3, 13.8, 16.0 and 18.3%. J's ring value comes from 1020 trials of one session against four in the paper, and its distance value from the vertical task only.

    How the test differed from the paper’s method

    • SVM hyperparameters are sklearn's defaults; the paper's tuned values are not reported. A tuned variant is shown.
    • The tolerance is 10%, not the paper's SEM over folds (see the claims file).
    • J ring is 1 of 4 sessions; J distance is the vertical task only.
  • Not reproducedResults, Accuracy of endpoint prediction from delay activity, paragraph 3; Figure 4c and 4d

    Angular errors of direction classification decrease with ring distance, but errors measured as arc length are not significantly different between rings (F test, p > .05 for J and R).

    When presenting the decoding accuracy in term of angular error, mean error decreases as a function target distance (Fig 4c). In contrast, when presenting the errors as arc-length ... the arc-length error distributions are not significantly different

    Results, Accuracy of endpoint prediction from delay activity, paragraph 3; Figure 4c and 4d

    What the paper reports

    • F test, p > 0.05 (n = J and R)

    Angular errors fall with ring distance (J 41.0, 24.3, 18.3 degrees; R 52.0, 42.4, 36.7), but arc errors grow (J 2.87, 3.40, 3.84 cm; R 3.63, 5.92, 7.69 cm), and the one-way ANOVA finds the rings different (J F = 22.15, p = 2.7e-10; R F = 52.16, p = 1.5e-22). Restricted to errors within the 4 cm ring's range, J's rings do not differ (p = 0.23) and R's do (p = 1.3e-03); the paper's Figure 4d inset uses that restriction for its means, so J matches there.

    How the test differed from the paper’s method

    • The paper's "F test" was read as a one-way ANOVA of per-trial arc errors; a variance test (Levene) and the restricted ANOVA are shown.
  • Partially reproducedMaterials and methods, Experimental design, paragraph 3; Figure 1b

    The sessions are those the Methods list for each target configuration; for monkey R, ring on 2017-01-24 and 25, horizontal on 2017-01-17 and 18, vertical on 2017-01-15 and 16, and 3-rings on 2017-01-19; for J, ring, vertical and 3-rings on the dates listed.

    36 for R (3577 over two sessions: 2017-01-24, 2017-01-25) ... For horizontal configuration ... 3699 trials over two sessions for R (2017-01-17, 2017-01-18). For vertical ... 4040 trials over two sessions for R (2017-01-15, 2017-01-16)

    Materials and methods, Experimental design, paragraph 3; Figure 1b

    8 of 12 sessions have the configuration the Methods give for their date. R's 2017-01-15 and 16 hold horizontal targets (Methods vertical) and 2017-01-17 and 18 vertical targets (Methods horizontal); all four mismatches are horizontal and vertical exchanged. 2017-01-17 also holds 69 successful trials to six ring targets outside its layout. The dandiset holds all 7 of R's sessions and 5 of J's 16.

    How the test differed from the paper’s method

    • The configuration is inferred from the target positions, as the files' task_id is 2 in every trial.
  • Partially reproducedMaterials and methods, Experimental design, paragraph 3

    Monkey R performed 3577 ring trials over two sessions, 3699 horizontal over two, 4040 vertical over two, and 912 3-rings trials in one session.

    36 for R (3577 over two sessions ... 3699 trials over two sessions for R ... 4040 trials over two sessions for R ... 912 trials from one session for R

    Materials and methods, Experimental design, paragraph 3

    Grouped by the Methods' dates, ring 3584 (paper 3577) and vertical 3842 (4040) are within 10% under all three readings, horizontal 4156 (3699) and 3-rings 1340 (912) are not. Measured values are reading (a), successful delayed center-out trials; readings (b) and (c) differ by at most 26 trials per configuration. Grouped by the file geometry, three of four are within 10% (horizontal 3842 against 3699, vertical 4156 against 4040), which fits the exchange of dates found in session-configurations. The 3-rings session holds 47% more successful delayed trials than the paper used, and no reading explains the difference. Reading (b) leaves out failed trials without a go cue; counting them gives ring 4568, horizontal 5250, vertical 5033 and 3-rings 1671, none within 10%.

    How the test differed from the paper’s method

    • Which trials the paper counted is not stated; three readings were tested.
  • Partially reproducedMaterials and methods, Experimental design, paragraph 1

    Delays were uniform from 300 to 700 ms for J and 400 to 900 ms for R, 10% of trials had no delay; average reaction time was 346 ms (J) and 311 ms (R), average reach time 582 ms (J) and 482 ms (R), and reaction time was negatively correlated with delay (p < 1e-4 for both).

    Delays were distributed uniformly from 300 to 700 ms for J and 400 to 900 ms for R ... The average reach time of successful trials was J: 582 ms, R: 482 ms and the average reaction time was J: 346 ms, R: 311 ms. Also, the reaction time was negatively correlated

    Materials and methods, Experimental design, paragraph 1

    What the paper reports

    • reaction time against delay, J and R: p < 10⁻⁴

    Reaction times (J 332, R 305 ms) and reach times read as acquisition minus go cue (J 587, R 486 ms) are within tolerance; read as acquisition minus movement onset they are 255 and 181 ms. R's mean reaction time includes the 5.9% of its trials whose onset precedes the go cue; its median (361 ms) and its mean over 100 to 800 ms (362 ms) are outside tolerance, and with onset searched only after the go cue the means are 342 ms (J, within tolerance) and 340 ms (R, just inside its bound of 342 ms). J's delays span 300 to 700 ms as stated; R's span 400 to 1000 ms, beyond the widened bound of 990 ms. No-delay trials are 19.8% of J's and 0.2% of R's successful center-out trials (26, all on 2017-01-19), not 10%. Reaction time falls with delay in R (r = -0.264, p = 1.8e-205) but not in J (r = -0.013, p = 0.21; with onset at 5% of peak speed r = -0.034, p = 1.5e-03; with onset after the go cue r = -0.029, p = 7.9e-03). J is 5 of its 16 sessions.

    How the test differed from the paper’s method

    • Movement onset (10% of peak speed) and maximum speed are defined here; the paper does not define them. Onsets at 5% and 20%, and onset searched only after the go cue, are shown.
    • Reach time is undefined in the paper; two readings were tested and (b) matches.
  • Partially reproducedResults, Structure of neural activity during delay period, paragraph 2; Figure 2c

    After target onset the neural state in the spatial plane starts at the center, diverges after 50 ms (R 75 ms) and converges to the steady delay state after 300 ms (R 400 ms).

    at target onset, the neural states projected onto this plane start at the center of this plane, start to diverge after 50 ms (R: 75 ms), and converge to the steady neural states after J: 300 ms, R: 400ms

    Results, Structure of neural activity during delay period, paragraph 2; Figure 2c

    Divergence (D below 0.9) at 70 ms (J) and 90 ms (R) after target onset; convergence (D below 0.1) at 270 ms (J) and 350 ms (R). All four are within tolerance, so by the letter of the criterion the result is reproduced; partially_reproduced is a judgment beyond the criterion, made because R's divergence cannot be told from baseline noise. Other thresholds move divergence between 10 and 150 ms and convergence between 230 and 390 ms, so both monkeys' times depend on the threshold; J diverges at 150 ms with threshold 0.8 and converges at 390 ms with threshold 0.05, outside tolerance. J's times hold under the two added checks (pre-onset D SD 0.03; with the converged state taken from trials whose delay window starts after the bin, 70 and 250 ms). R's D lies below 0.9 in 6 of the 10 bins before target onset (SD 0.09), and with the non-overlapping converged state it diverges at 10 ms and converges at 310 ms, both outside tolerance. The pre-target state sits at x -2.1, y -30.4 mm (J) and 17.6, -13.1 mm (R) in the plane's target coordinates, so "starts at the center" is not borne out.

    How the test differed from the paper’s method

    • The thresholds, bins and smoothing are chosen here; the paper reads the times from a figure.
  • Partially reproducedResults, Structure of neural activity during delay period, paragraph 4; Figure 2e and 2f

    The speed axis of delay activity varies with maximum speed for reaches to the same target distance, for 11 of 12 distances in J and 11 of 12 in R (p < 0.05), so it encodes speed independently of distance.

    We then tested whether the neural state in this dimension varied with speed for reaches to the same target, and found that for almost all distances it did (J: 11/12 distances in the vertical and horizontal tasks, R: 11/12, p < 0.05)

    Results, Structure of neural activity during delay period, paragraph 4; Figure 2e and 2f

    R 11 of 12 distances under both readings and with cross-validated projections; the exception is the nearest distance. J 6 of 12 under both readings (4 and 5 cross-validated), on 3211 trials of its two vertical sessions; its horizontal sessions are not in the dandiset. Each distance rank pools both line tasks and both sessions of each. Split by task and session, R has 1, 4, 10 and 7 of 12 and J 1 and 3, so the counts rest on pooling sessions.

    How the test differed from the paper’s method

    • J has only vertical sessions here, against horizontal and vertical in the paper.
    • R's horizontal targets lie at 5.2 + 10.4 k mm, so distances are matched by rank (1 to 12) within each task.
  • Partially reproducedResults, Neural state variability during delay, paragraph 2; Figure 3

    The mean absolute on-axis (distance) deviation of the neural state is larger than the off-axis (direction) deviation, in each task (J p = 9e-7 ring, 4e-7 horizontal, 1e-8 vertical, 3e-14 3-rings; R 8e-3, 4e-7, 2e-7 and 0.19).

    The mean absolute on-axis deviation for each target and for each monkey was systematically greater than the mean absolute off-axis deviations in the tasks

    Results, Neural state variability during delay, paragraph 2; Figure 3

    What the paper reports

    • J ring: p = 9 × 10⁻⁷
    • J horizontal: p = 4 × 10⁻⁷
    • J vertical: p = 10⁻⁸
    • J 3-rings: p = 3 × 10⁻¹⁴
    • R ring: p = 0.008
    • R horizontal: p = 4 × 10⁻⁷
    • R vertical: p = 2 × 10⁻⁷
    • R 3-rings: p = 0.19

    On-axis exceeds off-axis with p < 0.05 in 5 of 6 task and monkey pairs under readings (a) paired over targets and (c) Welch over trials, 4 of 6 under (b). The exception is J's ring task, one session of four here (13 of 24 targets with on > off, paired p = 0.69). R's 3-rings task is not significant (paired p = 0.38), as in the paper (0.19).

    How the test differed from the paper’s method

    • J's ring task is 1 of 4 sessions.
    • The test is not named; three readings were tested.
  • Partially reproducedResults, Accuracy of endpoint prediction from delay activity, paragraph 2; Figure 4c

    Classifying the 16 directions of each ring of the 3-rings task separately, accuracy increases with ring distance; 23, 36 and 43% for J and 22, 28 and 29% for R.

    when classifying targets in 3-rings separately for each distance, accuracy increases with ring distance (23%, 36%, 43% for J; 22%, 28%, 29% for R)

    Results, Accuracy of endpoint prediction from delay activity, paragraph 2; Figure 4c

    Accuracy rises with ring distance in both monkeys. J's values are within tolerance; R's are below it (paper 22, 28, 29%). With tuned hyperparameters R gives 20.2, 22.9 and 27.8%.

    How the test differed from the paper’s method

    • SVM hyperparameters are sklearn's defaults.
  • ReproducedMaterials and methods, Neural recording and signal preprocessing, paragraph 2

    Of the 96 PMd electrodes, 74 for J and 80 for R had activity significantly correlated with the x and y location of the target (p < .05, Bonferroni over 96).

    Of the 96 electrodes, we found that 74 for J and 80 for R had significant correlation (p < .05, with Bonferroni correction where n = 96) with the task

    Materials and methods, Neural recording and signal preprocessing, paragraph 2

    72 of 96 channels (J) and 80 of 96 (R) regress on target x and y at p < 0.05/96. Per session the counts are lower (J 39 to 70, R 30 to 70). On the units the file labels PMd (0 to 95) the counts are 74 (J) and 40 (R); R's labelled array has 29 silent channels.

    How the test differed from the paper’s method

    • PMd is taken as units 96 to 191, from the data check, against the units' electrode_group labels.
    • J is 5 of its 16 sessions.
  • ReproducedResults, Structure of neural activity during delay period, paragraph 3; Figure 2d

    On single trials, on-axis and off-axis deviations of the neural state in the spatial plane are positively correlated with the on-axis and off-axis deviations of the hand position at maximum speed (on-axis p = 1e-3 J, 4e-3 R; off-axis p = 4e-4 J, 2e-4 R).

    Both on-axis ... and off-axis ... deviations in the neural state were positively correlated with on-axis and off-axis deviations in the position of max-speed for both monkeys

    Results, Structure of neural activity during delay period, paragraph 3; Figure 2d

    What the paper reports

    • on-axis, J: p = 0.001
    • on-axis, R: p = 0.004
    • off-axis, J: p = 4 × 10⁻⁴
    • off-axis, R: p = 2 × 10⁻⁴

    Over all tasks (reading a), on-axis J r = 0.091 (p = 1.7e-17), R r = 0.022 (p = 1.3e-02); off-axis J r = 0.102 (p = 1.3e-21), R r = 0.072 (p = 2.2e-16). In the 3-rings task alone R's on-axis correlation is not significant (p = 6.5e-02). R's on-axis correlation is positive with p < 0.05 in one of seven sessions and negative in one (2017-01-16, r = -0.044, p = 4.9e-02). With deviations from session by condition means the correlations are nearly unchanged (R on-axis r = 0.020, p = 2.3e-02; off-axis 0.083, p = 3.0e-21; J on 0.092, off 0.101), so offsets between sessions do not produce them.

    How the test differed from the paper’s method

    • The position of maximum speed comes from hand positions smoothed here (Gaussian, SD 20 ms).
  • ReproducedResults, Structure of neural activity during delay period, paragraph 4

    After regressing out speed from the neural activity, a positive correlation with target distance remains (p < 1e-4 for J and R).

    we found that after regressing out speed from the neural activity, there is still a positive correlation

    Results, Structure of neural activity during delay period, paragraph 4

    What the paper reports

    • p < 10⁻⁴ (n = J and R)

    Cross-validated predictions of target distance from speed-residualised activity correlate with distance, J r = 0.123 (p = 1.2e-30), R r = 0.233 (p = 7.1e-158); in-sample r = 0.182 and 0.257. Within 3-rings the cross-validated r is 0.154 (J) and 0.265 (R).

    How the test differed from the paper’s method

    • The paper does not say how the correlation was tested; a 10-fold cross-validated linear regression was chosen.
  • ReproducedResults, Neural state variability during delay, paragraph 2; Figure 3a and 3c

    The distributions of single-trial neural states for each condition are elliptical, and for most conditions their major axis points along the direction of the mean neural state from the origin.

    we found that the ellipses for most conditions were oriented in the direction of the mean neural state from the origin

    Results, Neural state variability during delay, paragraph 2; Figure 3a and 3c

    Major axis within 45 degrees of the condition mean's direction for 69 of 96 conditions (J, median 21.0 degrees, Wilcoxon p = 4.1e-08) and 99 of 132 (R, median 19.3 degrees, p = 8.0e-10). Median ratio of major to minor axis 1.35 (J) and 1.56 (R).

  • ReproducedResults, Neural state variability during delay, paragraph 3; Figure 3b and 3d

    Relative deviations of the neural state (deviation as a fraction of reach distance), on and off axis, decrease with reach distance.

    we found that the relative deviations in both distance and direction in the neural state decreased as a function of reach distance

    Results, Neural state variability during delay, paragraph 3; Figure 3b and 3d

    Spearman rho of relative deviation with distance, on axis J -0.389 and R -0.404, off axis J -0.378 and R -0.402, all p below 1e-290. Absolute deviations grow slightly with distance (rho 0.073 to 0.103), so the decrease comes mainly from dividing by distance, and the test discriminates little.

  • ReproducedResults, Accuracy of endpoint prediction from delay activity, paragraph 4; Figure 4d

    Mean distance-classification error is slightly but significantly lower than the arc-length error of direction classification, and projecting the activity onto the spatial plane before classification raises it (J 2.64 to 3.03 cm; R given as 2.78 and 3.27 cm), to the level of the direction errors.

    we found that the mean classification error for distance classification was significantly (albeit only slightly) lower than the errors for directional classification ... The resulting classification error rate (Fig 4d, blue bars) is significantly greater than classification using the full dimensional neural activity

    Results, Accuracy of endpoint prediction from delay activity, paragraph 4; Figure 4d

    What the paper reports

    • plane-projected against full-dimensional distance error, J and R: p < 0.05
    • plane-projected distance error against direction errors, each distance in J and R: p > 0.05

    Under reading (a) all four values are within tolerance and projection raises the error (Welch J p = 2.7e-07, R p = 4.3e-41; paired over trials J p = 4.9e-12, R p = 1.2e-62). R's full value is inside its bound (2.502 cm) by 0.001 cm, so the match is fragile. The full-activity distance error is lower than the arc error of every ring in both monkeys (p at most 3.1e-05). After projection it equals the arc error only for J's 4 cm ring (p = 0.46), so the claim that projection brings it to the level of the direction errors holds for 1 of 6 rings. With arc errors restricted to the 4 cm ring's range, as in the Figure 4d inset, the full error is still lower than every ring's (p at most 3.1e-05), and the projected error equals the arc error for J's 4 and 8 cm rings (p = 0.46 and 0.11) but not J's 12 cm ring (p = 5.3e-03) or any of R's (p at most 1.0e-04), 2 of 6 rings.

    How the test differed from the paper’s method

    • The paper prints R's values in an order that contradicts its text; two readings were listed and (a) matches.
    • J's distance classification uses the vertical task only.
    • The plane is fit on all trials, including the test folds of the plane-projected classifier.

Not tested in a notebook 1

1 data missing from the dataset
  • BlockedMaterials and methods, Experimental design, paragraph 3· Data missing from the dataset

    Monkey J contributed 6161 ring, 5017 horizontal, 6110 vertical and 7722 3-rings trials, over four sessions each.

    24 targets for J (6161 trials over four sessions ... For horizontal configuration: 5017 trials over four sessions for J ... For vertical: 6110 trials over four sessions ... (7722 trials over four sessions for J

    Materials and methods, Experimental design, paragraph 3

    The dandiset holds 5 of the 16 sessions the Methods list for J (2015-10-01, 10-02, 10-15, 2016-01-27, 01-28). The 11 missing include all four horizontal sessions, three of four ring sessions and two of four in each of vertical and 3-rings. The data link in the paper (stanford.edu/~shenoy/Data/Even-ChenEtAlPLoSCompBio2019.zip) returns HTTP 403, and no other dandiset of this paper was found.

Findings about the data

What attempts found about the dataset itself, for its owner and the archive.

  • Metadata errorIn every file the units' electrode_group labels are swapped. Units 0 to 95 are labelled PMd but their electrodes column points to electrode rows of group 1 (M1), and units 96 to 191 are labelled M1 but point to rows of group 2 (PMd), the rows the PMd LFP uses. The data agree with the electrodes column; in R the array labelled PMd has 29 silent channels and carries almost no target information (at most 6 significant channels per session in the delay window and decoding at most 0.03 above chance), though 40 of its 96 channels regress on target position when R's sessions are pooled; this is as expected of the M1 array the paper calls unusable. The conversion code assigns electrode_group by the order of a dictionary of groups.
  • File format errorIn every J file the boundary between units 0 and 1 in units/spike_times_index is misplaced. Unit 0 stops 1323 to 3120 s into sessions of 3083 to 8106 s, and unit 1 holds two to five times the median number of spikes, with spike times that run to the session's end and restart near 0.
  • Metadata errorHand, cursor, eye and decoded positions are stated in meters with conversion NaN; the values are in mm (targets at up to 120, hand reaching them at about the same values).
  • Description missingtask_id is 2 in every trial and version_id is one value per monkey, so the files do not say which target configuration a session used; it must be inferred from target_pos, which has no unit. Electrode positions are NaN and impedances -1.
  • Metadata errorThe trials column is_successful is described as "if monkey started before the mandatory delay period after target shown"; it holds the rig's trial success flag.
  • Metadata errorThe dandiset description says the files hold sorted spikes; the units are threshold crossings of each electrode (one unit per electrode, built from the rig's spike rasters).
  • Metadata errorThe archive's assetsSummary for version 0.220124.2156 gives 15 files and 187.5 GB; the version holds 12 files and 39.5 GB.
  • Data missingThe dandiset holds 5 of the 16 sessions the paper uses for monkey J. All four horizontal sessions, three of four ring sessions and two of four in each other configuration are missing; the paper's own data link returns HTTP 403.
  • OtherThe files place R's horizontal targets on 2017-01-15 and 16 and vertical targets on 2017-01-17 and 18; the Methods give these dates the other way round. Grouped by the files' geometry, R's trial counts match the Methods' counts for horizontal and vertical better (within 4% and 3%) than grouped by date (12% and 5%).
  • OtherR's horizontal targets lie at 5.2 + 10.4 k mm from the center, not 1 cm apart as stated, and R's 2017-01-17 session holds 69 successful trials to six ring targets outside its layout.
  • OtherR's delays run from 400 to 1000 ms, against 400 to 900 ms in the Methods, and R has only 26 trials without a delay (all on 2017-01-19), against 10% in the Methods; J has 19.8%.
  • Selection not documentedR's 3-rings session holds 1340 successful delayed center-out trials where the paper used 912; neither the paper nor the dandiset says how the trials were selected.
  • OtherIn R's 2017-01-19 the cursor differs from the hand by more than 1 m in over 1% of samples; in the other sessions they agree to within 3 mm in 99% of samples.
  • Paper inconsistentThe Results give J's distance errors as "3.03 cm v.s. 2.64 cm" (projected, then full) and R's as "2.78 cm v.s. 3.27 cm", while stating that projection raises the error in both; the R values are printed in the reverse order or contradict the text.
  • Methods not availableThe paper names no analysis code, does not report its SVM hyperparameters or the nulls of its p-values, and its data link returns HTTP 403.

Notebooks

Every registered entry that uses this dataset. An entry can hold several notebooks, and can use more than one dataset.

Reproduction Structure and variability of delay activity in premotor cortex (Even-Chen et al. 2019)

Tests the findings of Even-Chen, Sheffer, Vyas, Ryu and Shenoy (2019) on dandiset 000121, which holds 12 sessions of delayed center-out reaching with Utah arrays in two monkeys (all 7 of R, 5 of the 16 of J). It checks the sessions and trial counts, the behaviour, the low-dimensional structure of PMd delay activity (PCA, the spatial plane, its time course, the speed axis), the variability of single-trial neural states on and off the reach axis, and SVM decoding of reach direction and distance. All files

Verification

  • Round 1
    2026-10-16    Agreed with 19 of 20 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: The executed notebook of the first run (commit d455c0d) with the conclusion cells and the summary removed, the helper module and the claims file.
    • pca-variance: the verifier’s outcome was “Not reproduced”. All 6 J values and R PCs 2-6 are within tolerance, but R PC1 is 72.8 against a lower bound of 72.9. The criterion's partial case requires all R values within, so by its letter this is not reproduced. It is a 0.14-point miss, and the result depends on the day-mean choice (the trial-window variant gives 80.5). The day mean is n_spikes / recorded_s, and recorded_s is the total in-trial hand time, while unit spans run to about 8300 s against 6520 s recorded; whether spikes exist between trials is never checked.
  • Round 2
    2026-10-16    Agreed with 20 of 20 outcomes. Independent agent run by CatalystNeuro (same harness, separate context, did not write the notebook). Reviewed: The full executed notebook of the revised run (commit 57c6966) with its conclusions and summary, the helper module, the claims file and this attempt record.

Method differences

How the tests differed from the paper’s method or data, and the claims each applies to.

Provenance

Attempt record
000121-even-chen-2019-01
Paper
Structure and variability of delay activity in premotor cortex
Date
2026-10-16
Made by
An agent, run by CatalystNeuro
Dataset
DANDI:000121
Version
0.220124.2156 (doi:10.48324/dandi.000121/0.220124.2156)
Run on
GitHub Actions
Harness
Claude Code (GitHub Actions)
Model
claude-opus-5-5
Protocol
pilot-7

Transcripts of the agent runs

Each run’s prompt, messages and tool calls, with long output cut and the text of the paper left out, and its final report.

← DANDI:000114DANDI:000127 →