DANDI:000127

Reproducing Chowdhury, Glaser & Miller (2020) from dandiset 000127 6

Notebook reproduce_chowdhury_2020.ipynb

• ◐Partially reproducedFigure 10

  Unlike in the two-workspace experiment, many neurons had no consistent relationship with arm kinematics across active and passive movements, shown by negative pseudo-R2 for the model trained on both conditions.

  Surprisingly however, unlike our results from the two-workspace experiment (see Figure 5), many neurons in the active/passive task did not have this consistent relationship, indicated by the many neurons with negative pR2 values for the model trained over both conditions.

  Figure 10

  19 of 50 neurons scored on active trials and 23 of 50 on passive trials have negative pseudo-R2 for the model trained on both conditions, but about as many (17 and 25) are negative when trained within one condition. Only 5 and 3 neurons are positive within a condition and negative across both. Training on both costs a little on active trials and nothing on passive trials.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
  • The whole-arm model is hand plus joint kinematics, as for fig9-whole-arm-fits-better.
  • Pseudo-R2 for one trial type is computed on a repeat's pooled held-out predictions, against that type's own mean firing; the paper does not say which mean it used.
• ◐Partially reproducedFigure 11B

  The consistency of the whole-arm model, its pseudo-R2 when trained on both conditions, correlated negatively with a neuron's separability index.

  Interestingly, we found the opposite result—model consistency actually correlated negatively with the separability index.

  Figure 11B

  Scored on active trials, model consistency correlates negatively with separability (Spearman -0.38, 95% interval -0.60 to -0.11). Scored on passive trials the correlation is -0.13 and on all trials together 0.14; neither can be told apart from zero.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
  • The whole-arm model is hand plus joint kinematics, as for fig9-whole-arm-fits-better.
• ✓ReproducedFigure 8Bverifier disagreed at first

  In the 120 ms after movement onset, average hand speed was similar in active reaches and passive bumps.

  However, average movement speed was similar between active and passive trials in this 120 ms window, which we used for data analysis.

  Figure 8B

  The speed traces of active and passive trials overlap in the 120 ms window and separate afterwards. Passive movements are 13% faster on average (0.207 against 0.183 m/s), half a standard deviation, which is statistically clear; the paper's "similar" is qualitative.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
• ✓ReproducedResults, Area 2 represents passive movements differently from active reaches, paragraph 3

  A linear discriminant classifier predicted whether a movement was active or passive 89% of the time from whole-arm kinematics in the analysis window.

  Averaged over the sessions, a linear discriminant analysis (LDA) classifier could predict the movement type 89% of the time, using only the whole-arm kinematics in the analysis window

  Results, Area 2 represents passive movements differently from active reaches, paragraph 3

  A linear discriminant tells active from passive trials 90% of the time from joint angles and velocities, against 58% from hand kinematics and 53% by chance. The paper reports 89% averaged over four sessions.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
  • Joint angles and velocities stand in for the hand and elbow marker kinematics, which are not in the dandiset.
• ✓ReproducedFigure 8E

  Many neurons had an above-chance separability index, the accuracy of a linear discriminant predicting the trial type from that neuron's firing.

  we used LDA to classify movement type based on individual neurons, calling this prediction rate the neuron's 'separability index' (Figure 8E). We found that many neurons had an above chance separability index

  Figure 8E

  20 of 50 neurons have a separability index above chance at p < 0.05, with a median of 0.56 and a highest value of 0.80.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
  • Neurons firing below 1 Hz in the window (15 of 65) are left out; the paper states no criterion.
• ✓ReproducedFigure 9

  In the active/passive experiment the whole-arm model again tended to out-perform the hand-only model, but for many more neurons than in the two-workspace experiment the difference was not significant.

  the whole-arm model again tended to out-perform the hand-only model (Filled circles above the dashed unity line in Figure 9). However, there were many more neurons (open circles) for which the difference between models was insignificant...

  Figure 9

  The whole-arm stand-in is significantly better than the hand-only model for 15 of 50 neurons and worse for none; 35 (70%) show no significant difference, against 17% in the paper's two-workspace experiment. That comparison uses the paper's number.

  How the test differed from the paper’s method

  • One session from one monkey; the paper has four sessions from two.
  • The whole-arm model is hand plus joint kinematics; the marker positions the paper's model uses are not in the dandiset.
  • The hand-only model is two-dimensional, the fits carry a small ridge penalty, neurons below 1 Hz are left out, and no Bonferroni correction is applied.