Dandiset 000727

Reproducing Brezovec et al. (2022) from dandiset 000727 14

Notebook reproduce_brezovec_2022.ipynb

• ✕Not reproducedResults, Demixing the unique contribution; Figure 5F

  When the fly turns, neural activity increases in specific regions of the ipsilateral hemisphere and decreases in the same regions of the contralateral hemisphere.

  Turning raises activity mostly on one side, opposite for the two directions (82% of the top 10% of blocks in half A for left turns, 25% for right turns), but the mirror blocks rise as well; their median change against slow walking is +0.0064 for left turns and +0.0022 for right turns, and 36% and 42% of them fall. The same holds with turning thresholds of 50 and 150 deg/s, and in 8 of 9 flies for left turns and 9 of 9 for right turns. With the change predicted by the demixing model of fig5a-unique-variance, as the paper derives Figure 5F, the mirror blocks rise as well (medians 0.0254 and 0.0111; 6.5% and 18.8% fall), as do the model's VL and VR coefficients of the mirror blocks (median 0.0064). The ipsilateral increase holds; the contralateral decrease does not.

  How the test differed from the paper’s method

  • The bounds for slow walking (1 to 3 mm/s, below 30 deg/s) and turning (beyond 100 deg/s) are choices; the paper does not state its own.
  • Neural activity and behaviour at the same volume time, without a lag.
  • The main test compares mean activity in turns and slow walking; the paper uses the demixing model, shown as a variant.
• ◐Partially reproducedResults, Neural encoding of velocity space is widespread; Figure 2C; Figure S2

  About 39% of the brain volume correlates with at least one of forward velocity, left and right angular velocity.

  Reading (a), single voxels at p < 1e-4, gives 31.1% of the 191,119 in-brain voxels of the 12 planes, below 35.1 to 42.9% and above 19.5%. Reading (b), 6 x 6 blocks at p < 1e-8, gives 53.5% of the same in-brain voxels, above the band. With each fly's behaviour rolled in time, 0.0% pass under either reading. With behaviour sampled at the start of each volume in place of its midpoint, reading (a) gives 39.2%, so the value depends on the neural times, which the dandiset does not hold. The fraction is 0.0 to 7.5% in the outermost planes and 21.0 to 39.1% in the others. Within single flies 2.4 to 19.7% of voxels pass the threshold for 3384 points. Almost no voxel is negatively correlated (at most 0.01% per component).

  How the test differed from the paper’s method

  • Processed volumes in the Functional Drosophila Atlas, not the paper's mean brain; 12 of 91 planes.
  • Behaviour sampled at the midpoint of each volume, as the dandiset has no per-plane times; the paper used each plane's acquisition time.
  • Blocks of 6 x 6 voxels in place of the paper's Ward supervoxels for reading (b).
  • In-brain voxels are those inside the brain in all nine flies; with voxels inside in any fly the fraction is 20.3%.
• ◐Partially reproducedResults, Neural encoding of velocity space is widespread; Figure 2C

  Of the brain volume, about 15% correlates only with forward velocity, 20% with forward velocity and turning, and 4% only with turning; only about 10% of correlated voxels correlate with all three components, so 90% show a tuning preference.

  Under reading (a), VF only 11.9% (expected 15), VF and turning 15.9% (20), turning only 3.2% (4), and 11.5% of correlated voxels correlated with all three (10). None is within its tolerance; the ranking holds and fewer than 20% of correlated voxels correlate with all three, so most show a tuning preference, as the statement says. Under reading (b) VF and turning is 39.4% and all three 32.2%. With behaviour at the start of each volume, reading (a) gives 11.6, 24.1, 3.5 and 25.7%.

  How the test differed from the paper’s method

  • As for fig2c-fraction-correlated.
• ◐Partially reproducedResults, Neural activity in the central brain is highly predictive; Figure 4F, G

  Linear models on the principal components predict forward, left and right velocity on held-out data with average R2 of about 0.3 using 100 PCs, performance improving over the first 100 PCs before declining.

  With 100 superfly PCs the median R2 over flies is 0.155 (VF), 0.196 (VL) and 0.212 (VR); the means are 0.038, 0.155 and 0.178. None is within 0.27 to 0.33; all medians are at least 0.15. Mean R2 rises to 100 PCs and declines after for each component, as stated. Three flies have negative VF R2 (-0.307, -0.306, -0.503). Variants; interleaved folds give medians 0.318, 0.252 and 0.271; alpha chosen within the training folds 0.134, 0.180, 0.212; each fly's own PCs 0.151, 0.210, 0.217. With behaviour rolled in time R2 is at most -0.002. With interleaved folds two of the three medians fall in the band (VF 0.318, VR 0.271; VL 0.252), so the folds, which the paper does not state, account for much of the gap; the outcome would stay partial.

  How the test differed from the paper’s method

  • Five contiguous folds within each fly, a choice; the code does not show its folds.
  • Alpha is chosen by the best held-out score, as the code does, which selects on the test folds.
  • The 1000 and 2000 PC models were not run (at most 500 PCs).
  • PCs of 5363 blocks from 12 planes.
• ◐Partially reproducedResults, Neural activity in the central brain is highly predictive; Figure 4H

  Flies that moved less along a velocity component were poorly predicted (R2 near 0), while flies that explored it more were well predicted, with R2 up to 0.6.

  The Spearman correlation of a fly's mean amount of a component with its R2 is positive for all three components (VF 0.017, VL 0.383, VR 0.683, p = 0.042 for VR only); the pooled within-component rank correlation is 0.361 with p = 0.064. The largest R2 is 0.534 (fly_094, VF), just below 0.54 to 0.66, and the smallest -0.503. Partially reproduced under the clause that the correlation is positive for at least two components.

  How the test differed from the paper’s method

  • Rests on the R2 of fig4fg-ridge-r2, with its contiguous folds.
• ◐Partially reproducedResults, Demixing the unique contribution; Figure 5A

  Demixing with linear models shows that forward velocity is uniquely encoded by a spatially organized subset of voxels, while locomotor state (walking or stopped) is not uniquely encoded anywhere, although the single-variable maps of forward velocity and locomotor state are very similar.

  At the 0.01 threshold, 5 blocks (0.09%) encode walking state uniquely and 40 (0.75%) encode VF uniquely, 8 times as many, but fewer than the 5% the criterion requires. Unique contributions are small everywhere (largest 0.032 for VF, 0.012 for walking state) and the full model's median held-out R2 is 0.0019. The single-variable maps of VF and walking state correlate at 0.882 over blocks, as the statement says. At a threshold of 0.005 walking state is unique in 1.88% of blocks and VF in 2.24%; with the walking state the code intended, in 0.97% and 0.91%, so the VF advantage depends on the 0.01 threshold, which lies far above the median full-model R2, and on the definition of walking state. Within single flies walking state is unique in up to 8.35% of blocks (fly_094).

  How the test differed from the paper’s method

  • Ridge alpha 1 on behaviour scaled to unit s.d., folds as fifths of every fly, and the 0.01 threshold are choices; the Methods give none of them.
  • Walking state as the code computes it, which uses forward velocity alone.
  • Blocks in place of supervoxels.
• ✓ReproducedResults, first section; Figure 1A legend; STAR Methods, Two-Photon Imaging and Behavior Tracking

  Nine flies were imaged for 30 minutes each at a volume rate of about 1.8 Hz, with 256 x 128 x 49 voxels of 2.6 x 2.6 x 5 um (about 1.6 million voxels per volume), giving 30,456 volumes in all, while the ball was tracked at 50 Hz.

  Nine flies, one session each, 3384 volumes per fly (30,456 in all) at 1.8797 Hz, 1,605,632 voxels of 2.61 x 2.61 x 5 um per raw volume, 30.0 minutes per session, FicTrac 90,000 frames at 50 Hz. The subject ids match the nine flies of the paper's code.

• ✓ReproducedResults, Neural encoding of velocity space is widespread; Figure 2D, E

  Forward velocity correlations are mirror-symmetric across the midline, while left and right turning correlations are anti-symmetric, with more correlated voxels in the hemisphere ipsilateral to the turn.

  The VF map correlates with its mirror image at r = 0.823. The VL map correlates with the mirrored VR map at 0.672 and with its own mirror at 0.449. Voxels significant for VL are 2.38 times as many in image half A (x < 157) as in half B, and voxels significant for VR 3.2 times as many in half B. Per fly, the VF mirror correlation is 0.166 to 0.580, summed VL correlation is larger in half A in 8 of 9 flies, summed VR correlation larger in half B in 7 of 9, and both in 6 of 9; fly_098 and fly_105 have VR on the same side as VL and fly_099 has VL on the other side. Against the pooled maps of the other eight flies, no fly matches better mirrored for both VL and VR, as a left-right flip would give; 7 of 9 match better unflipped for both. fly_099's VL map matches the mirrored map slightly better (0.158 against 0.190), its VR map clearly does not (0.439 against 0.199). For fly_105 the check does not decide (VL 0.274 against 0.234, VR 0.212 against 0.214), so a flip of that fly is not excluded; it would lower the pooled asymmetry. The ipsilateral part of the statement is not tested; it holds if half A is the fly's left, which the dandiset does not state.

  How the test differed from the paper’s method

  • Mirror about the centre of the FDA's left-right axis (156.5), chosen from the symmetry of the brain mask; the 0.7 threshold for symmetry is a choice of this attempt, made after the prototype run had shown r = 0.82.
• ✓ReproducedResults, Neural activity in the central brain is highly predictive; Figure 4B-D

  The first principal components of brain-wide activity are structured by behaviour; PC1 is low when the fly is stopped and high otherwise, PC2 increases with forward velocity and not with angular velocity, and PC3 carries the sign of angular velocity.

  All three parts hold under both readings of the threshold. Reading (a); PC1's below-threshold correlation with VF is 0.202 against 0.065 (PC2) and -0.045 (PC3); above the threshold PC2 has the largest correlation with VF (0.272, PC1 0.162, PC3 0.086) and 0.091 and 0.087 with VL and VR; PC3's above-threshold correlations are 0.114 with VL and -0.259 with VR (p = 3.5e-14 and 5.6e-72). Within each fly over all time points, PC1 correlates positively with VF (0.22 to 0.57) and PC3 more with VL than VR, in 9 of 9 flies. PC1 explains 2.22% of block variance.

  How the test differed from the paper’s method

  • PCA of 5363 blocks from 12 planes, not of 98,000 supervoxels from 49 planes.
• ✓ReproducedResults, Neural activity in the central brain is highly predictive; Figure 4I

  The model weights given to each PC are stereotyped across flies; PC1 is used for all components, PC2 mainly for forward velocity, and PC3 for left and right turning with opposite signs.

  Median correlation of the weights on PCs 1 to 20 between pairs of flies is 0.646 (VF), 0.732 (VL) and 0.761 (VR). PC3's weight is positive for VL and negative for VR in 9 of 9 flies, PC2's weight is largest for VF in 7 of 9, and PC1's weights have one sign for all three components in 9 of 9. With each fly's behaviour rolled in time (20 rolls), the median pair correlation is at most 0.151, PC3 has opposite signs in 3 to 8 of 9 flies and PC1 one sign in 0 to 6 of 9. Part (iv) follows in part from PC1's orientation, signed to correlate positively with VF, with which it also correlates positively with VL and VR (0.222 and 0.188).

  How the test differed from the paper’s method

  • Weights from a ridge fit to all of each fly's time points at the alpha chosen in fig4fg-ridge-r2, with each component scaled to unit s.d.
• ✓ReproducedResults, Demixing the unique contribution; Figure 5B, C; Figure S5Bverifier disagreed at first

  The functional regions defined by the unique contributions of forward, left and right velocity are all correlated with velocity when the fly is below the movement threshold, a widespread increase in activity at the initiation of movement, while above the threshold each region correlates most with its own velocity component.

  Under reading (b) of the threshold all nine below-threshold correlations of the forward, left and right regions (40, 4 and 8 blocks) with the three components are positive with p < 0.05 (0.013 to 0.221), and above the threshold each region correlates most with its own component (0.377, 0.170, 0.266). Under reading (a) the left region with VR is -0.008 (p = 0.2), so only reading (b) satisfies the criterion. The Pearson p-values treat time points as independent; against 200 rolls of each fly's behaviour in time, which keep the autocorrelation, all nine are above chance under reading (b), the weakest (left region with VR, r = 0.013) with p = 0.025 against a null 95th percentile of 0.010. The left region holds only 4 blocks.

  How the test differed from the paper’s method

  • Regions are the few blocks with a unique contribution of at least 0.01 in fig5a-unique-variance.
  • The threshold readings are two interpretations of the 0.75 s.d. of Figure 4E.
• ✓ReproducedResults, The temporal relationship between neural activity and behavior; Figure 6A-E

  The time of peak cross-correlation between supervoxel activity and forward velocity ranges from about 300 ms before the change in behaviour, the earliest, to more than 1 s after it.

  Reading (b); the 99th percentile of VF lead over the 2145 top blocks is 0.32 s, within 0.27 to 0.33 s, and the 1st percentile a lag of 1.23 s. Reading (a) gives an earliest lead of 0.44 s (outside tolerance) and a latest lag of 1.44 s. VL and VR give 99th percentiles of 0.30 and 0.22 s and 1st percentiles of lags of 1.24 and 1.22 s. With volume-start times the VF 99th percentile is 0.58 s. The median VF lead changes steadily with FDA plane from 0.36 s (plane 17) to -0.30 s (plane 80), a spread of 0.66 s against a volume period of 0.532 s, the pattern unknown plane acquisition times would produce; the dandiset gives neither the times nor the order of the planes. With each plane's median removed the 99th percentile is 0.50 s and the 1st -1.02 s, so the match within 0.03 s is not robust to the neural times; the lag of more than 1 s holds in both (1st percentile at least 0.84 s of lag in every plane from 38 to 80, more than 1 s in planes 59 to 73). Only the lag of more than 1 s is robust; the earliest lead cannot be tested with the timing the dandiset gives, and the outcome follows the criterion fixed in advance. Fitted per fly, the 99th percentile is 0.36 to 4.55 s.

  How the test differed from the paper’s method

  • Neural times at the volume midpoint, an error of up to 0.27 s per block.
  • Notch filter Q = 2 is a choice; the code's notch coefficients are not in the repository.
  • Top 40% by peak value of the deconvolved filter.
• ✓ReproducedResults, The temporal relationship between neural activity and behavior; Figure 6H

  Supervoxels whose peak correlation lags behaviour have broader temporal kernels, while those that anticipate behaviour have narrow kernels.

  Over the 2145 top VF blocks, lag and FWHM correlate at r = 0.672 (p = 2.2e-281, Spearman 0.703); mean FWHM is 1.38 s for blocks leading by up to 0.5 s and 2.62 s for blocks lagging by more than 1 s. VL and VR give r = 0.442 and 0.449; volume-start times 0.673. The plane means of lag and FWHM correlate at 0.979 over 10 planes, so plane accounts for part of the relation; with plane means removed r = 0.623 (p = 6.3e-231), and within single planes r is positive in all 10 planes with at least 10 blocks (0.222 to 0.751), with p < 0.05 in 8.

  How the test differed from the paper’s method

  • As for fig6-peak-time-range; FWHM measured from 0, around the peak.
• ✓ReproducedResults, The temporal relationship, paragraph on angular velocity; Figure 6F, G

  Across nearly all brain regions, the peak correlation with angular velocity comes first in the hemisphere ipsilateral to the turn, then in the contralateral one.

  For VL, over 868 mirror pairs, the block in half A leads its partner by a median 0.16 s and is earlier in 67% of pairs; for VR, over 849 pairs, the block in half B leads by 0.14 s and is earlier in 72%; both Wilcoxon p-values print as 0. The median difference is 0.26 to 0.33 s in planes 17 to 38 for VL and 0.19 to 0.29 s in planes 17 to 59 for VR, and -0.03 to 0.03 s in planes 73 and 80. Both blocks of a pair lie in one FDA plane; whether the two halves of a plane were acquired at the same time is not known.

  How the test differed from the paper’s method

  • The ipsilateral half is the image half with more significant correlations for that turn, from fig2-symmetry, since the dandiset does not say which half is left.