DANDI:001170

Reproducing Ling et al. (2023) from dandiset 001170 17

Notebook reproduce_ling_2023.ipynb

• ✕Not reproducedResults, OFF responses form anticorrelated odor afterimages across species, paragraph 6; Results, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 2D, 2F and 4C

  Time-averaged glomerular ON and OFF responses to the same odor are anticorrelated; glomeruli active during the odor tend to be suppressed after it and the reverse.

  glomeruli responding to odors during odor presentation tended to be suppressed following odor offset, and that glomeruli responding most strongly to odor offset were suppressed during the odor presentation ... odor ON and OFF responses were not just distinct, but were anticorrelated.

  Results, OFF responses form anticorrelated odor afterimages across species, paragraph 6; Results, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 2D, 2F and 4C

  The time-averaged ON and OFF vectors are positively correlated in all 28 session-odor pairs (median r 0.481, Wilcoxon p = 7.5e-09), for every odor (0.391 to 0.535) and every mouse (0.245 to 0.596). The median subtraction does not change r, by construction. With F z-scored on the 2 s before onset the median r is 0.526 and with frame-counted windows 0.446 (0 of 28 below 0 in both). With per-trial z-scored traces the median r is 0.068 (12 of 28 below 0, p = 0.33); on synthetic glomeruli with excitation only, that normalisation gives r = -0.90 between ON and 10 to 15 s after offset, against 0.30 with the pre-onset z-score. The angles of fig2h exceed 90 degrees only after centring, which makes the mean ON and OFF vectors opposite by construction.

  How the test differed from the paper’s method

  • dF/F with F0 from the 2 s before onset, not the paper's (undocumented) normalisation.
• ✕Not reproducedResults, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 4D

  Hierarchical clustering of glomerular responses cleanly separates ON responses from OFF responses.

  Supporting this, hierarchical clustering of mouse odor responses cleanly separated ON and OFF responses (Fig. 4D)

  Results, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 4D

  In 0 of 7 sessions, the 198-glomerulus session included, does the two-cluster cut put the four ON vectors apart from the four OFF vectors, with dF/F or with z-scored traces.

  How the test differed from the paper’s method

  • The paper shows one representative session without naming it; the 198-glomerulus session is taken as it, and all seven are reported.
• ✕Not reproducedResults, History-dependent suppression enhanced between sequentially presented odors, paragraph 3; Figure 6D and 6E (left)

  Methyl salicylate presented 1 s after another odor evokes a response less correlated with that odor's solitary response than the solitary methyl salicylate response is (history dependent contrast enhancement; one-tailed t-test p < 0.01, 3 sessions from 2 mice).

  Correlation values between solitary presentations of msc and other odors (left boxes in D) compared to correlation values between sequential msc and odors across imaging sessions (right boxes in D, n = 3 imaging sessions from 2 mice; one-tailed t test, p < 0.01)

  Results, History-dependent suppression enhanced between sequentially presented odors, paragraph 3; Figure 6D and 6E (left)

  What the paper reports

  • one-tailed t test, p < 0.01 (n = 3 imaging sessions from 2 mice)

  Over 21 session-odor pairs the solitary r (0.229) and the sequential r (0.219) do not differ (difference 0.009, positive in 10 of 21, one-tailed paired p = 0.39); per session p = 0.40, per first odor p = 0.33 to 0.51. On the three sessions the published version's Table 1 names (sub-148 ses-2, sub-148 ses-3, sub-149 ses-1) the solitary r is 0.133 and the sequential r 0.064 (p = 0.082). With F z-scored on the 2 s before the pair's onset there is no difference (0.251 against 0.258, p = 0.58). With per-trial z-scored traces the solitary r is 0.289 and the sequential r 0.118 (p = 5.6e-07; 0.0025 on the three sessions), as the claim states, but that normalisation can produce negative correlations with later activity by itself (fig2f).

  How the test differed from the paper’s method

  • All seven block sessions are used, as the preprint does not identify its three sessions from two mice; the three the published version names are shown as a variant.
  • dF/F with F0 from the 2 s before the pair's first onset.
• ✕Not reproducedResults, Odor OFF responses persist long after odor offset, paragraph 1; Figure 7B, 7D and S7C

  After odor offset the glomerular population activity stays negatively correlated with the odor-evoked ON response for several tens of seconds, although overall activity returns to baseline quickly.

  neuronal activity in the AL and OB maintained negative correlations with the odor-evoked ON responses for several tens of seconds following stimulus termination ... although the overall neural activity after stimulus termination returns rapidly to pre-stimulus levels, network activity remains negatively correlated

  Results, Odor OFF responses persist long after odor offset, paragraph 1; Figure 7B, 7D and S7C

  The median r with the ON vector over the 28 pairs is 0.481 at 0 to 5 s after offset, 0.312 to 0.328 from 5 to 20 s and 0.094 at 45 to 50 s, above 0 in every bin (p <= 0.014), while the mean dF/F falls from 0.071 to 0.005. With F0 from before each block r is positive in every bin (0.304 to 0.600); for the first block of sub-134 and sub-141 that baseline is only 14 to 24 s. With F z-scored on the 2 s before onset r is positive in every bin (0.152 to 0.527), and with frame-counted windows too (0.152 to 0.446). With per-trial z-scored traces r is negative from 5 s on (-0.344 to -0.480, 26 to 28 of 28), as the claim states, but on synthetic glomeruli with excitation only that normalisation gives negative values of the same sign and larger size (-0.90 and -0.93). Mice 148 and 149 keep r near 0.2 to 0.4 for the whole 50 s.

  How the test differed from the paper’s method

  • Correlation of the ON vector with 5-s bins, a summary of the paper's frame-by-frame matrix.
  • dF/F with F0 from the 2 s before onset.
• ◐Partially reproducedResults, OFF responses form anticorrelated odor afterimages across species, paragraph 6; Figure 2G and 2H

  In the mouse OB the angle between glomerular ON and OFF response vectors is greater than 90 degrees for each odor (mean 121.8 +/- 2.98 SEM, 4 odors, 7 sessions), so ON and OFF responses are inverted relative to each other.

  the angle between ON and OFF trajectories was greater than 90° for each odor presented (mean = 121.8° +/- 2.98 SEM, n = 4 odors, 7 sessions) (Fig. 2G, H), revealing that, as in the locust, odor ON and OFF responses in mouse OB glomeruli are inverted relative to each other

  Results, OFF responses form anticorrelated odor afterimages across species, paragraph 6; Figure 2G and 2H

  With the code's procedure (median subtraction, vectors centred over the 160 frames) the median ON-OFF angle exceeds 90 degrees in all 28 session-odor pairs (101.7 to 153.9) and in each odor's mean (111.0 to 140.0), but the mean is 127.9 (SEM 3.2), outside 121.8 +/- 2.98. After centring the mean ON vector is exactly opposite the mean OFF vector (cosine -1 in every case), and without centring the mean angle is 71.8 with 2 of 28 above 90. The same centred procedure gives 101.1 between the two halves of the ON window (28 of 28 above 90), 101.6 between the halves of OFF and 129.7 between the 2 s before onset and ON, and on synthetic frames around patterns 2.4 to 13.0 degrees apart it gives 91.6 to 115.6; so the angles above 90 follow the criterion but do not show that ON and OFF are inverted. Without the median subtraction the mean is 124.2 and with per-trial z-scored traces 119.5, both within tolerance; with F z-scored on the 2 s before onset 125.8 and with frame-counted windows 126.1. By mouse the means are 136.3 (134), 136.3 (141), 116.0 (148) and 117.9 (149).

  How the test differed from the paper’s method

  • dF/F with F0 from the 2 s before onset; the paper's deposited arrays are close to per-trial z-scores. The z-scored result is shown as a variant.
  • Windows placed in seconds on a 16 Hz grid; the code counts frames at a nominal 16 per second.
• ◐Partially reproducedResults, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 4B and S4B

  In the mouse OB the ON response to a reference odor is closer in angle to the ON responses of the other odors than to any OFF response, including its own, and this holds for every odor of the panel.

  As in locusts, angular separations between a reference odor and all other odor ON responses were smaller than angular separations between the reference ON response and all odor OFF responses (Fig. 4B). This result was true for all odors in the panel (Fig. S4B).

  Results, ON and OFF responses occupy distinct regions of feature space, paragraph 3; Figure 4B and S4B

  Reading (a) holds for acetophenone only (ON-ON 105.7, ON-OFF 127.5 degrees, paired p = 0.018). Methyl salicylate and eugenol show no difference (p = 0.76 and 0.83), and allyl sulfide goes the other way (ON-ON 112.1, ON-OFF 59.6, p = 0.00035). Reading (b) holds in none of the seven sessions for any reference odor. Full-dimensional vectors give the same pattern. The session-mean ON-ON angles are 85.6 to 123.1 degrees, against a median of 44.7 for the acetophenone reference in the published version (Figure 2G), so the paper's procedure probably differs from the one used here.

  How the test differed from the paper’s method

  • The mouse code for this figure is not in the Figshare record (it loads precomputed angles); the locust code's procedure (top three PCs, mean vectors) was used.
• ◐Partially reproducedResults, Odor OFF responses persist long after odor offset, paragraph 2; Figure 7G and 7H; Methods, Classification analysis of average ITI periods

  Glomerular activity averaged over the 50 s after each odor offset classifies the preceding odor with high accuracy (k-NN, k = 20).

  We observed a similar clustering of 55-second ITI activity from the mouse OB (Fig. 7G), which contributed to highly accurate classification of odors from ITI glomerular responses (Fig. 7H).

  Results, Odor OFF responses persist long after odor offset, paragraph 2; Figure 7G and 7H; Methods, Classification analysis of average ITI periods

  In sub-148 ses-3 (318 glomeruli) accuracy is 45%, above the null's 99th percentile (37.5%, null mean 16.7%) but below 50%; by odor 0.9, 0.0, 0.3 and 0.6. The code's choice (cosine on the top three PCs) gives 12.5%. Across the seven sessions accuracy is 22.5% to 47.5%, above the null's 99th percentile in three. The null mean of this leave-one-out vote (16.7% to 17.4%) is below the nominal 25%, as with k = 20 only 9 of the 39 other trials share the own odor; the paper gives no chance value for this panel. With k at or below the class size accuracy is 90.0% (k = 5) and 82.5% (k = 9) in sub-148 ses-3 and 80.0% to 100% in every session (null 99th percentiles 42.5% to 45.0%). All trials of one odor lie in one block, so neither k separates the odor from slow differences between blocks (fig7j).

  How the test differed from the paper’s method

  • k = 20 from the legend; the Methods' k = 50 exceeds the 39 other vectors.
  • dF/F with F0 the session median of each ROI.
• ◐Partially reproducedResults, Odor OFF responses persist long after odor offset, paragraph 4; Figure S8D and S8E

  When eleven odors are presented in random order (1 s pulses, 19 s apart), odor information after offset is variable across odors and on average declines to chance after several seconds.

  Classifier accuracy across the ITI was variable across odors (Fig. S8D), but, on average, declined to chance after several seconds (Fig. S8E).

  Results, Odor OFF responses persist long after odor offset, paragraph 4; Figure S8D and S8E

  In all five random-order sessions accuracy in the first 2 s after offset is above the null (24.4% to 56.8% against 99th percentiles of 10.6% to 12.3%, chance 9.1%), with large differences between odors. In the last 5 s it is 9.6% to 15.3%, still above the null's 95th percentile in four of five sessions (not in sub-134 ses-6). The published version's Table 1 lists random-order sessions with 175, 340, 323, 306, 286, 204, 250 and 280 glomeruli; only sub-126 ses-1 (323) and sub-134 ses-6 (306) have one of these counts. In those two early accuracy is 24.4% and 29.0%, and late accuracy is above the null's 95th percentile in sub-126 ses-1 (9.6% against 9.0%) and not in sub-134 ses-6 (9.9% against 10.0%).

  How the test differed from the paper’s method

  • The paper's random-order session (286 glomeruli, mouse 129 in the Figshare code) is not in the dandiset; five sessions with the same paradigm, four of them from mice not in the block sessions, were used.
  • In three of these sessions odors have different concentration values; the first 10 presentations of each odor were used regardless.
• ◐Partially reproducedResults, OFF responses form anticorrelated odor afterimages across species, paragraph 4; Figure 1F and 1G, S1data in Figshare doi:10.6084/m9.figshare.31435543.v2 (data2_locust_PN66.mat)

  In the locust antennal lobe the median angle between PN population ON and OFF response vectors exceeds 90 degrees for every one of 11 odors (mean 106.7 +/- 0.84 SEM).

  For every odor the median angle between the ON and OFF responses was more than 90 degrees (mean = 106.7° +/- 0.84 SEM, n = 11 odors)

  Results, OFF responses form anticorrelated odor afterimages across species, paragraph 4; Figure 1F and 1G, S1

  All 11 odors give a median ON-OFF angle above 90 degrees (99.2 to 108.7); the mean is 104.9 (SEM 0.87), 1.8 degrees below 106.7 +/- 0.84. Without centring the mean is 58.5; with each PN's pre-odor mean subtracted and no centring it is 91.9, with 9 of 11 above 90. The centring alone gives angles above 90 degrees; the two halves of the ON window are 96.0 apart on average (11 of 11 above 90).

  How the test differed from the paper’s method

  • Data from the Figshare record of the published version; the preprint's data may differ.
• ◐Partially reproducedResults, Odor OFF responses persist long after odor offset, paragraph 3; Figure 7Idata in Figshare doi:10.6084/m9.figshare.31435543.v2 (data5_locust_PN85.mat)

  In the locust a time-bin k-NN classifier's accuracy decays across the 20 s after offset to about 40% averaged over odors, but for most odors does not fall to chance.

  Classifier accuracy decreased to ∼40% across odors throughout the 20-second post stimulus ITI, but notably, for most odors, accuracy did not fall to chance levels during the entire ITI period (Fig.7I).

  Results, Odor OFF responses persist long after odor offset, paragraph 3; Figure 7I

  Accuracy decays from 78.7% in the first 2 s after offset to 29.0% averaged over odors in the last 5 s, below 40 +/- 4; every one of the 7 odors stays above its null's 99th percentile in the last 5 s (16.0% to 54.0% against 14.0% to 27.6%). Averaged over the whole 0 to 21 s the accuracy is 39.6%, a reading of "about 40%" the criterion did not list. As for fig7f, the files allow no control for the block structure.

  How the test differed from the paper’s method

  • The paper's ~40% is read from its text; the last 5 s of the ITI was taken as the end.
• ✓ReproducedResults, OFF responses form anticorrelated odor afterimages across species, paragraph 5; Figure 2H and 4B legends; Methods, Odor delivery to mice

  The mouse block-protocol data (four odors, 5 s pulses, 55 s between trials, blocks of ten, 15 min between blocks) come from seven imaging sessions in four mice, with a median of 215 glomeruli per imaging session.

  With this approach, we monitored calcium signals from hundreds of glomeruli (median: 215 glomeruli per imaging session) ... Summary of the median angular distances for 4 odors (n = 7 imaging sessions from 4 mice)

  Results, OFF responses form anticorrelated odor afterimages across species, paragraph 5; Figure 2H and 4B legends; Methods, Odor delivery to mice

  The trials tables give seven block-protocol sessions from mice 134, 141, 148 and 149. The median ROI count over these is 210 (reading a); over all twelve sessions it is 296 (reading b). The ROI counts of sub-141 ses-8 (198), sub-149 ses-1 (219) and sub-148 ses-3 (318) equal the glomerulus counts the Methods give for Figures 2F and 4C, 6D and 7. The published version's Table 1 gives the same seven counts in four mice, but puts the 318 session with the 219 session (mouse 4), where the dandiset puts it with the 286 session (sub-148).

  How the test differed from the paper’s method

  • Sessions are matched to the paper's figures by ROI count; no identifier links them.
• ✓ReproducedMethods, Time-bin-by-time-bin correlation analysis (n = 198 glomeruli for Fig. 2F); the paper's Figshare recordchecks the link to the paper’s data

  The dandiset's raw fluorescence of the 198-glomerulus session is the data the paper deposited for its Figure 2D to 2F and 4C (Figshare data1_mouse_glom198.mat), up to the normalisation, with glomeruli in the same order.

  For mouse data, dF/F values were taken to be a n-dimensional population glomerular activity vectors (where n=198 glomeruli for Fig. 2F

  Methods, Time-bin-by-time-bin correlation analysis (n = 198 glomeruli for Fig. 2F); the paper's Figshare record

  Matched by ROI count (sub-141 ses-8, the only 198-ROI session) and ROI order. With the code's frame-counted windows the median within-trial r between the dandiset's raw F and the deposited array is 0.968 (5th percentile 0.834, 0.947 to 0.994 by block), and every deposited glomerulus correlates best with the dandiset ROI of the same index. With windows placed in seconds the median r is 0.307, as the code counts 16 frames per second in a session imaged at 15.5 Hz. The deposited values are close to F z-scored within each trial window (r = 0.927). No other claim depends on this one.

• ✓ReproducedResults, Odor OFF responses persist long after odor offset, paragraph 3; Figure 7J and S8C

  A time-bin k-NN classifier identifies the preceding odor from glomerular activity throughout the 55 s inter-trial interval; accuracy stays high for its whole duration and falls to chance with shuffled labels.

  We found that classifier accuracy remained high for the duration of the 55-second ITI (Fig.7J) ... Shuffling odor labels results in chance-level classifier performance throughout the ITI (Fig. S8C).

  Results, Odor OFF responses persist long after odor offset, paragraph 3; Figure 7J and S8C

  In sub-148 ses-3 accuracy is 88.5% in the first 10 s after offset and 71.9% to 75.6% from 10 to 50 s, above 50% and the null's 99th percentile (32.5% to 35.9%); the other sessions give 72.2% to 99.4% in every segment. The statement's reading as odor memory is not supported by the outputs. The trial-label null does not keep the block structure, and as each odor has one block no label permutation that keeps whole blocks can serve as a null. On pseudo-trials within the 15 min after each block, where no odor is given, the same classifier names the preceding block at 69.6% to 96.9% (sub-148 ses-3 69.6% to 71.2%), nearly as high as in the ITIs (the pseudo-trial maximum is below the ITI minimum in 4 of 7 sessions), so slow differences between blocks are enough for most of this accuracy.

  How the test differed from the paper’s method

  • dF/F with F0 the session median of each ROI.
  • The shuffled-label control is a permutation of whole-trial labels with the neighbours kept.
• ✓ReproducedResults, ON and OFF responses occupy distinct regions of feature space, paragraph 1; Figure 3B and S4Adata in Figshare doi:10.6084/m9.figshare.31435543.v2 (data2_locust_PN66.mat)

  In the locust the ON response to a reference odor is closer to other odors' ON responses than to OFF responses for most reference odors, significantly for 5 of 11.

  While this finding was consistent for most reference odors, the difference in angular separation was significant for 5 out of 11 odors in the panel (Fig. S4A).

  Results, ON and OFF responses occupy distinct regions of feature space, paragraph 1; Figure 3B and S4A

  Four reference odors have ON angles significantly smaller than OFF angles (hex, hxa, phn, vny), and 9 of 11 have smaller ON angles on average. Ammonia differs significantly the other way (ON 107.3, OFF 52.5 degrees, p = 0.0033). The 11 ON angles include the reference's own 0 degrees, as the criterion states; without it the count is still 4, and ON is smaller on average for 8 of 11. Benzaldehyde (p = 0.052) and octanoic acid (p = 0.059) are near the threshold.

• ✓ReproducedResults, History-dependent suppression enhanced between sequentially presented odors, paragraph 2; Figure 5D and 5Edata in Figshare doi:10.6084/m9.figshare.31435543.v2 (data4_locust_PN85.mat)

  In the locust, PN responses to geraniol presented 0.5 s after another odor are less correlated with that odor's solitary response than solitary geraniol is (one-tailed t-test p < 0.05, 5 odor pairs).

  we found that the correlation between high-dimensional odor response vectors was lower when the odors were sequentially presented, compared to when the odors were presented in isolation (Fig. 5D, E).

  Results, History-dependent suppression enhanced between sequentially presented odors, paragraph 2; Figure 5D and 5E

  What the paper reports

  • one-tailed t test, p < 0.05 (n = 5 odor pairs)

  Solitary first odors correlate with solitary geraniol at 0.486 on average and with geraniol after them at 0.227, lower for all five (one-tailed paired p = 0.0055). With hexanol as the second odor the difference is smaller (0.407 against 0.321, p = 0.13).

• ✓ReproducedResults, Odor OFF responses persist long after odor offset, paragraph 2; Figure 7E and 7F; Methods, Classification analysis of average ITI periodsdata in Figshare doi:10.6084/m9.figshare.31435543.v2 (data5_locust_PN85.mat)

  In the locust, PN activity averaged over the 20 s after odor offset classifies the preceding odor (k-NN on angular distance), so ITI activity is odor specific.

  A k-nearest neighbor classifier, using angular distances of the full-dimensional ITI data, robustly discriminated odors (Fig. 7F). This confirmed the interpretation from the PCA analysis and showed that ITI activity was odor-specific.

  Results, Odor OFF responses persist long after odor offset, paragraph 2; Figure 7E and 7F; Methods, Classification analysis of average ITI periods

  The 20-s ITI averages classify all 70 trials correctly (null 99th percentile 25.7%, chance 14.3%); the code's choice (k = 4, cosine on three PCs) gives 97.1%. The published version's Table 1 gives this dataset a solitary block structure; the Figshare file holds only the 40 s around each trial, so no control for slow differences between blocks can be built, and the accuracy cannot be attributed to the odor alone.

• ⊘BlockedResults, Odor OFF responses persist long after odor offset, paragraph 2; Figure S7D and S7E, Figure 7K· Methods underspecifiedverifier disagreed at first

  During the 15 min between blocks odor-specific information is no longer present; classification of 1-min bins of the inter-block interval by the preceding odor is near chance (34.8%).

  during the IBI, network activity ceased to contain odor-specific information ... The overall classification rate of the glomerular IBI activity was 34.8%.

  Results, Odor OFF responses persist long after odor offset, paragraph 2; Figure S7D and S7E, Figure 7K

  With the Methods' k = 50 on 60 one-minute bins the rate is 0.0% to 13.3% (mean 3.8%) in the four sessions with complete IBIs, and this procedure's permutation null averages 10.8%, not the 25% chance of four odors, so the paper's 34.8% cannot come from it. With k = 20 the IBI bins are classified by the preceding block at 73.3% to 95.0% in those four sessions and 35.0% to 66.7% in the other three (permutation p <= 0.02), and with k at or below the 14 other bins of the own IBI at 69.4% to 98.3% (k = 10) and 90.0% to 98.3% (k = 5) in all seven (null means 21.8% to 25.5%, p < 0.001). These rates contradict the claim, but each odor has one IBI per session, so no validation can hold out a block, and slow drift between neighbouring minutes could give them.

  How the test differed from the paper’s method

  • The 318-glomerulus session has only three complete IBIs (54 bins), so k = 50 was run on the four sessions that have four.