Outcomes of the 1,149 claims that rest on the dandiset alone
Counted separately: 32 claims that also use data from outside the dandiset.
- 16 reproduced
- 8 partially reproduced
- 4 not reproduced
- 4 blocked
Counted separately: 62 claims on data the paper deposited in other archives.
- 29 reproduced
- 10 partially reproduced
- 2 not reproduced
- 21 blocked
A failure to reproduce is not a finding that the paper is wrong. It may be a failure of the attempt.
An outcome says what one attempt could recover from the data in a dandiset, with the methods available to it. A “not reproduced” or “partially reproduced” outcome can come from a mistake or a limitation in the agent’s analysis, from methods the paper does not describe fully, from data that is not in the dandiset, or from a dandiset that differs from what the paper analysed. “Blocked” means the claim could not be tested at all.
Most attempts are made by AI agents. Where a page says so, the attempt has been checked by a second, independent agent, but neither is infallible. Every attempt links to its code, its reasoning and the ways its analysis differs from the paper’s. Authors who think an attempt is wrong can open an issue.