organize: name files with BIDS labels (+), refuse indistinguishable identifiers - #1981

Summary

• BIDS labels. BIDS 1.11.0 allows + in labels (bids-standard/bids-specification#1926). dandi organize now builds labels by replacing every character other than an ASCII letter, an ASCII digit or + with +, one character at a time. For example, subject_id fly_226 becomes sub-fly+226; it used to be sub-fly-226, which is not a valid BIDS label. The function is the same as dandischema.utils.sanitize_label() from dandi/dandi-schema#447, and a test checks that they agree whenever dandischema provides it.
• Collision check. Distinct identifiers can now get the same label, so organize fails with a list of the offending IDs in two cases:
  • distinct subject IDs, or distinct session, tissue sample, slice or cell IDs of one subject, would get the same label
  • they would get labels differing only in case, which BIDS forbids and which collide on case-insensitive file systems
• Existing names are kept. get_existing_labels() collects the labels in the names of the sub-*/*.nwb files already in the Dandiset. A value whose older label is among them keeps that label. That covers the - replacement, and also a unique sanitize_label() match for names written by versions that kept , or spaces. Reorganizing in place renames nothing, and new files of an existing subject or session go next to its existing files.
• docs/source/cmdline/organize.rst and the organize help text describe all three.

Impact on DANDI

I computed this from the dandisets/<id>/draft/assets.jsonld manifests of the 1,001 public draft Dandisets (606,651 assets, 2026-11-05):

• Renames avoided. 226,518 assets in 365 Dandisets have labels that the new rule would change, so reorganizing them in place would rename them without the reuse of existing names. The characters responsible are mostly - (380,701 occurrences), then #, space, (, ), , and &.

• Dandisets that would now fail to organize from scratch:

  Dandiset	Distinct identifiers	Problem
  000246	AAV V15 D5, AAV V15-D5	same label; already merged into one folder today
  001250	vivo-marques-smith, vivo_marques_smith	same label; already merged today
  001702	FAD+_L_CA2 20X_SF2, FAD-_L_CA2 20X_SF2, …	same label (transgene ±); distinct today, so the new rule would merge them
  001428	Aged mice, aged mice	differ only in case
  001791	Barrel500 2, barrel500 2	differ only in case

  Reorganizing 001702 in place still works, because its existing distinct names are reused. No session IDs collide within a subject.

• No identifier contains non-ASCII characters, and none sanitizes to an empty label.

Notes and open questions

• Collapsing. Replacement is one character at a time, not collapsing runs or stripping a trailing +. Collapsing would merge more distinct IDs, e.g. …_-0.4V vs Eref… and …_0.4V vs Eref… in 000983, and 4484 and 4484? in 001195.
• + as list separator. dandi-cli already joins lists with +, as in _probe-1+2 and the modalities suffix _behavior+ecephys. probe-A+B can therefore mean two probes, or one probe named A_B. Either is a single valid BIDS label, but a reader can't tell which.
• + in URLs. In URL query strings + decodes to a space, so a path in ?path=… must be sent as %2B. requests does that; hand-written URLs may not.
• Validation. validate_organized_path() (LABELREGEX) is unchanged, so names from older versions, including those with , or spaces, still validate. A HINT-level "not a BIDS label" result could follow.
• Local files only. Existing names come from the local Dandiset directory. If its files were not downloaded, names already on the archive are unknown and new files get new-style labels.
• Suggested label: minor.

Testing

• pytest dandi/tests/test_organize.py: 68 passed. One test is skipped with dandischema 0.14.0, which lacks sanitize_label; with the Add BIDS sanitize_label(); use it to match file-name labels to metadata identifiers dandi-schema#447 branch installed, that test passes too.
• Full suite with DANDI_TESTS_NONETWORK=1 and -m "not integration": 941 passed, 11 failed, 402 skipped. The same 11 fail on master here: 10 need bids-validator-deno, which isn't installed, and test_helptext needs dandi on PATH. Network and Docker-based integration tests were not run; CI covers them.
• pre-commit (black, isort, codespell, flake8, reuse) is clean, and mypy reports no new errors.

Generated by Claude Code