scanpy · normalize_total · Numba · UnboundLocalError · reproducible builds
"UnboundLocalError: counts_per_cols" — What Disabling Numba Actually Broke in scanpy
Status as of writing: present in scanpy 1.11.5 (and versions around it), fixed via PR #4231 (merged 2026-07-27), shipped in 1.12.4 (2026-08-27). Reported in scanpy#4228.
If you disable Numba JIT — a common workaround when a container or CI environment can't write Numba's cache files — sc.pp.normalize_total() could crash outright on CSR matrices with the (default) setting exclude_highly_expressed=False:
NUMBA_DISABLE_JIT=1 python - <<'PY'
import numpy as np
from scipy.sparse import csr_matrix
from anndata import AnnData
import scanpy as sc
adata = AnnData(csr_matrix(np.array([[1, 2, 0], [0, 3, 4]], dtype=np.int32)))
sc.pp.normalize_total(adata, target_sum=1e4)
PY
# UnboundLocalError: cannot access local variable 'counts_per_cols'
# where it is not associated with a value
Why disabling Numba even matters
This isn't an exotic setup. People turn off Numba JIT (via NUMBA_DISABLE_JIT=1) for a few ordinary reasons:
- The runtime environment can't write to Numba's cache directory (common on locked-down CI runners or read-only containers)
- Debugging — JIT-compiled functions are harder to step through
- Cold-start latency in short-lived jobs where JIT compilation overhead isn't worth it
The actual bug
The CSR-specific fast path, _normalize_csr, only initialized counts_per_cols inside the if exclude_highly_expressed: block, but returned it unconditionally at the end of the function. With Numba's JIT active, this quirk of the compiled code path apparently never surfaced the same way; with JIT disabled, plain Python's stricter local-variable scoping raised the UnboundLocalError the moment the default-off branch was taken.
The fix (confirmed by reading the current source) simply initializes counts_per_cols = np.zeros(columns, dtype=np.int32) unconditionally before the branch, so it always has a value to return:
# current scanpy main, _normalization.py
def _normalize_csr(mat, rows, columns, exclude_highly_expressed=False, ...):
counts_per_cols = np.zeros(columns, dtype=np.int32) # <- now unconditional
...
if exclude_highly_expressed:
counts_per_cols_t = np.zeros((n_threads, columns), dtype=np.int32)
...
return counts_per_cell, counts_per_cols
What to do
Upgrade to scanpy ≥1.12.4. If you're stuck on an older pinned version and need Numba disabled, the workaround is to explicitly pass exclude_highly_expressed=True even if you don't need the exclusion logic — it forces the code path that does initialize the variable, sidestepping the crash (at the cost of a slightly different internal computation you should verify doesn't change your output if you truly don't want exclusion).
Verified against scanpy's main branch source and PR #4231, 2026-09. Source: scanpy#4228.
