napari-sim-processor vs the SHAPE pipeline

Swap the reconstruction, hold the SHAPE detector + tracker fixed. Does the biology survive? · shape2fate @ 2026-04-29 CPU · Windows

1 · Benchmark 2 · How this was built (AI + human-in-the-loop)3 · AI-HI summary ▶4 · SHAPE reconstruction

One annotated TIRF-SIM movie of clathrin-coated pits (RPE-1, EGFP-CLCa; 120 frames, 3 expert annotators). Five inputs, a plain diffraction-limited widefield and four SIM reconstructions of the same raw data, each run through the identical shape2fate CCP detector + MIP linker and scored (MOTA/HOTA) against the annotators. Every number is recomputed here or from the paper.

Baseline reproduced

Δ 3×10⁻⁵

numerically identical to the SHAPE reconstruction, the notebook hard-codes modulation = 1.0

Pipeline validated

MOTA 0.79 = 0.79

reproduces the paper's Supplementary Table 1 for Shape2Fate & the human band

Surprise finding

widefield > plugin SIM

plain 9-frame average (0.69) tracks better than every napari-plugin SIM reconstruction (≤0.68)

Headline. Only SHAPE's carefully-denoised super-resolution (MOTA 0.79) beats doing no SIM reconstruction at all (widefield, 0.69). The napari plugin's SIM reconstructions all score lower than the widefield, they add high-frequency noise the detector reads as false pits. Reconstruction quality, not resolution, moves the downstream biology.

1 · The five inputs

Same raw SIM, six views of the annotated region (frame 40), magma, contrast-stretched per image. recomputed here

How SIM reconstruction works, a 6-slide explainer

Optics, read from the DeltaVision acquisition header (not the filename): green EGFP, emission 515 nm, sample pixel 0.0613 µm. The shipped parameters.json (603 nm / 0.0791 µm) is wrong on both, but immaterial: with the SHAPE method, reconstructing at the true green optics vs the shipped ones changes MOTA by <0.01 (0.769 vs 0.770), because the SIM carrier is estimated from the data, not the assumed optics. Results below use the shipped values to match the paper; the correct optics give the same. Full saga on page 2.

2 · How the reconstructions differ

All five estimate the same data-driven illumination carrier (three orientations ~60° apart; the plugin recovers all three on 94% of frames, SHAPE on all). They differ only in the final processing. recomputed here

Estimated illumination parameters, what each pipeline computes from the data on pressing "Start"

Per orientation, first time-point. The three plugin variants (Wiener / OS / WF) share this one calibration; LR widefield estimates nothing (no SIM). SHAPE recovers all three orientations ~60° apart (≈22/82/142°); on this first frame the plugin mis-locks orientation 3 (78° ⚠ vs the true ~22°), it succeeds on 94% of frames. |k| is at the pixel used for the benchmark (0.0791 µm); phase conventions differ between the two estimators, so phases are not directly comparable. Everything downstream (detection + linking) is identical for every input.

3 · Accuracy vs manual ground truth

120 frames · 3 annotators · annotated region · match 5 px. Same detector + linker on each input. recomputed here

Metric radar

MOTA & HOTA vs the human band

Grey = inter-annotator range. SHAPE is at the human band (in-band on MOTA/DetA/mTIOU, marginally below on HOTA/1−MOTP, and below on AssA). The plain widefield (0.69) beats every plugin SIM variant; among those filtered widefield (0.68) is best, Wiener super-res noisy (0.44), filtered OS worst (0.32).

4 · My recomputation vs the published table

Shape2Fate preprint, Supplementary Table 1 (same movie). cmeAnalysis & TraCKer are external MATLAB tools I did not re-run. from the paper recomputed here

My independent SHAPE reproduces the paper's Shape2Fate column and my human band its Annotators column, the pipeline is verified. cmeAnalysis (0.50) & TraCKer (0.59) are prior TIRF trackers run on the same widefield; the shape2fate detector on that widefield scores 0.69, the detector, not the reconstruction, carries most of the gain.

5 · Detection & tracking, seen directly

Annotated region. Green = ground truth; blue dots / colored lines = the pipeline. recomputed here

Detections (frame 40), all five inputs

More dots ≠ better: SHAPE & widefield place a clean dot per pit; the SIM variants scatter false detections across the grainy background.

Detections over time (animated), SHAPE vs plugin Wiener

SHAPE detections — **SHAPE**, clean (MOTA 0.79)

plugin Wiener detections — **plugin Wiener**, a cloud of false detections (MOTA 0.44)

Track overlays (max-projection; green = GT), all five

6 · What comes out of this

Doing no SIM reconstruction beats the plugin's SIM. The plain 9-frame widefield (MOTA 0.69) outscores every napari-plugin SIM variant (0.32–0.68); their super-resolution adds noise the detector mistakes for pits.
The detector carries the pipeline. shape2fate's detector on a plain widefield already scores 0.69 vs 0.50–0.59 for classical trackers on the same widefield. Most of the quality is the deep detector, not the reconstruction.
SHAPE's edge is denoising, not resolution. RL-deconvolution + apodization + a fixed modulation lift the widefield's 0.69 to a human-level 0.79.
Practical takeaway: if a full SHAPE reconstruction isn't available, feed the detector a plain widefield, not a naive SIM reconstruction. Cheaper and more accurate here.

7 · Idea: bring SHAPE reconstruction into the napari plugin

The gap between the plugin's best SIM (0.68) and SHAPE (0.79), and the fact that the plugin's SIM underperforms a plain widefield, is closed by three self-contained steps that shape2fate.reconstruction already implements in pure NumPy:

A "SHAPE-Wiener" reconstructor in baseSimProcessor: after carrier calibration, route to shape2fate's Wiener with Richardson–Lucy pre-deconvolution + apodization + fixed modulation = 1.0. Each is <60 lines, GPU-optional.
Harden carrier estimation, reject a candidate whose orientation duplicates another (the plugin's ~6% per-frame failure).
Expose as a mode beside the current Wiener / OS / WF buttons, knobs as widgets.
Optional: ship the detector + linker as a companion widget, this dashboard is the feasibility proof (all headless, CPU).

Provenance. recomputed here = computed in this session (all reconstructions, detections, tracks, metrics, the inter-annotator band). from the paper = Shape2Fate preprint (bioRxiv 10.64898/2026.03.29.715120), Supplementary Table 1. Detector ccp-detector-sandy-wildflower-269.pt. CPU only.