widefield

Image processing, analysis and interactive viewers for widefield calcium imaging.

This is the SteinmetzLab fork of cortex-lab/widefield. The original MATLAB is preserved unchanged under matlab/; new development is in Python under src/widefield/.

matlab/              the original MATLAB, moved but not modified
src/widefield/       the Python package
tests/               pytest suite, incl. golden data generated by the MATLAB itself
tests/matlab_ref/    MATLAB scripts that generate the golden reference and the benchmark

Install

pip install -e ".[gui,dev]"

The numerics (widefield.svd, .events, .correlation, .hemo, .compress, .io) need only numpy and scipy and stay importable with no Qt installed — batch preprocessing runs headless. The gui extra adds PySide6 + pyqtgraph for the viewers. Recording a movie additionally needs pip install 'imageio[ffmpeg]'.

The data representation

A widefield movie is far too large to hold in memory, so it is kept SVD-compressed throughout:

M = U @ V

where U is (Ypix, Xpix, nSV) spatial components and V is (nSV, nFrames) temporal components. Following the original MATLAB, the singular values are folded into V — so what this package calls V is really S @ V — and frames are reconstructed on demand rather than ever being materialized.

from widefield import load_uvt, svd_frame_reconstruct

d = load_uvt(r"Y:\Subjects\AB_0004\2021-03-24\1", nsv=500)   # blue/violet/corr layout
frame = svd_frame_reconstruct(d.u, d.v[:, 1000])             # one (Ypix, Xpix) image
stack = svd_frame_reconstruct(d.u, d.v[:, 1000:1100])         # (Ypix, Xpix, 100)

nsv matters: the .npy files are Fortran-ordered, so truncating components is a contiguous prefix read. Asking for 500 of 2000 components transfers 500 MB instead of 2 GB, and reads sequentially — the single biggest lever on load time over a network mount. Loaded arrays are cached locally (~/.widefield/cache, or $WIDEFIELD_CACHE), never beside the read-only server data.

The viewers

Each is a plain function you call on arrays you already have, exactly like the MATLAB, and each also exposes a QWidget class for embedding in a larger application:

from widefield.gui import (
    pixel_correlation_viewer,   # pixelCorrelationViewerSVD
    pixel_tuning_curve_viewer,  # pixelTuningCurveViewerSVD
    movie_with_traces,          # movieWithTracesSVD
    svd_viewer,                 # svdViewer
)

pixel_correlation_viewer(d.u, d.v)

To try all four on one session:

python examples/try_viewers.py            # or --demo for synthetic data, no server needed

They have full keyboard/mouse parity with the MATLAB versions — see each module's docstring for the key table. Deliberate differences:

• Arrow and i/j/k/l keys move relative to the screen, so they stay correct after you rotate the image with alt+arrows. The MATLAB's data-space keys invert visually once rotated.

• Adding a pixel in the movie viewer is ctrl+click, not right-click: in pyqtgraph right-click is the view's own context menu, so the MATLAB binding produced a menu and a new point.

• The tuning viewer's ROI is a live draggable polygon whose mask and traces stay available as viewer.roi, rather than being written into the base workspace by assignin.

• hemo_correct_* do not plot. The MATLAB versions pop figures mid-computation, which makes them unusable in a batch pipeline; the scale-factor map and transform are returned instead.

• Hover is on by default in the correlation viewer. A map costs ~9 ms, so sweeping the mouse over cortex and watching the structure move is the fastest way to read a session. h toggles it; pass hover=False for the MATLAB's click-to-place behavior.

Hotkeys work wherever focus happens to be inside the window — except while the caret is in a text box, where they stay out of the way.

All three viewers have a temporal band-pass: type cutoffs in Hz (0 and inf by default). Filtering V filters every pixel, so it re-applies quickly. In the correlation viewer it changes the covariance and so rebuilds the precompute — worth it, because removing the slow drift before correlating usually sharpens the functional boundaries, and restricting to a band is how you ask which timescale the correlation structure lives on.

• The tuning viewer high-passes causally (forwards only); the movie and correlation viewers filter zero-phase. A zero-phase high-pass runs backwards as well as forwards, so part of every response lands before the event that caused it: on an opto session the pre-stimulus baselines fan out in proportion to laser power and look like anticipation. Filtering forwards only cannot move anything backwards. Only the high-pass is made causal — a causal low-pass would delay every measured peak, and its backward smear is small and does not grow with response size. The status line names whichever combination is in force. See bandpass_filt(..., causal_highpass=True).

Also in the movie viewer: a Follow toggle so a manual zoom on the trace plots survives playback instead of being reset every frame, a scrub slider, and nsv_display to cap components per frame if full-rank playback stutters.

Seeing behind a condition average

The tuning viewer has a fourth panel the MATLAB does not: every individual trial behind the selected condition, in white, with the condition's average over them. It exists because a condition average is not evidence on its own — one trial that wandered off is enough to move a 90-trial mean far enough to invent an effect, and nothing in the average shows you that. Two things to do about it:

• m swaps the mean for a median. If a condition's average collapses back in line with its neighbors, one or two trials were carrying it. The band swaps with the statistic: mean ± s.e.m., or median + 95% CI.
• Click a trial to isolate it. That trial's own movie replaces the condition average in the brain panel, reconstructed from its own components rather than approximated. [/] step through trials, esc goes back to the average.

The median's interval is the distribution-free order-statistic interval (invert the sign test), not a bootstrap: exact rather than asymptotic, built only from values trials actually reached, and one sort rather than ~1000 resampled medians per condition — which matters because it recomputes on every pixel you click. Below 6 trials no such interval reaches 95% and the band is left blank rather than quietly widened. See stats.py.

The brain panel stays a mean even in median mode, because a pixelwise median cannot be built from component medians (the median does not commute with U @ V) and doing it honestly would mean reconstructing every trial at full resolution for every frame — seconds each, against the ~8 ms that makes scrubbing feel live. Isolate the trial instead; that shows it exactly.

Performance

Measured against the MATLAB on a real session (AB_0004/2021-03-24/1, 512x512, 200 components; MATLAB numbers from tests/matlab_ref/bench_matlab.m):

operation	MATLAB	Python
load U + V	29.1 s	1.6 s cold / 0.21 s cached	18x / 141x
reconstruct one frame (scrubbing)	8.03 ms	7.69 ms	parity
playback frame, incl. full redraw	8.03 ms	3.56 ms	2.3x
correlation precompute	0.51 s	0.31 s	1.6x
seed correlation map	11.26 ms	9.29 ms	1.2x
event-locked average	0.16 s	0.09 s	1.8x
tuning viewer, step one time point	8.03 ms	0.008 ms	1000x

Testing

pytest

369 tests, no MATLAB or server access required. The suite's backbone is tests/data/matlab_reference.mat, which holds inputs and the MATLAB implementations' outputs for those inputs, generated by tests/matlab_ref/gen_reference.m. Every ported function is checked against it, so the port is verified against real MATLAB behavior on any machine. Regenerate after changing the MATLAB:

matlab -batch "run('tests/matlab_ref/gen_reference.m'); run('tests/matlab_ref/gen_svd_reference.m')"

What is ported

Python	from MATLAB
svd.svd_frame_reconstruct, pixel_timecourse	svdFrameReconstruct
svd.change_u, dff_from_svd	ChangeU, dffFromSVD
svd.hp_filt, detrend_and_filt	hpFilt, detrendAndFilt
svd.subsample_shift	SubSampleShift (the unmerged interp1 fix)
svd.bin_image	binImage
events.event_locked_avg_svd + helpers	eventLockedAvgSVD
correlation.SeedCorrelation, correlation_map_raw	pixelCorrelationViewer[SVD] math
hemo.hemo_correct_local, hemo_correct_nonlocal	HemoCorrectLocal, HemoCorrectNonlocal
compress.svd_compress	get_svdcomps
io.load_uvt, read_u_from_npy, read_v_from_npy	loadUVt (Pipelines), readUfromNPY, readVfromNPY
signals.schmitt, schmitt_times	schmitt, schmittTimes
colormaps.*	colormap_blueblackred, colormap_redblackblue, colormap_*WhiteRed/Blue
utils.find_nearest_point	findNearestPoint
gui.* (4 viewers)	pixelCorrelationViewerSVD, pixelTuningCurveViewerSVD, movieWithTracesSVD, svdViewer

Not yet ported

Still MATLAB-only under matlab/, with the reasoning:

• Registration / motion correction — alignToTarget, align_iterative, determineTargetFrame, generateRegistrationTarget, pick_reg_init, register_movie, registerDatFile, registration_offsets. Pipelines/widefield/registerImages3.m has largely superseded these in production; port that rather than these if registration is needed.
• Kernel regression — kernelRegression{,2,3}, makeContPredictor, makeKernelRegPredictor{,Cos,Exp}.
• Raw acquisition readers — loadRawToDat, tifToDat, timeFromPCOBinaryMulti, LoadCustomPCO, readB16, readOneCustomPCO, loadTiffStack, getNFramesFrom*. These are camera- and rig-specific and need real raw files to test meaningfully.
• sparseRetinotopy, reSVD, svdVideoObj, +svdVid (UDP acquisition).
• Remaining viewers — the raw (non-SVD) pixelCorrelationViewer / pixelTuningCurveViewer (their math is ported: correlation_map_raw, peri_event_series), movieWithTracesSVDmulti, transformationViewerSVD, quickMovieWithVids, frameGen_divide.
• rotateImages is an interactive click-through script rather than a function, so it has no direct equivalent; it would need redesigning as a proper alignment tool.

Related repositories

• SteinmetzLab/Pipelines widefield/ — the live MATLAB preprocessing pipeline. It writes the blue/, violet/, corr/ layout that widefield.io reads, and it is the successor to this repo's scripts/.
• SteinmetzLab/DataBrowser — the session browser. Its widefield panel predates this package and duplicates some of these numerics; the plan is for it to call into this package instead.

Notes on fidelity

Where MATLAB behavior is surprising, the port reproduces it and says so in a comment rather than quietly "fixing" it — these numbers feed comparisons against MATLAB-derived results:

• bin_image reproduces conv2 'same' plus decimation exactly, including zero-padded edge windows. It coincides with a block mean at B=2 but differs by >100% at B=4.
• detrend_and_filt keeps causal one-pass filtering (MATLAB filter, not filtfilt), so the output is phase-shifted. Production depends on this.
• MATLAB's filtfilt padding differs from scipy's default and is matched explicitly.
• The movie viewer restarts at frame 1 when it overruns while the tuning viewer wraps modulo — the two MATLAB files genuinely disagree, and so do we.
• MATLAB flattens pixels column-major and numpy row-major, so flat per-pixel vectors are permutations of MATLAB's even where every derived image is identical.