Segmentation and Tracking

← Back to Notebook Examples

The notebooks below all use spice.segmentation.<method>(). Each method builds a new SPICEData that shares the input image’s history, and records what it did automatically.

notebooks/segmentation/example_unet.ipynb

Trains a U-Net segmentation model (a ResNet-34 encoder, Dice loss) to tell “particle” from “background” in TiO₂ SEM images, using one call: spice.segmentation.train_unet(). Preparing the dataset, building the DataLoader, setting up the model, training with cosine annealing, and (optionally) checking accuracy on a test set with IoU scoring all happen inside that one call. The returned SPICEData holds the per-epoch [train_loss, val_loss] numbers as data, with the model settings in metadata.

notebooks/segmentation/example_sam1.ipynb

Segments objects with no training needed at all, using Meta’s Segment Anything Model (SAM v1), via spice.segmentation.run_sam1(). Shows two ways to use it: (1) automatically finding masks across the whole image (SamAutomaticMaskGenerator, automatic=True), keeping the largest one found; and (2) segmenting from a single point you click (SamPredictor). Both return a new SPICEData with the binary mask and a confidence score in metadata, recorded in spice.history.

notebooks/segmentation/example_sam2.ipynb

The same idea as SAM v1, but with Meta’s newer Segment Anything Model 2 (SAM2), via spice.segmentation.run_sam2(). One point you click predicts a binary mask on a SEM image. Covers setting up SAM2 (its checkpoint file and YAML config) and running the predictor on the same image and point as the SAM1 notebook, so the two masks can be compared. The returned SPICEData holds the mask and confidence score, recorded in spice.history.

notebooks/segmentation/example_nasa_micronet.ipynb

Trains a UNetPlusPlus model (with a se_resnext50_32x4d encoder, pretrained on NASA’s MicroNet microscopy dataset) to segment EBC1 SEM images of an environmental barrier coating (oxide layer vs. background), using spice.segmentation.run_micronet(). Covers preparing the dataset with Albumentations augmentation, training with early stopping (DiceBCE loss), checking accuracy on a test set (IoU), viewing predictions per image, and saving the first test prediction. Needs the pretrained_microscopy_models package, which isn’t on PyPI and installs straight from GitHub — see Special cases on the installation pages.

notebooks/segmentation/example_trackpy.ipynb

Tracks many particles moving across a sequence of fluorescence microscopy frames, using spice.segmentation.run_trackpy(). The frame sequence loads as one dataset_type='multiple_frames' SPICEData via load_frame_sequence(), and then the whole Trackpy pipeline runs in a single call: finding particles in each frame, linking them into trajectories across frames, dropping short trajectories, optional quality filtering (by mass, size, or roundness), correcting for overall drift, and computing mean-squared displacement (MSD) — both per particle and averaged across all of them. The returned SPICEData holds the averaged MSD values as data, with per-run statistics in metadata.