Illustration generated with AI; not experimental data.

Image Segmentation

Image segmentation assigns a class label to every pixel in an image, partitioning it into meaningful regions. In electrochemical microscopy, segmentation is used to identify and quantify phases, particles, pores, and other structural features.

Classical vs. ML-Based Approaches

Classical approaches:

• Thresholding (Otsu, adaptive): effective for bimodal histograms.

• Watershed transform: separates touching objects using topological flooding; sensitive to over-segmentation.

• Active contours / level sets: iteratively deform a contour to minimize an energy functional.

• Graph-cut methods: formulate segmentation as a min-cut problem on a pixel graph.

Machine-learning approaches:

• Random forests on handcrafted features: fast and interpretable; used in tools such as ilastik.

• Convolutional neural networks (CNNs): learn hierarchical features end-to-end from annotated data; far more powerful than handcrafted features for complex textures and shapes.

• Foundation models: large models pretrained on web-scale or domain-specific data; applicable with zero or very few labels.

The classical, unsupervised approaches above are covered on the Image Clustering page. JuSPICE’s deep-learning and foundation-model pipelines are described below — pick one.

U-Net

Encoder-decoder CNN with skip connections; the workhorse of trainable segmentation.

Segment Anything Model (SAM v1)

Point/box-prompted foundation model, trained on over 1 billion masks.

SAM 2

SAM extended with a memory mechanism for propagating masks across frames.

NASA MicroNet

Pretrained encoder backbones specialized for materials microscopy.

TrackPy

Crocker–Grier particle detection and trajectory linking across frames.

JuSPICE integration

In JuSPICE, “segmentation” and “clustering” refer to two distinct modules. The classical approaches listed above (thresholding, watershed, and the other unsupervised region-partitioning methods) are provided by juspice.clustering_module via the spice.clustering accessor — see Image Clustering and Clustering. The deep-learning and foundation-model pipelines (U-Net, SAM-1, SAM-2, NASA MicroNet) and TrackPy particle tracking are exposed through juspice.segmentation_module via the spice.segmentation accessor. Images should be loaded via load_data() and results saved via save_data() to preserve provenance.

See Segmentation for a module-level explanation and juspice.segmentation_module for full API details.

Segmentation Notebooks

The following Jupyter notebooks demonstrate each segmentation and tracking approach:

• notebooks/segmentation/example_unet.ipynb

• notebooks/segmentation/example_sam1.ipynb

• notebooks/segmentation/example_sam2.ipynb

• notebooks/segmentation/example_nasa_micronet.ipynb

• notebooks/segmentation/example_trackpy.ipynb

See Notebook Examples for descriptions of each notebook.

References

Libraries

• PyTorch — deep-learning framework for U-Net training and inference

• segmentation-models-pytorch — high-level U-Net and encoder-decoder API

• Segment Anything (SAM) — Meta AI foundation model for zero-shot segmentation

• SAM 2 — extended SAM with memory for video and image segmentation

• pretrained-microscopy-models (MicroNet) — NASA pretrained encoders for materials microscopy

• trackpy — particle tracking and MSD analysis

See Bibliography for the publications behind the methods on this page.