Abstract for this paper: Cell segmentation is important for various downstream tasks. This paper introduces a generalist segmentation method called Cellpose that can segment cells from various ranges of image types without any fine-tuning. They also collect a large dataset containing over 70000 segmented objects to train the model. Finally, they built software to assist with labeling and sharing of data.


Base Information

  • Motivation: Traditional methods, such as the watershed algorithm, work well only when the segmented objects form a single basin. However, the cell forms multiple intensity basins due to the borders and protuberances.

  • Data preparation: Using the masks drawn manually to generate the topological map through a process of simulated diffusion.

    Data generation
  • Model: The neural network was trained to predict

    • Horizontal gradient of the topological map
    • Vertical gradient of the topological map
    • The binary map indicates the probability of whether there are objects here.

    Totally three channel outputs from the model.

  • Predict: Using the gradients to build vector fields and using gradient tracking which can assign the center point to each pixel. Finally, the pixels that have the same center point will build the mask.

Method Structure


To train the model, this work collects a large dataset, including

  • 316 images of fluorescently labeled proteins
  • 50 images from bright field microscopy
  • 58 images of membrane-labeled cells
  • 86 images from other types of microscopy (not cell)
  • 98 images of non-microscopy images

The dataset can be divided into two parts:

  • Specialized data

    100 images pre-segmented as part of the Cell Image Library have two-channel images (cytoplasm and nucleus). The model will train on this dataset alone.

  • Generalized data

    All 608 images including specialized data. 69 images were reserved for testing.

Distribution of dataset. Legend is as follows: dark blue, Cell Image Library; blue, cytoplasm; cyan, membrane; green, nonfluorescent cells; orange, microscopy other and red, non-microscopy.

Two models will be trained/tested on this dataset.

Data usage

Mask & Flow Field

From Mask to Flow Field

To build the label of the network’s output, the mask should be transformed into the Flow Field by:

  • Find each individual instance in Mask (different instances are represented by different numbers, starting with 1, 0 for background)

  • Calculate the center point and average diameter of all instances

  • Flow Field is obtained by using the center point as the heat source and iteratively performing multiple diffusions.

  • Compute the gradient of the thermal map on the x/y axis to obtain the gradient map (the output of the model)

  • mask directly as a binary graph (and model output as a cross-entropy loss)

From Flow Field to Mask

The output of the network is the gradient map of the x/y axis and probability map. To build the masks should follow:

  • Using the probability map to find the pixel needs to be considered
  • Compute the dynamic mesh and find local hot spots using Euler integrals
  • Use filtering and flooding to get the final hot spot
  • Assign a hot spot to each pixel, belonging to the same hot spot belongs to a mask

Network Structure

To predict the three channel outputs having the same size as the input, this paper chose U-Net architecture and adjusted the following:

  • Replacing the standard building block of U-Net with residual blocks
  • Using global average pooling to obtain the style of the image
Effect of network architecture


Generalist Cellpose Model
Comparing with other methods


[1] C. Stringer, T. Wang, M. Michaelos, and M. Pachitariu, “Cellpose: a generalist algorithm for cellular segmentation,” bioRxiv,bioRxiv, Feb. 2020. doi: 10.1101/2020.02.02.931238.

[2] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation,” in Lecture Notes in Computer Science,Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 2015, pp. 234–241. doi: 10.1007/978-3-319-24574-4_28.