Microplankton life histories revealed by holographic microscopy and deep learning
- Department of Physics, University of Gothenburg, Sweden
- Department of Marine Sciences, University of Gothenburg, Sweden
Figures
Figure 1 with 3 supplements
Experimental setup and deep-learning data analysis.
(a) Holographic microscope: Planktons suspended in a miniature sample well are imaged with an inline holographic microscope. The (cropped) example holographic image features two different plankton species: Oxyrrhis marina and Dunaliella tertiolecta (full image in Figure 1—figure supplement 1). (b) Deep-learning network 1: A regression U-Net (RU-Net, see details in Figure 1—figure supplement 2), trained on simulated holograms, uses individual holograms to predict output maps containing the segmentation of the planktons, their -position, their dry mass , and the distances and from the closest plankton for each pixel (to be used for the accurate localization of planktons). (c) Plankton 3D position and dry mass: The information obtained by the RU-Net permits us to reconstruct the 3D position of the planktons along with their dry mass (color bar). (d) Plankton sequences: Using the plankton positions obtained by the RU-Net, we extract sequences of -pixel holograms centered on an individual plankton. (e) Deep-learning network 2: The sequences are then used by a weighted-average convolutional neural network (WAC-Net, see details in Figure 1—figure supplement 3), trained on simulated data, to refine the estimations of and . (f) Dry mass time series: Example of a refined dry mass prediction in picograms (pg) for a micro-zooplankton (Oxyrrhis marina, orange line) and a phytoplankton (Dunaliella tertiolecta, blue line) obtained by the WAC-Net.
Figure 1—figure supplement 1
Holographic microscope and full-scale view of an experimental holographic image.
(a) Detailed view of the inline holographic microscope shown in Figure 1a: A plane wave generated by a monochromatic LED source illuminates the sample well that contains two different species of planktons (Dunaliella tertiolecta and Oxyrrhis marina), whose bright-field microscope images are shown in the insets. The holograms of the planktons are shown in the cropped holographic image at the bottom. (b) Full-scale hologram: Inline holographic microscopes offer a large field of view; in our case, the active region of the sensor has the dimensions . The cropped region in Figure 1a is marked by the dashed box.
Figure 1—figure supplement 2
Simulated holographic images and RU-Net architecture.
(a) Examples of simulated plankton holograms generated by DeepTrack 2.0 (Midtvedt et al., 2021a): Each image () contains plankton holograms generated over a wide range of sizes and refractive indices. The background noise is varied randomly with a Gaussian random noise to match the experimental images. (b) Input–output pairs to train the RU-Net: The input of the RU-Net is a simulated holographic image. The output maps contain the segmentation of the planktons, their -position, their dry mass , and the distances and from the closest plankton for each pixel (to be used for the accurate localization of planktons). (c) RU-Net architecture: The downsampling part consists of a sequence of six convolutional blocks containing a convolutional layer (depicted in brown; the number of convolutional filters is indicated at the bottom of each convolutional layer) followed by an ReLU activation and a max-pooling layer (depicted in purple). In the upsampling path, the RU-Net is divided into two branches. Each upsampling branch contains a sequence of four upsampling blocks, where each upsampling block contains an upsampling layer (depicted in red) followed by a convolutional block (depicted in blown), and an ReLU activation. The final upsampling block used for the plankton segmentation employs a sigmoid activation (depicted in black). The outputs of both upsampling paths (depicted in green) are merged together, after which a final convolutional block is added to predict the five-channel output image. See also Methods, ‘RU-Net architecture and training’.
Figure 1—figure supplement 3
Simulated plankton sequence and weighted-average convolutional neural network (WAC-Net) architecture.
(a) Simulated time sequence of holographic images cropped around an individual plankton: The 15-frame simulated sequence is an example input sequence for training the WAC-Net to estimate the plankton dry mass and radius. The images are centered on the target plankton, while other planktons in the background are considered as noise that the WAC-Net should learn to ignore. (b) WAC-Net architecture: The input image sequence is processed by a time-distributed block consisting of a series of four convolutional blocks. Each convolutional block has a convolutional layer (depicted in brown; the number of convolutional filters is indicated at the bottom of each convolutional layer), followed by an ReLU activation and pooling layer (depicted in purple). The features are flattened (depicted in blue) followed by a series of two dense layers (depicted in red; the number of nodes is indicated at the bottom), which return an output vector (depicted in green) for each image in the sequence ( images). This is followed by two convolutional layers (depicted in yellow; with 128 and 1 filters, respectively), which estimate the weight for each image in the sequence. A weighted average is then obtained for the vectors to extract single value outputs (depicted in green). Finally, the outputs are processed with a series of dense layers to obtain the final result for the dry mass and radius. See also Methods, ‘WAC-Net architecture and training’.
Figure 2 with 1 supplement
Dry mass estimates.
(a) Phytoplankton species Oxyrrhis marina as detected by RU-Net on a portion of experimental hologram (see Figure 1—figure supplement 1 for the complete hologram). (b) Dry mass distributions for O. marina (illustrated in the inset) obtained by applying weighted-average convolutional neural network (WAC-Net) to the experimental holograms (orange) and by volume-to-carbon relationships (gray, Menden-Deuer and Lessard, 2000). The red line is the value of the average mass estimate obtained from elemental analysis. (c) Comparison of the dry mass estimations obtained by WAC-Net and by the volume-to-carbon method for nine different species of diatoms (Thalassiosira pseudonana, Thalassiosira weissflogii), phytoplantons (Isochrysis galbana, Rhodomonas salina, Dunaliella tertiolecta), and micro-zooplanktons (Oxyrrhis marina, Kryptoperidinium triquetrum, Alexandrium minutum, Scrippsiella acuminata). The two measurements have a correlation coefficient of . The dashed line represents the best fit and the error bars show the standard deviations of the distributions. The insets illustrate each species.
Figure 2—figure supplement 1
Dry mass and equivalent spherical radius (ESR) estimates for different species of planktons.
Comparison between dry mass (left panels) and ESR (right panels) estimated from volume-to-carbon relationships (Menden-Deuer and Lessard, 2000) (gray histograms) and dry mass estimated using our method (orange histograms) for nine species of plankton. The plankton species include phytoplankton species (a: D. tertiolecta, b: R. salina, c: I. galbana), micro-zooplankton species (d: O. marina, e: A. minutum, f: S. acuminata, g: K. triquetrum), and diatoms (h: T. weissflogii, i: T. pseudonana).
Figure 3 with 3 supplements
Feeding events.
3D recording of a feeding event where (a) a predator micro-zooplankton (Oxyrrhis marina, orange traces) approaches a prey phytoplankton (Dunaliella tertiolecta, blue traces), (b) feeds on it, and (c) finally moves away (see Video 1 and Figure 3—figure supplement 2). The 2D projection of traces is superimposed on the holographic images in the bottom (see also Figure 3—figure supplement 1). (d) Dry mass time series of predator (orange trace) and prey (blue trace) estimated by weighted-average convolutional neural network (WAC-Net) in the three different phases. (e) The pre-feeding dry mass distributions of the predator Oxyrrhis marina (O. m) and the prey Dunaliella tertiolecta (D. t), and the post-feeding dry mass distribution of predator are represented in the box plots. The dry mass increase between pre- and post-feeding phases of the predator is indicated in the plot. The post-feeding dry mass increment of the predator (O. m) matches the dry mass of the prey (D. t). (f) There is a high correlation () between dry mass increments of predators and dry mass of prey for 26 feeding events. The dashed line represents the best fit.
Figure 3—figure supplement 1
2D projection of traces in a feeding event.
2D recording of a feeding event where (a) a predator micro-zooplankton (Oxyrrhis marina, orange traces) approaches a prey phytoplankton (Dunaliella tertiolecta, blue traces), (b) feeds on it, and (c) finally moves away (see Figure 3a–c). The trajectories show the path of planktons in last 32 frames; the current position is highlighted with a rectangle (orange for predator and blue for prey).
Figure 3—figure supplement 2
Feeding event main.
(a–c) Snapshots of video (Video 1) showcasing the feeding event discussed in Figure 3: 3D motion of the prey (D. tertiolecta, blue) and predator (O. marina, orange) with holographic image reproduced on the bottom of the 3D volume on the left. Their 2D motion at a reconstructed holographic plane is shown on the right. The choice of the reconstructed plane (at from the sensor) ensures a clear view of most planktons. The feeding phase and experimental time are indicated at the top of the video.
Figure 3—figure supplement 3
Feeding event additional.
(a–c) Snapshots of video (Video 2) showcasing an additional feeding event.
Figure 4 with 1 supplement
Growth and cell division of a diatom.
Different life stages of a diatom (Thallasiosira weissflogii) and its daughter cells: (a) the parent cell (blue), (b) divides into two daughter cells (orange); (c) the daughter cells continue to grow, (d, e) until another cell division occurs (green). (f) Dry mass time series through generations estimated by weighted-average convolutional neural network (WAC-Net) (see also Video 3 and Figure 4—figure supplement 1). Each cell dry mass is estimated when it has at least 3.6 μm () of empty space around it to ensure optimal performance of the WAC-Net; the corresponding times are indicated by the gray dashed lines. A drop in the dry mass values can be noticed with the daughter cells in subsequent divisions. (g) Correlation plot showing the relation between the sum of the dry masses of the daughter cells and the dry mass of the parent cell for 11 different division events (). The dashed line represents the best fit.
Figure 4—figure supplement 1
Division event.
(a-c) Snapshots of video (Video 3) showcasing the life history of plankton in Figure 4. The holographic image of a diatom (T. weissflogii) undergoing successive cell division events is shown by the holographic image on the left. The image on the right is the reconstructed image on the plane from the sensor. The generation and experimental time are indicated at the top of the video.
Videos
Video 1
Feeding event 1.
Video 2
Feeding event 2.
Video 3
Division event.
Tables
Table 1
Planktons used in the experiments.
Strain identifier denotes strain code in Gothenburg University Marine Algae Culture Collection (GUMACC) and synonym strain identifier in parenthesis. The Oxyrrhis marina culture was kindly provided by Denmark Technical University (DTU-Aqua) and does not have a strain ID. Equivalent Spherical Diameter (ESD) denotes the spherical diameter based on Coulter counts (Beckman multisizer III) of pure cultures.
| Scientific name | Strain identifier | Class | ESD (mean ± SD) |
|---|---|---|---|
| Alexandrium minutum | GUMACC83 (CCMP113) | Dinophyceae | 18.3 ± 2.5 μm |
| Dunaliella tertiolecta | GUMACC5 | Cholorphceae | 6.7 ± 0.9 μm |
| Isochrysis galbana | GUMACC108 (CCMP1323) | Prymnesiophyceae | 4.0 ± 0.7 μm |
| Kryptoperidinium triquetrum | GUMACC71 (LAC20, KA86) | Dinophyceae | 14.9 ± 1.2 μm |
| Oxyrrhis marina | DTU-Aqua | Dinophyceae | 14.5 ± 1.5 μm |
| Rhodomonas salina | GUMACC126 (CCAP978/27) | Cryptophyceae | 7.4 ± 1.0 μm |
| Scripsiella acuminata | GUMACC110 (CCMP1331) | Dinophyceae | 17.7 ± 4.9 μm |
| Thallassiosira pseudonana | GUMACC132 (CCAP1085/12) | Cosconodiscophyceae | 4.8 ± 0.9 μm |
| Thallassiosira (Conticribra) weissflogii | GUMACC162 (CCAP1085/18) | Cosconodiscophyceae | 12.9 ± 1.4 μm |
Additional files
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Microplankton life histories revealed by holographic microscopy and deep learning
eLife 11:e79760.
https://doi.org/10.7554/eLife.79760
