Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Mar;59(2):211-226.
doi: 10.1177/03009858211067478. Epub 2021 Dec 30.

Computer-assisted mitotic count using a deep learning-based algorithm improves interobserver reproducibility and accuracy

Christof A Bertram  1   2 , Marc Aubreville  3 , Taryn A Donovan  4 , Alexander Bartel  2 , Frauke Wilm  5 , Christian Marzahl  5 , Charles-Antoine Assenmacher  6 , Kathrin Becker  7 , Mark Bennett  8 , Sarah Corner  9 , Brieuc Cossic  10 , Daniela Denk  11 , Martina Dettwiler  12 , Beatriz Garcia Gonzalez  8 , Corinne Gurtner  12 , Ann-Kathrin Haverkamp  7 , Annabelle Heier  13 , Annika Lehmbecker  13 , Sophie Merz  13 , Erica L Noland  9 , Stephanie Plog  8 , Anja Schmidt  13 , Franziska Sebastian  13 , Dodd G Sledge  9 , Rebecca C Smedley  9 , Marco Tecilla  14 , Tuddow Thaiwong  9 , Andrea Fuchs-Baumgartinger  1 , Donald J Meuten  15 , Katharina Breininger  5 , Matti Kiupel  9 , Andreas Maier  5 , Robert Klopfleisch  2
Affiliations

Computer-assisted mitotic count using a deep learning-based algorithm improves interobserver reproducibility and accuracy

Christof A Bertram et al. Vet Pathol. 2022 Mar.

Abstract

The mitotic count (MC) is an important histological parameter for prognostication of malignant neoplasms. However, it has inter- and intraobserver discrepancies due to difficulties in selecting the region of interest (MC-ROI) and in identifying or classifying mitotic figures (MFs). Recent progress in the field of artificial intelligence has allowed the development of high-performance algorithms that may improve standardization of the MC. As algorithmic predictions are not flawless, computer-assisted review by pathologists may ensure reliability. In the present study, we compared partial (MC-ROI preselection) and full (additional visualization of MF candidates and display of algorithmic confidence values) computer-assisted MC analysis to the routine (unaided) MC analysis by 23 pathologists for whole-slide images of 50 canine cutaneous mast cell tumors (ccMCTs). Algorithmic predictions aimed to assist pathologists in detecting mitotic hotspot locations, reducing omission of MFs, and improving classification against imposters. The interobserver consistency for the MC significantly increased with computer assistance (interobserver correlation coefficient, ICC = 0.92) compared to the unaided approach (ICC = 0.70). Classification into prognostic stratifications had a higher accuracy with computer assistance. The algorithmically preselected hotspot MC-ROIs had a consistently higher MCs than the manually selected MC-ROIs. Compared to a ground truth (developed with immunohistochemistry for phosphohistone H3), pathologist performance in detecting individual MF was augmented when using computer assistance (F1-score of 0.68 increased to 0.79) with a reduction in false negatives by 38%. The results of this study demonstrate that computer assistance may lead to more reproducible and accurate MCs in ccMCTs.

Keywords: artificial intelligence; automated image analysis; canine cutaneous mast cell tumors; computer assistance; deep learning; digital pathology; mitotic count; mitotic figures.

PubMed Disclaimer

Conflict of interest statement

Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Figures

Figure 1.
Figure 1.
Overview of the course of the study (stages 1–3) with different degrees of computer assistance (red arrows) of the study participants in 3 examination time points (stages). The deep learning model (concatenation of 2 convolutional neural networks) that was used in this study for computer-assistance was developed in a previous study using an independent training and test dataset with different WSI from the same laboratory that provided the study cases. ccMCT, canine cutaneous mast cell tumors; MC-ROI, mitotic count region of interest; MF, mitotic figure.
Figures 2–3.
Figures 2–3.
Immunohistochemistry-assisted ground truth. Figure 2. Labeling method of the ground truth dataset. The histological sections (hematoxylin and eosin stain, HE) were destained and relabeled with immunohistochemistry against phosphohistone H3 (pHH3). Subsequently, whole-slide images of both staining methods were aligned on the cellular level via automated image registration and combined to decide if a tumor cell has a mitotic figure (MF) or not. Ground truth annotations comprised pHH3-positive cells that were recognizable on HE images (green circles) or were not readily identifiable on HE images (blue circles; especially prophase MF) as well as unambiguous late phase (especially telophase) MF that were pHH3-negative (red circles). Here these patterns are displayed as 3 distinct colors but in the ground truth dataset those structures were labeled as one label class. Figure 3a. High-magnification image of a pHH3-stained tumor section used for creating the ground truth with 3 positive tumor cells (green circles) and a pHH3-negative mitotic figure imposter (arrow). Figure 3b. Histological image (HE stain) of the same tumor location as in Figure 3a with exemplary annotations by one of the study participants (blue circles). Compared to the pHH3-assisted ground truth, 2 annotations are true positives (TP), 1 annotation is a false positive (FP), and 1 annotation MF was missed (false negative, FN, arrow).
Figure 4.
Figure 4.
Scatterplots of the participant’s mitotic count (MC) values (stages 2 and 3) and the algorithmic (unverified) MC compared with the pHH3-assisted ground truth MC (all obtained in the same mitotic hotspot MC-ROI based on the algorithmic heatmap). The black line in the scatterplots indicate equal values for ground truth and pathologists or algorithmic MCs.
Figures 5–8.
Figures 5–8.
Approximate location of the mitotic count region of interest (MC-ROI) selected manually by each study participant (represented by the rectangular boxes) in the whole-slide images. The black box with the dashed line represents the algorithmically preselected MC-ROIs (algorithmic hotspot). The estimated MC heatmap is visualized by variable opacity of a green overlay (scale on the right side of image) on the histological image (hematoxylin and eosin stain) and is based on algorithmic mitotic figure predictions. Dark green areas represent mitotic hotspots. Figure 5. Case no. 33 with widely distributed MC-ROIs. Figure 6. Case no. 46 with widely distributed MC-ROIs. Figure 7. Case no. 38 with MC-ROIs mostly along the tumor periphery. Figure 8. Case no. 5 with similar MC-ROIs at a site of local tumor invasion.
Figures 9–11.
Figures 9–11.
Object detection performance (identification and classification of mitotic figures) of the 23 participants and the deep learning–based algorithm in stages 2 and 3. Figure 9. Micro-averaged F1-score (upper graph), precision (middle graph), and recall (lower graph) with their 95% confidence intervals. The difference is considered significant if the intervals do not overlap. Figure 10. Macro-averaged recall and precision for the individual participants in stages 2 and 3 (connected by a black arrow) and the precision-recall curve for the algorithm (at different classification thresholds). For the algorithmic predictions for the present study, a single classification threshold was used that resulted in a recall of 0.81 and precision of 0.84. Figure 11. Macro-averaged F1-scores for the individual participants for stages 2 and 3. The dashed black line represents the F1-score of the algorithmic predictions.
See this image and copyright information in PMC

References

    1. Al-Janabi S, van Slooten HJ, Visser M, et al. Evaluation of mitotic activity index in breast cancer using whole slide digital images. PLoS One. 2013;8(12):e82576. - PMC - PubMed
    1. Aubreville M, Bertram C, Klopfleisch R, et al. SlideRunner—a tool for massive cell annotations in whole slide images. In: Maier, Deserno A, Handels TMH, eds. Bildverarbeitung Für Die Medizin 2018. Springer; 2018:309–314.
    1. Aubreville M, Bertram CA, Donovan TA, et al. A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research. Sci Data. 2020;7(1):417. - PMC - PubMed
    1. Aubreville M, Bertram CA, Jabari S, et al. Inter-species, inter-tissue domain adaptation for mitotic figure assessment—learning new tricks from old dogs. In: Tolxdorff, Deserno T, Handels TMH, eds. Bildverarbeitung Für Die Medizin 2020: Springer Vieweg; 2020:1–7.
    1. Aubreville M, Bertram CA, Marzahl C, et al. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region. Sci Rep. 2020;10(1):16447. - PMC - PubMed

Publication types

LinkOut - more resources