Introduction

During training, pathologists are taught to recognize microscopic and macroscopic patterns that are established diagnostic features of disease indicative of disease progression. Advancements in biomedical research have led to the discovery of cell biological and genetic drivers underlying pathology, the diagnostic features of which may not be quantifiable by routine visual inspection. In recent years, artificial intelligence and machine learning approaches have demonstrated impressive ability to quantify and classify data derived from histologically stained tissue images, with improvements in speed, objectivity, and stratification, compared with traditional diagnostic routines. Such tools include automated segmentation pipelines and diagnostic workflows designed by digital pathology companies such as PathAI, Visiopharm, Aiforia, and others1,2,3. These approaches may offer a bridge between laboratory findings and clinical practice4,5. To make this translation of computer-based approaches into clinical pathology settings a success, a platform that allows interpretation of the results of computationally derived features within the context of a biological framework is needed6,7.

Existing computational image analysis tools rely heavily on convolutional neural network (CNN)-based approaches, extracting features from acquired images and enabling continuous accretion of knowledge8,9. Whether or how the resulting image features are relevant to disease diagnosis, stage or mechanism is often unclear when using a CNN. By contrast, the selection of hand-engineered image features to quantify the size, shape, color, and texture characteristics (the latter is defined as spatial variation of pixel intensity levels within histological sub-compartments) can be informative data in the evaluation of disease processes10. These features can be analyzed by advanced machine learning tools that have been borrowed from areas other than image analysis, such as text data mining using recurrent neural networks and transformer models11,12. By selectively providing computational models with quantitative values describing histological sub-compartments of interest, the resulting models are more interpretable compared to CNNs that receive raw image data as input11,12,13,14,15. Making computational models accessible to pathologists using a set of interpretable features in an easy-to-use, plug-and-play format will further engage pathologists in computational research, integrating the most robust characteristics of both classical and modern diagnostic approaches and delivering a translational tool for research and clinical applications.

Toward that aim, we have developed HistoLens, an open-source graphical user interface tool to enhance histological image investigation and biomarker discovery. Using an extensive set of 448 hand-engineered features and their associated visualizations, this application enables users to interpret combinations of quantitative image features and visualize the informative regions within images (Supplemental Table 1). As these informative regions are determined using image analysis methods that are closely tied to the calculation of specific hand-engineered features, we can be sure of their consistency and robustness. This visualization for hand-engineered features greatly increases the interoperability of features identified by HistoLens to be of value. Each of these features are designed to be summaries of a particular quantitative characteristic for a sub-compartment within a single structure. We also provide an example for future developers on how they can add their own features to HistoLens both for calculating different characteristics of Periodic Acid Schiff (PAS)-stained histology or for quantifying other types of common histological stains such as Silver (Supplemental Documents 4).

The ability of HistoLens to identify and visualize differences among human glomeruli is demonstrated in two distinct nodular glomerular diseases—diabetic nephropathy (DN) and amyloid nephropathy (AN)—from the histologically unremarkable glomeruli observed in minimal change disease (MCD). Additionally, we demonstrate the ability of HistoLens to characterize subtle glomerular differences in the Tg26 mouse model of HIV-associated nephropathy (HIVAN). In Supplemental Document 4, we provide an additional example of how HistoLens can be used to characterize Silver staining in subjects diagnosed with IgA nephropathy.

Methods

Human data collection followed a research protocol approved by the IRBs at the University of Florida, Gainesville, FL and Johns Hopkins Medical Institutions, Baltimore, MD and University of Michigan, Ann Arbor, MI. Animal studies were performed in accordance with protocols approved in advance by the Institutional Animal Care and Use Committee at the National Institutes of Health, Bethesda, MD.

Software overview

HistoLens is a stand-alone desktop application for hand-engineered image feature analysis, quantification, classification, and visualization for digital pathology images (Fig. 1). HistoLens accepts as input digital histology whole-slide images (WSIs) along with WSI micro-compartmental annotations at varying scales (cell nuclei to larger structures, such as glomeruli, in the context of kidney pathology) in Aperio ImageScope Extensible Markup Language (XML) or JavaScript Object Notation (GeoJSON/JSON) QuPath annotation format. Annotations can be generated manually or through the use of deep learning models implemented in cloud platforms such as Histo-Cloud or FUSION (Functional Unit State Identification for WSIs)16,17. Software applications that support the recording of digital pathology annotations include both open-source applications, such as QuPath, Automated Slide Analysis Platform (ASAP), and commercial applications, including Pathomation (Antwerp Belgium), and Aperio ImageScope (Leica Biosystems, Wetzlar Germany) among others18,19,20. If no annotations are found in the same folder as the WSIs, users can also use the annotation tool included in HistoLens which allows users to interactively annotate multiple different types of structures. The resulting annotations can then be saved in either Aperio ImageScope XML format, GeoJSON format, or Histomics format which is employed in Histo-Cloud.

Figure 1
figure 1

HistoLens workflow. A new experiment using HistoLens is initialized by providing the file paths to a directory of whole slide images (WSIs), together with their associated annotation files and slide metadata. User-provided file paths and a record of analyses are saved into an experiment file that can be quickly loaded, so as to pick up where a previous analysis ended.

Full size image

Computational analysis in HistoLens includes interactive sub-compartment segmentation (extracting biologically relevant sub-compartments in each image; for example, nuclei, eosinophilic regions, and luminal space), color normalization, feature extraction, and feature ranking, all linked to patient level or tissue level labels21. Sub-compartment segmentation is performed using an interactive procedure that minimizes the amount of user parameters required and therefore simplifies the image analysis process. These parameters include colorspace, intensity threshold, minimum object size, and a splitting parameter applied only to overlapping nuclei. Users can also modify sub-compartment segmentation parameters for individual slides if there is variation in staining or imaging conditions within their dataset. If a user is confident that the staining characteristics are consistent across their slides, they may also bypass the remaining slides and project their current segmentation parameters onto subsequent slides prior to feature extraction. The resulting data (sub-compartmental segmentation parameters and their associated quantitative hand-engineered features) can be analyzed using HistoLens for structure classification and interactive visualization for biological discovery.

The main window of HistoLens consists of three major panels (Fig. 2). The first panel presents an annotated microanatomic structure pulled from the WSI, together with any hand-engineered feature visualizations that are displayed as an overlaid heatmap contained within a rectangular region of interest (ROI). The annotated structure in this display can be changed in three ways. Users can (1) select a different annotated image pulled from the WSI using the accompanied annotation file by selecting its label in the list below the displayed image, (2) click the “Next Image” or “Previous Image” buttons, or (3) tap the right or left arrow keys on the keyboard.

Figure 2
figure 2

HistoLens main window with select interactive components highlighted. (A) Shown is the main image viewing area. Here the current microanatomic structure pulled from the whole slide images (WSI) in accordance to the annotation in the provided annotation text file is displayed, as well as the feature visualization (with adjustable transparency) and the relative feature intensity region of interest (ROI). (B) The toolbox, including feature distribution plot, statistical summaries, annotations, and modeling capabilities, is shown. (C) The selected structure’s feature value relative to the whole dataset is indicated in the Feature Distribution Plot. (D) Hand-engineered features are separated into a hierarchy by sub-compartment and feature type. Users are able to either select individual features in the specific feature column for visualization on the right panel (B) or they are able to combine features from sub-compartment and feature type, or by adding select features to a custom list (E). (E) Custom list where user can add features across sub-compartments and categories for specific analysis.

Full size image

The second panel, located under the main visualization window, is a series of lists with titles, “Sub-compartment,” “Feature Type,” “Specific Feature,” and “Custom List.” By selecting an item in the “Sub-compartment” list, users can access the relevant sub-compartments of interest for quantifying associated hand-engineered features and feature visualizations. The “Feature Type” list contains the categories of features (e.g., size, morphology, texture, color) that are included in that sub-compartment. In the “Specific Feature” list, users can make specific image feature selections. Clicking the “Add to Custom” button, located beneath “Custom List,” adds the selected specific feature to the “Custom List” list. Adding features to this list allows users to select combinations of hand-engineered features that are not in the same sub-compartment or feature type. Located next to the four lists are four buttons labeled, “View Sub-Compartment Features,” “View Feature Type,” “View Specific Feature,” and “View Custom List.” Clicking on these buttons controls which features are currently under shown in the visualization panel as discussed next.

The third panel is a toolbox which allows users to generate figures, build classification models, and generate image annotations for downstream analysis. The major tools are separated on tabs labeled, “Feature Distribution Plot,” “Statistical Measures,” “Classification Models,” “Relative Feature Intensity,” “Add Annotations,” and “Feature Definitions.” Details of these tabs are provided below.

Feature distribution plot This tab provides plots of the distribution of the current/selected hand-engineered features across a dataset. Labels on the plot show users the location of the selected image in the main display relative to the rest of the dataset. When a single feature is selected using the “View Specific Feature” button as part of the second panel, the data distribution for that feature is presented as a violin plot in which the points correspond to the annotated structures in the dataset.

When selecting more than one feature using the “View Feature Type,” “View Sub-Compartment Features,” or “View Custom List” buttons, scatterplots are used to show either the differences between two features or between the first and second principal components of the dimensionally reduced feature set. Included in this tab are a second set of tabs, located below the feature distribution axes, for selecting the data that included in the plot. These tabs include the following five options: switching how each point is labeled, selecting regions in the feature space for making qualitative comparisons between sub-clusters displayed in the Feature Distribution Plot, labeling the data scatter according to slide metadata, adding additional labels from an external file or classification model, and removing outliers (with a table displaying the image name removed). Using these tools, trends in the current/selected features can more easily be identified and evaluated.

Statistical measures The second tab provides statistical comparisons, including a summary of descriptive statistics for the current feature (minimum, maximum, mean, median, etc.) identified by current labels in the feature distribution plot. When single features are viewed, a statistical test is implemented to provide users with a quantitative measure of statistical significance. The test implemented depends on the number of classes in the current labels. For two classes, a Student t-test is performed and for three or more classes, a one-way ANOVA is performed22,23. For multiple features, the silhouette index for each class is displayed to quantify data clustering relative to other classes24. Principal component analysis is also provided in the form of percentage variance explained in the first two principal components, together with the coefficients for each feature. To view the individual contributions of specific features on the principal components, users are able to select features for a biplot (a two-variable scatterplot) on the feature distribution plot25. Arrows are used to indicate the direction and magnitude of coefficients for each feature.

Classification models Using the features displayed in the feature distribution plot, a user can implement several classification models, including decision trees, discriminant analysis, a naïve Bayes classifier, and a simple neural network26,27,28,29. The performance of each of these models for a particular image set is displayed using a confusion matrix for categorical labels and a violin plot of error for continuous labels. A performance report is also generated that includes other performance metrics of the model, including accuracy, sensitivity, specificity, F1 score (defined as the harmonic mean between precision and recall, with values ranging from 0 to 1), and the number of structures in each slide that are predicted to be from a certain class30. Saving the model, by clicking the “Save Model” button, generates a new directory in the slide directory containing the model object and the performance report (‘.xlsx’ file). This report also includes the features used to generate the model and predicted labels for each member of the test set. Additional information on the Classification Models workflow is provided in Supplemental Document 1.

Relative feature intensity This tab provides users with a quantitative description of the intensity, measured here as the area of the hand-engineered feature visualization present in the rectangular ROI in the first panel of each feature. For individual features, this plot displays the distribution of intensities contained within the feature visualization for that image. For multiple features, the quantification method instead shows the relative content for each feature in the enclosed region compared to the rest of the image. Additionally, the relative values displayed in the histogram are recorded along with feature name and feature rank in table format. Users can use this tab to locate which regions are influential towards highly ranked features as determined using a chi-square test (default) when new labels are imported through the “Add Labels” tab31.

Adding annotations In this tab, users can define custom classes and annotate more specific regions in the current image. These regions are saved locally as binary masks for each user-specified class where annotated pixels have a value of one and non-annotated, background pixels have a value of zero. The resulting folder structure separates out annotated images from different slides with subfolders corresponding to the name of the annotated class in that mask. These binary masks can be used in downstream deep-learning pipelines for segmenting different types of lesions or specific areas within annotated compartments. This tab also allows users to transfer the current image annotation to another annotation layer in the original annotation file associated with the WSI. This feature can be used to assign structures to a separate group within the study or to create a new structure in the WSIs.

Feature definitions The last tab contains written definitions for the current feature or features under observation as well as a description of how each feature’s visualization was generated (see Supplemental Table 1). A key for interpreting the columns of Supplemental Table 1a is provided in Supplemental Table 1b.

Experimental design

We analyzed the performance of HistoLens to recognize biologically significant characteristics of histological structures in two distinct experiments: one using a human dataset consisting of DN, MCD, and AN cases, and the other using a murine dataset consisting of HIVAN and wild-type (WT) control mice.

Tissue sectioning, staining, & imaging Tissues were sectioned at 2–3 µm thickness for staining and imaging. The tissues were stained primarily with PAS and hematoxylin counterstain. The slides were scanned using a brightfield microscopy WSI scanner, NanoZoomer 2.0-HT (Hamamatsu, Shizuoka, Japan). The pixel resolutions of the images used were 0.26, 0.25, and 0.51 µm/pixel.

Human DN, MCD, and AN In the first experiment, we examined whether and how HistoLens distinguishes features between nodular sclerotic glomerular pathologies (DN and AN) and histologically normal glomeruli (MCD). This particular comparison was selected as the visual appearances of glomeruli in DN and AN are readily distinguishable from MCD using PAS staining visualized by light microscopy (Fig. 3)32,33. Additionally, there already exists a well-described general set of distinct diagnostic features that identifies the presence of DN and AN33. The primary feature of significance is the extent and quality of the glomerular mesangium. For glomeruli manifesting nodular diabetic glomerulosclerosis, nodular mesangial matrix expansion is apparent, in extreme cases forming Kimmelstiel–Wilson nodules33. In contrast, amyloid glomerulopathy, also a nodular pattern, is characterized by deposition of non-matrix material and along capillary walls34. By light microscopy, amyloid is generally PAS-weak compared to strong PAS-positivity in nodular diabetic glomerulopathy.

Figure 3
figure 3

Respective nine example glomeruli from cases of (A) MCD, (B) DN, and (C) AN. (D) Arrows highlighting regions of nodular glomerulosclerosis in DN (PAS-strong) and amyloid nephropathy (PAS-weak) that are not present in MCD. Scale bar indicates 50 µm.

Full size image

In HistoLens, the distinguishing features among DN, AN, and MCD can be grouped into two categories, mesangial size features and mesangial color features. For each of these categories, several metrics can be quantified to identify more detailed and objective differences among the samples. These features include measures of PAS + area (measured using number of pixels contained within segmented PAS + regions) and PAS + region color statistics (mean and standard deviation of red, green, and blue [RGB] values).

Human lupus nephritis (LN) and IgA nephropathy To demonstrate the ability of HistoLens to extend to other diseases and stain types, we further examined two additional datasets and include that abridged analysis in Supplemental Documents 3 and 4. The first dataset contained PAS-stained slides from patients with Lupus Nephritis (LN) who exhibited different responses to treatment (Complete Response (CR) and No Response (NR)). The next dataset consisted of Silver-stained slides from patients with IgA Nephropathy (IgAN), where HistoLens was employed to quantify differences in silver deposition between patients.

Murine HIV-associated nephropathy (HIVAN) model Next we used HistoLens to characterize differences between diseased and WT groups in a mouse study set. The mouse cohort was an HIVAN model. In this model, Tg26 mice from the FVB/N strain contain a transgene with a gag-pol-deleted HIV-1 genome and manifest a sclerosing glomerulopathy (Fig. 4)35. We sought to test whether quantitative assessment and visualization in HistoLens would be able to achieve a sufficiently granular analysis of the effects of experimental factors on tissue compartments. See Supplemental Table 2 for dataset composition details.

Figure 4
figure 4

Example glomeruli from the HIV-associated nephropathy (HIVAN) dataset. (A) Nine example glomeruli taken from wild-type mice. (B) Nine example glomeruli taken from HIVAN mice. (C) Example of global and focal glomerulosclerosis in HIVAN mice compared with a wild-type mouse glomerulus. Scale bar indicates 50 µm.

Full size image

Sources for other quantitative methods applied in this work, including references and MATLAB implementation documentation, can be found in Supplemental Table 3.

Approval for animal experiments

All animal studies were performed in accordance with protocols approved by the Laboratory of Animal Science Section (LASS) at NIH NIDDK, and are consistent with federal guidelines and regulations and in accordance with recommendations of the American Veterinary Medical Association guidelines on euthanasia. This study is reported in accordance with ARRIVE guidelines.

Approval for human experiments

Human data collection followed a protocol approved by the Institutional Review Board at Johns Hopkins, University of Florida, and University of Michigan prior to commencement. All methods were performed in accordance with the relevant federal guidelines and regulations. Participants were required to be over 18 years of age, and with a diagnosis of chronic kidney disease that required renal biopsy. Special populations (vulnerable) such as minors, pregnant women, neonates, prisoners, children, and cognitively impaired patients were not included. All patients provided written informed consent.

Results

HistoLens performance was tested using human and murine model data; details of the experiments are discussed under Experimental Design in the Methods section. A description of the datasets used—including sources, disease pathology, tissue thickness, staining, and image acquisition—are summarized in Supplemental Table 2.

Differentiating human DN, MCD, and AN

We first examined variations in PAS-stained renal glomerular mesangial color intensity among three human diseases: MCD, DN, and AN. Hand-engineered features that measure the color properties of the mesangial area include the mean and standard deviation of RGB intensities assessed on PAS-stained sections. Across this dataset, we found the standard deviation of blue intensities to be the most significant PAS + color feature distinguishing AN and DN from MCD (Fig. 5) (ANOVA, p = 5.74*10−27; MCD vs DN, p = 0.35; MCD vs AN, p = 3.47*10−23; and DN vs AN, p = 1.43*10−15). In general, glomeruli from AN cases had a lower standard deviation in blue intensities than MCD. A similar trend was also observed for red and green values. AN and DN glomeruli generally had a more homogenous distribution of color intensities in mesangial pixels, a finding likely driven by increased mesangial area, consistent with the known acellular/pauci-cellular mesangial expansion in these entities33.

Figure 5
figure 5

Standard deviation of PAS + blue intensities in MCD, DN, and AN. (A) Violin plot of the standard deviation of blue intensities in PAS + regions. * indicates p-value < 0.05. (B) Example images taken from each distribution. In all three groups, glomeruli with a larger standard deviation of PAS + blue values have more variation in glomerular tuft architecture, including more capillary lumens (DN 2) or more nuclei (AN 2) compared to glomeruli with a lower standard deviation of PAS + blue values. Scale bar indicates 50 µm.

Full size image

We next examined features relevant to mesangial expansion, which HistoLens quantifies using the distance transform. The distance transform allows us to efficiently calculate the distance between pixels from adjacent sub-compartments (e.g., nuclei or luminal space). Instead of having a single value for total mesangial expansion present in a particular glomerulus, we then have a value for each pixel within the PAS + sub-compartment quantifying expansion at a particular point. These PAS + distance transform values are then binned into pixel counts containing pixel distance transforms within certain ranges. Using these subdivisions, we quantitatively mapped the cut-off point at which there is a large difference between MCD, DN, and AN glomeruli (Fig. 6). At smaller PAS + distance transform ranges, we anticipated that the glomerular area spanning those values would be similar among the three diseases, relative to the PAS + areas of each glomerulus. Of the slides in this dataset, only a few glomeruli contained greatly expanded mesangium. These glomeruli were found predominantly in the DN and AN groups. We observed that the statistical significance of PAS + distance transform features increased for larger values, indicating that these features are increasingly significant in the presence of higher degrees of mesangial expansion.

Figure 6
figure 6

PAS + sub-compartment distance transform values to delineate relative mesangial expansion. (A) Plot indicating difference in number of pixels with PAS + distance transform within a certain range. Statistical significance is indicated on the right axis, where the log p-values decrease with increasing levels of PAS + thickness. . (B) Example image of PAS + distance transforms for a glomerulus from each class. Color bar indicates value of distance transform at specific point. Distance transform values below 8 pixels are set to transparent. Using quantitative measures output by HistoLens, quantitative determinations of relative mesangial expansion were made. Scale bar indicates 50 µm.

Full size image

There were noticeable differences between DN and AN versus MCD (as expected) but also between DN and AN (two sclerosing diseases). DN samples generally had higher mesangial pixel counts compared to the other two diseases, with MCD having the lowest values. This relationship is illustrated in Fig. 6B, where the PAS + distance transform values greater than eight pixels are delineated using respective example images from each histologic class. The areas on each image that are highlighted in red correspond to the greatest values of the PAS + distance transform, overlapping with areas of nodular mesangial sclerosis. In the sub-classification of DN, these large, paucicellular/acellular sclerotic nodules are referred to as Kimmelstiel-Wilson nodules and have been used to delineate the transition from Tervaert stage IIb to III33.

After confirming histopathologic features that are already known to be clinically relevant, we expanded our search to uncover digitally encoded image biomarkers that could assist in making classification decisions between one or multiple classes. These additional features include quantitative characteristics that may be difficult for a human to appreciate in an image but can be assessed with digital feature quantification. A prime example of such diagnostic features are texture properties.

With a computational aid like HistoLens, the measurement and tabulation of texture features can be accomplished with the click of a button. This way, users can concentrate on the task of interpretation if the newly derived digital image feature is found to be statistically significant. Texture features are especially relevant in the comparison between sclerotic glomerular pathologies like DN, AN, and histologically unremarkable MCD. To distinguish a region of sclerotic from healthy mesangial matrix, pathologists look for areas with an increased distance between cell nuclei, with the space occupied by uniformly PAS + material. Figure 7 shows an example quantification of PAS + energy between MCD, DN, and AN with p values < 0.05 (ANOVA, p = 6.15*10−27; MCD vs DN, p = 5.80*10−24; MCD vs AN p = 5.79*10−18; and DN vs AN, p = 1.20*10−3). In the context of texture quantification, energy refers to the sum of squared elements in the gray-level co-occurrence matrix and is also known as the angular second moment or uniformity. From a qualitative perspective, we observed that glomeruli that featured a higher textural energy had a greater number of capillary lumens present and generally lacked sclerotic nodules. Figure 7 highlights image regions where the textural energy is highest in each glomerulus. From this visualization, it can be determined that glomeruli from MCD cases have a generally higher, uniformly distributed textural energy compared to glomeruli from DN or AN cases.

Figure 7
figure 7

PAS + energy is an objective measure of mesangial texture. (A) Violin plot of PAS + energy values. * indicates a p-value less than 0.05. (B) Example glomeruli from each distribution with corresponding feature visualization focusing on nodular sclerotic regions with a higher energy value than surrounding regions in each image. Using HistoLens, the quantitative distinction between MCD glomeruli and both DN and AN glomeruli is appreciable through statistical analysis, example image extraction, and feature visualization. Scale bar indicates 50 µm.

Full size image

Interactive glomerular sub-classification

We next demonstrate a pathologist’s use of HistoLens to aid in glomerular sub-classification for specific lesions using established quantitative definitions. For this analysis, we focused on nodular glomerulosclerosis in the DN, MCD, and AN study set. Nodular glomerulosclerosis can be identified by localized regions of mesangial expansion, and exists in both DN and AN, among other diseases36,37,38. Without HistoLens, annotation of nodular glomerulosclerosis involves searching through all WSIs or narrowing down the search according to some other clinical metric. This process is error-prone and inefficient, especially when multiple annotators are involved, and requires significant coordination and consensus. HistoLens, in contrast, enables users to define specific quantitative criteria for annotation and facilitates comparisons between multiple users’ labels for agreement.

As a first step, we identified some key histologic features of glomeruli with nodular sclerosis. Because the primary identifying feature of nodularity is mesangial expansion, we initially selected hand-engineered features relating to PAS + region size. For example, the distribution of the number of pixels with a PAS + distance transform value between 13 and 15 pixels exhibited vast differentiation between MCD glomeruli compared with those from patients with DN and AN (Fig. 8A). After reviewing 50 glomerular images, we had annotations for nodular, non-nodular, and globally sclerotic glomeruli (see the red box in Fig. 8A). All glomeruli from the DN and AN study sets found inside of the red box were deemed to contain nodular sclerosis, with visual confirmation by our pathologist collaborator who pulled the images using HistoLens (see Fig. 8B and C). All glomeruli from the MCD cohort were deemed non-nodular. Next, HistoLens was used to increase the number of features used to represent each glomerulus using principal component analysis. The Feature Distribution Plot in HistoLens was used to identify probable nodular glomeruli near already-labeled nodular glomeruli (Fig. 8D), while actively confirming the labels of these new glomeruli, as determined by our pathologist. The proximity of points in this dimensionally reduced feature space indicated similarity in appearance.

Figure 8
figure 8

Glomerular sub-classification using HistoLens via active collaboration between HistoLens and an expert pathologist. (A) Initial feature selection and manual annotation of selected glomeruli. (B) Feature visualization showing regions of image included in calculating feature value. (C) Screenshot of interactive session between pathologist and first author. (D) Using relative positioning of labeled examples for bulk annotation. (E) Performance of multilayer perceptron classifier on validation set of labeled images. The trained model was applied to unlabeled glomeruli for labeling. (F) Merging disease and nodularity labels to examine differential appearances of nodular glomeruli in DN and AN. This study highlights how HistoLens facilitates slide-level annotations to dataset-level observations and hypothesis testing. Scale bars indicate 50 µm.

Full size image

With a larger set of labeled glomeruli, we designed and trained a simple multilayer perceptron in HistoLens using all hand-engineered features and achieved 0.94 and 0.93 F1-score and accuracy measure on labeled samples in a holdout set of glomeruli (Fig. 8E)29s. The holdout set in this case comprised a random sampling of 20% of the original glomeruli. This classifier was then used to label the remaining unlabeled glomeruli. Finally, with the resulting glomerular sub-classification labels achieved with the aid of HistoLens, we analyzed the distinctions between glomeruli presenting with nodular glomerulosclerosis across disease types. One feature we found to be distinctive between nodular glomeruli in DN and AN was the number of nuclei. The difference in nuclei number is related to the pathological mechanism behind nodularity in both DN and AN, where in AN the nodular appearance is caused by amyloid protein deposits in the glomerular tuft and in DN the nodularity is caused by the increased cellularity of aggressively expanding mesangium38,39. This feature distinction is shown in Fig. 8F, which depicts the histopathological differences between respective glomeruli.

HIV-associated nephropathy mouse model

In the previous use-case experiments, we primarily focused on capturing the pervasive, systemic, and diffuse characteristics of each disease type across all glomeruli from each slide. In the Tg26 HIVAN model, however, glomerular pathological changes are known to be focal, with only select glomeruli depicting changes corresponding to focal segmental glomerulosclerosis (FSGS) lesions in human HIVAN, on occasion forming collapsing lesions40,41. Because Tg26 mice can have many normal-appearing glomeruli, the presence of outliers is sufficient to denote a disease phenotype. In our assessment, we focused on the PAS + sub-compartment features as these features appear to exhibit the greatest degree of spread between the Tg26 and WT genotypes (Fig. 9). Clustering of only the PAS + features from Tg26 and WT glomeruli was quantified using silhouette score and is reported in the “Statistical Measures” tab of HistoLens (WT = 0.522; Tg26 = − 0.451). Silhouette score, a metric used to quantify intra-cluster similarity and inter-cluster variability, ranges between − 1 and + 1 where a high positive value indicates that members of that specific cluster are both very similar to each other and very different from members of other clusters.24 The large negative value calculated for Tg26 glomeruli is indicative of the greater intra-class variability compared with WT glomeruli.

Figure 9
figure 9

Scatter plot of principal component analysis (PCA) dimensionally reduced PAS + features to differentiate Tg26 HIV-associated nephropathy (HIVAN) vs wild-type (WT) glomeruli. Relative clustering between WT and HIVAN glomeruli indicates the high degree of variation present in HIVAN mice, with the greatest outliers manifesting the most severe focal lesions. Scale bars indicate 50 µm.

Full size image

In this use-case study, the quantitative influence of differences in staining color and image brightness was particularly pronounced. Looking at the principal components plot for all features related to PAS + color resulted in strong clustering behavior according to the slide from which each glomerulus originated. Without the level of interactivity that is available in HistoLens, such dependence on small deviations in stain coloration may be mistaken as being derived from differences in pathology. Color normalization, using the method described by Macenko et al.21 was used in primary experiments to circumvent significant staining differences. However, we noticed that after applying color normalization images took on an artificially muted appearance that suppressed other observations of mesangial texture. This observation was also noted in the Human MCD, DN, and AN datasets where a color space shift is biologically linked to the binding of proteoglycans to amyloid fibrils and is a primary distinguishing feature between DN and AN sclerotic nodules42,43. In this particular work, we are utilizing highly specific quantitative measures of color from both diseased and control samples. Further exploration of how to handle color and stain variation in the presence of disease for the purpose of quantitative feature extraction and comparison is needed. Although, in general, we observed the Tg26 mice to exhibit lighter PAS + and nuclear staining compared to WT mice, some slides exhibited globally abnormal staining intensity, which would be considered outliers within both groups. Due to the previously identified dependence on slide-level staining properties, we focused our analysis of this dataset on morphological features.

These findings suggest that HistoLens can aid in determining whether mouse models can mimic similar levels of human phenotypic changes for various disease types, while also informing users on the presence of batch artifacts and mitigating such issues by focusing quantitative analysis on morphological features during the validation process.

Comparison of HistoLens with existing platforms

A variety of open-source platforms have been developed in recent years to aid in the analysis of digital pathology WSIs. Among these, ImageJ (v.2.1.4.0), CellProfiler (v.4.2.5), and QuPath (v.0.4.3) are the most widely used.6,7,8,9 Like HistoLens, each of these stand-alone desktop applications allows various users with unique levels of computational expertise to apply advanced image analysis approaches to complex biological data. These applications facilitate slide annotation, feature extraction, experiment tracking, and some machine learning functionality.

HistoLens differs from the aforementioned applications because it implements extensive high-dimensional data visualization, including feature distribution plots of hand-engineered image features. HistoLens also allows for classification of structures and clustering analysis with user-provided metadata and is optimized to identify disease-related changes in histological datasets. This focus on structural sub-categorization and slide-level classification allows researchers to obtain insights into their annotated datasets. QuPath includes many useful operations for efficiently annotating images as well as some built-in segmentation; but it does not allow users to extensively quantify sub-compartments within segmented regions without additional plugin development that is not accessible to a non-technical user. CellProfiler does allow for very specific quantification of nuclei within images; however, it is optimized for cell-specific segmentation and not tissue structures as an aggregate. Furthermore, the hand-engineered features calculated in HistoLens describe more pathologically relevant properties of tissue sub-compartments without relying on highly granular measures of image texture.

Discussion

HistoLens is a desktop application that enables pathology investigators to efficiently obtain a finely detailed, quantitative assessment of the structural characteristics of large histologic datasets. In addition to providing standard tools for data visualization, HistoLens offers several features that are designed specifically for histologic WSI data. These include the ability to pull representative images directly from annotated regions within WSIs and provide visual context for each of the 448 features that can be measured in HistoLens. In this work, we demonstrate the ability of HistoLens to validate known discriminative features in human kidney biopsies. Quantitative features that are used in existing clinical diagnostics were assessed and novel features were identified using computational image analysis techniques. These novel features include quantitative measures of texture, packing distribution of certain sub-compartments within object boundaries, and measurements of width and color. Visual evidence for these features was provided using visualizations of hand-engineered features, delivering localization for complex quantitative features.

Through the use of these specific and objective measures of biological structures, existing clinical diagnostic criteria can be updated to incorporate modern computational advances. Although advanced modeling techniques such as convolutional neural networks have proven adept at tasks like image classification, interpreting what features these models identify as important is a difficult task that is vulnerable to factors such as training set biases and the presence of noise (biologically irrelevant information). With these issues in mind, HistoLens is presented as a connection between the existing knowledge of biological relationships and complex computational methods. The ability to return to the image and validate preliminary observations is a critical function in computational pathology and implementing this function was a key design consideration in HistoLens.

Although this manuscript focuses on the analysis of renal histology, the same analytical procedures may be applied to stained histology tissues from any organ. By selecting the type of stain applied to WSIs while processing a dataset, users can interactively segment sub-compartments and design features that quantify characteristics of the structures that they are interested in. In Supplemental Document 2 an example is given for a mouse study that uses an alternative nuclear stain to localize a biologically meaningful cell type and in Supplemental Document 4 this process is extended to the design of custom features which quantify the distribution of Silver stain in glomeruli of IgA nephropathy patients44. In Supplemental Document 3, we performed a preliminary analysis of LN cases with patients who responded to treatment and those who did not. Further extension of this tool to include other histological stains and organ-specific workflows will be addressed in future releases. Additionally, we will investigate the feasibility of deploying HistoLens as a cloud-based web application to reach more users without requiring download of specific releases of HistoLens. Initial deployment of HistoLens is limited to stand-alone desktop applications (for Windows, Mac, and Linux operating systems) to allow users to leverage local data without sharing externally, which can be beneficial if sensitive protected health information is used.

HistoLens has limitations. This desktop application presumes that structures are already annotated prior to sub-compartment segmentation and feature extraction. Future iterations of the software may extend functionality to other tissues, particularly those manifesting progressive fibrotic diseases. Annotation on a WSI is a very complex procedure from a tool-design perspective. As mentioned previously, several companies have already devoted significant resources towards the design and deployment of annotation software both in a web browser and in other stand-alone tools. In order to best serve pathologist users who have put significant time into annotation on a particular platform and computational scientists whose algorithms output specific formats of annotations, HistoLens is designed to be receptive to commonly used annotation formats as a secondary tool focused on quantitative feature extraction and analysis. Therefore, some features which are included in other software may not be present in HistoLens’ own WSI annotation tool. Nevertheless, the included WSI annotation tool in HistoLens elevates this tool to an all-in-one for interactive analysis. More details on annotating WSIs in HistoLens can be found starting on page 6 of the included User Manual. Expansion of this functionality as well as the integration of automated segmentation algorithms will be revisited in future releases as more feedback from users is collected.

Increasing the accessibility of quantitative techniques to individuals working in human and animal pathology is critical to fostering collaborative engagement among researchers from different subspecialties and backgrounds. HistoLens empowers clinicians and researchers to answer various questions, facilitating more objective, efficient, and effective workflows into clinical and research pathology.