Background

Hospital information systems (HIS) have evolved from closed and proprietary-linked systems into open and standards-based. This change has encouraged medical equipment vendors and software developers to create uniform and interoperable systems. Standardized interfaces let developers link different internal department information systems together: for example, connecting the Laboratory Information System (LIS) of a clinical chemistry or a pathology department with the Radiology Information System in a radiology department. Linked data repositories make all patient-related material (e.g., clinical history and images from different modalities) available to all institution personnel. For instance, pathologists and radiologists can view breast ultrasound and X-ray images simultaneously with corresponding histological specimens.

The most widely used medical imaging standard is the Digital Imaging and Communications in Medicine (DICOM)1, which is routinely used in several medical specialties, especially radiology. In pathology, digitization of whole microscope specimens has only recently become possible with high-throughput slide scanners2. The digitized versions of microscope glass slides are called “virtual slides” or “whole-slide images” (WSIs). Acquiring, handling, and displaying WSIs is commonly called “virtual microscopy” (or whole-slide imaging)3,4. WSIs can be used for local viewing or, more practically, for remote viewing by transmitting them over networks5. Within the internal network of a hospital or pathology department, personnel can use WSIs in case meetings, slide seminars, and instructional live-audience presentations6. By allowing access over the Internet, WSIs can also be used more widely in second-opinion consultations, national and international conferences, and inter-laboratory quality assurance programs7.

Since microscope specimens are often up to 20 × 30 mm in size, a WSI can contain up to 40 GB of uncompressed image data (with a scanning resolution 0.2–0.5 μm per pixel)2. The amount of data increases further if scanning is done at a higher optical magnification and/or if several focus layers (along Z-axis) are scanned (e.g., in cytopathology)8. Due to the large size of WSIs, all viewing systems described to date apply the “on-demand” principle: that is, only a user-requested area (with a desired resolution) of the WSI is decoded and displayed. Moreover, the large image size necessitates the use of lossy image compression. Lossy compression can yield a 10- to 30-fold compression ratio compared to lossless compression, without affecting the diagnostic properties of a WSI9. Thus, a suitable image format for virtual microscopy needs to be based on an effective image compression algorithm, as well as to provide a sophisticated random access technique.

Virtual microscopy currently lacks a universally accepted WSI format. There are several proprietary image formats that are tied to specific scanner vendors, such as SVS (by Aperio Technologies, USA), NDP (by Hamamatsu Photonics, Japan), and Mirax (by Carl Zeiss MicroImaging, USA). The interoperability between different vendor formats and viewing software is practically non-existent. We have previously shown that the open JPEG2000 standard is a suitable format for WSIs, allowing fast random slide access and efficient lossy compression10. Although JPEG2000 compression is computationally intensive, the process can be matched with current slide scanner speeds by utilizing multi-core processor environments10. JPEG2000 is a family of standards supervised by the Joint Photographic Experts Group standardization committee11,12. The standard family currently consists of 13 parts, three of which are essential for virtual microscopy. Part 1 (Core Coding System)13 specifies the code-stream syntax and the JP2 file format, which uses “jp2” as the common file extension. Part 2 (Extensions)14 provides extensions for the first part. Part 9 (Interactivity Tools, APIs, and Protocols)15 introduces the JPEG2000 Interactive Protocol (JPIP) for remote serving and viewing of JPEG2000 images. We have previously developed and released a free JPEG2000 software package (called JVS, for JPEG2000 Virtual Slide) comprising WSI compression, viewing, and network server applications10. Della Mea et al.16 have presented a survey of currently available JPEG2000 viewing software.

For clinical diagnostic use, WSIs must be compatible with existing imaging standards, such as DICOM17. Although a DICOM Object definition for visible light microscopy exists18, WSIs are too large (both pixel dimensions and byte size) to be used directly as these objects. A base standard Correction Item (CP 896)19 to overcome the image dimension limitation was recently rejected by the DICOM Standards Committee20. Nevertheless, the committee will continue to consider the possibility of introducing this change to the standard as a supplement. In addition to image size, the access characteristics of WSIs differ from conventional DICOM images. Panning and zooming within a huge WSI require fast random access with on-demand decoding. Currently, the DICOM standard includes the basic parts of the JPEG2000 standard in Supplements 6121 and 10522. Supplement 106 (JPEG 2000 Interactive Protocol)23 describes two JPIP-based Transfer Syntaxes as methods of delivering image pixel data apart from patient data: the noncompressed JPIP Referenced Transfer Syntax and the Deflate-compressed24 JPIP Referenced Deflate Transfer Syntax. Thus, the DICOM standard specification already contains necessary elements for transmitting WSIs over JPIP.

When using the JPIP Transfer Syntaxes in a DICOM-based Picture Archiving and Communication System (PACS), a DICOM server sends its client a Uniform Resource Locator (URL) string that refers to the WSI pixel data provider (i.e., a JPIP server), together with the image name, which can be arbitrary and unrelated to patient data (as shown in Fig. 1). Upon receiving the pixel data provider reference, the client DICOM workstation can either use a built-in JPIP viewer or invoke an external one for retrieving the WSI from the specified JPIP server. All network messaging between the PACS and the client end is done according to the DICOM protocol, except the JPIP transmission, which is by default performed on top of the Hypertext Transfer Protocol (HTTP/1.1)25 for compatibility with existing Web infrastructure, but it can also be done using a lower-level transport protocol (such as Transmission Control Protocol, TCP)26. Image serving performance of JPIP has been demonstrated to be excellent and upwards scalable in multi-client systems10,27.

Fig 1
figure 1

The principle of transmitting whole-slide images (WSIs) within a DICOM-based PACS by using JPEG2000 Interactive Protocol (JPIP).

Full size image

Although JPIP is described in the DICOM standard specification and a brief description of a commercial application exists28, we are not aware of any open DICOM software solutions or libraries supporting it. In this study, we explore the feasibility of linking JPEG2000 WSIs with a DICOM-based PACS by using JPIP. First, we modified an open-source DICOM library by adding support for JPIP as described in the Supplement 106. Second, the modified library was used as a basis for a software package (JVSdicom), which provides a proof-of-concept for a DICOM client-server system that can transmit patient data, conventional DICOM imagery, and JPEG2000 WSIs over JPIP. The software package consists of a compression application (JVSdicom Compressor) for producing DICOM-compatible JPEG2000 WSIs, a DICOM PACS server application (JVSdicom Server), and a DICOM PACS client application (JVSdicom Workstation). Finally, we present a DICOM-compatible whole-slide imaging system based on the software developed.

Materials and Methods

An open-source DICOM library (OFFIS DCMTK DICOM toolkit, version 3.5.4)29 was modified by adding support for JPIP, as described in the DICOM Supplement 106. The modified library was used to embed DICOM functionality to the developed JVSdicom software package. JVSdicom Compressor is based on our previously described JPEG2000 compression application (JVScomp, version 2.1)10. JVSdicom Server and Workstation utilize the Qt open-source software framework (version 4.3)30 for core application functionality and the Tango Icon Library31 for graphical user interface elements. The software package was written in C++ programming language and built for 32-bit Windows® platforms but can be run under a 64-bit Windows® platform as well.

Reference material representing typical diagnostic imagery of breast cancer was obtained from Tampere University Hospital. The material comprises radiology imagery (ultrasound, mammography, bone scan, and magnetic resonance imaging) and histological specimen slides (routine H&E stains and immunohistochemistry). The standard-sized slides (75 × 25 mm) were scanned with Aperio ScanScope® XT (Aperio Technologies, USA) using uncompressed BigTIFF32 as the primary output format. Whole-mount section slides (75 × 50 mm) were acquired with a Zeiss Axioskop40 microscope (Carl Zeiss MicroImaging, USA) as described equipped with a charge-coupled device color camera (QICAM Fast; QImaging, Canada) and a motorized specimen stage (Märzhäuser Wetzlar GmbH, Germany). The automated image acquisition was controlled by the Surveyor imaging system (Objective Imaging, UK) using uncompressed bitmap as the primary output format. The developed JVSdicom Compressor application was used to convert the histological reference material into the DICOM-compatible JPEG2000 WSI format.

Results

The JVSdicom software package was developed as a proof-of-concept for a DICOM client–server system that can transmit patient data, conventional DICOM imagery (e.g., radiological), and JPEG2000 WSIs using the JPIP Referenced Transfer Syntax. The software package consists of a compression application (JVSdicom Compressor), a DICOM PACS server application (JVSdicom Server), and a DICOM PACS client application (JVSdicom Workstation).

JVSdicom Compressor is a free, command line-based image compression application capable of converting multiple image formats (e.g., BMP, PPM, and BigTIFF) into the DICOM-compatible JPEG2000 WSI format. The output JPEG2000 file follows the optimized code-stream parameterization we have previously described10. In addition to the JPEG2000 WSI file, the application generates a supplementary DICOM file containing medical information, image properties, and a JPIP server reference to the WSI file location, which is used by the DICOM client to access the WSI. The resulting DICOM file uses the General Microscopy modality18 and Visible Light Microscopic Image Information Object Definition (IOD)18 with a minimal set of required attributes.

JVSdicom Server is an open-source DICOM PACS server application that acts as a Storage Service Class Provider (SCP) and as a Query/Retrieve SCP (Table 1). The server is capable of accepting multiple associations simultaneously. Server can be configured to contain several filesystem-based storage areas with different Application Entity (AE) Titles, as well as to limit access to these areas from a predefined AE network (Fig. 2). Alternatively, the server features a public mode, which can be used to grant open access to the server. For open access, the calling AE is assumed to have a receiving Storage SCP set up. New DICOM entries can be added into the server storage with pixel data either coming from existing image files or replaced with a JPIP reference (i.e., in case of JPEG2000 WSIs). JVSdicom Server supports several Storage Service Object Pair (SOP) Classes. A comprehensive conformance statement appears in the software documentation.

Fig 2
figure 2

A screenshot of JVSdicom Server storage management view. DICOM image objects can be added to the storage by importing existing image files or by creating a new JPIP-referenced object.

Full size image
Table 1 Main features of JVSdicom Server
Full size table

JVSdicom Workstation is an open-source DICOM PACS client application that acts as a Query/Retrieve Service Class User (SCU) and a Storage SCU (Table 2). With it, users can query and retrieve images from a DICOM-compatible PACS server (Fig. 3). Users can view the images with several image enhancement options, as well as view a summary of patient- and treatment-related information. JVSdicom Workstation interacts with a DICOM server as a conventional DICOM client, but upon receiving a JPIP reference to a JPEG2000 WSI, it invokes an external JPEG2000 viewing application. The external viewing application displays the image pixel data, while JVSdicom displays the associated DICOM medical information. Thus, by having an external viewer for WSIs, users can view conventional DICOM imagery and corresponding histopathologic specimens side by side (Fig. 4). The external viewing application can be chosen by the user. The default viewer is JVSview10. Regions of interests from the DICOM images and WSIs can both be opened in a public domain image analysis software, ImageJ33, which features a multitude of analysis tools for medical imaging. JVSdicom Workstation is compatible with commercial-grade DICOM servers, supporting several Storage SOP Classes (a comprehensive conformance statement appears in the software documentation).

Fig 3
figure 3

A screenshot of JVSdicom Workstation showing the PACS query view. Conventional DICOM imagery is displayed in the thumbnail window, while JPEG2000 WSIs are opened in an external JPIP viewer.

Full size image
Fig 4
figure 4

Viewing of a breast cancer specimen X-ray and corresponding histological WSI (whole-mount section) side by side with JVSdicom Workstation and an external JPIP viewer. Within JVSdicom, users can rotate the image, adjust width and center values, and measure distances by using the ruler tool. The JPIP viewer displays the WSI using an overview and a main navigation window.

Full size image
Table 2 Main features of JVSdicom Workstation
Full size table

The JVSdicom software package is fully DICOM-compliant and designed to run on Windows® XP but is also Windows Vista™ compatible. The binaries and source code, as well as a comprehensive conformance statement, are available on our Web site (http://jvsmicroscope.uta.fi/). The Web site also features a public JVSdicom server, containing example images produced in the diagnostics of breast cancer.

Discussion

We developed the JVSdicom software package to study the feasibility of linking JPEG2000 WSIs with a DICOM-based PACS by using JPIP. JVSdicom Compressor is used for creating DICOM-compatible JPEG2000 WSIs, which consist of the actual JPEG2000 image file and a supplementary DICOM file containing patient-related information. The supplementary DICOM file can be served using the JVSdicom Server, while the JPEG2000 image files are served using a separate JPIP server. Querying and retrieving images from the JVSdicom Server can be done using the JVSdicom Workstation, which handles conventional DICOM imagery directly but uses an external JPIP client application for JPEG2000 WSI viewing.

A complete, DICOM-compatible whole-slide imaging system can be constructed by combining JVSdicom with our previously described JPEG2000 viewing application (JVSview) and JPIP network serving application (JVSserv; Fig. 5). In this model system, a WSI scanner produces raw image data, which is processed by JVSdicom Compressor. JVSdicom Compressor produces a JPEG2000 file containing the actual WSI image data and a DICOM file containing the associated medical data (i.e., patient information) as well as some mandatory image properties, such as width and height. By default, the produced DICOM file contains anonymized DICOM entries, but it could be linked with a LIS or a HIS for retrieving patient information. A straightforward way to name the JPEG2000 WSI file is to use the microscope slide label identification string, which can be read automatically if bar-coded labels are used. The JPEG2000 WSI file is then moved into a JVSserv server, and the DICOM file is moved into a JVSdicom Server, which both are parts of the same PACS. Both files can be stored separately inside a server-specific storage area within the PACS. JVSdicom can also receive imagery from other imaging modalities, which are in turn linked with the LIS or HIS. End-users (e.g., pathologists or physicians) query the JVSdicom Server with a JVSdicom Workstation and retrieve patient-linked image objects. They can view and analyze conventional DICOM imagery within JVSdicom Workstation, while WSIs are opened with JVSview in another viewing window. The DICOM data is transmitted using the JPIP Referenced Transfer Syntax, and the JPEG2000 WSI data is transmitted via an auxiliary channel over JPIP. The system architecture makes it possible to use JVSserv separately outside the PACS, since WSIs do not contain any DICOM references.

Fig 5
figure 5

A model system for linking whole-slide images (WSIs) with DICOM by using JPEG2000, JPIP, and the JVS software. The WSI scanner produces raw image data, which JVSdicom Compressor processes. The resulting DICOM and JPEG2000 WSI files are moved into the PACS to JVSdicom Server and JVSserv, from which they are queried with JVSdicom Workstation and viewed with JVSview.

Full size image

An open issue with using JPEG2000 WSIs and JPIP is the image dimension restriction in the DICOM standard specification. Since image width and height tags are currently described using 16-bit values, oversized (width or height >65,535 pixels) WSIs will show false information with regards to these tags. However, since the definitive image width and height information are always also stored in the JPEG2000 code-stream header, the DICOM counterpart values can be bypassed. This requires the JPIP client software to use the code-stream values instead of the DICOM-specific values. The software described in this study follows this principle. In the future, if a DICOM Supplement equivalent to the CP 896 (to eliminate 16-bit image dimension restrictions) is introduced to the standard, the JPIP viewing software can also operate using the DICOM-specific image dimension tags. Current limitations for DICOM image object size due to 32-bit addressing (4 GB) do not affect JPEG2000 WSIs as long as they are stored in a separate JPIP server, such as the one described in this study. As such, the JPEG2000 standard does not limit individual file size and image dimensions, even in the largest future WSIs11,13.

The DICOM Working Group for pathology imaging (WG-26) is currently making an effort to consolidate pathology concepts and whole-slide imaging with DICOM34. WG-26 has recently finalized Supplement 122 (Specimen Module and Revised Pathology SOP Classes), introducing a new mechanism for pathology specimen identification in DICOM35. The new mechanism revises the concept of “specimen” within DICOM framework. Zwönitzer et al.36 have also proposed a similar approach. The scope of our study was to exemplify a DICOM WSI solution based on JPIP. Thus, the software described does not currently implement Supplement 122.

WG-26 has also recently started preparing a draft for a new DICOM Supplement for whole-slide imaging containing an IOD and SOP Classes for WSIs (as discussed in the WG-26 meetings)37. At its current state, the supplement utilizes a “pyramidal” approach in which the original resolution WSI is split into small image tiles (usually thousands of them). In addition, a number of successively lower resolution versions of the original tiles may be precomputed, creating a pyramid-shaped resolution structure. All the image tiles are stored as a conventional DICOM Series and accessed with the DICOM WSI object, which provides a mapping between the Series and the conceptual image pyramid. The pyramidal approach is similar to what various WSI scanner vendors have used in their web viewing solutions for several years. Prior to its finalization, the supplement draft is subject to changes.

Both approaches for whole-slide imaging in DICOM, the pyramidal and the JPIP-based, as described in this study, can be implemented with minor modifications to the existing standard specification. An advantage of the pyramidal approach is that WSIs are treated equal to other DICOM imagery residing within the same server. In the JPIP-based approach, an additional JPIP server must be set up within the PACS. However, a disadvantage of the pyramidal approach is that because WSIs are stored in a DICOM-specific data structure using lossy compression, as is required in virtual microscopy, relocating WSIs from the PACS (e.g., for teaching purposes) requires lossy recompression. This results in image quality degradation. In the JPIP-based approach, on the other hand, the JPEG2000 WSIs are not stored using a DICOM-specific data structure, making them directly interchangeable with non-DICOM systems. Moreover, since patient-related DICOM data is not embedded in the JPEG2000 WSI files, the same JPIP server can be readily used inside and outside of a PACS without the need for anonymization.

Since the DICOM standard specification already includes support for JPIP, the pyramidal approach and the JPIP-based approach for whole-slide imaging are not mutually exclusive and can be used simultaneously in the same PACS. Regardless of the approach, changes in existing PACS workstation software are also required because of the WSI-specific image viewing characteristics. At the time of preparation of this article, the detailed contents of the upcoming WG-26 WSI supplement, as well as its expected release date, are open. In the future, when the supplement is finalized and implemented in practice, comparisons between the two WSI approaches might turn out useful.

Conclusions

To our knowledge, the software package described in this study is the first practical solution to overcome the limitations of DICOM in virtual microscopy. Compared to other approaches, such as the pyramidal approach of the DICOM WG-26, JPEG2000 with JPIP is a good alternative, enabling use of WSI archives either with or without a linked DICOM system. Further, since JPEG2000 has also many other advantageous features over those of existing WSI image formats, we anticipate that JPEG2000 will become a widely-accepted standard WSI format in virtual microscopy.