Introduction to International Forum on Medical Imaging in Asia
Back to ten years ago, we invited some of the most experienced authors in the field and published the Springer book “Biomedical Imaging” , we did not foresee such a dramatic change in the theoretical research and practical development. We have since then witnessed the overwhelming activities in both academia and industry, remarkably the biannual International Forum on Medical Imaging in Asia (IFMIA). Upon the success in IFMIA 2019 in Singapore, we believe it is of value to provide the readers with a comprehensive coverage on medical imaging, with a selection of both original research and critical reviews in a Special Issue, thanks to the agreement of the JSPS Editorial Office.
Therefore, this article is to briefly introduce technical advancements in all imaging modalities for molecular, cellular, anatomical and functional imaging, with the case studies reported by some leading groups in Asia. Research topics include imaging instrumentation, registration, reconstruction, multimodality methods, noise filtering and image enhancement, segmentation, classification and feature detection, model based imaging, as well as system development and acceleration technologies.
Intelligent Signal Processing and Medical Instrumentation
Oral lesions are conventionally diagnosed using white light endoscopy and histopathology. In National Cancer Centre Singapore, a clinic protocol of virtual histology using an in vivo cellular imaging and real-time processing system (see Fig. 1) is reported by Cheong etc. , in which photoactivation and miniaturized confocal image scanning are utilized. In their experiments, fluorescence imaging of the human and murine oral cavities was carried out using the fluorescent dyes fluorescein sodium  and produced discriminant image signals. Embedded computational intelligence with real-time image processing, feature detection and visualization is demonstrated. Nevertheless, main constraints seem to be the memory limitation for FPGA implementation of complex tissue image alignment and 3-dimensional reconstruction.
In the optical image feature detection with optical interferometry, fringe projection is a commonly used technique, and has brought a key issue, i.e. fringe pattern analysis , to the researchers in the domain. Extracting phase distribution from arbitrary phase-shifted fringe patterns is found useful in phase-shifting interferometry. The advanced iterative algorithm (AIA) and the windowed Fourier ridges and least squares fitting (WFRLSF) is invented, but both of the signal processing algorithms are sensitive to noise, which limits their applications to almost perfect fringe patterns. The windowed Fourier filtering (WFF) algorithm is proposed by Qian etc. from Nanyang Technological University (NTU) for both pre-filtering and post-filtering to suppress the noise . Their simulation results show that with the effective noise suppression, the phase error is reduced to less than 0.1 rad.
For the optical medical instrumentation, design and fabrication of fiber-axicons for in vivo and in vitro cellular imaging and real-time processing system is often the key to a high-resolution system. A direct-laser writing fabrication process for micro-axicons is reported by Huang etc. from Zhejiang University (ZJU) . A fiber-axicon-generated Bessel beam is utilized to write on UV-curable optical epoxy to form new axicons and axicon arrays, with satisfactory apex angle and proximity of the writing axicons. The fabricated axicons are capable of generating a quality Bessel beam with an excellent focusing performance.
In the quantitative image analysis and diagnostic applications in radiologic imaging, radiomics has emerged as a novel framework which allows to extract a large number of quantitative features from radiologic images and promises to improve the characterization of lesions providing potentially valuable information in the context of personalized medicine. However, radiomic features are known to be affected by technical parameters and feature extraction methodology. Refer to Fig. 2, a study by Jin et al. evaluates the robustness of CT radiomic features against the technical parameters involved in CT acquisition and feature extraction procedures using a standardized phantom, and verifies the feature robustness by using patient cases . A total of 47 radiomic features of textures and first-order statistics are extracted on the homogeneous region from all scans. Intrinsic variability is measured to identify unstable features vulnerable to inherent CT noise and texture. Susceptibility index is defined to represent the susceptibility to the variation of a given technical parameter. Eighteen radiomic features are shown to be intrinsically unstable on reference condition. The features are more susceptible to the reconstruction kernel variation than to other sources of variation. The feature robustness evaluated on the phantom CT correlates with those evaluated on clinical CT scans. This study reveals that a number of scan parameters could significantly affect the radiomic features. These characteristics should be considered in a radiomic study when different scan parameters are used in a clinical dataset.
Multi-Modal Image Reconstruction and Registration
Another important research topic in medical imaging is mathematical methodology of CT reconstruction. Intensive studies on this topic have been conducted at University of Tsukuba and other institutions [29, 30]. A fast iterative image reconstruction algorithm for short-scan fan-beam computed tomography is developed by minimizing a data-fidelity term regularized with a total variation penalty, in collaboration with ZJU . The prior information obtained from probabilistic atlas constructed from earlier scans of different patients is effectively utilized in low-dose CT imaging, in collaboration with Suez Canal University, Egypt .
Besides, multiple modalities of medical imaging are often used in clinical diagnoses and interventional imaging processes. The most challenging task is registration of the acquired anatomical structure images and the surgical tools or designed implants. This is through voxelization of freeform models and synthesis of complex objects (see Fig. 3). A NURBS volume representation and its voxelization algorithm  are invented by Lin etc. between NTU and Beijing Normal University (BNU). The key is estimation of a forward difference bound to maximize the parameter step, thus speeding up the voxelization . Although the mathematical proof of the bound is given in the work, a statistical model and adaptive bound may produce a more effective forward difference algorithm; and the model can be trained via 3D convolutional neural networks.
To verify the volumetric image registration, a sub-voxel digital volume correlation (DVC) method combining the 3D inverse compositional Gauss-Newton (ICGN) algorithm with the 3D fast Fourier transform-based cross correlation (FFT-CC) algorithm is developed by Wang etc. from NTU . The new algorithm can eliminate path-dependence in the conventional iterative DVC methods caused by the initial guess transfer scheme.
Machine Learning for Medical Segmentation and Classification
Medical segmentation and classification is largely based on feature extraction and detection. On cellular image feature extraction, pattern recognition and classification, a Springer monograph is published by Xu etc. between NTU and ZJU . Using the antinuclear antibodies (ANAs) in patient serum as the subjects and the Indirect Immunofluorescence (IIF) technique as the imaging protocol, the Bag-of-Words (BoW) framework and a Linear Local Distance Coding (LLDC) method is introduced. A rotation invariant textural feature of Pairwise Local Ternary Patterns with Spatial Rotation Invariant (PLTP-SRI) is also defined which is robust to noise and weak illumination. While the proposed PLTP-SRI feature extracts local feature, the BoW framework builds a global image representation, thus aggregation of the two kinds of features in different aspects achieves excellent classification.
Also based on the feature detection, a statistical model for segmentation and identification of hormone response elements (HREs) in genomic sequences is reported by Stepanova etc. . Based on the verified HREs carrying di-nucleotide preservation in comparison with uniform nucleotide distributions, both mono and di-nucleotide Position Weight Matrices are computed to extract the statistic pattern of the positions.
For temporal autocorrelation present in functional magnetic resonance images (fMRI), a mixed spectrum analysis (MSA) of the brain voxel time-series is proposed by Arun etc. . It can segment the discrete component corresponding to input stimuli and the continuous component carrying temporal autocorrelation. In their experiments, varying correlation structure among the brain regions does not affect the efficiency of the method. Brain activation is detected by predicting the likelihood of activation by comparing the amplitude of discrete component at stimulus frequency across the brain voxels by using normal distribution and modelling spatial correlations among the likelihood with a conditional random field.
Aimed at segmenting and tracing filamentary structures in both neuronal and retinal images (see Fig. 4), a two-step graph-theoretical approach is proposed by Jedeep etc. between NTU and A-Star Bioinformatics Institute [13, 14]. The key idea is that the problem can be reformulated as label propagation over directed graphs, such that the graph is to be partitioned into disjoint sub-graphs, or equivalently, each of the neurons (vessel trees) is separated from the rest of the neuronal (vessel) network.
Recently, a generic and robust low-rank nonlinear kernelization in the framework of statistical shape models (SSM) is presented in  by Ma etc. between NTU and Fraunhofer Institute Singapore. It effectively solves data contamination and arbitrary corruptions in 3D medical image segmentation. The SSM and Deep Neural Networks are incorporated via Bayesian inference with the shape prior from SSM and initial structure localization from deep learning. The model shows great potentials for those use cases where training datasets are large enough.
Accurate segmentation of brain in MRI has been an important task in neuroimaging analysis and yet remained as a challenging issue due to the presence of equipment noise and the complexity of the brain structure. A new method based on the back propagation (BP) neural network and the AdaBoost algorithm is presented by Chao et al. in . The system is trained using a gravitational search algorithm to establish 10 groups of back propagation neural network (BPNN) by applying 10 groups of different data. Subsequently, the AdaBoost algorithm is adopted to obtain the weight of each BPNN. In a comparison experiment using a group of brain MRI datasets, the proposed method outperforms the four state-of-the-art segmentation methods through subjective observation and objective evaluation indexes.
Endoscopic image analysis has an increasing importance due to the wide spread of minimally-invasive surgery. Deep learning-based real-time pathology classifications of endoscopic images have been pioneered by Nagoya University . Musculoskeletal applications are becoming important in the super-aging society in Japan. CNN-based segmentation of vertebrae of X-ray video during swallowing is addressed by University of Tsukuba , while segmentation of individual muscles, bones and implants as well as metal artifact reduction from CT are addressed by Nara Institute of Science and Technology (NAIST) and Osaka University [33, 34] whose particular advantage is prediction of segmentation accuracy using uncertainty estimated from Bayesian U-net . Statistical shape models are used for mandibular segmentation at NAIST in collaboration with University of Tehran . The lung has been addressed by Yamaguchi University and Osaka University and machine-learning approaches are utilized for lung disease classification [36, 37] partly in collaboration with Dalian University of Technology .
Intelligent Computer-Aided Diagnosis and Interventional Imaging
Maximum Likelihood (ML) is a popular optimization criterion in phylogenetics and basic medicine. However, inference of phylogenies with ML is NP-hard. Recursive-Iterative-DCM3 (Rec-I-DCM3) is a divide-and-conquer framework that divides a dataset into smaller subsets (subproblems), applies an external base method to infer subtrees, merges the subtrees into a comprehensive tree, and then refines the global tree with an external global method. In  Du etc. present a novel parallel implementation of Rec-I-DCM3 for inference of large trees with ML. In diagnostic processes, they use RAxML as external base and global search. 6 large real-data alignments containing 500 to 7769 sequences are tested with satisfactory diagnostic accuracy. In the basic medicine, probability and statistic models are also proposed by Stepanova etc. for specialized transcription factors to recognize specific DNA sequences . A Hopfield neural classifier is developed with the flexibility of internal structure being adapted recurrently for the target motif structure.
For telemedicine with mobile devices, a single-pass volume rendering algorithm is developed for the popular WebGL platform by Movania etc. . A remarkable advantage is that it can directly run on most embedded and mobile devices, thanks to interface by the OpenGL ES 2.0 shading API, thus can be implemented efficiently on the embedded GPU in the mobile device, as shown in Fig. 5.
Considering multiple features of medical images, Yu etc. between Xiamen University and NTU define a set of visual features to represent the information of its color, texture and shape . With the patch alignment framework, a new subspace learning method, termed Semi-Supervised Multimodal Subspace Learning (SS-MMSL), is invented to encode different features from different modalities into the subspace. It adopts the discriminative information from the labeled data to construct local patches and aligns these patches to get the optimal low dimensional subspace for each modality, achieving improved medical diagnostic accuracy. This approach is also taken by Jadeep etc.  in the filamentary tracing problem, in which the matrix-forest theorem is applied.
In cardiovascular disease diagnosis and prognosis, arrhythmia heartbeat classification is crucial in electrocardiogram to help prevent stroke or sudden cardiac death. A novel ECG arrhythmia classification method is reported by Yang etc. from Hebei University , addressing stacked sparse auto-encoders (SSAEs) and a softmax regression model. Via deep learning, the algorithm can hierarchically extract high-level features from huge amount of ECG data.
Interventional imaging is a crucial technique for computer-aided therapy and surgery. For real-time synthesis of medical objects, dynamics simulation of deformable models with anisotropic materials is introduced in a monograph by Cai etc. from NTU . Their fibre-field incorporated corotational finite element model (CLFEM) can work directly with a constitutive model of transversely isotropic materials, displaying adaptive dynamic features of the anatomical structures, as illustrated by the video clips in Fig. 6.
As confocal laser endomicroscopy (CLE) is a minimally invasive optical technique that enables in vivo imaging of tissue structures, and also holds potential for guided biopsy procedures, application of CLE to imaging oral cavity lesions is reported by Thong etc. from Singapore General Hospital . Along the direction of accurate interventional imaging is the development of a fiber-optic bending sensor based on the propagation of LP21 mode by Fan etc. from ZJU . In the experiments, the new sensor achieves a sensitivity of 4.13 rad/m−1 and exhibits the temperature-immune, thus can detect both bending direction and bending angle with a large dynamic range.
Upon acquisition of images and sensed surgical tools, an interactive process relies on real-time feedback of the augmented imaging systems. GPU based volume rendering algorithms have attracted researchers, especially in a cloud environment to have ubiquitous data processing and visualization capability. A pervasive computing solution is presented by Movanian etc. [25, 26] for highly accurate and real-time volume rendering.
Research topics addressed in the above sections include imaging instrumentation, registration, reconstruction, multimodality methods, noise filtering and image enhancement, segmentation, classification and feature detection, model based imaging, as well as system development and acceleration technologies. While we are positive to recent reports on machine intelligence, we would also present to the readers the achievements and challenging issues in much talked data mining and deep learning; we would thus leave open for the readers the question of whether machine intelligence can effectively work in medical imaging and clinical diagnosis.
Lin, F, Olivo, M., Kung, S.-Y., Ed. (2009). Biomedical Imaging, Springer Science+Business Media, LLC, ISBN 978-1-4419-1165-0.
Sing, C. L., Feng, L., Soon, S. H., Kemao, Q., Feng, Z., Thong, P. S. P., Chee, S. K., Olivo, M., & Sun-Yuan, K. (2009). Embedded Computing for Fluorescence Confocal Endomicroscopy Imaging. Journal of Signal Processing Systems, 55(1–3), 217–228.
Thong, P. S. P., Tandjung, S. S., Mobeen, M. M., Chiew, W.-M., Olivo, M., Bhuvaneswari, R., Lin, F., Qian, K., Seah, H.-S., & Soo, K.-C. (2012). Towards Real-time Virtual Biopsy of Oral Lesions using Confocal Laser Endomicroscopy Interfaced with Embedded Computing. Journal of Biomedical Optics, 17(5), 056009. https://doi.org/10.1117/1.JBO.17.5.056009.
Wenjing Gao, Qian Kemao, Haixia Wang, Feng Lin, Hock soon Seah, parallel computing for fringe pattern processing: A multicore CPU approach in MATLAB® environment, Optics and Lasers in Engineering, Vol. 47, No. 11, 2009, pp1286–1292.
Kemao, Q., Wang, H., Gao, W., Lin, F., & Soon, S. H. (2010). Phase Extraction from Arbitrary Phase-shifted Fringe Patterns with Noise Suppression. Optics and Lasers in Engineering, 48(6), 684–689.
He, H., Chen, S., Zhou, H., Li, Q., Jian, F., Lin, F., & Wu, X. (2014). Fabrication of micro-axicons using direct-laser writing. Optics Express, 22(9), 11035–11042.
Lin, F., Seah, H. S., Wu, Z. K., & Ma, D. (2007). Voxelisation and fabrication of freeform models. Virtual and Physical Prototyping, 2(2), 65–73.
Wu, Z. K., Lin, F., Seah, H. S., & Chan, K. Y. (2004). Evaluation of difference bounds for computing rational Bezier curves and surfaces. Computers & Graphics, 28(4), 551–558.
Wang, T., Jiang, Z., Kemao, Q., Lin, F., & Soon, S. H. (2016). GPU-Based Parallel Computing Accelerated Digital Volume Correlation. Experimental Mechanics, 56(2), 297–309. https://doi.org/10.1007/s11340-015-0091-4.
Xu, X., Wu, X., Lin, F. (2017). Cellular Image Classification, Springer International Publishing AG, ISBN 978-3-319-47628-5, https://doi.org/10.1007/978-3-319-47629-2.
Stepanova, M., Lin, F., & Valerie, L. C. L. (2006). Establishing a Statistic Model for Recognition of Hormone Response Elements. Computational Biology and Chemistry, 30(5), 339–347.
Kumar, A., Lin, F., & Rajapakse, J. C. (2016). Mixed Spectrum Analysis on fMRI Time-Series. IEEE Transactions on Medical Imaging, 35(6), 1555–1564. https://doi.org/10.1109/TMI.2016.2520024.
De, J., Cheng, L., Zhang, X., Lin, F., Li, H., Ong, K. H., Yu, W., Yu, Y., & Ahmed, S. (2016). A Graph-theoretical Approach for Tracing Filamentary Structures in Neuronal and Retinal Images. IEEE Transactions on Medical Imaging, 35(1), 257–272. https://doi.org/10.1109/TMI.2015.2465962.
De, J., Zhang, X., Lin, F., & Cheng, L. (2018). Transduction on directed graphs via absorbing random walks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(7), 1770–1784. https://doi.org/10.1109/TPAMI.2017.2730871.
Ma, J., Wang, A., Lin, F., Wesarg, S., & Erdt, M. (2019). A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data. Computerized Medical Imaging and Graphics, 77, 101638. https://doi.org/10.1016/j.compmedimag.2019.05.006.
Du, Z. H., Stamatakis, A., Lin, F., Roshan, U. W. and Nakhleh, L. (2005). Parallel Divide-and-Conquer Phylogeny Reconstruction by Maximum Likelihood, The 2005 International conference on high performance computing and communications (HPCC-05), Capri-Sorrento Penisular, Italy, September 21-24.
Stepanova, M., Lin, F., & Valerie, L. C. L. (2007). A Hopfield Neural Classifier and Its FPGA Implementation for Identification of Symmetrically Structured DNA Motifs. The Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology, 48(3), 239–254.
Movania, M.M., Lin, F. (2012). Ubiquitous Medical Volume Rendering on Mobile Devices, International Conference on Information Society (i-Society’12), London, UK, 25–28 June.
Yu, J., Lin, F., Seah, H. S., & Lin, Z. (2012). Image Classification by Multimodal Subspace Learning. Pattern Recognition Letters, 33(9), 1196–1204.
Cheng, L., De, J., Zhang, X., Lin, F., Li, H. (2014). Tracing retinal blood vessels by matrix-Forest theorem of directed graphs, 17th international conference on medical image computing and computer assisted intervention (MICCAI’14), Boston, USA, 14–19 September, https://doi.org/10.1007/978-3-319-10404.
Yang, J., Yang, B., Lin, F., Liu, M., Hou, Z., & Liu, X. (2018). A novel electrocardiogram arrhythmia classification method based on stacked sparse auto-encoders and softmax regression. International Journal of Machine Learning and Cybernetics, 9(10), 1733–1740. https://doi.org/10.1007/s13042-017-0677-5.
Cai, J., Lin, F., Seah, H.S. (2016). Graphical Simulation of Deformable Models, Springer International Publishing Switzerland, ISBN 978-3-319-51030-9, https://doi.org/10.1007/978-3-319-51031-6.
Thong, P. S. P., Olivo, M., Tandjung, S. S., Movania, M. M., Lin, F., Qian, K., Seah, H.-S., & Soo, K.-C. (2011). Review of Confocal Fluorescence Endomicroscopy for Cancer Detection. IEEE Photonics Society (IPS) Journal of Selected Topics in Quantum Electronics, PP(99). https://doi.org/10.1109/JSTQE.2011.2177447.
Fan, Y., Wu, G., Wei, W., Yuan, Y., & Lin, F. (2012). X. Wu, fiber-optic bend sensor using LP21 mode operation. Optics Express, 20(24), 26127–26134.
Movania, M.M., Lin, F. (2012). High-Performance Volume Rendering on the Ubiquitous WebGL Platform, 14th IEEE International Conference on High Performance Computing and Communications (HPCC’12), Liverpool, UK, 25–27 June.
Movania, M. M., Chiew, W. M., & Lin, F. (2014). On-Site Volume Rendering with GPU-Enabled Devices. Wireless Personal Communications, 76(4), 795–812.
Chao, Z., Kim, H.J. (2019). Brain image segmentation based on the hybrid of back propagation neural network and AdaBoost system, Journal of Signal Processing and Systems. https://doi.org/10.1007/s11265-019-01497-y
Jin, H., Kim, J.H. (2019). Evaluation of feature robustness against technical parameters in CT radiomics: Verification of phantom study with patient dataset, Journal of Signal Processing and Systems. https://doi.org/10.1007/s11265-019-01496-z.
Wang, T., Kudo, H., Yamazaki, F., Liu, H. (2019). A fast regularized iterative algorithm for fan-beam CT reconstruction. Physics in Medicine and Biology, 64(14) https://iopscience.iop.org/article/10.1088/1361-6560/ab22ed/pdf.
Rashed, E. A., & Kudo, H. (2016). Probabilistic atlas prior for CT image reconstruction. Computer Methods and Programs in Biomedicine, 128, 119–136.
Misawa, M., Kudo, S. E., Mori, Y., Nakamura, H., Kataoka, S., Maeda, Y., Kudo, T., Hayashi, T., Wakamura, K., Miyachi, H., Katagiri, A., Baba, T., Ishida, F., Inoue, H., Nimura, Y., & Mori, K. (2016). Characterization of colorectal lesions using a computer-aided diagnostic system for narrow-band imaging endocytoscopy. Gastroenterology., 150(7), 1531–1532.
Fujinaka, A., Mekata, K., Takizawa, H., Kudo, H. (2019). Automated segmentation of cervical intervertebral disks from video fluorography using a convolutional neural network and its performance evaluation, Journal of Signal Processing and Systems, https://doi.org/10.1007/s10464-007-9108-14 [Special Issue]
Sakamoto, M., Hiasa, Y., Otake, Y., Takao, M., Suzuki, Y., Sugano, N., Sato, Y. (2019). Bayesian segmentation of hip and thigh muscles in metal artifact-contaminated CT using convolutional neural network-enhanced normalized metal artifact reduction, Journal of Signal Processing and Systems, https://doi.org/10.1007/s11265-019-01507-z [Special Issue]
Hiasa, Y, Otake, Y, Takao, M, Ogawa, T, Sugano, N, Sato, Y. (2019). Automated muscle segmentation from clinical CT using Bayesian U-net for personalized musculoskeletal modeling. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/TMI.2019.2940555.
Abdolali, F., Zoroofi, R. A., Abdolali, M., Yokota, F., Otake, Y., & Sato, Y. (2017). Automatic segmentation of mandibular canal in cone beam CT images using conditional statistical shape model and fast marching. International Journal of Computer Assisted Radiology and Surgery, 12(4), 581–593.
Mabu, S., Atsumo, A., Kido, S., Kuremoto, T., Hirano, Y. (2019) Investigating the Effects of Transfer Learning on ROI-based Classification of Chest CT Images: A Case Study on Diffuse Lung Diseases, Journal of Signal Processing and Systems, https://doi.org/10.1007/s11265-019-01499-w [Special Issue]
Xu, R., Cong, Z., Ye, X., Hirano, Y., Kido, S., Gyobu, T., Kawata, Y., Honda, O., Tomiyama, N. (2019). Pulmonary textures Classification via a multi-scale attention network. IEEE Journal of Biomedical and Health Informatics. https://doi.org/10.1109/JBHI.2019.2950006
Source of Financial Support
This work is partially supported by a grant AcRF RGC 2017-T1–001-053 by Ministry of Education, Singapore.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lin, F., Kim, J.H., Sato, Y. et al. Recent Collaborative R&D in Medical Imaging in Asia. J Sign Process Syst (2020) doi:10.1007/s11265-019-01514-0