We describe a contactless optical technique selectively enhancing superficial blood vessels below variously pigmented intact human skin by combining images in different spectral bands. Two CMOS-cameras, with apochromatic lenses and dual-band LED-arrays, simultaneously streamed Left (L) and Right (R) image data to a dual-processor PC. Both cameras captured color images within the visible range (VIS, 400–780 nm) and grey-scale images within the near infrared range (NIR, 910–920 nm) by sequentially switching between LED-array emission bands. Image-size-settings of 1280 × 1024 for VIS & 640 × 512 for NIR produced 12 cycles/s (1 cycle = 1 VIS L&R-pair + 1 NIR L&R-pair). Decreasing image-size-settings (640 × 512 for VIS and 320 × 256 for NIR) increased camera-speed to 25 cycles/s. Contrasts from below the tissue surface were algorithmically distinguished from surface shadows, reflections, etc. Thus blood vessels were selectively enhanced and back-projected into the stereoscopic VIS-color-image using either a 3D-display or conventional shutter glasses.
As a first usability reconnaissance we applied this custom-built mobile stereoscopic camera for several clinical settings:
• blood withdrawal;
• vein inspection in dark skin;
• vein detection through iodide;
• varicose vein and nevi pigmentosum inspection.
Our technique improves blood vessel visualization compared to the naked eye, and supports depth perception.
Visually guided procedures provide instant feedback and meantime afford insights that would otherwise be difficult or even impossible to obtain.26 This is a primary reason for mankind’s continuous desire to extend vision beyond the boundaries of the human eye, which has resulted in many successful diagnostic imaging modalities like X-ray imaging, endoscopy, thermography, ultrasound scans and MRI that all found their way into the clinic.
While investigating the principal feasability of a camera for imaging blood oxygenation levels, we noticed that our multispectral images also contained information about subcutaneous vasculature, with improved contrast in the near infrared.24 Near infrared imaging of superficial blood vessels in itself is not new; soon after infrared photography2,9,25 early electronic cameras were used for experiments.13,15 Since then, numerous near infrared imaging methods have been developed using transillumination mode and/or reflection mode.5,6,10,14, 22 Our multispectral camera, however, provides the opportunity to obtain normal color images within the visible range (VIS) which are pixel-to-pixel matched with images obtained in the invisible near infrared range (NIR). By combining the image information content of both spectral bands with a processing algorithm, we developed a new technique that allows selective enhancement of superficial blood vessels with selectable pigment suppression within a normal color image.23
The underlying principle does not necessarily require stereoscopic image acquisition to derive increased vessel contrasts from the tissue. When, however, developing a new medical imaging technique, deriving image information from patient tissue is not the only issue. Especially for visually guided procedures it is also crucial to put effort in an ergonomic human interface and avoid conflicts between the proprioceptional and visual perceptions of the user. The added value of stereoscopic image recordings already was recognized and successfully used to document medical cases more than a century ago.4 We likewise reasoned that offering enhanced blood vessel contrast at the cost of depth perception would restrict the usefulness of our technique and therefore decided to realize a stereoscopic version of our vessel contrast enhancement device.
Visually guided procedures typically require a combination of well trained eyes and specific fine-motoric skills, since the coordination between eyes and hands is task dependent.16 For eye-hand coordination, brain processes for perception and action interact so closely that they cannot be separated and the influence of visual illusions (like human depth perception) to motoric tasks becomes stronger when input takes place via lower levels in the brain.3 The match between a human interface device and the visual and motoric brain processes strongly defines whether a technology can be applied intuitively or not. A new technology can be classified as intuitive if it speeds up the learning process for novices in a certain skill, without impairing the performance of persons already skilled in the existing art.20 Thus a useful increase in vascular contrast should neither imply a task-impairing decrease in depth perception (like in monoscopic techniques) nor distort spatial clues for human vision (like shadows) nor introduce false depth information (like projection parallax).
As a first reconnaissance of practical feasibility, we applied our new technique to several visual clinical procedures for which we expected that improved visualization of blood vessels would be of interest, being:
vein inspection in dark skin;
detection of veins through iodide;
inspection of varicose veins and nevi pigmentosum.
The instrumental hardware setup is schematically drawn in Fig. 1. Two synchronized identical single-chip CMOS-cameras (Vector Technologies Belgium, custom built) were equipped with apochromatic lenses and two identical custom-built dual-band LED-arrays (O2-View, the Netherlands). These LED-arrays were current controlled and each had two individually programmable channels for the emission of in total ±1.2 Cd visible white light with adjustable color temperature (consisting of 40 broadband white LEDs with a yellow accent and 20 broadband white LEDs with a blue accent) as well as one programmable channel for the emission of near infrared radiation (20 LEDs, 920 nm, max. 32 mW per LED). The light sources were constructed so that the geometrical beam profiles of VIS and NIR matched very closely (and thus also any resulting shadows and/or reflections). Left (L) and Right (R) image data was simultaneously acquired and streamed to a dual processor PC equipped with a stereoscopic monitor (Sharp LL-151–3D). It was also possible to connect a conventional cathode ray tube (CRT) monitor equipped with shutter glasses, for comparison of stereoscopic representation.
A schematical representation of image aquisition and data structure is drawn in Fig. 2. The CMOS-camera detector chips were equipped with a Bayer filter mosaic to obtain an RGB color image within the VIS. The camera pixels, however, were also sensitive to near infrared radiation because all three filter channels of the Bayer mosaic (red, green and blue) were designed as highly transparant within the NIR. Thus four NIR-pixels were acquired for each VIS RGB color pixel group. By sequentially switching between emission bands of the LED-arrays, images within the Visual range (VIS, 400–780 nm) and Near Infrared range (NIR, 910–920 nm) were acquired in an alternating fashion. VIS and NIR image-size-settings could be varied independantly. For 8-bit encoding depth, at image-size-settings of 1280 × 1024 for VIS and 640 × 512 for NIR, 12 cycles/s were obtained (1 cycle = 1 VIS L&R-pair + 1 NIR L&R-pair). At the cost of decreasing image-size-settings downto 640 × 512 for VIS and 320 × 256 for NIR, camera speed could be increased up to 25 cycles/s. Encoding depths of 10-bit and 12-bit were also available, but only used for stills (due to the lower obtainable framerate).
Multispectral stereoscopic movies were recorded in several typical clinical settings for which the technique was considered as possibly useful. LED currents and diaphragm settings were chosen so that for each movie saturated pixels were avoided. A preview mode allowed aiming, adjustment of converging angle α and focusing of the cameras. After software triggering the stereo-camera streamed a sequence of 8-bit digitally encoded image cycles to PC-memory (using auto-incremental numbering). All images were automatically saved on a fast SATA harddisk-array. Camera and light source settings were automatically stored in a text file and located in the same directory.
All patients and volunteers gave their informed consent for filming as well as for publishing the resulting image material. No diagnosis or therapy was based upon any of our results.
Processing was performed with custom developed software (programming language C++) using the stored text file with camera and light source settings as additional input values. Figure 3 schematically represents this process.
The processing method allowed discrimination between image information obtained from the tissue surface versus image information obtained from within the tissue. In order to achieve this, a distinction was made between shadows, reflections and absorption contrasts for both VIS and NIR.
Suppressing Shadows on the Surface
Since NIR and VIS beams were matched closely, shadows produced by irregular shapes at the tissue surface (e.g. skin structure, skin folds, nevi, hair, etc.) or by objects between the light sources and the tissue (fingers, needles, surgical tools, etc.) also matched well in both wavelength ranges. Our algorithm excluded such matching shadows from enhancement and left all useful aspects of shadows unaffected (e.g. depth clues).
Useful Effects of Shadows and Lighting Geometry
Due to the fact that the VIS and IR shadows matched very closely, the algorithm was able to selectively enhance contrast from below the surface while leaving shadows on the surface unaffected (thus not distorting these important depth clues).
Due to the shallow angle of the lightbeams, the skin texture was pronounced. Due to the lighting from two sides, objects that were brought towards the tissue surface (e.g. needles, scalpels, probes, etc.) when lighted from both sides, could produce two separate (not too heavy) shadows. These shadows met and typically formed a “V” pattern when an object touched the surface in the middle of the field of view, thus providing extra information for depth perception.
Enhancing Absorption Contrast of Blood Vessel Walls
Edge detection by a Prewitt image filter was performed on each VIS and NIR image. Pixel positions containing edge information above a certain adjustable threshold in both spectral regions were classified as surface artefacts and excluded from enhancement. The boundary regions of absorption contrasts caused by structures below the tissue surface, produced edges that were mainly present in the NIR image. Enhancement was selectively performed only for pixel positions where the NIR image contained more edge information than the corresponding VIS image. These “valid” pixels identified the vessel boundaries. The positions of these valid pixels were stored in a 1st NIR mask.
Discarding Reflections on the Surface
Shiny areas that produced reflections and/or saturated pixels also matched in both wavelength ranges. This characteristic allowed to calculate a 2nd NIR mask in which superficial reflections were also excluded from enhancement. Neighboring pixels of identified saturated pixels were excluded from enhancement. The radius of this exclusion region was programmable, but due to the favorable anti-blooming behavior of the CMOS-camera chips, suppressing direct neighbor pixels appeared sufficient.
Enhancing Absorption Contrast of Blood Vessel Lumen
The “content” of blood vessels was separately enhanced by raising pixel values of the normalized NIR image to the power of N (with N user adjustable between 0.5 and 2.5) while discriminating NIR pixels below a freely adjustable noise threshold and excluding information from identified shadows. Multiplication with the 2nd NIR mask then produced a final enhancement mask for subsequential backprojection into the VIS image by pixel-to-pixel multiplication.
Suppressing Contrasts Originating from Melanin Pigment
Superficial contrasts within the VIS, originating from melanin pigment concentrations, could either be filtered out or left unchanged. Figure 4 illustrates the normalized distribution of the intensities for the aquired separate spectral bands (R, G, B and NIR) in relation to the intensity (IVIS) of the composed visible RGB-image. Intensities were calculated using the Intel ippiRGBtoGray function.11 Four clouds of data points can be discerned, being IR/IVIS (red channel), IG/IVIS (green channel), IB/IVIS (blue channel) and INIR/IVIS (NIR channel). By calculating the ratio of (IR/IVIS )/(INIR/IVIS ) and comparing the result for each pixel with an adjustable threshold, it can be decided whether or not to apply backprojection to a pixel. This concerns the pixels located within the overlapping region of IR/IVIS and INIR/IVIS within Fig. 4. The resulting difference is clear when comparing between Fig. 9c and d with regard to the visualization of nevi pigmentosum and hair.
The images could either be displayed on an auto-stereoscopic liquid crystal display (LCD) monitor or on a conventional CRT monitor equipped with shutter glasses.
The auto-stereoscopic LCD-monitor (Sharp LL-151–3D) was equipped with software controllable switching between stereoscopic (or 3D) and normal (monoscopic or 2D) mode. Monitor resolution was 1280 × 768 in monoscopic mode (XGA). In stereoscopic mode the available pixels were split-up in two separate images (L&R) by activation of a vertical LCD parallax barrier. The principle behind this technology is illustrated in Fig. 5.
The CRT-monitor (iiyama vision master 21) was used at a resolution setting of 1280 × 1024 in combination with wireless shutter glasses (e-Dimensional) and thus provided stereoscopic information without sacrificing resolution.
The enhancement algorithm occurs on a pixel-to-pixel basis and does not affect resolution.
Our device offered several imaging modes for data presentation:
Monocopic raw VIS preview (normal full color vision)
Monoscopic raw NIR preview (greyscale)
Off-line stereoscopic looped VIS view with and without enhanced blood vessel back-projection (with freely adjustable enhancement settings).
Off-line stereoscopic looped raw NIR view or enhanced NIR view (with freely adjustable enhancement settings)
Stereoscopic stills in all modes
Monoscopic stills in all modes (freely switchable between Left and Right)
All modes offered the possibility for pause and scrolling forward or reverse frame-by-frame. By means of virtual slider controls and virtual pushbuttons the user interface allowed freely adjustable settings for shadow suppression, pigment suppression, noise threshold and vessel lumen fill-in contrast.
Results for Blood Withdrawal
During routine blood withdrawal the inserted needle tip remained only slightly visible on the VIS image due to skin surface deformation (see Fig. 6a). On the raw NIR image, however, the inserted needle remained visible within the tissue for a few mm with some metallic reflection. Subcutaneous bleeding during needle removal could be detected in the raw NIR image while the needle tip was still in the tissue (see Fig. 6b). This was clearly highlighted by backprojection in the VIS image (see Fig. 6c). The processing settings used to obtain this backprojection mainly laid the accent upon enhancing the absorption contrast of blood vessel lumen, whereas edge enhancement was set to minimum. Reflections on the thumb nail did not lead to image distortion and the blood volume in the nail bed showed more contrast.
Results for Dark Skin
Hardly any vascular contrast is present within the VIS image (see Fig. 7a). The NIR image, however, is not affected by skin pigmentation and reveals a subcutaneous vascular pattern (see Fig. 7b). The combined information results in an enhanced image (see Fig. 7c).
Results for Vein Detection Through Iodide
Iodizing skin portions before surgery is a common clinical procedure which darkens the skin and lowers the visibility of blood vessels. Our method offers imaging right through iodide. To demonstrate this, we filmed a glass Petri dish, placed horizontally upon a volunteers arm while filling it up to a 3 mm thick layer of iodide solution. From the VIS image no vessels could be detected through the iodide solution (see Fig. 8b) and hardly any through the empty Petri dish (see Fig. 8a). The raw NIR image, however, clearly showed superficial blood vessels even through the Petri dish (see Fig. 8c).
Results for Varicose Vein and Nevi Pigmentosum Inspection
The device was used to film varicosis patients during dermatological outpatient clinic. The visualization of varicose veins could be drastically improved. Even in cases where the blood vessels were covered by subcutaneous fat, they could be detected quite well (see Fig. 9). By either de-activating or activating the first NIR mask in the processing algorithm (see Fig. 3) it was also possible to choose whether pigment contrasts in the visual range were left intact or filtered out. The difference is clear when comparing between Fig. 9c and d with regard to the visualization of nevi pigmentosum and hair. By clicking a software screen button, this suppression of pigment contrast could be freely switched on and off during viewing.
From the figures presented in this article, it is clear that all experiments resulted in improved visualization of superficial blood vessels. Based upon the absorption coefficients of Hb versus HbO2, arteries provide superior contrasts at 920 nm. The presented images, however, show enhanced venous contrasts, because the larger arteries are buried deeper underneath the skin. Stereoscopic movies provide a more lifely impression of the underlying, but presentation of stereoscopic movies is not possible in a printed journal. We therefore offer the possibility to download viewer software and movies via the internet http://www.erasmusmc.nl/ThoraxcenterBME/html/research/additional/bloodvesselcamera.htm
For both the autostereoscopic LCD and the CRT display plus shutter glasses, we found that the perception of depth was less for stills than for movies. This qualitative observation matches research findings on stereoscopic display techniques in X-ray technology8 and is also consistent with the differences between static and dynamic stereoacuity described by Mathias and Rudolf Sachsenweger.17 Stereoscopic movies also showed an increase in apparent image sharpness compared to stereoscopic stills, which may be explained by Shipley’s description of stereoscopic contour integration over blur.18 Compared to the autostereoscopic LCD-display, the increase in perceived sharpness was much stronger when using a CRT with shutter glasses. For monoscopic stills, however, the LCD-display (switched to monoscopic mode) was superior to the CRT. These observations are in agreement with the technology overview by Szold19 and can be explained by the sacrifice of resolution when using the LCD display in autostereoscopic mode, which price has not to be paid when using the CRT with shutter glasses.
The beneficial effect of separately adjustable enhancement paths for blood vessel contours and blood vessel lumen, which we experienced during the development of our processing algorithm, is consistent with the findings of Yin et al. that surface-features and edge processes make different contributions in determining an object’s unity and shape.27 The perceived usefulness of stereoscopic information, however, also supports the concept formulated by Tse that ‘‘mergeable’’ volumes, rather than relatable contours, are the critical elements in completion.21
With our technique we have aimed to avoid parallax errors and loss of vessel contrast by shadows which are inherent to other blood vessel contrast enhancement techniques projecting the vascular pattern via a projector onto the skin.28 We exclude parallax errors because the user truly looks beneath the tissue surface trough two camera’s. Absence of vessel contrast in shadows is avoided by lighting from two sides and by discrimination of superficial artifacts versus contrasts originating from below the surface.
This ability to selectively enhance contrasts from beneath the surface, while preserving the natural depth clues of shadows on the surface, supports both static and dynamic transmittance anchoring of the visual system which is crucial for depth perception.1 Preservation of natural shadows is furthermore important for the correct interpretation of depth clues by occlusion (e.g. from a hand or an instrument positioned between observer and tissue) thus precluding errors in depth perception.12
Since only carefully balanced white light is projected on the skin, there is no impairment of color perception. It is up to the user to freely switch between normal full color vision, color vision plus superimposed blood vessel backprojection and stereoscopic near infrared greyscale vision (with or without enhancement features).
The embodiment described in this article still has some drawbacks:
Due to the fact that the pertaining configuration aquires VIS and NIR images sequentially, motion artifacts can lead to a backprojection shift.
The deeper the vessels lie under the surface, the less we can visualize them. In the pertaining configuration, the depth range is limited to about 1 mm. Based upon the absorption coefficients of Hb and HbO2, arteries provide superior contrasts at 920 nm. The presented images, however, show enhanced venous contrasts, because the larger arteries are buried deeper underneath the skin.
The above mentioned drawbacks require further development. Fortunately, many technological improvents still can be added. Allthough the device worked quite well under normal ambient lighting conditions (100–200 Lux), the use of a switchable filter might further improve spectral separation. Furthermore, known techniques like application of crossed polarizers, in combination with more powerful light sources, can offer a considerably larger penetration depth.7
Compared to inspection with the naked eye under normal lighting conditions, the tested stereoscopic blood vessel contrast enhancer offered improved visualization in all investigated settings, providing the best stereoscopic image quality when using the CRT monitor with shutter glasses and the best monoscopic image quality when using the LCD set to monoscopic mode.
Our technique supports perception of depth, 3-dimensional motion and discrimination between tissue surface and underlying structures. It also has potential as an educational tool by offering the possibility to look and record trough the eyes of an experienced specialist. Further improvements on penetration depth, frame-rate and focus depth-of-field form targets for momentary ongoing further research.
Anderson B. L., Singh M., Meng J. 2006. The perceived transmittance of inhomogenous surfaces and media. Vision Res. 46:1982–1995
Dallow R. L., McMeel J. W. 1974. Penetration of retinal and vitreous opacities in diabetic retinopathy. Use of infrared fundus photography. Arc. Ophthalmol. 92:531–534
de Grave, D. D. The use of illusory visual information in perception and action. Erasmus, Rotterdam, 2005, pp. 112.
Elschnig, Elschnig demonstrirt ferner stereoskopische Photographieen. Presented at Achtundzwansigte Versammlung der Opthalmologischen Gesellschaft, Heidelberg, 1900.
Elsner A. E., Burns S. A., Weiter J. J., Delori F. C. 1996. Infrared imaging of sub-retinal structures in the human ocular fundus. Vision Res. 36:191–205
Gostout C. J., Jacques S. L. 1995. Infrared video imaging of subsurface vessels: a feasibility study for the endoscopic management of gastrointestinal bleeding. Gastrointest. Endosc. 41:218–224
Groner W., Winkelman J. W., Harris A. G., Ince C., Bouma G. J., et al. 1999. Orthogonal polarization spectral imaging: a new method for study of the microcirculation. Nat. Med. 5:1209–1212
Hardy J., Dodds S., Roberts A. 1996. An objective evaluation of the effectiveness of different methods of displaying three-dimensional information with medical X-Ray images. Invest. Radiol. 31:433–445
Haustein U. F. 1967. Infrared photodocumentation of the superficial venous system of the leg. Ästhetische Medizin 16:155–160
Hayashi N., Kawano S., Tsuji S., Tsujii M., Takai Y., et al. 1994. Identification and diameter assessment of gastric submucosal vessels using infrared electronic endoscopy. Endoscopy 26(8):686–689
Intel. Intel integrated performance primitives for Intel architecture—Reference manual A70805–017US, 2006
Johnson L. G., Edwards P., Hawkes D. 2002. Surface transparency makes stereo overlays unpredictable: The implications for augmented reality. Stud. Health Technol. Inform. 94:131–136
Jones C. H., Newbery S. P. 1977. Visualization of superficial vasculature using a Vidicon camera tube with silicon target. British J. Radiol. 50:209–210
Kono M., Ueki H., Umemura S. 2002. Near-infrared finger vein patterns for personal identification. Appl. Optics 41:7429–7436
Ontikova N. M., Iaroslavtsev D. A., Lirman A. V. Increase in the information content of the image of the surface veins by using a television infrascope. Meditsinskaia tekhnika: 20–22, 1976
Rotman G. 2005. Localization during pursuit eye movement. Erasmus, Rotterdam, pp 102
Sachsenweger M, Sachsenweger U. 1991. Stereoscopic acuity in ocular pursuit of moving objects. Dynamic stereoscopy and movement parallax: relevance to road safety and occupational medicine. Documenta Opthalmologica 78:1–133
Shipley T. F. 1987. Field processes in stereovision. A description of stereopsis appropriate to opthalmology and visual perception. Documenta Opthalmologica 66:95–170
Szold A. 2005. Seeing is believing - Visualization systems in endoscopic surgery (video, HDTV, stereoscopy, and beyond). Surg. Endosc. 19:730–733
Taffinder N., Smith S. G. T., Huber J., Russel R. C. G., Darzi A. 1999. The effect of a second-generation 3D endoscope on the laparoscopic precision of novices and experienced surgeons. Surg. Endosc 13:1087–1092
Tse P. U. 1999. Volume Completion. Cognit. Psychol. 39:37–68
Webb S. Imaging by diaphanography. In The physics of medical imaging, ed. S Webb. Bristol: Institute of physics publishing
Wieringa F. P., D. Bakker, A. F. W. van der Steen, F. Mastik, and R. G. M. van Melick. Patent No. EP1566142A1, 2005.
Wieringa F. P., Mastik F., Van der Steen A. F. W. 2005. Contactless multiple wavelength photoplethysmographic imaging: A first step towards “SpO2 camera” technology. Ann. Biomed. Eng. 33:1034–1041
Wissmanns H. F. 1968. Prophylaxis and treatment of venous diseases in general practice. Objective evaluation by means of infrared photography. die Medizinische Welt 20:191–196
Woods D. D., E. S. Patterson, and E. M. Roth. Can we ever escape from data overload? A cognitive systems diagnosis. Cognition Technol.Work: 22–36, 2002.
Yin C., Kellman P. J., Shipley T. F. 2000. Surface integration influences depth discrimination. Vision Res. 40:1969–1978
Zeman H. D., G. Lovhoiden, C. Vrancken. Prototype vein contrast enhancer. Presented at Photonics West 2004: Biomedical Optics (BiOS), 2004.
This study was sponsored by TNO Quality of Life and by TNO O2 View BV (both based in Leiden, the Netherlands). Custom mechanical constructions were made by Mr. L. Bekkering and support on electronics was given by Mr. J. Honkoop (both from Biomedical Engineering, Thorax Centre). We greatly appreciate the possibilities to record data in clinical settings and the assistance of the blood withdrawal team of the cardiology outpatient clinic as well as the staff of the dermatology outpatient clinic (all within Erasmus MC Rotterdam, the Netherlands).
About this article
Cite this article
Wieringa, F.P., Mastik, F., Cate, F.J.t. et al. Remote Non-invasive Stereoscopic Imaging of Blood Vessels: First In-vivo Results of a New Multispectral Contrast Enhancement Technology. Ann Biomed Eng 34, 1870–1878 (2006). https://doi.org/10.1007/s10439-006-9198-1