A Low-Cost Multi-Camera System With Multi-Spectral Illumination

. Certain characteristics of prints are invisible for the naked eye, e. g. elements printed at small print sizes or reﬂection/absorption characteristics of print inks, which are typical for security prints such as banknotes. Detailed inspections of these characteristics are usually carried out using special and expensive equipment. Often image acquisition and processing are carried out on diﬀerent systems. This contribution proposes a compact and modular standalone multi-camera image processing system with multi-spectral illumination for such document inspection applications. It describes its standard oﬀ-the-shelf components, their designs, dependencies, and links between them. The system allows to acquire planar test objects at various optical resolutions and process the images internally using standard software or application-speciﬁc algorithms, executed on the integrated processing platform. The functionality of a demonstration device is shown in the scope of a banknote authentication application based on print method detection with implementations of the Sound of Intaglio and Sound of Oﬀset algorithms. The beneﬁts of multi-spectral illumination are demonstrated in additional banknote tests.


Introduction
Certain characteristics of prints are invisible for the naked eye, e. g. elements printed at small print sizes or reflection/absorption characteristics of special print inks, which are typical for security prints such as banknotes [Hei12,SFF14,Hei17,CELB18]. Detailed inspections of these characteristics are carried out using special and expensive equipment. Often image acquisition and processing are carried out on different systems. This contribution describes a compact and modular standalone multi-camera image acquisition and processing system with multi-spectral illumination that has been realised for such inspection applications.
The proposed system is based on the outlook concepts presented in [GHD + 16], where a standalone camera-based system for banknote authentication in the context of Point of Sales (PoS) applications was proposed. Similarities of the new design to the PoS design in [GHD + 16] include the compact form factor, the use of cost-effective hardware and the algorithms used to process the captured image data. Instead of only detecting the authenticity of a banknote, as was the case for the PoS design, the approach presented in this contribution aims at enabling the user to carry out further forensic investigations while maintaining a low cost point.
It allows to acquire planar test objects (like prints) at various optical resolutions (between 1 400 dpi and 2 700 dpi) with a camera array of 12 cameras with 5 Mpx resolution and process the images internally using standard or applicationspecific software, executed on the integrated processing platform. In addition, the device offers several interfaces for output, communication, user interaction, and data transfer. All components are commercial off-the-shelf (COTS) components integrated in a single housing as a standalone device. The application of COTS components, which can be used in various other installations and applications, helps to keep the necessity of using bespoke electronic designs at a low level. This also maintained a short development process resulting in effective device costs, which are significantly lower compared to available products.
To the best of knowledge there are no comparable systems on the market, especially regarding multi-camera systems that integrate sophisticated image analysis, although a number of multi-camera systems exist on the market. The VRmagic C [Ste18] system, for example, employs 4 cameras with up to 4 Mpx resolution, a system by Texas Instruments uses 6 cameras with 1 Mpx each [Tex18]. Analysis systems usually use one camera (e. g. 5 Mpx) and provide basic insights, such as appearance under certain illumination conditions (like UV, IR, Lumi) in combination with applied optical filters (like colour filters). The systems themselves provide no decision-making, instead the user has to be an expert to take that part [Ult18,Fos18]. These systems are costly in the range of 50 000 to 100 000 EUR.
Sect. 2 describes the components of the multi-camera system with multispectral illumination, their designs, dependencies, and links between them. A prototype has been realised to show its functionality concept and prove the feasibility. Evaluation of the device is carried out in the scope of a banknote authentication application using the Sound of Intaglio [GGSL09,Loh10,LDH + 13] and Sound of Offset [PL16] algorithms, which are implemented and executed on the integrated processing unit. The effects of multi-spectral illumination are demonstrated in additional banknote tests. These experiments and results are presented in Sect. 3. Finally, the findings are discussed and open points for further research and development are presented in Sect. 4.

Approach
This section describes the approach followed in design and realisation of a lowcost multi-camera system with multi-spectral illumination for document inspection applications. It consists of 12 RGB cameras, a dedicated illumination device, a staged multiplexed image acquisition system, and a powerful image processing platform, on which the application-specific image processing software is executed. All integral parts of the device are described in the following subsections.

Hardware
As for every image processing application, the crucial part of the processing chain lies within its initial step: image acquisition. The image quality requirement is determined by the subsequent applied algorithm used for analysis. In this case, the analysis of commercial raster print using the Sound of Offset (SoO) method as well as Intaglio print analysis using Sound of Intaglio (SoI) need to be supported.
Experiments showed that SoO requires images at resolutions of at least 2 500 dpi when using an area-scan imaging module to allow identification of individual dots of commercial raster print. An alternative to area-scan imaging modules are line-scan sensors (e. g., contact imaging sensors (CIS)). With respect to SoO, usage of a line-scan imaging module decreases the minimal necessary resolution to 1 200 dpi per colour. This is due to its main advantage, namely one dedicated sensor per colour channel. Therefore, no colour subsampling takes place during image acquisition for RGB image acquisition, and, consequently, colour interpolation is not necessary. In contrast, this is the case for area-scan imaging modules, which typically implement a Bayer-pattern-based approach to retrieve colour images. By the utilisation of such colour reconstruction algorithms, sharpness and consequently details in the image are lost. In other words, line-scan imaging sensors reveal more details in RGB images than area-scan imaging sensors working at the same resolution (cf. Fig. 1). However, COTS line-scan imaging modules, which would be applicable in the presented system, are not available on the market. An alternative is the design of a bespoke line-scan imaging module, which also requires moving components. Integration of moving components is to be avoided, as error-free acquisition of images requires high-precision mechanical components along with precise synchronisation of sensor triggering and data readout. In addition, the development of such component takes long and is costly.
In consequence, the system design is based on COTS area-scan sensors. The entire image acquisition part is implemented based on hardware from the Raspberry Pi [Ras18] ecosystem. The 5 Mpx RGB cameras [Ard18b] are standard Raspberry Pi camera modules. Their optics are improved by replaceing the standard M12 lenses with high-quality lenses, having focal lengths between 4.0 and 6.0 mm. The distance between camera and document plane is variable. By application of these lenses and adjustment of the distance between camera and object, it is possible to facilitate optical resolutions between 1 400 dpi and 2 700 dpi with the 5 Mpx cameras. One camera covers approx. 47 × 36 mm at 1 400 dpi of the entire acquisition area (160 × 70 mm). Thus, to cover the entire area, 12 cameras are arranged in a 4 × 3 matrix. This setup allows for flexible adaptations to the requirements of a specific application. If, for example, it is necessary to acquire the entire object and at the same time inspect the central part of the object at high resolution, the central columns are equipped with lenses allowing higher resolutions compared to the rest.
These 12 cameras need to be controlled by a single device that handles the image acquisition. There are no products available on the market that can control this number of cameras out of the box (cf. Sect. 1). The choice for carrying out image acquisition with components from the Raspberry Pi ecosystem is advantageous in this regard. ArduCAM offers camera multiplexer boards [Ard18a] that facilitate the operation of up to 4 cameras with a single Raspberry Pi. These multiplexer boards can also be serially cascaded to extend the number of cameras connected to the single Raspberry Pi. This has been exploited in the design of the proposed system: Sets of 4 camera modules are each connected to one of three ArduCAM camera multiplexer boards, all serially connected to one standard Raspberry Pi Zero that controls the image acquisition. When the Raspberry Pi receives the command to acquire the images of each camera, it iterates sequentially over the 12 cameras, receiving one image per camera. Cameras, multiplexers, and Raspberry Pi communicate over the standard MIPI (Mobile Industry Processor Interface) port.
A main challenge arising with this setup is the illumination design. Using a single source of light would be insufficient since it fails to illuminate a document homogeneously and create shadows or over-exposure in the images. To avoid such drawbacks, a custom illumination concept has been created. The illumination consists of a matrix of 39 RGB LEDs (type: WS2812B [Wor18]). It allows for a uniform illumination of the tested objects. To tackle shadow casting and over-exposure, the ability to control individual LEDs comes in handy. By only activating those LEDs that lead to optimal image quality for a specific camera also assures images free of reflections and scattered light. In addition, the applied LEDs support free RGB parameterisation. Each of the colour channels is set up with an 8-bit parameter vector. This capability enables a free choice of the light colour and intensity for each of the 39 LEDs, resulting in multi-spectral illumination of the test object. Such type of illumination (light emission at a certain range of the spectrum) is referred to as lumi illumination. The control of the LED matrix board is realised over SPI (Serial Parallel Interface), connected to the Raspberry Pi used for image acquisition.
Another requirement is the processing of the acquired images within the system itself. This is to be carried out on a powerful, user-programmable COTS processing unit, which offers manifold communication and interaction interfaces at small scale. The Raspberry Pi platform is a possible candidate. It is however not utilised due to computation power constraints. An UP 2 board [AAE18] has been chosen instead. It is equipped with an Intel N4200 CPU (up to 2.5 GHz core frequency), 8 GB DDR4 RAM, and 64 GB eMMC storage. Its interfaces (3 × USB 3.0 (Type A) + 1 × USB 3.0 OTG (Micro B), 2 × USB 2.0, 2 × Gb Ethernet, WiFi, Bluetooth, HDMI, DisplayPort) allow interaction with the user along with flexible result output to screen or printers, and interconnections with other devices. It is, for example, connected to the Raspberry Pi by USB to control the image acquisition and receive the images from the cameras. Its operating system is Microsoft Windows 10 Pro allowing all standard and also userprogrammed software to be executed on the system. As such, it was the most powerful compact general-purpose computation device at the time of system specification. All is included within its 86 × 90 mm footprint, which makes it possible to integrate all hardware components in one housing. Altogether, the system consists of these core parts, which are schematically sketched in Fig. 2: cameras, illumination, processing unit. All of these hardware components need power supply and communication interconnections. This task can be realised quickly by connecting the components with wires and attach dedicated power supplies to them. However, such approach demands more space in the housing than designing a bespoke main board, which integrates the electric connections for power and communication. Another advantage of a dedicated main board is the reduction of points of failure, thus simplifying search for failures and increasing overall operation stability, by integrating as many wired connections as possible. It is the "heart" of the system, powering the components on and off, controlling their operation mode (including boot and shut-down), and managing the entire communication between the components.
All components are integrated to a stand-alone device in the compact housing shown in Fig. 3. The housing's size of 250 × 165 × 110 mm facilitates easy portability. It features a switch on the top side to turn the device on and off, and trigger image acquisition. User interaction is realised by bringing out the UP 2 's HDMI and USB ports (for display, input devices, and external storage) on the back of the housing.

Software
The image acquisition and processing software is implemented in a modular client-server structure. The server side acts as the interface to the user and implements the image processing, whereas the client side is responsible for image acquisition. Figure 4 shows the software structure diagram. Image acquisition and authentication processes are controlled by and visualised in a Microsoft Windows graphical user interface (GUI) on the UP 2 board, the server side. It also implements the image inspection and result output. In addition, all images are automatically stored to the device for backup and traceability reasons. A screenshot of the GUI is depicted in Fig. 5. The entire image acquisition process is controlled from within the GUI. A test object is placed on the glass plate for inspection. After starting the process (e. g., by pressing "Start Analysis" in the GUI), the software sends the command to trigger image acquisition from the UP 2 board over USB to the client part of the software executed on the Raspberry Pi. It is responsible for controlling the cameras and LED matrix board over custom scripts implemented in Python. The client iterates over the 12 cameras to acquire one image from every single camera and at the same time activates only those LEDs, which allow for homogeneous illumination of the test object, depending on the currently active camera. The images received by the Raspberry Pi are then sent to the server side for further processing. After reception, the images are immediately shown in the GUI to allow inspection by the user.
The GUI also forwards the acquired images to a banknote authentication application, which is also executed on the server side, and collects and outputs its results. It is based on the concepts presented in [GHD + 16], where the authentication is realised by print method detection. One part is the identification of banknote-typical Intaglio print using the Sound of Intaglio [GL10,LGS14,GDL15] algorithm. It exploits the detection of inherent characteristics of the Intaglio print to authenticate banknotes worldwide and is trained beforehand to adapt to the specific characteristics of the given camera system. As large-volume counterfeits are produced using commercial offset print, the second authentication part implements the Sound of Offset [PGLS16,PL16] algorithm. It is capable of detecting commercial offset print by looking for intrinsic periodic artefacts in an acquired image. Training is not necessary for SoO.
Both analysis parts are combined in the implementation similar to the procedures presented in [PL16]. First, the image is acquired on the device (cf. above). Each acquired image is divided into regions of interest (ROI), which are each subsequently authenticated using SoI and SoO. In opposition to the procedure presented in [PL16], regions of size 400 × 400 px are analysed. The final results of both authentication algorithms are combined and afterwards presented to the user. A flow chart depicting the authentication process is shown in Fig. 6. Results of the authentication process are visualised in the GUI by colour overlays in the processed image. This allows for immediately linking the authentication result to the processed data.
The entire system's functionality has been validated as described in the following section.

Experiments and Results
A demonstrator system has been built to validate the functionality of the system design presented in the previous section. Its performance for the authentication of banknotes with both the Sound of Intaglio and Sound of Offset methods is determined. In addition, capabilities and benefits of the integrated multi-spectral illumination are evaluated.
Arranged in a rectangular raster of four columns and three rows, the twelve cameras are individually equipped with lenses, adjusted to the specific use case. In this demonstration system, the Intaglio-printed main motif of the genuine banknote is analysed by SoI, whereas SoO analysis is carried out on the remainder of the note that contains no Intaglio print. The SoI algorithm demands images acquired at an optical resolution of at least 1 200 dpi (area-scan sensor); for SoO images of at least 2 500 dpi are required. Fig. 7b shows an acquired image that was digitized with 2 700 dpi. Raster dots are clearly visible.
For demonstration purposes, the SoI method has been trained for authentication of all euro denominations (5, 10, 20, 50, 100, 200, 500 EUR) of the first series. Each camera image is split into quadratic regions of interest (RoI) of 400 × 400 px. Three of these regions per image are authenticated with the respective authentication method to handle the trade-off between precision and execution time of the authentication. A test set of 10 genuine banknotes per denomination was used, i. e., 70 banknotes in total. These notes were taken from circulation and are thus in different conditions. Intaglio print was detected correctly by SoI on each of the banknotes and only for those regions printed with Intaglio. In addition, 330 reproductions were evaluated. These reproductions were created on SOHO (small office/home office) ink jet and laser printers. The system was not able to identify Intaglio print on any of these prints, which represents the ideal outcome. The SoO analysis also worked correctly for tested samples: Those bearing periodic print rasters were correctly identified.
Besides accuracy, the authentication duration represents an important measure for each algorithm. Durations for SoI and SoO for the analysis of a single RoI are shown in Table 1. In addition, authentication durations on a Raspberry Pi 3B board, as a typical representative of a low-cost single-board computation device, are given as well, to achieve comparability to [PL16]. As was expected, the implementations on the UP 2 board are executed faster. The multi-spectral illumination integrated in the demonstrator has also been evaluated using different prints of the same detail of a specimen banknote. It is available in its original (genuine) form, and as ink jet and laser print reproduction. When inspected under white light, the prints look similar. Acquired under lumi illumination conditions (i. e.,coloured illumination), the images reveal significant differences depending on the type of print, substrate, ink, printer model, etc. (cf. and [SFF14,GDL15]). Table 2 shows the images obtained under white light and cyan illumination (approx. 450 nm).
No substantial difference is visible in the RGB images under both illumination conditions. Though, the CMYK (cyan, magenta, yellow, black) decomposition of the images reveals a significant difference in the Y channel under cyan illumination. Whereas no Y intensity is detected for the genuine, the ink jet bears contributions to the Y channel. Such a clear distinction is not detectable for any other channel, regardless of the illumination. Hence, lumi analyses are possible and beneficial on the device.

Conclusion and Outlook
Overall, the current hardware configuration of the system is usable for existing algorithms. Functionality has been shown in the scope of a banknote authentication application applying the Sound of Intaglio and Sound of Offset methods. In addition, image acquisition under lumi illumination conditions is shown.
The entire system design has been targeted at a low-cost, modular system that can be adjusted to specific application requirements by incorporation of COTS components. This target is achieved with the presented system. Whereas commercially available analysis system are in the range of multiple 10 000 EUR, the hardware involved in the assembly of a demonstration device presented above costs approx. 2 000 EUR. When achieving serial production state, this is expected to easily come down to below 1 000 EUR due to scale effects. All components can be exchanged by other standard components without the need of a system redesign. For example, the number of applied cameras can be changed arbitrarily in the range from 1 to 12, and the LED matrix can be adjusted by applying other LEDs (e. g., IR or UV LEDs). Along with the possibility of the integration of application-specific software, flexibility and future security is achieved.
A couple of restrictions have been identified, as well. By use of the 12 cameras, the entire acquisition area (160 × 70 mm) can be acquired at an resolution of around 1 300 dpi. While this resolution is sufficient for SoI, it will not be sufficient for the detection of high-resolution raster prints by SoO because single printed dots can not be distinguished from each other. For detection of stochastic printing methods it is necessary to use area-scan sensors with a resolution of at least 2 500 dpi. If however such a high resolution was employed, it would not be possible to cover the complete area. For a complete coverage of a 500 EUR banknote at 2 500 dpi, it would be necessary to either use 48 cameras with 5 Mpx resolution, arranged in a 8 × 6 matrix or 15 cameras with 14 Mpx resolution (5 × 3 matrix). The capture time increases linearly with the number of cameras when Raspberry Pi camera modules are used. Therefore, it is advisable to use industry cameras with USB 3 interfaces. These could be connected directly to the analysis pc, which in turn would make the comparably slow Raspberry Pi camera relay unnecessary. To achieve good analysis durations, the use of a PC with a high performance processor is advised. In addition, there are multiple enhancements that could be implemented to improve robustness or facilitate further features (multiplexer integration to mainboard; RGBW instead of RGB LEDs; add IR and UV LEDs).