Introduction

The geometric distortion in an image of an optoelectronic display system like head-up display (HUD) is mainly caused by different display sources, such as, cathode ray tube (CRT), the electronic assembly, the optical assembly, and the mechanical interfacing mismatch between the CRT plane and optical plane. The basic approach for correcting geometric distortion is to take image from the display system, compare it with the reference image, and correct it using suitable methods. The conventional solution of this task is manual assessment using high precision instruments like theodolites, collimators, microscopes and photometers that are not only expensive but also object-specific. The evaluation and testing through these equipment is tedious and time consuming along with the dependency on human factors. Also, the measurement accuracies vary from user to user [1–7].

The testing using these equipment is done mostly by a person who has minimal understanding regarding optical testing methods. This results in skewed analysis of outcomes. However, in a few systems, a mechanism is used to correct geometric distortion of an optoelectronic system by matching the image of unit under test (UUT) with the desired image pattern display which is fixed. This method results in a fair degree of accuracy, but manual intervention is still there in the system. The quality of the whole process is based on the handling human who has limited accuracy, low efficiency, and low reliability. Also, the total time spent by a human being is limited on his abilities and intelligence [8–15].

Now-a-days, various computational techniques like coordinate transformation and collinearity equations are available for geometric estimation and correction in an image. These mathematical techniques provide a higher precision as compared to the equivalent techniques. Coordinate transformation technique is usually implemented by first selection of coordinate transform formulae that includes various transforms like, Helmet transform, affine transform, pseudo affine transform, projection transform, or using polynomials. The accuracy of geometric correction is characterized by standard deviation in pixel units of the image. Another method uses collinearity equations that represents the physical model is based on the geometry of the sensor, ground coordinates and image coordinates. These techniques are accurate but require prior knowledge of the X/Y coordinates of the image. Recently, various image processing algorithms, based on these techniques are developed for real-time applications. Such systems are widely used for industrial applications. There are a large number of manual and automated theodolite systems which are mainly used for larger objects in industry. The current image-editing tools allow basic functions in a single action which are applied globally and uniformly to the image. Different operations such as increase/decrease the brightness/contrast of an image, measurement of X/Y coordinates or pixels of an image, add/remove a certain colorcast from the entire image, etc. can be done using the toolbox. A few image-editing techniques can be combined with these existing tools but these type of combinations limit to single image adjustment at a time along with the implementation difficulty [16–24]. Therefore, an application program interface is required, which allows the placement of one image over the other for the analysis of distortion in the images.

The present development meets the requirements by enabling users to perform complex image-editing operations in a single action. A graphical user interface (GUI) is developed that allows the user to place multiple reference points in an image, and provides all information necessary to analyze the distortion in an image relative to the reference. The GUI deals with the measurement, visualization, and storage of X/Y coordinates of the images that are located at distinct locations and have properties that can be measured or quantified. It also enables the user to superimpose an image over the previous image or vice versa and also to move the image in up/down or left/right direction for maintaining the same reference with respect to another image. The complete X/Y coordinates database can be further processed for the analysis of distortion in an image with respect to the distortion free template image.

Existing Methods of Coordinates Measurement

The measurement of X/Y coordinates of the image is one of the main prerequisites for applying geometric error correction using various computational techniques. Various GUIs are available for the measurement of X/Y coordinates with reduced design complexity and increased system accuracy and reliability. The most of available GUI’s and image processing/editing tools like Adobe Photoshop, Paint, Picasa, Microsoft Office picture manager etc. provide the option for X/Y coordinate measurement. Even though it display the coordinates on the bottom left or right corner of an image, these tools do not have the facility to pin/mark the coordinates or desired pixels. These tools provide coordinate data based on where the cursor is pointed/placed which is further difficult to select due to sensitivity of the cursor of any laptop or computer [25–30]. Other high processing tools like MATLAB also provides an image viewer tool for determining the X/Y coordinates. It displays the information regarding the location of coordinates of an image in the bottom left corner of the tool. It also has the flexibility to record the X/Y coordinates to the clipboard. Data cursor tool is also a MATLAB tool which enables to read the data directly from the graph or image by displaying the X/Y coordinates of the selected points. This tool allow to interpolate the data points, select and display the multiple data points, displays the coordinates in a cursor window, export the data points to the workspace and edit the data point display function [31–33]. IMAQ vision tool is another powerful tool by National Instruments LabVIEW and is currently used for many image processing and analysis applications. IMAQ vision tool uses the image display control function to display the image information on the bottom of an image information indicator box which includes the size of the image, image type and the pixel information. The point function in the ROI tool available in IMAQ vision enables to view the X/Y coordinates of the selected pixel on the right side of the image viewer window [34–37]. The developed GUI outcasts all these tools due to its multiple pin marking coordinate features, flexibility for image pixel adjustment to change the image width and height, ability to record all marked points in a Microsoft Excel file. The developed GUI also has the ability to work on two images simultaneously by fusing both the images together to analyses the geometric distortion between the reference and the distorted image.

Test Set-up for Image Capturing

Geometric correction is a method to correct the abrasions of the display in which a distortion free image is compared with the distorted image. Keeping all other parameters fixed, to determine the correction parameters of the distorted images, following steps are need to be carried out [4–7, 21–24]:

  • (a) Place the test set-up for geometric distortion correction at a fixed location. This test setup consists of HUD mounting tray, camera, and camera holding mechanical assembly.

  • (b) Capture the image projected by the corrected HUD assembly (by suitable correction mechanism). This image acts as the template image.

  • (c) Keeping all other parameters such as mounting tray, camera, camera holding assembly, etc., fixed, replace the HUD by another HUD whose image has to be corrected for geometric distortion. Obtained image will be compared with the reference image taken in the above step for determining correction parameters.

  • (d) Repeat this process for different HUDs whose image requires geometric distortion correction.

  • (e) All the images taken above will be compared and processed in geometric distortion correction utility.

Graphical User Interface (GUI) for Coordinates Measurement

The geometric distortion correction utility is developed to measure the errors between the reference image and the images with geometric distortion. Primarily, the GUI application is developed for the measurement of X/Y coordinates/pixels of an image from an optoelectronic display system which had to be checked and analyzed for a correct fit. The interface also enables a user to measure the X/Y coordinates or pixels at the different locations in the images, movement of an image(s) in up/down or left/right direction for the centering or maintaining the same reference point, and several other image-editing functions. It allows the users to determine the coordinates of various point just by clicking on the points intended.

The main feature of proposed GUI is its modular concept that can be controlled by a user who can perform the various operations by selecting particular buttons. The complete application is designed on a Visual C++ platform and serves as major interface between user and system. The GUI can accommodate an image of resolution up to 500 × 675 pixels. It provides all state-of-the-art features of modern interfaces such as image control, mouse positioning control, image up/down and left/right control, X/Y coordinates modification control, pincushion distortion processing window and database management, etc. for the analysis of geometric distortion of an image with respect to its template image.

This GUI comprises of four key elements that act together to provide an accurate measure of the extent of geometric correction and also provide ease in the processing of correction data. These key elements include image display window; two image processing blocks: one for reference image and second for distorted image and the pincushion data file block which enables a user to store the coordinates selected in the two image processing blocks directly into an information file with ‘.ini’ extension or to update an already existing data file. User can also export this ‘.ini’ file into an excel file for further processing of the acquired data. Figure 1 shows the GUI utility used for measurement of image coordinates and further processing.

Fig. 1
figure 1

GUI utility for coordinate measurement and processing

Full size image

Although many geometric distortion correction methods and tools that are available today provide reliable coordinate measurement up to some extent, they appear wanting in one way or the other. Some coordinate measurement systems that are designed on java-script can reliably reproduce the coordinates of each pixel of the image. The problem with this kind of utility is that it provides no means to adjust the position of an image whose coordinates are to be measured. The application uploads the image and adjusts it according to a predefined ratio. This poses a problem when the image is larger or smaller than what is generally accepted by the application. This problem may have grave consequences when the application is used to compare the coordinates of an image with some reference image. If the width and height of the image does not match with the reference image at the time of taking the coordinates of the points, it may produce inaccurate correction parameters as these parameters are directly correlated with the error between coordinates of test and reference image. The geometric distortion GUI developed here enables the user to position images manually thereby allowing adapting to the image size. This GUI integrates all features that facilitate coordinate measure, coordinate error measurement and data processing.

There are some other applications which uses reference grid pattern method for coordinate measurement and error determination. In such cases, the coordinates of distorted image is compared with the coordinate of a grid pattern which is assumed to be free from errors. The difference in the corresponding coordinates is then calculated and least square method is employed for error determination. This method is not only tedious but also more susceptible to errors as the grid pattern used is need not be actually free form distortions. Whereas this GUI provides more accurate results as the actual desired image which has been corrected before taking its coordinates is used as the reference for the coordinates of other images. In addition to all the features that GUI offers over other conventional techniques, there is yet another feature that contributes to an increased accuracy for this GUI. The data obtained from coordinates measurement by this GUI is stored as ‘.ini’ information file and when needed can be exported to the excel format. This provides ease of image data processing and obviates any human errors involved in documentation as it minimizes human intervention and obviates the need to manually record all the data entries. During measurement, all relevant information of X/Y coordinates of the images is directly stored in the database as the File ‘Data.ini’. However, the GUI has a flexibility to store this database with any other name by using the browse button in the ‘Pin Cushion Processing’ block. To provide GUI with different options and characteristics, button assignments, text indicators and image display blocks are provided. The flowchart of the proposed GUI is shown in Fig. 2.

Fig. 2
figure 2

Flowchart of the GUI for geometric distortion correction

Full size image

Results and Discussions

Proposed GUI brings the flexibility to adjust the mouse pointer easily for desired X/Y coordinates of the image by following the left/right or up/down topology. This is achieved by clicking the left/right and up/down buttons provided in the GUI window after selecting any nearby coordinates. The GUI offers flexibility to change the location of any X/Y coordinates by selecting particular coordinate data from database dialog box and subsequently using the left/right button for modification of X coordinate and up/down button for modification of Y coordinate and finally press the ‘redraw’ button. The GUI also offers a flexibility to clear the complete database of X/Y coordinates of an image by using the ‘empty’ button in the GUI. However, the selection of the X/Y coordinates in the reference and distorted image shall be followed in the same manner. It allows the user to store the X/Y coordinates of the corresponding data points of both images for further use and processing. The data of both images can be checked, analyzed and exported to Excel sheet. These coordinate’s database is used to make the statistical analysis of the distortion in an image at the different measuring points with reference to the corresponding points of the reference image. The GUIs measures X/Y coordinates of the image and estimate the geometric distortion between the two different images. The results of different functions present in GUI are depicted in Figs. 3 and 4.

Fig. 3
figure 3

GUI window with selected points of the Image 1

Full size image
Fig. 4
figure 4

GUI window with selected points of the image to be evaluated, that is, Image 2

Full size image

With the designed GUI, multiple X/Y coordinates of the reference image as well as distorted image were marked successfully. Estimation of the geometric distortion in a distorted image with respect to the reference image of a CRT and active-matrix liquid-crystal display (AMLCD) based system by superimposing one image over the other is shown in Figs. 5 and 6, respectively. The geometric distortion error could be calculated with the help of qualitative difference in the form of single fused image as well as quantitative difference in the form of selected coordinates. The desired computational techniques can be subsequently to correct the distortion errors. Design and implementation of the proposed GUI provides the solutions for geometric distortion measurement and correction in optoelectronic display systems like HUD.

Fig. 5
figure 5

Reference and distorted images of CRT based system displayed simultaneously on the GUI for the estimation of geometric distortion

Full size image
Fig. 6
figure 6

Reference and distorted images of AMLCD based system displayed simultaneously on the GUI for the estimation of geometric distortion

Full size image

Conclusion

Various reliable coordinate measurement methods are available. It is not possible to adjust position and size of an image whose coordinates are to be measured using these methods. The limitations of these conventional methods provide the background for the need for a GUI based coordinate measurement system. The basic approach for the distortion correction takes the reference image from the reference HUD, corrects it and uses it as template image for the correction of images from other HUDs. The proposed GUI uses the same approach where the coordinates of the template image and the distorted image can be evaluated using various widgets and icons provided in the utility. The error between the coordinates of the respective points corresponds to the measure of distortion and can be employed further to determine the correction parameters for the image distortion correction. The proposed GUI minimizes human interference in executing the process of geometric correction making the technique swift and accurate as compared to the conventional methods of geometric correction.