Pose-relay videometrics based ship deformation measurement system and sea trials

The measurement of ship deformation has been a difficult problem for the global ship building industry and navigation field research. So far, an effective and proven method for measuring 3D position and attitude deformation remains unavailable. This article proposes an innovative pose-relay videometrics measurement system for ship deformation on the basis of primary research and experimental validation. The system offers solutions to the important technical problems such as infrared marker manufacture, high-precision detection and multi-camera synchronization and control. Sea trials were carried out, and reasonable, long-term, and continuous 3D data of position and attitude deformations between a given deck point and the ship reference, and between the ship’s head and the stern were acquired for the first time. The experiments have highlighted the considerable applicability of the proposed method of deformation measurement and long-term monitoring of ships and similar large structures.

When voyaging, ships are subject to inevitable deformations caused by the external stress of loads, waves and/or environmental temperature changes. Research shows that static angular deformation because of changes in cargo and fuel distribution, and uneven exposure to sunlight can amount to 1°-1.5°. In addition, hull motion, wave impact, helm steering, etc. will cause dynamic deformations of the order of units of angular minutes for yawing angles, dozens of angular minutes for pitching angles, and fractions of an angular minute for rolling angles [1].
Apart from its influence on the precision of shipboard equipment, ship hull deformation leads to alignment errors of strap-down inertial navigation systems. In recent years, researchers have studied and developed ship deformation measurement and correction technologies, among which the inertial measurement units matching method [1] and the Fiber Bragg grating method [2][3][4] are the most frequently *Corresponding author (email: yuqifeng@vip.sina.com) used. The former requires a precise Kalman filtering model to estimate the deformation; the latter, a strain-sensor measurement in essence, applies to local deformation only and requires an accurate model that converts the strain into an attitude angle to ultimately obtain the deflection angle with the purpose of eliminating the attitude angle error of the deck surface. In China, the Changchun Institute of Optics, Fine Mechanics and Physics designed an auto-collimationinterferometry based optical measurement system [5][6][7][8][9], which has found application in a type of survey ships [8]. The system, enclosed by big steel pipes which provide an optical path and eliminate atmosphere disturbances, is easily operated and is designed with proven technology. However, its steel pipes are heavy and voluminous thus it has to be installed on the deck, which usually blocks passageways [9]. Some researchers have also investigated methods such as the polarized energy measuring, the double lamp-houses and CCD, the double frequency polarized, the liquid pressure, the GPS and the united base methods, most of which cannot satisfy real-time measurement requirements [10,11]. DInSAR techniques [12] have been applied to measure deformations of slow urban deformations; however, they do not seem to be appropriate for ship deformation measurements.
So far, measurement of ship deformation and compensation remains problematic in the navigation field because of the difficulties of continuous large-scale measurement. An effective and proven method to measure the 3D position and attitude deformation of long-distance and nonintervisible targets over a long term has yet to be found. To address the issue, we proposed a pose-relay videometric method (previously called the fold-ray videometric method and camera-relay videometric method) using a series of cameras [13][14][15]. Our method transfers 3D geometric information of random regions with a flexible fold-ray optical path. Thus it is capable of real-time measurements of 3D positions, attitudes, and deformations between nonintervisible objects and those of intervisible objects with a very large angle of view. Therefore, it should find application in the deformation measurement of ships and other large structures. Our earlier research [13][14][15] focused on the working principles and laboratory validation of the proposed system. We then made technological breakthroughs in camera and relay-station calibration [16], infrared marker manufacture and highprecision detection, multi-cameras synchronization and control [17], etc. in our attempts to put it to actual use in large-structure deformation measurement. Over the past several years, we developed a videometric system for ship deformation measurement and carried out multiple sea trials. For the first time, we acquired reasonable, long-term, and continuous 3D data of position and attitude deformation that provided references for the analysis of the influence that deformation has on observation systems and its appropriate compensation. This helped evaluate the mechanical properties of deformation including rigidity, fatigue and fracture, with the final purpose of improving ship maintenance.

Principle of pose-relay videometrics with camera-series
The principle of our pose-relay videometric method with the camera-series follows.
Conventional videometrics are founded on linear optical path measurements which means that an intervisible optical path between two objects is necessary. When it involves two or more nonintervisible objects or large view angles between the objects and the camera, the conventional videometrics become invalid for measuring their 3D positions and attitudes because of the camera's failure to capture images of the objects.
The pose-relay videometric method with camera-series has the following outline: (1) it can build a fold-ray camera-series of cameras and markers to connect arbitrary areas flexibly; and (2) it can transmit and calculate the position, attitude, and deformation of targets relative to reference base. In our earlier studies, the method was called camera-relay videometrics and fold-ray videometrics as (1) implies. It is now renamed the pose-relay videometric method with camera-series to better illustrate its functionality.
As shown in Figure 1, the target and the reference are connected by a series of cameras and markers. S i (i=0, 1,···, n) is called the measuring station, and the chain from S 0 to S n is the camera series. The target's attitude and position relative to the reference base can be expressed as [15] 1 where i+1 R i and i+1 T i are the rotation matrix and the translation vector between coordinate systems S i and S i+1 . From eqs. (1) and (2), the attitude and position of the target relative to the reference base can be calculated as long as the attitude and position of each relay station are obtained [18][19][20][21][22][23] via measurement or calibration. Because each relay station is a rigid body, neither the rotation nor the translation of the stations as a whole brought by the installation deformation will affect the final measurement results.

Ship deformations videometric measurement system
Given that the pose-relay videometrics method is capable of measuring the position and attitude of nonintervisible objects, it can be used to measure the displacement and deformation of the attitude angles of shipboard observation equipment. Figure 2 illustrates a deformation measurement system for an on-deck observation system.
In Figure 2, p is the reference base, which can be an inertial navigation system that serves as a navigation reference, or another reference such as the keel of the ship's hull that is determined by the ship designer or measurement personnel; and t is the target to be measured, which can be an observation system or any other point of interest. Adaptability and precision are critical to the practicability of a pose-relay videometrics based measurement system for ship deformation.
Because of the diversity of the fold-ray relay form and the ease of varying the shooting distance by adjusting the focal length of the camera lens, the target to be measured is of arbitrary orientation. A certain number of relay stations are set up between the target and the base; the number and relay form are determined by the actual environment and the flexibility required. So the pose-relay videometrics system with camera-series can measure the pose and deformations of a nonintervisible target without causing damage to the ship's structure. Any reconstruction will be minor if required. That means the system's adaptability is assured.
The measuring precision is ensured mainly by the following functions: camera calibration, relay station calibration, marker manufacture and high-precision detection, and multi-camera synchronization and control. Both precision and maneuverability of the system are the top priorities in the study of the key functions. Considering the conditions under which ships deform, we designed and manufactured illuminated markers with an infrared LED and passive reflective markers with highly reflective sheets lit by an infrared source. A band-pass filter with a narrow corresponding wave length was installed in front of the lens to remove the environment's influence. A camera calibration technique based on controllable rotation was implemented to calibrate the accurate interior parameters. A synchronous image acquisition and processing system for multiple cameras was designed and implemented, to ensure that the synchronous frames of every camera could be processed synchronously. This realizes instantaneous synchronous freezing at a given time and avoids the influence of deformation at some installing points of the relay station in the camera-series.
Based on studies of the above key functions, a pose-relay videometric measurement system for ship deformation was developed. Validation experiments and sea trials were carried out to ensure the viability of the system. The cameras used in the trials were the GRAS-50S5M manufactured by Point Grey Research ® Inc. with a resolution of 2448 by 2048 pixels. The results showed that the angular accuracy of the measuring system was less than 30 arc-seconds. Precision could be improved if higher resolution cameras and a more accurate marker detection method were adopted. The following section discusses the sea trials and results.

Sea trials and data analysis
Three sea trials were carried out after the measurement system was developed. In two trials, deformation measurement between the ship's head and the stern, and another between a given deck point and the ship's reference, were carried out.

Deformation measurement between a ship's head and stern
Attitude angular deformation exerts a major influence on the precision of shipboard systems. This section thus focuses on angular deformation data. It is expressed by three angles; pitch, yaw and roll [7]. Head-and-stern deformation measurement was undertaken in the first sea trial, and was conducted on a 130 m long and about 13 m wide ship. The setup of the measurement system is shown in Figure 3. There are 5 relay stations at which the measurement reference S 0 is at a distance of about 70 m from the target S 4 . The various items of equipment making up the measurement system are shown in Figure 4, which shows relay station S 1 (left side) and S 3 (right side).
The sea trial involved a voyage lasting 8 h in a 3-4 level sea state. Part of the deformation curve of the pitch, yaw, and roll angles within a 2 h period is given in Figure 5.
The major inflexions marked as points A, B, and C in Figure 5 correspond to the helm operations that can be observed in the recorded information. The maximum deformations of the roll, pitch and yaw angles are 7.2, 2.8 and 2.5 arc-minutes, respectively. Because of the absence of exact records of the voyage operations and sea states, reasons for other minor inflexions cannot be presented here.

Deformation measurement between two points above and under the deck
The measurement system connects a given deck point and the ship reference on the fore deck, the vertical distance between them is about 8 m and the horizontal distance about 16 m. The experiment was divided into two parts. The first was carried out when the ship was berthed, and the second when the ship was at sea. As winds and waves have relatively less influence under berthed conditions, the measurement   data for the first part are compared with those obtained under at sea conditions to observe the influence of waves and helm operations.
The angular deformation at berth in a harbor is shown in Figure 6, in which ΔAx, ΔAy, ΔAz represent the target's three angular deformations relative to the reference coordinate system. Figure 6 indicates a relatively small angular deformation at berth.
Data of angular deformation at sea are illustrated in Figure 7. Data were collected from 10:00 to 13:00 the day after measurements were made with the ship at berth.
Parts of the helm operations extracted from the navigation log are shown in Figure 7. It is observed that nearly all the larger deformation points correspond to certain helm operations. When the ship is at anchor (shown as region E-F in Figure 7), the angular deformation, ΔAz, still remains large; and the analysis indicates that it is probably related to the ship's yaw angle and the wind direction. The ship sailed a relatively short distance in about a 1 to 2 level sea state, so the deformation was relatively small. In higher sea states, it would presumably be greater.

Implications of sea trials
The pose-relay videometric system with camera-series for ship deformation measurement developed after primary  research and technique innovation has been validated in three sea trials. Reasonable, long-term, continuous 3D data of position and attitude deformation were acquired for the first time. The conclusions drawn from the sea trials are as follows.
(1) The proposed system for ship deformation measurement is capable of measuring the deformation between a ship's head and stern.
(2) The proposed system is capable of measuring the relative pose and deformation between observation equipments. After real-time measurement and data processing, the data collected by the observation equipment can be compensated for and transmitted to another system.
(3) The proposed system is capable of connecting a key point of interest such as equipment bases with a selected reference base such as a compass platform, and measuring the angular deformation at an arbitrary point relative to a reference base to facilitate the modification of angular information of a key point.
(4) The proposed system can yield useful measurement data. The evident inflexions of the measured data are found to correspond to voyage controls such as helm and anchor operations. The roll angle deformation caused by helm operation in the head-and-stern measurement exceeds 7 arcminutes. Deformation will obviously be greater as the sea state becomes heavier.

Conclusions
In this report, a pose-relay videometric method with a camera-series based on a new concept of optical measurement was proposed as way to make deformation measurements of large structures such as ships. The system was realized after research into important functional problems such as infrared marker manufacture and high-precision detection, and multi-cameras synchronization and control. The simulation experiments showed that the precision of the attitude angle measurements was less than 30 arc-seconds, which would be improved, for example, by employing higher resolution cameras, and more accurate location methods. Sea trials were then carried out and long-term 6-D ship deformation data obtained. The sea trials validated the viability, reliability and practicability of the proposed deformation measurement system..
The proposed system provides an effective, easily implemented and highly-accurate solution to ship deformation measurement and correction. After further research through experiments, and with improvements, the system is expected to play an important role in deformation measurement and structural monitoring of ships' hulls and other large structures.