1 Introduction

Pipelines are the primary assets of the oil and gas midstream industry. There are currently 2.8 million miles of regulated pipelines in the United States alone (PHMSA 2021a). The oil and gas pipelines, which typically serve a greater transportation demand than trucks and trains can handle, are responsible for moving about 64% of the U.S. energy commodities across the nation (PHMSA 2021a). In the present era, the role of pipelines is crucial as the consumption of natural gas rises every year. Natural gas consumption in 2020 increased by approximately 21% to 30.4 trillion cubic feet, compared with 24.08 trillion cubic feet in 2010 (EIA 2021). Despite their indispensable role, pipelines are vulnerable to various types of risk. According to PHMSA, about 12,505 pipeline incidents occurred in the United States between 2001 and 2020, resulting in 270 fatalities, 1176 injuries, and USD 9.9 billion in property damages (PHMSA 2021b). Apart from the property damages and environmental loss, Jackson et al. (2014) stated that more than USD 3.1 billion worth of natural gas was lost and unaccounted for annually between 2005 and 2010 in the United States.

Pipelines can fail due to various causes, such as corrosion (internal and external), pipeline misalignment, deterioration of support structures, overgrown vegetation, equipment failure, improper operation, and other external forces (Fazzini et al. 2009; PHMSA 2021c). Meanwhile, a large portion of failures can be prevented by increasing pipeline system resilience through timely problem detection and preventive maintenance. To this end, this study proposes DARTS (drone and artificial intelligence reconsolidated technological solution) to monitor and inspect pipeline conditions and support preventive maintenance of pipelines. This solution is particularly aimed at identifying the targeted root causes of pipeline failure—misalignment and deterioration of supporting structures—and assessing the pipeline system resilience by processing image data. The research objectives of this study are: (1) to examine the current practices for pipeline inspection; (2) to create a model for integrating artificial intelligence and drone technologies for image data collection; (3) to develop deep learning algorithms for detecting the targeted problems from collected image data; and (4) to develop a resilience assessment model. The results of our research indicate that the proposed methodology helps evaluate pipeline conditions and in assessing strategies for preventive maintenance.

This article is organized as follows. Section 2 summarizes the literature related to the pipeline system and inspection practices, drone-based image data collection systems, and the resilience of pipelines. Section 3 discusses the research methodology, including a testbed for experiments and drone technology used for data collection. The proposed pipeline inspection method for detecting the root causes of pipeline failures is outlined in Sect. 4. Section 5 describes the resilience model. Section 6 presents the test results of the proposed method, followed by the discussions and conclusion of the present study in Sects. 7 and 8.

2 Literature Review

This section summarizes the literature review in three subject areas: the pipeline systems and their inspection practices, the drone-based image data collection system, and the resilience of pipelines.

2.1 Pipelines and Inspection

A pipeline is comprised of various parts, including pipes, pipe connectors, and pipe support systems. All connected pipes must stay in their design positions, supported by pipe support systems. The alignment is the centerline of the actual pipeline on a scale in any direction, and deviation from the centerline can lead to misalignment in pipelines. The ASME B31.3 Standard (ASME 2020) provides the allowable tolerance for angular misalignment, lateral misalignment, and combined angular-lateral misalignments, which must be identified early. In a pipe rack, pipes are aligned parallel to each other, and the piping engineers must ensure a minimum gap between the pipes to provide the sideway thermal movement of pipes. Jia and Feng (2011) stated that the minimum safe distance between two pipelines is critical for protecting the pipelines.

Proper design and maintenance of these supports per the codes and guidelines are required to produce an acceptable life cycle for the piping systems (ASME 2020). Nayyar (1999) described three types of loads acting on the pipe and support systems:

  1. (1)

    Sustained loads are likely to exist throughout regular plant operation, including gravitational forces acting on the pipe, pressure, fluid weight, and so on;

  2. (2)

    Expansion loads are due to displacement of piping from its designed positions, caused by seismic movements, thermal expansion, contraction, and so on; and

  3. (3)

    Occasional loads are expected at irregular intervals during operation, such as from wind, earthquake, and fluid transients.

Pipe support systems can fail when these loads exceed the maximum principal stress and shear stress. Nayyar (1999) specified five failure modes that occur in piping systems: failure by general yielding, yielding at sub-elevated temperature, yielding at elevated temperature, brittle fracture, and fatigue loading.

A pipe support system typically consists of pipe fasteners, a supporting structure, and a foundation. Pipe fasteners are used to hold pipes in position on the supporting structure. A supporting structure is a structural system that carries pipe loads (dead and live) down to a foundation. A foundation is a structure that supports the supporting structure and transfers all loads exerted on it to the ground. Pipe support systems are constructed with protective measures to resist severe blows, absorb shocks, and stand extreme events with tolerable loss levels (Peng and Peng 2009). Structures of a pipe support system should also be thoroughly planned to discard water; otherwise, stainless steel can act as a cathode that can accelerate the corrosion process of support structures (Paczkowski and Skibicki 2019). Mohan and Poobal (2018) pointed out that cracks on the concrete surfaces are the signs of degradation of the structure, which can lead to servere damage to the entire pipe support system.

Fatigue damage can alter the shapes and structural integrity of pipes and components of a pipe supporting system. For instance, when a spring-type fastener fails due to accumulated fatigue, the resulting impact can ultimately cause damage to pipes (Huff et al. 2015). In the Denali fault zone, bumper restraint supports were applied to control lateral seismic movement, and the pipeline shoes were lengthened to dampen seismic energy (Sorensen and Meyer 2003). The collapse limit state of structural members rose from 0 to 59% when the dynamic analysis was used as an assessment approach for determining fragility curves regarding soil movements in near and far-field conditions (Di Sarno and Karagiannakis 2020). As per the American Society of Mechanical Engineers (ASME), the allowable stress is dependent on the specified minimum yield strength of the pipe material (ASME 2019). The allowable stress in ASME for gas transmission and distribution piping systems depends on specified minimum yield strength and temperature, reducing the permissible values when the temperature increases more than 120 °C (ASME 2021).

The abovementioned issues are commonly inspected via visual observations by the third-party inspection contractor. Inspectors usually walk in pre-established areas to detect pipeline leakages and various defects in pipelines and pipe support systems (Ramalli et al. 2016). The existing practice is inherently limited due to the potential failure to notice the damage during visual observations and heavy concentration on relatively more vulnerable areas, leaving some areas unchecked. Other significant drawbacks include inconsistent inspection standards, a shortage of qualified inspection workforce, and increasing labor costs and resulting financial burden.

2.2 Application of Drone Technology for Image Data Collection

Drone technology has received intensive attention in many research areas recently because of its data collection capability. Unmanned aerial vehicles can provide high-resolution photos and videos in a much more convenient and flexible way than traditional methods. Meanwhile, the rapid development of advanced machine learning (ML) and deep learning (DL) algorithms has achieved many breakthroughs in various research areas. Artificial intelligence (AI)-based approaches have naturally been applied to solve challenging pipeline inspection and monitoring problems. For example, Alharam et al. (2020) implemented a system that uses a drone equipped with a thermal camera that can monitor oil and gas pipelines to detect leakages and cracks in pipelines in remote and risky areas. Different machine learning algorithms were tested and compared. Mattar and Kalai (2018) developed a wall-sticking aerial robot platform that can approach any place of the structure by flying and sticking to the target place. The robot is equipped with electromagnetic hold mount elements to stick the sensor probe on the structure’s surface and performs ultrasonic thickness testing at random spots.

Deep learning models VGG-16 and mask R-CNN were employed in Ghorbani and Behzadan (2021) to identify and locate oil spills. The VGG16 model was used for image classification to predict the existence of an oil spill in an image. The mask R-CNN model and PSPNet model were used to detect oil spills and mark their boundaries at the pixel level. A dataset named Nafta containing 1,292 images taken by UAVs, satellites, or first-person cameras from previous oil spill accidents is used for performance evaluation. A bilateral segmentation network (Chen et al. 2022) and a fast R-CNN model (Jiao et al. 2019) were also applied for oil spill detection. Fedorova et al. (2020) developed a mathematical model to determine the composition of the group of drones and its basing method for the oil pipeline monitoring task. The model can also calculate the cost of surveillance for a specified period. It is suitable for oil pipelines, gas pipelines, and other similar objects. Sudevan et al. (2018) presented research focusing on vertical structure inspection in the oil and gas industry. The introduction of robotic technologies such as UAVs and wall climbing drones provides a possible solution for challenges in vertical structure inspection by increasing efficiency, reducing risks, and lowering costs.

Lee et al. (2017) proposed a method to model the response to pipeline failure regions detected by fixed sensors using UAV-enabled sensing with ground-based maintenance crews. When a failure occurs, a fixed sensor network first refines the search area to a certain subset of the network, and then two mixed-integer programming formulations were used to optimize the UAV exploration time and maintenance vehicle travel time to minimize the time elapsed from the time of failure to the time of repair. Phung et al. (2019) presented a real-time control system for surface inspection using multiple UAVs. The UAV formation is established using the angle-encoded particle swarm optimization that generates an inspecting path and redistributes it to each UAV based on an Internet of Things board. An online image processing technique was developed and implemented using histograms to detect potential damage or defects.

2.3 Resilience of Pipelines

The literature reveals a variety of explanations for resilience from different perspectives depending on the nature and characteristics of fields. Concerning pipeline resilience, a large group of researchers has focused on the ability of a system to absorb and adapt to disruptive events and to recover after events (Hosseini et al. 2016). Taleb-Berrouane and Khan (2019) proposed a dynamic resilience model for pipe corrosion by applying Stochastic Petri-nets (SPN) linked with Monte Carlo simulation to model and measure resilience metrics. Kim et al. (2020) investigated the feasibility of replacing the current contact displacement gauges with an image processing technology. This study concluded that non-contact displacement measurement was possible with an error rate of 3.3%. Timashev and Bushinskaya (2016) took into consideration pipelines’ limit states concerning the ultimate permissible lateral displacement by a structural element due to local deformations and extended fracture caused by crack propagation. Despite these previous studies, there is still much room to explore to increase pipeline resilience.

3 Research Methodology

This section describes the drone used for image data collection and the experimental details about common defects identified through the literature review and implemented in the testbed, as shown in Table 1.

Table 1 Categories and types of root causes of pipeline failure targeted in this research
Full size table

3.1 Testbed

The testbed was constructed to create the root causes listed in Table 1. The testbed simulates a typical raised pipeline system that consists of pipes, pipe fasteners, supporting structures, and foundations. The typical defects identified through the literature review were applied to the testbed. The data collected from the experimental bed concern three common types of defects—misalignment, missing components, and defective components. Table 1 lists the types of defects implemented. Figure 1a presents the pipe support design. Figure 1b and c show the alignment defect where the spacing between the pipes from the center should have a minimum clearance of 1 inch. Figure 1d demonstrates that the pipe has moved to the edge of the supporting structure due to thermal expansion and is about to fall off. Figure 1e shows displaced pipe shoe support from its designed position. Since the supporting structure can deteriorate over time, attachments can drop from the support rapidly with the corrosion catalyst, as shown in Fig. 1f. Vibration loads from fluid transients, wind, and surroundings can loosen the bolts and nuts in the pipe foundation and pipe attachments, as shown in Fig. 1g, h, and i. A pipe guide allows a pipe to expand and contract within the limits of the guide due to thermal, mechanical, wind, or seismic loadings without a rigid attachment to the structure. Figure 1j, k, and l illustrate the missing and deformed guides. Early detection of these components can prevent the alignment defect and ultimately damage to the pipe.

Fig. 1
figure 1

The testbed: a pipeline support structure design; b misaligned pipes; c lack of minimum clearance between pipes; d pipe on the verge of falling off from the pipe rack; e displacement of pipe shoe; f missing bracing; g missing bolt; h missing nut of U-bolt; i missing nut in pipe shoe; j deformed guide; k deformed guide; and l displaced guide.

Full size image

3.2 Data Collection

We used a DJI Mavic Pro drone to collect the videos in this study. The captured drone videos have a resolution of 3840 × 2160 pixels at 30 frames/s. Mavic Pro extends Mavic’s maximum range to around four miles and stays airborne for 24 consecutive minutes. The top and side videos were recorded to segment pipe and pipe support for each defect. The top view video was sufficient to detect the alignment issues of the pipe. However, the top and side view videos were required to notice missing and deformed components. DJI Mavic Pro is equipped with a Global Positioning System (GPS) that records the GPS data and can determine the defect’s location for timely maintenance. This study used Dash-ware software to link the video and the GPS data, giving the location per frame with longitudinal and latitudinal coordinates and showing the drone height from the ground, vertical speed, and takeoff distance. The experiment was assembled at the Reese Construction Management Program Lab, Beaumont, Texas, United States. The experiment was conducted in the early morning, midday, and late evening. The drone videos were taken when the weather was clear, partially cloudy, and overcast. The experiment was conducted at an average speed of 1.5 m/s (5.1 ft/s) and an elevation of 4.5 m (15 ft) from the pipeline, complying with Federal Aviation Administration (FAA) regulations (FAA 2020).

3.3 Application of Deep Learning Neural Network

Advanced deep learning neural network is applied to detect and segment pipelines and other supporting components from drone videos. SegNet (Badrinarayanan et al. 2017), FCN (Long et al. 2015), DeconvNet (Noh et al. 2015), CRFASRNN (Hong et al. 2015), and DeepLab (Chen et al. 2018) are widely used segmentation network architectures. To achieve pixel-wise image segmentation, these network models have similar encoder-decoder designs. The encoder network takes the original image as the input and generates a feature-level representation in a latent space. In contrast, the decoder network maps the feature-level representations back to the image space with pixel-wise segmentation. The main difference among these segmentation networks lies in the decoder part. The DeepLabV3+ semantic segmentation model from Chen et al. (2018) was adopted in this study for pipeline segmentation due to its high performance on street view image segmentation and efficiency regarding memory and computation time. DeepLabV3+ has no fully connected layers and can vastly reduce the number of trainable parameters of the entire network. This measure is effective in minimizing memory consumption and improving inference time without sacrificing performance. This adopted network was pre-trained using the PASCAL VOC dataset (Everingham et al. 2010), which has 11,530 annotated images with 20 different classes. Those images include airplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dog, horse, motorbike, person, sheep, sofa, table, potted plant, train, and TV/monitor. This dataset is a widely used benchmark for semantic segmentation. Networks pre-trained by PASCAL VOC have been successfully applied for street view image segmentation, medical image analysis, gait classification, and so on.

4 Proposed Pipeline Inspection Method

Computer vision algorithms are used to analyze the extracted pipeline system for damage and defect inspection. Figure 2 presents the flowchart of the proposed method. The proposed pipeline inspection procedure consists of the following four significant steps:

  1. (1)

    Preprocessing: The captured drone videos are first decoded into frames. Then, the drone video frames are preprocessed to remove image noise. Image filtering techniques are applied in the preprocessing step;

  2. (2)

    Segmentation: A deep learning neural network is modified to segment the pipeline system from drone videos. Transfer learning is used to retrain the network to create a model for the pipeline segmentation using the manually created ground-truth dataset;

  3. (3)

    Postprocessing: After the pipeline segmentation, postprocessing steps are applied to remove misclassified objects by geometric information, such as area, shape, and aspect ratio; and

  4. (4)

    Inspection: Based on the prior knowledge and the properties of the pipeline system, the detected pipelines and their supporting components are inspected to find defects or damages.

Fig. 2
figure 2

Flowchart of the proposed drone and artificial intelligence (AI)-based pipeline inspection system

Full size image

4.1 Preprocessing for Drone Video

The captured drone videos are saved in the MOV format, so they must be decoded into video frames before being input into the neural network for pipeline segmentation at the image level. Meanwhile, because the testbed has a complex background and many surrounding objects, a 5 × 5 averaging filter is applied to smooth the image and remove the noise in the image.

4.2 Pipeline and Supporting Component Segmentation

The filtered video frames are fed into the deep learning neural network for the pipeline and its supporting component segmentation, which is the most critical step in the presented system. In this study, the captured pipeline frames are classified into eight classes in order to inspect each component: (1) Pipeline; (2) Supporting structural frame; (3) U-bolt; (4) Crossbar; (5) Bolt; (6) Nut; (7) Concrete foundation; and (8) Other (all other objects except previous seven classes). Figure 3a and b show an input image and its manual segmentation, used as the ground truth. Figure 3c shows the segmentation result of the proposed system using Fig. 3a as the input image. The segmentation output has a high similarity to the ground truth shown in Fig. 3b.

Fig. 3
figure 3

Pipeline segmentation process: a input drone frame; b ground truth of a drone frame segmentation with eight classes; and c segmentation output of the proposed system.

Full size image

The original DeepLabV3+ model is modified and uses the transfer learning technique to retrain the network to make it applicable for pipeline segmentation:

  1. (1)

    The class labels are imbalanced because the number of pixels in the pipeline and supporting structure are much more than those in other classes. This imbalance can impact the learning process and cause a bias in favor of the dominant classes. The distribution of class labels and class weight is computed and used for training to avoid the problem.

  2. (2)

    The original DeepLabV3+ model has 101 convolutional, ReLu, pooling, batch normalization, and classification layers. A new convolution layer replaces the last convolutional layer with 4 × 4 stride and 2 × 2 cropping.

  3. (3)

    The pixel classification layer in the original network is replaced by a new pixel classification layer using computed class weights.

  4. (4)

    The study annotated 51 video frames manually to create a training dataset. Data augmentation is applied to enlarge the sample numbers and add variety to the training dataset by randomly transforming the original data. The augmented training dataset has 1,581 images. Instead of training the network from scratch that needs a very large training dataset, the transfer learning technique is used for the network training, which can transfer the knowledge learned from the PASCAL VOC dataset to the collected pipeline image dataset for this project. The parameters in the modified network are optimized during the training phase.

4.3 Postprocessing

Postprocessing is applied after the pipeline segmentation to remove misclassified regions by geometric information, such as area, shape, and aspect ratio. The image morphological erosion, dilation, and open and close operations are used in the step. The isolated pixel groups are removed, and the spikes on the edges are also eliminated from the input. The postprocessed images are fed into the next step for pipeline component inspection.

4.4 Inspection

After the segmentation, the detected pipelines and supporting components need to be separated for defects and damage inspection. The three types of defects were detected as follows:

  1. (1)

    Pipeline alignment issues: Each pipe is marked as a connected component to get property information. Then, the skeleton is extracted from each corresponding component by medial axis transform. The Hough transform is applied to find a straight-line segment to approximate the pipe. Finally, the angles of all line functions computed from pipes are analyzed to detect misalignment.

  2. (2)

    Crossbar issues: Each support structure has two braces (prior knowledge) that can be used to detect missing braces. Similar steps used for misalignment analysis are applied to get the line segment approximation of each bracing. Because two braces are crossed, the corresponding two-line segments should have an intersection and a 90° angle between them. Therefore, if a support structure has only one brace or the braces are not well fixed, then no intersection exists, or the angle is not close to 90°, which indicates a missing brace.

  3. (3)

    Pipeline corrosion issues: Corrosion can cause color changes on the pipeline surface, so a color-based clustering method is used to cluster the pipeline surface into regions based on their color similarities. The regions with average colors (that is, cluster center) close to standard corrosion colors are marked as the corrosion regions. Their areas can be measured for corrosion and resilience analysis.

5 Resilience Model

Pipelines, including support structures, are constructed with protective measures to resist severe blows, absorb shocks, and stand extreme events with tolerable loss levels. The pipeline resilience framework was developed to address five critical issues—misalignment, deformation, missing parts, cracks, and broken components—that can affect the performance and safety of pipelines. Figure 4 illustrates the resilience framework based on the five critical performance-affecting issues. Since color change on the pipe can be due to dust or grease, resilience for corrosion issues is not evaluated due to the higher possibility of false detection. Any misalignment, missing, cracked, or broken components can induce imminent, large-scale damage to the pipeline. Therefore, a pipeline system having such issues is considered “non-resilient,” as shown in Fig. 4. On the contrary, the deformed components are further evaluated to determine the state of the pipeline system. Based on the detection of the alignment issue, deformed components can be categorized into two different resilient states. If a deformed component is identified and the pipeline alignment issue is not detected, the system is in an alarming resilient state as the deformation is within the elastic region. However, if a deformed component and the alignment issue are detected together, then the system is in a non-resilient state, as deformation might be in the plastic region or near to rupture phase.

Fig. 4
figure 4

Resilience framework based on the DARTS application

Full size image

6 Results

This section explains the datasets used for training and validation. The results and details of segmentation and inspection test results are shown in subsections.

6.1 Drone System and Datasets

From a drone video, 51 frames were selected, and eight classes were labeled in each frame. Of these, 41 images were used for training and 10 images for validation. Data augmentation was applied to enlarge the training data size and add more variety to the training dataset by randomly transforming the original training samples, improving the network’s segmentation performance. The same random transformations were used for both image and pixel label data. The random functions applied are random left/right reflection and random horizontal/vertical translation from − 10 to + 10 pixels. Each training sample image and its label are augmented to 40 images, so there are 1681 (41 + 41 × 40) images. Each validation sample image is also augmented to 20 images, resulting in 210 images.

6.2 Segmentation Results

The algorithm was implemented using Matlab R2020b. The computer has Ubuntu Unix operating system, 16 GB memory, and Nvidia 1080GT GPU. The training parameter settings are listed in Table 2. The training time is 125 min and 20 s. These resulted in the final training accuracy of 99.20, the final training loss of 0.08, the final validation accuracy of 97.58, and the final validation loss of 0.14. Some segmentation results are shown in Fig. 5, which overlap with the original drone video frames.

Table 2 Parameter setting for the network training
Full size table
Fig. 5
figure 5

Pipeline and supporting component segmentation results

Full size image

Table 3 lists the Intersection over Union (IoU) for each class to show segmentation performance. Pipeline, frame, crossbar, and foundation classes are well segmented with decent IoUs because they have big areas and are robust to small segmentation errors. But U-bolt, bolt, and nut classes are much smaller and, therefore, very sensitive to segmentation errors—a small error (for example, a few pixels) can cause a big IoU drop. In this study, we focused on the inspection of pipeline and crossbar-related issues using segmented drone frames.

Table 3  Segmentation accuracies of pipeline and supporting components
Full size table

6.3 Inspection Results

The proposed inspection method was evaluated using the prepared testbed. Videos were taken from the top and side views with a camera angle of approximately 60°. The resilience model was used to define the system state of the pipelines. The following discusses the results of the inspection experiments:

  1. (1)

    Pipeline alignment issues: The misaligned pipelines in the testbed were applied, as shown in Fig. 6a. It shows the detected pipelines highlighted in red and the line segment generated by the Hough transform of each pipeline. It can be seen that two-line segments are not parallel to each other and have an intersection, which indicates that the pipelines are misaligned and the system state is non-resilient. Therefore, pipeline alignment problems can be detected by the proposed system.

  2. (2)

    Crossbar issues: There are two supporting structures in the testbed. One has two crossbars, and the other has only one. Figure 6b shows the crossbar detection results highlighted in red color. It shows the line segments generated by the Hough transform. The first supporting structure has only one line segment, and the second one has two crossed line segments with a 90° angle. Therefore, the missing component of the first supporting structure can be detected, and the system state is defined as non-resilient.

  3. (3)

    Pipeline corrosion issues: The corrosion detection result is shown in Fig. 6c. The detected pipelines are highlighted in red color, and the detected corrosion regions are highlighted in blue. A closeup of the detected corrosion on the pipeline surface can be seen in Fig. 6c. The resilience is not evaluated because the color contrast can also be due to dust or grease on the pipeline.

Fig. 6
figure 6

Inspection results: a pipeline alignment detection; b missing crossbar detection; c pipeline corrosion detection.

Full size image

7 Discussions

The limitations of human observation-based inspection of oil and gas pipelines have been well recognized in terms of time, cost, accuracy, safety, and accessibility (Ramalli et al. 2016; Alharam et al. 2020). To overcome these limitations, various research and development efforts have been undertaken in the academia and industry. Some of these efforts resulted in meaningful progress in advancing pipeline monitoring and inspection practices by utilizing advanced technologies such as UAVs, photogrammetry, sensors, and artificial intelligence. This advancement has consequently contributed to improving pipeline inspection and thereby allowing more timely problem detection and preventive maintenance. Meanwhile, most of the previous studies on the application of advanced technologies for monitoring and inspection of oil and gas pipelines have heavily focused on oil and gas leakage (El-Abbasy et al. 2014; Iqbal et al. 2017; Alharam et al. 2020; Asadzadeh et al. 2022). The present study applied UAV and artificial intelligence to fill a research gap by focusing on the advancement of the inspection of the potential root causes—misalignment and deterioration of pipeline supporting structures—of various pipeline failures, including oil and gas leakage.

Compared with traditional inspection methods, the proposed DARTS system has significant advantages. Drones can efficiently capture the images and videos of pipelines that are difficult to access. Computer-based automatic visual inspection can avoid human errors with low cost. Data are stored in a database for timely assessment. The scalability is robust because of non-contact detection.

The experimental results demonstrate that DARTS can achieve decent inspection performance on the constructed testbed. Furthermore, DARTS is also a generalized framework that can be applied to monitor a wide range of aboveground pipeline systems with attention to the following issues and limitations. Since there are a variety of pipeline systems with different structures and pipe supports as per loads acting on the pipeline, the deep learning neural network needs to be retrained using a new training dataset collected from every new pipeline system. Some defects may not be completely detected using single-angle drone videos where drone images have blind spots from each angle. Therefore, multiple drone videos with different angles are required to detect specific issues in the pipe support systems. Dust, dirt, or grease residue can change the color and appearance of pipeline components and thereby decrease detection accuracy. The application of the drone image-based method can be affected by adversarial weather conditions.

8 Conclusion

In this study, we proposed a method to integrate drone technology for automatic field data collection with AI-based image processing concerning the status of pipelines. The findings of this study are expected to benefit the oil and gas midstream industry in various ways, providing the enhanced ability to monitor pipeline conditions and conduct resilience assessments. The proposed method can improve the monitoring and inspection of pipelines by saving time and workforce for collecting field data, improving response time for correcting or removing problems, improving pipeline monitoring access using a drone, and enhancing the assessment of pipeline resilience.

The authors plan on improving the segmentation and inspection methods, especially for small parts (that is, bolts and nuts), by optimizing network structure and parameters. Datasets collected from real pipeline systems will be used for comprehensive performance evaluation. A professional drone with an onboard computer and high-resolution camera will be utilized to achieve real-time analysis.