Joint self-supervised and reference-guided learning for depth inpainting

Depth information can benefit various computer vision tasks on both images and videos. However, depth maps may suffer from invalid values in many pixels, and also large holes. To improve such data, we propose a joint self-supervised and reference-guided learning approach for depth inpainting. For the self-supervised learning strategy, we introduce an improved spatial convolutional sparse coding module in which total variation regularization is employed to enhance the structural information while preserving edge information. This module alternately learns a convolutional dictionary and sparse coding from a corrupted depth map. Then, both the learned convolutional dictionary and sparse coding are convolved to yield an initial depth map, which is effectively smoothed using local contextual information. The reference-guided learning part is inspired by the fact that adjacent pixels with close colors in the RGB image tend to have similar depth values. We thus construct a hierarchical joint bilateral filter module using the corresponding color image to fill in large holes. In summary, our approach integrates a convolutional sparse coding module to preserve local contextual information and a hierarchical joint bilateral filter module for filling using specific adjacent information. Experimental results show that the proposed approach works well for both invalid value restoration and large hole inpainting.


Introduction
Depth information has been successfully used in many computer vision tasks in recent years, such as scene understanding [1], augmented reality [2], autonomous driving [3], and robotics [4]. However, the acquisition of depth information heavily depends on RGB-D sensors, e.g., Microsoft Kinect, Zed Stereo, Intel RealSense, and Velodyne Lidar. Their output usually suffers from missing or invalid values caused by surface reflection or insufficient illumination. Preserving local contextual information throughout the corrupted depth image itself and adding specific adjacent information by considering relationships between the pixel values in the color image can be used to correct such problems. A crucial challenge for depth inpainting is how to capture and unify local contextual information and specific adjacent information to improve correction.
There is a trend to utilize deep learning models [5][6][7][8][9][10][11] to fill depth maps due to their powerful feature representation ability and flexibility. A common approach is to model an intermediate representation such as dense surface normals and occlusion boundaries. With the intermediate representation, the corrupted depth map is filled in by integrating contextual information and global optimization using the raw depth maps and RGB images [5]. The depth regression model [7] also performs well in calculating full-resolution depth maps from sparse samples and RGB images. Although these learning-based models can work well, they can be impractical in real applications, since annotating the required massive training data is time-consuming and laborious.
Heuristic models [12][13][14][15][16][17][18][19] based on comprehensive rules provide acceptable results for depth inpainting, at acceptable cost. Edge-aware smoothing can be enhanced via a bilateral solver [16] that combines the flexibility and speed of filtering. In Ref. [17], a total generalized variation operation is adopted as a global energy optimization strategy for depth map completion. A low gradient regularization method [18] can be adopted to fill in the depth map without a corresponding color image. These heuristics do not require large amounts of data or memory footprints. However, in the process of depth map inpainting, they often neglect their depth adjacency relationships and hierarchical relationships between color image pixels, which may result in poor performance. Therefore, it is necessary to develop a model that can effectively utilize contextual information by self-supervised learning and fill in large holes in a hierarchical reference-guided manner.
With these issues in mind, in this paper, we propose to exploit a joint self-supervised and reference-guided learning approach to fill in the depth map. For selfsupervised learning, we propose an improved spatial convolutional sparse coding module incorporating the total variation (TV) regularization, the latter being used to enhance structural information and preserve edge information (as shown in Fig. 1). In previous works, the conventional convolutional sparse coding approaches rely on an alternating direction multiplier minimization formulation in the Fourier domain [20], which loses the connection to the patchbased processing paradigm and may constrain output improvements. Therefore, our improved convolution sparse coding is devoted to learning localization strategies in the spatial domain. The pipeline of the proposed approach is shown in Fig. 2, with two modules: the convolutional sparse coding module and the hierarchical joint bilateral filter module. In the former, a convolutional dictionary and sparse coding are learnt in a self-supervised manner. Note that the convolutional dictionary and sparse coding are alternately updated from a captured depth map in the spatial domain. In this manner, the contextual information of the depth map is effectively preserved and integrated through global optimization, which generates intermediate depth maps step by step. After that, the learned convolutional dictionary and sparse coding are convolved to yield an initial depth map, which is effectively smoothed using Fig. 2 Pipeline, consisting of two modules, performing convolutional sparse coding and hierarchical joint bilateral filtering. Given a captured depth map, iteratively obtain the initial depths by convolutional sparse coding. These are further filled via a hierarchical joint bilateral filter for final depth inpainting. local contextual information. The corresponding hierarchical joint bilateral filter module is designed using a reference-guided learning strategy. This module takes the initial depth map and original RGB image as input to progressively fill in large holes in the initial depth map: missing information in large holes is filled successively by iterating from neighboring pixels.
The main contributions of this paper are three-fold: • a system that obtains an initial depth map by self-supervised learning and further fills in the initial depth map by guided hierarchically by RGB values, • an improved spatial convolutional sparse coding module to enhance structural information and preserve edge information for depth inpainting, and • experimental results showing that the proposed approach both restores invalid values and inpaints large holes well. The rest of this paper is organized as follows. A brief review of related work is presented in Section 2. In Section 3, we present our proposed approach. Section 4 tests our approach. Finally, the paper is concluded in Section 5.

Related work
In this section, we present a brief literature review related to depth map inpainting, depth reconstruction from sparse samples, and convolutional sparse coding.

Depth map inpainting
Depth map inpainting methods can be roughly categorized into two groups: intrinsic-informationbased and extrinsic-information-based. The models in the first group focus on filling missing depth values through contextual information from the surrounding pixels. Hornácek et al. [21] proposed a depth map inpainting approach which identified and merged patch correspondences within the input depth map itself. In Ref. [18], Xue et al. proposed integrating low gradient regularization and low rank regularization into a single approach for depth inpainting. This can reduce the penalty for gradients while penalizing nonzero gradients to allow for gradual depth changes. In most cases, these models work well for depth inpainting but are unsatisfactory when filling large holes.
Extrinsic-information-based models can better address large hole inpainting challenge than the former group. Gong et al. [13] incorporated color information for depth inpainting by extending the inpainting model and the propagation strategy; an edge-preserving guided filter served as a denoiser. Chen et al. [22] proposed to improve the quality of the depth map through a region growing method that fills large holes using a bilateral filter. In Ref. [23], Qi et al. developed a non-local filtering scheme integrated with conventional inpainting, which achieves a good balance between depth and color information to guarantee an accurate inpainting result. As an extension to Refs. [22,23], Matyunin et al. [24] utilized the information about the motion and color of objects from several consecutive frames. Ferstl et al. [17] formulated a convex optimization problem via higher-order regularization, which breaks limitations in lateral resolution. Zhang and Funkhouser [5] took the dense surface normal and occlusion boundary as an intermediate representation, and combined raw depth map and RGB image information to yield a final depth map via global optimization. These models improve large hole inpainting, but still do not provide ideal results as the introduced external information is imperfect.

Depth reconstruction from sparse samples
Depth reconstruction from sparse samples aims to predict depth values from the RGB image and depth sparse samples. Hawe et al. [25] proposed to reconstruct a dense disparity image with a conjugate sub-gradient method by considering where disparity maps are sparse in the Wavelet basis. Liu et al. [26] proposed a model incorporating wavelet and contourlet dictionaries to achieve better results.
In recent decades, the boom in deep learning has also led to the development of depth reconstruction methods. Uhrig et al. [27] presented a sparsity invariant CNN architecture using partially normalized convolutions on the input sparse depth image. Two innovative improvements to that approach were proposed by Ma et al. In Ref. [7], a deep regression model is employed to compute full-resolution depth maps from a sparse set of depth samples and RGB images, while in Ref. [28], Ma et al. presented a self-supervised training pipeline given sequences of color and sparse depth images. Jaritz et al. [29] learned sparse features through an encoder-decoder scheme without the need for an additional validity mask. Cheng et al. [30] proposed a convolutional spatial propagation network for depth prediction and learned an affinity matrix using a linear propagation model and affinity between neighboring pixels. Ku et al. [31] designed a heuristic approach to highlight the effectiveness of crafted feature extraction that produces dense depth information by using only commonly available morphological operations.

Convolutional sparse coding
Convolutional sparse coding has been widely applied in image processing and computer vision [20,[32][33][34][35][36][37]. Zeiler et al. [32] proposed deconvolutional networks that capture mid-level cues spontaneously emerging from image data; this is one of the earliest works based on a sparse coding convolutional pipeline for the whole image. Bristow et al. [33] proposed a fast algorithm that is designed upon ideas from signal processing and augmented Lagrange methods. The problem in the spatial domain is transformed into a problem in the Fourier domain. An optimization method is proposed in Ref. [34] for solving convolutional sparse coding problems and extending it to different application domains. In Ref. [20], Heide et al. proposed a fast and flexible convolutional sparse coding method that easily merges proper boundary conditions and learns features for an incomplete signal. Affara et al. [38] computed supervised convolution sparse coding in a supervised dictionary learning manner. Papyan et al. [39] exploited a slice-based dictionary learning model that incorporates traditional patch-based local and global structure.
Generally speaking, conventional sparse coding approaches work in the Fourier domain, which loses the connection to the patch-based processing paradigm and so may constrain improvements in the output. Therefore, inspired by Refs. [39,40], we propose an improved convolutional sparse coding module for depth inpainting in the spatial domain. TV regularization is introduced to enhance structural information and preserve edge information.

Method
In this section, we present details of the method. The proposed method involves two stages, an initial depth calculation, then determining the final depth map. We first briefly overview the whole pipeline and then present our model definition. Finally, we discuss the convolutional sparse coding module and the hierarchical joint bilateral filter module, in turn.

Overview
The overall pipeline is shown in Fig. 2, and contains two modules: a convolutional sparse coding module and a hierarchical joint bilateral filtering module. The convolutional sparse coding module focuses on filling in small areas of invalid values to give the initial depth map in a self-supervised learning manner. Specifically, we first learn a convolutional dictionary and sparse coding from the corrupted depth map; the dictionary and sparse coding are alternately updated. The spatial sparse coding serves as an importance weighting to combine the sparse representations in the convolutional dictionary to generate the initial depth map. Unlike conventional sparse coding, in addition to learning the dictionary and sparse coding, we incorporate TV regularization to correct the depth map and preserve the local contextual information and edge structures in the spatial domain. TV correction is beneficial for generating adaptive and efficient representations. The hierarchical joint bilateral filtering module fills large holes in the initial depth map by a reference-guided learning strategy. This module takes the original RGB image as additional input, to introduce extrinsicinformation to progressively fill in the depth map. In this manner, the hierarchical joint bilateral filter can guarantee structural consistency and neighborhood consistency.

Convolutional sparse coding
Assume x ∈ R N is a vectorized corrupted depth map and B ∈ R N ×N indicates the corresponding binary diagonal mask. B(n, n) = 0 indicates that the depth value x n at location n is invalid, while B(n, n) = 1 indicates it is valid. The goal of depth inpainting is to restore the invalid values in x. We propose to adopt a convolutional sparse coding (CSC) model to recover invalid values in the depth map. CSC can be performed by solving the problem in Eq. (1): where d c ∈ R M represents the vectorized 2D dictionary filter, z c ∈ R N is the vectorized feature map corresponding to d c , and * denotes one-dimensional convolution. λ is a regularization parameter that balances the data fidelity term and the regularization term.

Adding TV regularization
Essentially, Eq. (1) is a CSC pursuit task to find the feature representations {z c } C c=1 , given the dictionary filters {d c } C c=1 . To enhance structural information and preserve edge information, we add a total variation regularization term and reformulate Eq. (1) as where TV is the total variation regularization (i.e., the Euclidean norm of horizontal and vertical differences in variables) and β is the regularization parameter.
Most CSC methods construct a global dictionary D G ∈ R N ×NC as the concatenation of C banded circulant matrices, where each matrix is a convolution with a corresponding filter d c . By permuting its columns, the global dictionary D G consists of all shifted forms of a local dictionary D L ∈ R M ×C , containing the filters {d c } C c=1 as its columns. The global sparse coding Γ ∈ R NC represents the interlaced concatenation of all feature maps {z c } C c=1 . Using the above formulation, Eq. (2) can be written in an intuitive version: which is a global optimization problem. Generally speaking, such global optimization problems are complex and hard to solve in the Fourier domain, so we convert it to a local optimization problem that is solvable in the spatial domain instead: where P T n ∈ R N ×M is the operator that positions D L α n in the n-th location. The {α n ∈ R C } N n=1 are called needles that point to N non-overlapping C dimensional local vector sets.

Inpainting via self-supervised learning
Equation (4) contains two group of variables, {α n } N n=1 and D L , so the objective function in Eq. (4) is a non-convex constrained optimization problem. Such a problem can be transformed into an alternative solvable convex optimization problem: one group of variables can be solved by fixing another. In particular, we ignore the TV term in Eq. (4) and employ TV correction [41] to refine the depth map by a redundant Haar wavelet-based shrinkage estimate, as follows: 1. Update step for α n . To optimize each α n separately with a local strategy, we transform Eq. (4) into Eq. (5), in which the local dictionary D L is fixed for finding the needles: This can be solved using projected steepest descent [40], in which the iterative solution process can be formulated as is a projection operator which projects all atoms of dictionary D L to the unit ball, and η is the step size. Defining the restored depth map as x = N n=1 P T n D L α n , the gradient of D L can be found using 3. TV correction. Once {α n } N n=1 and D L have been estimated, we apply TV correction to refine the restored depth map. The first and the third terms in Eq. (4) can not be equivalently solved with Haar wavelets [42] since finding them is not a simple denoising optimization problem based on TV regularization. Thus, we modify Eq. (4) to turn it into a denoising optimization problem: (8) where B is the transpose of B. We define y = x + B x and transform Eq. (8) to a simpler form: Using the Haar wavelet-based shrinkage estimate [42], we can calculate a approximate solution via where S β (·) = sign(·)(|·| − β) + is the elementwise soft-thresholding operator with threshold β. H(·) is the Haar wavelet transform and H T (·) is the inverse wavelet transform. x * is the result refined by the TV correction operation.

Hierarchical RGB guided inpainting
As mentioned above, the convolutional sparse coding module is capable of encoding contextual information in a self-supervised manner. This module is a feasible solution for obtaining the initial depth inpainting. However, the output of the convolutional sparse coding module is unsatisfactory when it encounters large holes since the convolutional sparse coding module is unable to mine enough information. Some approaches overcome this dilemma by introducing extrinsic information. Our approach is inspired by a joint bilateral filter (JBF) method, which takes RGB color images as input to introducing spatial positions and RGB color pixels as extrinsic information to guarantee structural consistency and neighborhood consistency. It formulates the process of calculating the inpainted depth map x h as where p and q indicate coordinates from the set of surrounding pixels Ω. k p is the number of corresponding non-zero depth values for the bilateral kernel centered on pixel p. x * (p) and x rgb (p) represent the depth and RGB values corresponding to pixel p, respectively. f (·) and g(·) are weight distribution functions. Both are of the form: However, this JBF may need a large bilateral kernel to fill a large hole and may fail if irrelevant RGB color information disturbs the original depth value at the current position. As a result, the inpainted depth map may become over-smoothed or structurally inconsistent with valid depth values. We propose a hierarchical joint bilateral filter (HJBF) module to address this issue, as shown in Fig. 2, which fills in the depth map from the outside to the inside using a small filter kernel K in a progressive manner. The difference between JBF and HJBF is schematically illustrated in Fig. 3. We concisely present the overall process of our approach in Algorithm 1.

Experiments
Several experiments were conducted to verify the effectiveness of the proposed approach on two depth map datasets. We first introduce the experimental setting and benchmark against some existing models, and then give a detailed discussion.

Dataset and evaluation metric
We tested our approach on two public depth map datasets: RMS-depth (rendered Middlebury stereo depth) [18] and NYU-depth-v2 [43]. The RMS-depth dataset is composed of 14 depth maps, including Adirondack, Jadeplant, Motorcycle, Piano, Playtable, Playroom, Recycle, Shelves, Teddy, Pipes, Vintage, MotorcycleE, PianoL, and PlaytableP, used to build corrupted depth maps including 14 with random gap masks and 2 with textual masks. The NYU-depth-v2 dataset contains 1449 densely labeled pairs of aligned RGB and depth images of size 427 × 561, comprising 464 indoor scenes taken with a Microsoft Kinect camera, including 249 training and 215 testing scenes. We randomly selected 100 pairs of aligned RGB and depth images for testing and then directly conducted experimental analysis on the realistic corrupted depth maps in the NYU-depth-v2 dataset.
To evaluate the quality of the proposed approach, we compared our approach to the LRL0 φ algorithm [18]. Using the NYU-depth-v2 dataset, we compared our approach to six existing methods which can be categorized into two groups: a heuristic group containing GBF [44], JBF [44], FB [16], and TGV [17], and the learning-based group containing STD [7] and CSPN [30].
Following previous works [18,29], in this paper, we quantitatively evaluate four commonly used metrics. Let y and y * be the inpainted depth map and the corresponding ground truth, respectively. y max is the maximum of y. P is the total number of pixels in the depth map. y i and y * i are predicted and the ground truth values, respectively. "card" is the cardinality of a set and δ i = 1.25, 1.25 2 , 1.25 3 . Metrics are computed only on the missing area of the depth map.
Peak signal to noise ratio (PSNR) is a widely used objective standard to evaluate image quality. A higher PNSR means greater similarity to the ground truth, and is formulated as PSNR = 10 log 10 P y 2 max y − y * 2 2 (13) Relative error (Rel), root mean squared error (RMSE), log 10 metric (Log 10 ), and percentage of predicted pixels δ i reflect the reliability of measurement. They are formulated as Log 10 = 1 P log 10 (y) − log 10 (y * ) 1 (16)

Implementation and parameters
The proposed model was implemented using MATLAB and experiments performed on a PC with a 3.6 GHz Core i7 CPU with 16 GB memory. In all experiments, we adopted a similar approach to Ref. [40], in which corrupted depth boundaries are padded with M − 1 elements. At the end of the process, the added padding is discarded by cropping the M − 1 boundary elements from the restored depths and the resulting feature maps. The number of atoms in the convolutional dictionary was 81. The regularization parameters λ and β were set to 0.01 and 1, respectively. We used a hierarchical filter kernel K of size 7 × 7.

Effectiveness
We conducted quantitative and qualitative experiments from two different perspectives to verify the effectiveness of the proposed approach. One used random sampling and textual patterns on the RMSdepth dataset, while the other used random sampling and missing block inpainting on the NYU-depth-v2 dataset.

Random sampling and textual inpainting
PSNR results from the LRL0 φ algorithm [18] and our approach, using the RMS-depth dataset, are presented in Table 1. In this comparison, we only exploit the convolutional sparse coding module to fill in the depth maps as the invalid regions in the dataset are only tiny holes. From Table 1, we see that our approach is superior to the LRL0 φ algorithm for all depth maps. Some representative results on the RMS-depth dataset are shown in Fig. 4, with close ups of local areas shown in Fig. 5. The figure reveals that for edge denoising, the results of the LRL0 φ algorithm still have some residual noise, again indicating that the LRL0 φ algorithm is inferior to the proposed approach. Similarly, for the text removal cases, the LRL0 φ algorithm cannot eliminate artifacts  from text-corrupted depth maps, unlike the proposed approach. Again, our approach is more effective than the LRL0 φ algorithm. This can be explained by the fact that the proposed approach iteratively extracts the contextual information from the corrupted depth maps in a selfsupervised manner. This preserves the contextual information well and leads to impressive edge denoising ability in the convolutional sparse coding module. In particular, artifacts are avoided in the textual elimination task since the contextual information is so helpful. In summary, the proposed approach works well both for edge denoising and text removal, verifying the effectiveness of the convolutional sparse coding module in our approach.

Random sampling and missing block inpainting
To verify the effectiveness of inpainting a large hole in the depth map, we conducted three diagnostic experiments on NYU-depth-v2.
The first experiment was conducted to verifying the effectiveness with random sampling. We present results for six models on NYU-depth-v2 in Table 2 in which the ground-truth maps were 50% random sampled. Similar results with 30% random sampling are presented in Table 3. From Table 2, we find that the proposed approach is superior to all other models in terms of Rel, Log 10 , RMSE, δ i = 1.25 2 , and δ i = 1.25 3 , while slightly inferior to TGV in terms of δ i = 1.25. In Table 3, the proposed approach is basically superior to other models for the first three and last metrics, while slightly inferior to TGV in terms of δ i = 1.25 and δ i = 1.25 2 . This can be interpreted as the powerful representations extracted by convolutional sparse coding being beneficial when there is random sampling. Particularly, on comparing Table 2 to Table 3, we find our results with high rate random sampling are better than with low rate random sampling: the convolutional sparse coding module in our approach works better given more samples. Representative output is presented in Fig. 6. The figure shows that GBF and FB produce obvious artifacts while STD and CSPN oversmooth. In contrast, JBF, TGV, and our proposed approach effectively eliminate artifacts and avoid oversmoothing.
The second experiment aimed to assess the ability of our approach to fill large holes. Results for seven models from NYU-depth-v2 are given in Table 4 in which the ground-truth maps were 50% randomly sampled with a 75 × 75 missing block. Representative output is presented in Fig. 4. Table 4 shows that our approach is promising, due to the hierarchical filter taking the RGB image as additional input and filling the depth map from the outside to the inside, which effectively extracts and integrates extrinsic information from neighboring pixels. From the figure, we find that JBF results are too smooth when the depth map contains large holes while TGV fails. Our method achieves the best visual effect while maintaining the original depth structure.
To verify the ability of our approach to process multiple holes, we conducted the third experiment on NYU-depth-v2. Results from seven models are presented in Table 5 in which the ground-truth maps were 50% randomly sampled, with multiple 35 × 35 missing blocks. Representative output is presented in Fig. 7. The proposed approach is superior to all others in all metrics in Table 5. Our small filter kernel with a hierarchical filling strategy is capable of dealing with holes at various scales. This is consistent with the output illustrated in Fig. 7.

Realistic depth inpainting
To explore the utility of our approach for realistic corrupted depth maps, we qualitatively conducted an experimental comparison and exhibit the inpainted results for such depth maps in Fig. 8. We see that our approach results in better contextual information and edge structures than the others. To sum up, these experimental results, on both simulated and realistic depth maps, demonstrate the flexibility and adaptability of the proposed approach.

Analysis
We start by analyzing the hierarchical kernel size and then present an ablation study for the TV regularization. Finally, we consider the running time of our approach and a baseline model.

Impact of hierarchical filter kernel
We analyzed the size of the hierarchical filter kernel K to find the best choice. Curves of Rel and Log 10 are shown in the first two columns of Fig. 9: the overall quality of our approach monotonically decreases as K increases. We can conclude that the smaller K is, the better. To analyze speed, we present the runtime curves in the last column of Fig. 9. When K has size in the range of [7,13], the runtime is relatively fast,   but outside this range leads to higher computational cost. Overall, we set use a 7 × 7 kernel as the best speed-accuracy trade-off.

Impact of TV regularization
The performance of our approach with/without TV regularization in four different mask types on NYUdepth-v2 dataset is presented in Table 6. From this table, we find that in all four mask type, the models with TV regularization achieves a better performance in all terms of metrics. This verifies the effectiveness of the TV regularization.

Speed
We also compared the running time of our approach to a baseline model, consisting of standard CSC and HJBF. Table 7 provides results for realistic corrupted depth maps of test scenes from the NYU-depth-v2 dataset. We can see that the running time of our approach basically is unchanged from that of the baseline model. The time for TV correction is negligible.

Conclusions
Existing depth inpainting approaches may suffer from poor performance in the presence of invalid values and large holes. To alleviate these issues, we propose a joint self-supervised and reference-guided learning approach for depth inpainting. In this approach, we first leverage an improved convolutional sparse coding operation based on self-supervised learning to fill in the corrupted depth map. TV regularization is employed to enhance structural information and preserve edge information. In this operation, the initial depth map is generated by alternately learning the convolutional dictionary and sparse coding. Then, the initial depth map is refined by a hierarchical joint bilateral filter in a reference-guided learning manner. In this manner, the local contextual information and specific adjacent information can be effectively extracted and integrated into the depth inpainting process. Experimental results show that the proposed approach achieves strong depth inpainting results. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www. editorialmanager.com/cvmj.