Detection and localization of inter-frame video forgeries based on inconsistency in correlation distribution between Haralick coded frames

Abstract

With the immensely growing rate of cyber forgery today, the integrity and authenticity of digital multimedia data are highly at stake. In this work, we deal with forensic investigation of cyber forgery in digital videos. The most common types of inter-frame forgery in digital videos are frame insertion, deletion and duplication attacks. A number of significant researches have been carried out in this direction, in the past few years. In this paper, we propose a two-step forensic technique to detect frame insertion, deletion and duplication types of video forgery. In the first step, we detect outlier frames, based on Haralick coded frame correlation; and in the second step, we perform a finer degree of detection, to eliminate false positives, hence to optimize the forgery detection accuracy. Our experimental results prove that the proposed method outperforms the state–of–the–art with an average F₁ score of 0.97 in terms of inter–frame video forgery detection accuracy.

Appendix

In this section, we present the fourteen Haralick features proposed in [11], which have been adopted in our work. They are defined based on the following notations:

P(i,j) = (i,j)th entry in normalized Gray Level Co-occurrence Matrix (GLCM).

P_x(i) = i th entry in the Marginal–Probability Matrix obtained by summing the rows of P(i,j), i.e, $P_{x}(i)={\sum }_{j}^{N_{g}} P(i,j)$, where N_g = number of distinct gray levels in the quantized image.

P_y(j) = j th entry in the Marginal–Probability Matrix obtained by summing the columns of P(i,j), i.e, $P_{y}(j)={\sum }_{i}^{N_{g}} P(i,j)$.

$P_{x+y}(k) = {\sum }_{i = 1}^{N_{g}}{\sum }_{j = 1}^{N_{g}}P(i,j)$, where k = i + j ∈ [2,⋯ , 2N_g].

$P_{x-y}(k)= {\sum }_{i = 1}^{N_{g}}{\sum }_{j = 1}^{N_{g}}P(i,j)$, where k = |i − j|∈ [0,⋯ ,N_g − 1].

Mathematical equations to compute the fourteen Haralick features, based on the above notations, are as follows:

• 1. Angular Second Moment (Energy): This feature measures the uniformity of an image, computes as:

  $$Angular Second Moment = \sum\limits_{i = 1}^{N_{g}} \sum\limits_{j = 1}^{N_{g}} P^{2}(i,j) $$

• 2. Contrast: It measures the frequency of local changes in an image, by computing intensity contrast between a pixel and its neighborhood. This represents the neighborhood based gray tone linear dependencies in an image. This feature is computed as follows:

  $$Contrast=\sum\limits_{|i-j|= 0}^{N_{g} - 1}|i-j|^{2}\left( \sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}}P(i,j)\right) $$

  For a constant image, contrast is 0.

• 3. Sum of Squares (Variance): This is a measure of heterogeneity in an image, and strength of its correlation to first order statistical variables such as standard deviation. Variance is proportional to difference between gray level values and their mean, computed as:

  $$Variance= \sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}}(i-\mu)^{2}P(i,j) $$

  where μ is the mean of P_x(i).

• 4. Correlation: This feature measures how correlated a pixel is to its neighborhood. It is the measure of gray tone linear dependencies in the image.

  $$Correlation= \frac{{\sum}_{i = 1}^{N_{g}}{\sum}_{j = 1}^{N_{g}} (i,j)P(i,j) - \mu_{x}\mu_{y}}{\sigma_{x}\sigma_{y}} $$

  where μ_x, μ_y, σ_x and σ_y are the means and standard deviations of P_x and P_y, respectively.

• 5. Sum Average:

  $$\sum\limits_{i = 2}^{2N_{g}}i ~P_{x+y}(i) $$

  where x and y represent the row and column respectively of a GLCM entry, and P_{x + y}(i) is the probability of GLCM coordinates summing to x + y.

• 6. Sum Entropy:

  $$\sum\limits_{i = 2}^{2N_{g}}P_{x+y}(i)\log{P_{x+y}(i)} $$

• 7. Sum Variance:

  $$\sum\limits_{i = 2}^{2N_{g}}(1-SE)^{2} P_{x+y}(i) $$

  where SE = Sum Entropy.

• 8. Inverse Difference Moment (Homogeneity): This feature measures the similarity between pixels and their neighborhoods. Local textures having minimal changes, lead to high homogeneity. It is computed as:

  $$Inverse ~Difference ~Moment= \sum\limits_{i = 1}^{N_{g}} \sum\limits_{j = 1}^{N_{g}}\frac{P(i,j)}{1+(i-j)^{2}} $$

• 9. Entropy: Entropy measures the textural randomness within an image. Entropy is computed as:

  $$Entropy = -\sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}}P(i,j)\log_{2}(P(i,j)) $$

• 10. Difference Variance:

  $$\sum\limits_{i = 0}^{N_{g}-1} i^{2} P_{x-y}(i) $$

• 11. Difference Entropy:

  $$-\sum\limits_{i = 0}^{N_{g}-1} P_{x-y}(i)\log({P_{x-y}(i)}) $$

• 12. Information Measure of Correlation 1:

  $$\frac{HXY-HXY1}{max\{HX,HY\}} $$

  where,

  $$\begin{array}{@{}rcl@{}} HXY&=&-\sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}} P(i,j)\log(P(i,j))\\ HXY1&=&-\sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}} P(i,j)\log{P_{x}(i)P_{y}(j)}\\ HX&=&\text{Entropy of}~P_{x}(i), HY = \text{Entropy of} ~P_{y}(j) \end{array} $$

• 13. Information Measure of Correlation 2:

  $$(1-\exp {[-2(HXY2-HXY)]})^{\frac{1}{2}}$$

  where

  $$HXY2=-\sum\limits_{i = 1}^{N_{g}}\sum\limits_{j = 1}^{N_{g}} P_{x}(i) P_{y}(j) \log ({P_{x}(i) P_{y}(j)}) $$

• 14. Max Correlation Coefficient:

  $$\sqrt{\text{Second largest eigenvalue of Q}} $$

  where,

  $$Q(i,j)=\sum\limits_{k = 1}^{N_{g}} \frac{P(i,k) P(j,k)} {P_{x}(i) P_{y}(k)} $$