Skip to main content
SpringerLink
Log in

FauxWard: a graph neural network approach to fauxtography detection using social media comments

  • Original Article
  • Published:
Social Network Analysis and Mining Aims and scope Submit manuscript

Abstract

Online social media has been a popular source for people to consume and share news content. More recently, the spread of misinformation online has caused widespread concerns. In this work, we focus on a critical task of detecting fauxtography on social media where the image and associated text together convey misleading information. Many efforts have been made to mitigate misinformation online, but we found that the fauxtography problem has not been fully addressed by existing work. Solutions focusing on detecting fake images or misinformed texts alone on social media often fail to identify the misinformation delivered together by the image and the associated text of a fauxtography post. In this paper, we develop FauxWard, a novel graph convolutional neural network framework that explicitly explores the complex information extracted from a user comment network of a social media post to effectively identify fauxtography. FauxWard is content-free in the sense that it does not analyze the visual or textual contents of the post itself, which makes it robust against sophisticated fauxtography uploaders who intentionally craft image-centric posts by editing either the text or image content. We evaluate FauxWard on two real-world datasets collected from mainstream social media platforms (i.e., Reddit and Twitter). The results show that FauxWard is both effective and efficient in identifying fauxtography posts on social media.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://www.theverge.com/2016/10/7/13191788/clown-attack-threats-2016-panic-hoax-debunked.

  2. http://www.globaltimes.cn/content/759679.shtml.

  3. https://www.independent.co.uk/news/world/australasia/family-took-refuge-in-a-lake-to-escape-the-aussie-bushfires-8444881.html.

  4. https://www.reddit.com/.

  5. https://www.twitter.com/

  6. https://cloud.google.com/vision/.

References

  • Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp 5–10

  • Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:180110247

  • Cooper SD (2007) A concise history of the fauxtography blogstorm in the 2006 lebanon war. Am Commun J 9

  • Cooper BB (2013) Fast company. How twitter’s expanded images increase clicks, retweets and favorites. https://www.fastcompany.com/3022116/what-twitters-expanded-images-mean-for-clicks-retweets-and-favorites

  • Cui L, Wang S, Lee D (2019) Same: sentiment-aware multi-modal embedding for detecting fake news. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pp 41–48

  • Fout A, Byrd J, Shariat B, Ben-Hur A (2017) Protein interface prediction using graph convolutional networks. In: Advances in neural information processing systems, pp 6530–6539

  • Fridrich AJ, Soukal BD, Lukáš AJ (2003) Detection of copy-move forgery in digital images. In: Proceedings of digital forensic research workshop, Citeseer

  • Gupta A, Lamba H, Kumaraguru P, Joshi A (2013) Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd international conference on World Wide Web, pp 729–736

  • Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:170605674

  • Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  • Hill C (2017) Market Watch. Facebook and google’s war with fake news heats up. https://www.marketwatch.com/story/facebook-and-googles-war-with-fake-news-heats-up-2017-04-14

  • Huynh-Kha T, Le-Tien T, Ha-Viet-Uyen S, Huynh-Van K, Luong M (2016) A robust algorithm of forgery detection in copy-move and spliced images. Int J Adv Comput Sci Appl 7(3)

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980

  • Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:160902907

  • Kwak H, Lee C, Park H, Moon S (2010) What is twitter, a social network or a news media? In: Proceedings of the 19th international conference on World wide web, pp 591–600

  • Lazer DM, Baum MA, Benkler Y, Berinsky AJ, Greenhill KM, Menczer F, Metzger MJ, Nyhan B, Pennycook G, Rothschild D et al (2018) The science of fake news. Science 359(6380):1094–1096

    Article  Google Scholar 

  • Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Thirty-second AAAI conference on artificial intelligence

  • Mallonee L (2016) Wired. Photo fuels spread of fake news. https://www.wired.com/2016/12/photos-fuel-spread-fake-news/

  • Matern F, Riess C, Stamminger M (2019) Gradient-based illumination description for image forgery detection. IEEE Transactions on Information Forensics and Security 15:1303–1317

    Article  Google Scholar 

  • Murphy N (2019) Reddit. Reddit’s 2019 year in review. https://redditblog.com/2019/12/04/reddits-2019-year-in-review/

  • Nguyen TH, Grishman R (2018) Graph convolutional networks with argument-aware pooling for event detection. In: Thirty-second AAAI conference on artificial intelligence

  • Pérez-Rosas V, Kleinberg B, Lefevre A, Mihalcea R (2017) Automatic detection of fake news. arXiv preprint arXiv:170807104

  • Priya S, Sequeira R, Chandra J, Dandapat SK (2019) Where should one get news updates: Twitter or reddit. Online Soc Netw Media 9:17–29

    Article  Google Scholar 

  • Pun CM, Yuan XC, Bi XL (2015) Image forgery detection using adaptive oversegmentation and feature point matching. IEEE Trans Inf Forensics Secur 10(8):1705–1716

    Article  Google Scholar 

  • Qian F, Gong C, Sharma K, Liu Y (2018) Neural user response generator: fake news detection with collective user intelligence. IJCAI 18:3834–3840

    Google Scholar 

  • Rashid MT, Wang D (2020) Covidsens: a vision on reliable social sensing for covid-19. Artif Intell Rev, pp 1–25

  • Saleh K (2015) Invesp. Social media engagement statistics and trends. https://www.invespcro.com/blog/social-media-engagement/. Accessed 07 Aug 2018

  • Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, Springer, Berlin, pp 593–607

  • Shang L, Zhang DY, Wang M, Lai S, Wang D (2019a) Towards reliable online clickbait video detection: a content-agnostic approach. Knowl-Based Syst 182:104851

    Article  Google Scholar 

  • Shang L, Zhang DY, Wang M, Wang D (2019b) Vulnercheck: a content-agnostic detector for online hatred-vulnerable videos. In: 2019 IEEE international conference on big data (Big Data), IEEE, pp 573–582

  • Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor Newsl 19(1):22–36

    Article  Google Scholar 

  • Smith K (2020) Brand watch. 60 incredible and interesting twitter stats and statistics. https://www.brandwatch.com/blog/twitter-stats-and-statistics/

  • Vo N, Lee K (2018) The rise of guardians: fact-checking url recommendation to combat fake news. arXiv preprint arXiv:180607516

  • Volkova S, Shaffer K, Jang JY, Hodas N (2017) Separating facts from fiction: linguistic models to classify suspicious and trusted news posts on twitter. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 2: short papers), pp 647–653

  • Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2015b) Reliable social sensing with physical constraints: analytic bounds and performance evaluation. Real-Time Syst 51(6):724–762

    Article  Google Scholar 

  • Wang D, Szymanski BK, Abdelzaher T, Ji H, Kaplan L (2019) The age of social sensing. Computer 52(1):36–45

    Article  Google Scholar 

  • Wang D, Abdelzaher T, Kaplan L (2015a) Social sensing: building reliable systems on unreliable data. Morgan Kaufmann

  • Wang D, Abdelzaher T, Kaplan L, Aggarwal CC (2013a) Recursive fact-finding: a streaming approach to truth estimation in crowdsourcing applications. In: 2013 IEEE 33rd international conference on distributed computing systems, IEEE, pp 530–539

  • Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2013b) Exploitation of physical constraints for reliable social sensing. In: Real-time systems symposium (RTSS), 2013 IEEE 34th, IEEE, pp 212–223

  • Wang D, Amin MT, Li S, Abdelzaher T, Kaplan L, Gu S, Pan C, Liu H, Aggarwal CC, Ganti R, et al. (2014) Using humans as sensors: an estimation-theoretic perspective. In: Information processing in sensor networks, IPSN-14 proceedings of the 13th international symposium on, IEEE, pp 35–46

  • Wang D, Kaplan L, Le H, Abdelzaher T (2012) On truth discovery in social sensing: a maximum likelihood estimation approach. In: Proceedings of ACM/IEEE 11th international information processing in sensor networks (IPSN) conference, pp 233–244. https://doi.org/10.1109/IPSN.2012.6920960

  • Wang D, Szymanski BK, Abdelzaher T, Ji H, Kaplan L (2018) The age of social sensing. arXiv preprint arXiv:180109116

  • Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2019) A comprehensive survey on graph neural networks. arXiv preprint arXiv:190100596

  • Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu PS (2018) Ti-cnn: Convolutional neural networks for fake news detection. arXiv preprint arXiv:180600749

  • Yao QQ, Perlmutter DD, Liu JZ (2017) What are shaping the ethical bottom line?: Identifying factors influencing young readers’ acceptance of digital news photo alteration. Telematics Inform 34(1):124–132

    Article  Google Scholar 

  • Yin X, Han J, Yu PS (2008) Truth discovery with multiple conflicting information providers on the web. IEEE Trans Knowl Data Eng 20(6):796–808. https://doi.org/10.1109/TKDE.2007.190745

    Article  Google Scholar 

  • Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018a) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, pp 974–983

  • Ying Z, You J, Morris C, Ren X, Hamilton W, Leskovec J (2018b) Hierarchical graph representation learning with differentiable pooling. In: Advances in neural information processing systems, pp 4800–4810

  • Zhang DY, Badilla J, Zhang Y, Wang D (2018b) Towards reliable missing truth discovery in online social media sensing applications. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 143–150

  • Zhang DY, Li Q, Tong H, Badilla J, Zhang Y, Wang D (2018c) Crowdsourcing-based copyright infringement detection in live video streams. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 367–374

  • Zhang DY, Shang L, Geng B, Lai S, Li K, Zhu H, Amin MT, Wang D (2018d) Fauxbuster: a content-free fauxtography detector using social media comments. In: 2018 IEEE international conference on big data (Big Data), IEEE, pp 891–900

  • Zhang DY, Song L, Li Q, Zhang Y, Wang D (2018e) Streamguard: a bayesian network approach to copyright infringement detection problem in large-scale live video sharing systems. In: 2018 IEEE international conference on big data (big data), IEEE, pp 901–910

  • Zhang DY, Wang D, Zhang Y (2017) Constraint-aware dynamic truth discovery in big data social media sensing. In: Big data (big data), 2017 IEEE international conference on, IEEE, pp 57–66

  • Zhang D, Wang D, Vance N, Zhang Y, Mike S (2018a) On scalable and robust truth discovery in big data social media sensing applications. IEEE Trans Big Data 5(2):195–208

    Article  Google Scholar 

  • Zhang J, Cui L, Fu Y, Gouza FB (2018f) Fake news detection with deep diffusive network model. arXiv preprint arXiv:180508751

  • Zhang Y, Dong X, Rashid MT, Shang L, Han J, Zhang D, Wang D (2020a) Pqa-cnn: Towards perceptual quality assured single-image super-resolution in remote sensing. In: 2020 IEEE/ACM international symposium on quality of service, IEEE

  • Zhang Y, Dong X, Shang L, Zhang D, Wang D (2020b) A multi-modal graph neural network approach to traffic risk forecasting in smart urban sensing. In: The 17th annual IEEE international conference on sensing, communication and networking, IEEE

  • Zhang D, Vance N, Wang D (2019) When social sensing meets edge computing: vision and challenges. In: 2019 28th International conference on computer communication and networks (ICCCN), IEEE, pp 1–9

  • Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2018) Graph neural networks: a review of methods and applications. arXiv preprint arXiv:181208434

  • Zhou X, Zafarani R (2018) Fake news: a survey of research, detection methods, and opportunities. arXiv preprint arXiv:181200315

Download references

Acknowledgements

This research is supported in part by the National Science Foundation under Grant No. CNS-1845639, CNS-1831669, Army Research Office under Grant W911NF-17-1-0409. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN, 46556, USA

    Lanyu Shang, Yang Zhang, Daniel Zhang & Dong Wang

Authors
  1. Lanyu Shang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dong Wang.

Additional information

Publisher's Note

Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shang, L., Zhang, Y., Zhang, D. et al. FauxWard: a graph neural network approach to fauxtography detection using social media comments. Soc. Netw. Anal. Min. 10, 76 (2020). https://doi.org/10.1007/s13278-020-00689-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13278-020-00689-w

Keywords

Search

Navigation