Skip to main content

Automatic Recognition of Native Advertisements for the Slovak Language

  • Conference paper
  • First Online:
Systems, Signals and Image Processing (IWSSIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1527))

Included in the following conference series:

Abstract

In recent years the native advertisement is becoming more and more prevalent in online spaces. Differentiating between genuine content and native advertisement using Natural Language Processing is therefore also becoming a very interesting research topic. In this paper, we examine the possibilities of using deep textual representation for the Slovak language to recognize the “PR (Public relations) articles” (that serve as a native advertisement in this context) from authentic news articles on popular Slovak news websites. We show that the BERT (Bidirectional Encoder Representations from Transformers) embeddings as a text representation are sufficient for this task (achieving accuracy over 80% even with a statistical model - Logistic Regression) and that the models generally perform better without prior lemmatization.

We have scraped three Slovak news websites (for a total of 5455 news articles containing both paid-for content and a wide variety of genuine categories), and we have evaluated multiple binary classification methods (Logistic Regression, Random forest classifier and Support Vector Machines) trained on top of generated RoBERTa sentence embeddings. On our testing set, we were able to achieve an accuracy of 85.13%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Report available at: https://www.iabslovakia.sk/vydavky-do-reklamy/vydavky-do-internetovej-reklamy-2020-sk/ (Last accessed 09 Apr 2021).

  2. 2.

    Available at URLs: https://www.aktuality.sk/, https://www.cas.sk/, https://techbox.dennikn.sk/ (Last accessed 09 Apr 2021).

  3. 3.

    Available at https://nlp.johnsnowlabs.com/2020/05/05/lemma_sk.html (Last accessed 09 Apr 2021).

  4. 4.

    Known as stsb-xlm-r-multilingual; available at: https://huggingface.co/sentence-transformers/stsb-xlm-r-multilingual (Last accessed 09 Apr 2021).

  5. 5.

    Available at https://github.com/google/sentencepiece (Last accessed 09 Apr 2021).

References

  1. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326 (2015)

  2. Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L.: SemEval-2017 task 1: semantic textual similarity multilingual and crosslingual focused evaluation. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 1–14. Association for Computational Linguistics, Vancouver, Canada, August 2017. https://doi.org/10.18653/v1/S17-2001, https://www.aclweb.org/anthology/S17-2001

  3. Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale (2020)

    Google Scholar 

  4. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805 (2018). http://arxiv.org/abs/1810.04805

  5. Hládek, D., Staš, J., Ondáš, S.: Comparison of recurrent neural networks for Slovak punctuation restoration. In: 2019 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pp. 95–100 (2019). https://doi.org/10.1109/CogInfoCom47531.2019.9089903

  6. Kocaman, V., Talby, D.: Spark NLP: natural language understanding at scale. Softw. Impacts, 100058 (2021). https://doi.org/10.1016/j.simpa.2021.100058

  7. Kudo, T., Richardson, J.: SentencePiece: a simple and language independent subword tokenizer and detokenizer for neural text processing. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 66–71. Association for Computational Linguistics, Brussels, Belgium, November 2018. https://doi.org/10.18653/v1/D18-2012, https://www.aclweb.org/anthology/D18-2012

  8. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite BERT for self-supervised learning of language representations (2020)

    Google Scholar 

  9. Liu, Y., et al.: Roberta: a robustly optimized BERT pretraining approach (2019)

    Google Scholar 

  10. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)

    Google Scholar 

  11. Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.: Deep learning based text classification: A comprehensive review. arXiv preprint arXiv:2004.03705 (2020)

  12. Pecar, S., Simko, M., Bielikova, M.: Sentiment analysis of customer reviews: impact of text pre-processing. In: 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA), pp. 251–256 (2018). https://doi.org/10.1109/DISA.2018.8490619

  13. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar, October 2014. https://doi.org/10.3115/v1/D14-1162, https://www.aclweb.org/anthology/D14-1162

  14. Reddy, S., Yu, Y., Pappu, A., Sivaraman, A., Rezapour, R., Jones, R.: Detecting extraneous content in podcasts (2021)

    Google Scholar 

  15. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. arXiv preprint arxiv:1908.10084 (2019)

  16. Reimers, N., Gurevych, I.: Making monolingual sentence embeddings multilingual using knowledge distillation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, November 2020. https://arxiv.org/abs/2004.09813

  17. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter (2020)

    Google Scholar 

  18. Suppa, M., Adamec, J.: A summarization dataset of Slovak news articles. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 6725–6730 (2020)

    Google Scholar 

  19. Taylor, C.R.: Native advertising: the black sheep of the marketing family (2017)

    Google Scholar 

  20. Vaswani, A., et al..: Attention is all you need (2017)

    Google Scholar 

  21. Williams, A., Nangia, N., Bowman, S.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122. Association for Computational Linguistics (2018). http://aclweb.org/anthology/N18-1101

  22. Wu, S., Dredze, M.: Beto, Bentz, Becas: the surprising cross-lingual effectiveness of BERT (2019)

    Google Scholar 

Download references

Acknowledgment

The research described in the paper was done within the International Center of Excellence for Research on Intelligent and Secure Information and Communication Technologies and Systems - II. stage, ITMS code: 313021W404, co-financed by the European Regional Development Fund.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zuzana Bukovcikova .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Andicsova, V., Bukovcikova, Z., Sopiak, D., Oravec, M. (2022). Automatic Recognition of Native Advertisements for the Slovak Language. In: Rozinaj, G., Vargic, R. (eds) Systems, Signals and Image Processing. IWSSIP 2021. Communications in Computer and Information Science, vol 1527. Springer, Cham. https://doi.org/10.1007/978-3-030-96878-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-96878-6_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-96877-9

  • Online ISBN: 978-3-030-96878-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics