Abstract
Review scores collect users’ opinions in a simple and intuitive manner. However, review scores are also easily manipulable, hence they are often accompanied by explanations. A substantial amount of research has been devoted to ascertaining the quality of reviews, to identify the most useful and authentic scores through explanation analysis. In this paper, we advance the state of the art in review quality analysis. We introduce a rating system to identify review arguments and to define an appropriate weighted semantics through formal argumentation theory. We introduce an algorithm to construct a corresponding graph, based on a selection of weighted arguments, their semantic similarity, and the supported ratings. We provide an algorithm to identify the model of such an argumentation graph, maximizing the overall weight of the admitted nodes and edges. We evaluate these contributions on the Amazon review dataset by McAuley et al. [15], by comparing the results of our argumentation assessment with the upvotes received by the reviews. Also, we deepen the evaluation by crowdsourcing a multidimensional assessment of reviews and comparing it to the argumentation assessment. Lastly, we perform a user study to evaluate the explainability of our method. Our method achieves two goals: (1) it identifies reviews that are considered useful, comprehensible, truthful by online users and does so in an unsupervised manner, and (2) it provides an explanation of quality assessments.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Source code available at: https://github.com/davideceolin/FAReviews.
- 2.
- 3.
- 4.
The questionnaire is available at https://forms.gle/srGJpGyYBzWd9RTaA.
References
Amgoud, L., Cayrol, C.: A reasoning model based on the production of acceptable arguments. Ann. Math. Artif. Intell. 34, 197–215 (2002)
Amgoud, L., Vesic, S.: Two roles of preferences in argumentation frameworks. In: Liu, W. (ed.) ECSQARU 2011. LNCS (LNAI), vol. 6717, pp. 86–97. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22152-1_8
Baroni, P., Caminada, M., Giacomin, M.: Abstract argumentation frameworks and their semantics. In: Baroni, P., Gabbay, D., Giacomin, M. (eds.) Handbook of Formal Argumentation, Chap. 4. College Publications, London (2018)
Bench-Capon, T.J.M.: Value-based argumentation frameworks. In: Proceedings of NMR Workshop, pp. 443–454 (2002)
Bench-Capon, T.J.M.: Persuasion in practical argument using value-based argumentation frameworks. J. Logic Comput. 13(3), 429–448 (2003)
Ceolin, D., Noordegraaf, J., Aroyo, L.: Capturing the ineffable: collecting, analysing, and automating web document quality assessments. In: Blomqvist, E., Ciancarini, P., Poggi, F., Vitali, F. (eds.) EKAW 2016. LNCS (LNAI), vol. 10024, pp. 83–97. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49004-5_6
Coste-Marquis, S., Konieczny, S., Marquis, P., Ouali, M.A.: Selecting extensions in weighted argumentation frameworks. In: Proceedings of COMMA. IOS Press (2012)
Coste-Marquis, S., Konieczny, S., Marquis, P., Ouali, M.A.: Weighted attacks in argumentation frameworks. In: Proceedings of KR, pp. 593–597. AAAI Press (2012)
Dunne, P.E., Hunter, A., McBurney, P., Parsons, S., Wooldridge, M.: Weighted argument systems: basic definitions, algorithms, and complexity results. Artif. Intell. 175(2), 457–486 (2011)
Ghose, A., Ipeirotis, P.G.: Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans. Knowl. Data Eng. 23(10), 1498–1512 (2011)
Kincaid, J., Fishburne, R., Rogers, R., Chissom, B.: Derivation of new readability formulas for navy enlisted personnel. Research branch report 8–75. Technical report, Chief of Naval Technical Training: Naval Air Station Memphis (1975)
Korfiatis, N., García-Bariocanal, E., Sánchez-Alonso, S.: Evaluating content quality and helpfulness of online product reviews: the interplay of review helpfulness vs. review content. Electron. Commer. Res. Appl. 11(3), 205–217 (2012)
Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. In: Proceedings of ICML, pp. 957–966. JMLR.org (2015)
Martínez, D.C., García, A.J., Simari, G.R.: An abstract argumentation framework with varied-strength attacks. In: Proceedings of KR, pp. 135–144. AAAI Press (2008)
McAuley, J.J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of SIGIR, pp. 43–52. ACM (2015)
Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of EMNLP, pp. 404–411. ACL (2004)
Modgil, S.: Reasoning about preferences in argumentation frameworks. Artif. Intell. 173(9), 901–934 (2009)
Ocampo Diaz, G., Ng, V.: Modeling and prediction of online product review helpfulness: a survey. In: Proceedings of ACL, vol. 1, pp. 698–708. ACL (2018)
Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of NLP Frameworks Workshop, pp. 45–50. ELRA (2010)
Roitero, K., Soprano, M., Fan, S., Spina, D., Mizzaro, S., Demartini, G.: Can the crowd identify misinformation objectively? The effects of judgment scale and assessor’s background. In: Proceedings of SIGIR, pp. 439–448. ACM (2020)
Wathen, C.N., Burkell, J.: Believe it or not: factors influencing credibility on the web. J. Am. Soc. Inf. Sci. Technol. 53(2), 134–144 (2002)
Wu, P., Van Der, Heijden, H., Korfiatis, N.: The influences of negativity and review quality on the helpfulness of online reviews. In: Proceedings of ICIS, pp. 3710–3719 (2011)
Wyner, A., Schneider, J., Atkinson, K., Bench-Capon, T.: Semi-automated argumentative analysis of online product reviews. In: Proceedings of COMMA, pp. 43–50. IOS Press (2012)
Acknowledgements
This work is partially supported by The Credibility Coalition.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ceolin, D., Primiero, G., Wielemaker, J., Soprano, M. (2021). Assessing the Quality of Online Reviews Using Formal Argumentation Theory. In: Brambilla, M., Chbeir, R., Frasincar, F., Manolescu, I. (eds) Web Engineering. ICWE 2021. Lecture Notes in Computer Science(), vol 12706. Springer, Cham. https://doi.org/10.1007/978-3-030-74296-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-74296-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-74295-9
Online ISBN: 978-3-030-74296-6
eBook Packages: Computer ScienceComputer Science (R0)