Finding and Ranking High-Quality Answers in Community Question Answering Sites
- 218 Downloads
Community Question Answering (CQA) sites have become a very popular place to ask questions and give answers to a large community of users on the Internet. Stack Exchange is one of the popular CQA sites where a large amount of contents are posted every day in the form of questions, answers and comments. The answers on Stack Exchange are listed by their recent occurrences, time of posting or votes obtained by peer users under three tabs called active, oldest and votes, respectively. Votes tab is the default setting on the site and is also preferred tab of users because answers under this tab are voted as good answers by other users. The problem of voting-based sorting is that new answers which are yet to receive any vote are placed at the bottom in vote tab. The new answer may be of sufficiently high-quality to be placed at the top but no or fewer votes (later posting) have made them stay at the bottom. We introduce a new tab called promising answers tab where answers are listed based on their usefulness, which is calculated by our proposed system using the classification and regression models. Several textual features of answers and users reputation are used as features to predict the usefulness of the answers. The results are validated with good values of precision, recall, F1-score, area under the receiver operating characteristic curve (AUC) and root mean squared error. We also compare the top ten answers predicted by our system to the actual top ten answers based on votes and found that they are in high agreement.
KeywordsAnswer ranking Classification Community Question Answering Data imbalance Regression
- Alalwan, A., Rana, N. P., & Dwivedi, Y. K., Algharabat, R. (2017). Social media in marketing: A review and analysis of the existing literature, telematics and informatics, Available at http://www.sciencedirect.com/science/article/pii/S0736585317301077.
- Bian, J., Liu, Y., Zhou, D., Agitating, E., & Zha, H. (2009). Learning to recognize reliable users and content in social media with coupled mutual reinforcement. In Proceedings of the 18th international conference on world wide web (pp. 51–60).Google Scholar
- Burel, G., He, Y., & Alani, H. (2012). Automatic identification of best answers in online enquiry communities. The Semantic Web: Research and Applications, 7295, 514–529. (ESWC 2012. Lecture Notes in Computer Science).Google Scholar
- Calefato, F., Lanubile, F., & Novielli, N. (2016). Moving to stack overflow: Best-answer prediction in legacy developer forums. In Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement. Article 13 (pp. 1–10). ACM.Google Scholar
- Chall, J. S., & Dale, E. (1995). Manual for use of the new Dale–Chall readability formula. Brookline: Brookline Books.Google Scholar
- Chen, B. C., Dasgupta, A., Wang, X., & Yang, J. (2012). Vote calibration in community question-answering systems. In Proceedings of the 35th international ACM SIGIR conference on research and development in information retrieval (pp. 781–790). ACM.Google Scholar
- Craswell, N. (2009). Mean reciprocal rank. In L. Liu, M. T. Özsu (Eds.), Encyclopedia of database systems. (pp. 1703–1703). Springer US.Google Scholar
- Davis, J., & Goadrich, M. (2006). The relationship between Precision–Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning (pp. 233–240).Google Scholar
- Dong, H., Wang, J., Lin, H., Xu, B., & Yang, Z. (2015). Predicting best answerers for new questions: an approach leveraging distributed representations of words in community question answering. In 2015 ninth international conference on frontier of computer science and technology (FCST) (pp. 13–18). IEEE.Google Scholar
- Kincaid, J. P., Fishburne, R. P., Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel (No. RBR-8-75). Millington: Naval Technical Training Command Millington TN Research Branch.CrossRefGoogle Scholar
- Liu, Q., Agichtein, E., Dror, G., Gabrilovich, E., Maarek, Y., Pelleg, D., & Szpektor, I. (2011). Predicting web searcher satisfaction with existing community-based answers. In Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval. (pp. 415–424). ACM.Google Scholar
- Liu, L., & Ozsu, M. T. (Eds.). (2009). Mean average precision (p. 1703). Boston, MA: Springer.Google Scholar
- Plume, C. J., Dwivedi, Y. K., & Slade, E. L. (2016). Social media in the marketing context: A state of the art analysis and future directions (1st ed.). Oxford: Chandos Publishing Ltd.Google Scholar
- Sakai, T., Ishikawa, D., Kando, N., Seki, Y., Kuriyama, K., & Lin, C. Y. (2011). Using graded-relevance metrics for evaluating community QA answer selection. In Proceedings of the fourth ACM international conference on web search and data mining. (pp. 187–196). ACM.Google Scholar
- Shah, C., & Pomerantz, J. (2010). Evaluating and predicting answer quality in community QA. In Proceedings of the 33rd international ACM SIGIR conference on research and development in information retrieval. (pp. 411–418). ACM.Google Scholar
- Singh, J. P., Dwivedi, Y. K., Rana, N. P., Kumar, A., & Kapoor, K. K. (2017a). Event classification and location prediction from tweets during disasters. Annals of Operations Research. https://doi.org/10.1007/s10479-017-2522-3.
- The stack exchange dataset. (2017). Retrived from https://archive.org/details/stackexchange/. Accessed on March 13, 2017.