Unsupervised learning blocking keys technique for indexing Arabic entity resolution

  • Marwah AlianEmail author
  • Arafat Awajan
  • Bandan Ramadan


Attribute values in textual datasets are subjects of different types of errors due to the data entry processes such as typographical errors, pronunciation errors or dialects alterations. These errors make the entity resolution process more challenging. The iterative blocking indexing technique can be used for correcting this type of errors mainly in query access where the records are stored into more than one block. Blocking indexing technique selects a subset of object pairs saved in the same block for later detailed computation for similarity discarding other pairs in other blocks considering them as irrelevant. This work aims to solving such problems for Arabic texts. It proposes to adapt a specific model for learning blocking keys and analyze its performance for Arabic datasets. The resulted blocking keys are passed as blocking keys to Dynamic Aware Inverted Index (DySimII) that worked efficiently with Arabic datasets. The model is tested against a telephone book dataset that contains duplicates and errors in attribute values according to phonetic and typing errors. The results reach a matching accuracy of 84% for using learned keys with small number of corrupted attributes while the performance is declined with the increase of the number of corrupted attributes.


Arabic entity resolution Learning keys Indexing Arabic datasets 


  1. Attia, M. (1999). A large scale computational processor of Arabic morphology and applications. Master’s Dissertation, Computer Engineering. Egypt: Cairo University.Google Scholar
  2. Ramadan, B. (2016). Indexing techniques for Real-time entity resolution on large dynamic databases, PhD Thesis, Ed.: Australian National University.Google Scholar
  3. Ramadan, B., Christen, P. (2014). Forest-based dynamic sorted neighborhood indexing for real-time entity resolution, in Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, Shanghai, pp. 1787–1790.Google Scholar
  4. Chelli, A. (2016). ASem Light Stemmer, available at:
  5. Christen, P. (2012). Data matching: Concepts and techniques for record link-age, entity resolution, and duplicate detection. New York: Springer.CrossRefGoogle Scholar
  6. Christen, P. (2012). A survey of indexing techniques for scalable record linkage and deduplication. IEEE Transactions on Knowledge and Data Engineering, 24(9), 1537–1555.CrossRefGoogle Scholar
  7. Christen, P., & Goiser, K. (2007). Quality and complexity measures for data linkage and deduplication, In Hamilton, F., Guillet, H. J. (Ed.) Quality measures in data mining, ser. studies in computational intelligence, New York: Springer, pp. 127–151.CrossRefGoogle Scholar
  8. Farghaly, A. F. (1987). Three level morphology for Arabic,” in the Arabic Morphology Workshop (AMW’87), Italy.Google Scholar
  9. Fellegi, I., & Sunter, A. (1969). A theory for record linkage. Journal of the American Statistical Association, 64(328), 1183–1210.CrossRefzbMATHGoogle Scholar
  10. Al Ameed, H. K., Al Ketbi, S. O., Al Kaabi, A. A., Al Shebli, K. S. A., Al Shamsi, N. F., Al Nuaimi, N. H., Al Muhairi S. S., (2005). Arabic Light Stemmer: Anew Enhanced Approach,” in The Second International Conference on Innovations in Information Technology (IIT’05),, pp. 1–9.Google Scholar
  11. Hernandez, M. A., & Stolfo, S. J. (1998). Real-world data is dirty: Data cleansing and the merge/purge problem. Data Mining and Knowledge Discovery, 2(1): 9–37.CrossRefGoogle Scholar
  12. Alian, M., Al-Naymat, G., Ramadan B. (2017). Using transliteration with entity resolution for Arabic datasets, in 14th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA’2017), Hammamet, Tunisia, 2017, pp. 593–597.
  13. Kejriwal M., Miranker, D. P. (2013). An unsupervised algorithm for learning blocking schemes, in 2013 IEEE 13th International Conference on Data Mining, pp. 340–349.Google Scholar
  14. Bilenko, M., Kamath, B., Mooney, R. J. (2006). Adaptive blocking: Learning to scale up record linkage, in Sixth IEEE International Conference on Data Mining (ICDM-06), Hong Kong, pp. 87–96.Google Scholar
  15. Christen, P., Gayler, R., & Hawking, D. (2009). Similarity-aware indexing for real-time entity resolution. in 18th ACM conference on Information and knowledge management, Hong Kong, China, pp. 1565–1568.Google Scholar
  16. Ramadan, B., & Christen, P. (2015). Unsupervised blocking key selection for real-time entity resolution in advances in knowledge discovery and data mining PAKDD 2015. Lecture Notes in Computer Science, Cham, Vol. 9078, pp. 574–585.Google Scholar
  17. Ramadan, B., Christen, P., Liang, H., & Gayler, R. W. (2013). Dynamic similarity-aware inverted indexing for real-time entity resolution in The series of lecture notes in computer science,trends and applications in knowledge discovery and data mining. New York: Springer, Vol. 7867, pp. 47–58.CrossRefGoogle Scholar
  18. Farghaly, A., Shaalan, K. (2009). Arabic natural language processing: Challenges and solutions. ACM Transactions on Asian Language Information Processing, 8(4), 22.CrossRefGoogle Scholar
  19. Tran, K. N., Vatsalan, D., Christen, P., (2013). GeCo—An online personal data Generator and Corruptor, in ACM Conference on Information and Knowledge Management (ICIKM’13), San Francisco, CA, USA, pp. 2473–2475,
  20. Vogel, T., Naumann, F. (2012). Automatic blocking key selection for duplicate detection based on unigram combinations, in the international workshop on quality in databases (QDB).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Hashemite UniversityZarqaJordan
  2. 2.Princess Sumaya University for TechnologyAmmanJordan
  3. 3.Prince Sultan UniversityRiyadhSaudi Arabia

Personalised recommendations