Multimodal Language Independent App Classification Using Images and Text

  • Kushal Singla
  • Niloy Mukherjee
  • Joy Bose
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10859)


There are a number of methods for classification of mobile apps, but most of them rely on a fixed set of app categories and text descriptions associated with the apps. Often, one may need to classify apps into a different taxonomy and might have limited app usage data for the purpose. In this paper, we present an app classification system that uses object detection and recognition in images associated with apps, along with text based metadata of the apps, to generate a more accurate classification for a given app according to a given taxonomy. Our image based approach can, in principle, complement any existing text based approach for app classification. We train a fast RCNN to learn the coordinates of bounding boxes in an app image for effective object detection, as well as labels for the objects. We then use the detected objects in the app images in an ensemble with a text based system that uses a hierarchical supervised active learning pipeline based on uncertainty sampling for generating the training samples for a classifier. Using the ensemble, we are able to obtain better classification accuracy than if either of the text or image systems are used on their own.


User modelling App classification Object recognition Object detection 


  1. 1.
    Zhu, H., Cao, H., Chen, E., Xiong, H., Tian, J.: Exploiting enriched contextual information for mobile app classification. In: Proceedings of the 21st ACM international conference on Information and knowledge management, pp. 1617–1621. ACM, 29 Oct 2012Google Scholar
  2. 2.
    Zhu, H., Chen, E., Xiong, H., Cao, H., Tian, J.: Mobile app classification with enriched contextual information. IEEE Trans. Mob. Comput. 13(7), 1550–1563 (2014)CrossRefGoogle Scholar
  3. 3.
    Lindorfer, M., Neugschwandtner, M., Platzer, C.: Marvin: Efficient and comprehensive mobile app classification through static and dynamic analysis. In: 39th Annual Computer Software and Applications Conference (COMPSAC), Vol. 2, pp. 422–433. IEEE (2015)Google Scholar
  4. 4.
    Lokhande, P.P., Shivaji, R.L.: A review on risk score based app classification using enriched contextual information of app context. Int. J. Comput. Sci. Inf. Technol. (IJCSIT), 5(6), 7063–7066 (2014)Google Scholar
  5. 5.
    Seneviratne, S., Seneviratne, A., Kaafar, M.A., Mahanti, A., Mohapatra, P.: Spam mobile apps: characteristics, detection, and in the wild analysis. ACM Trans. Web 11(1), 1–29 (2017). Article 4CrossRefGoogle Scholar
  6. 6.
    Olabenjo, B.: Applying Naive Bayes Classification to Google Play Apps Categorization. Arxiv.
  7. 7.
    Radosavljevic, V., Grbovic, M.: Smartphone App categorization for interest targeting in advertising marketplace. In Proceedings of the WWW (2016)Google Scholar
  8. 8.
    Shewale, S.K., Gayakee, V.V., Ugale, P.D., Sonawane, H.D.: Personalized App service system algorithm for effective classification of mobile applications. Int. J. Eng. Tech. Res. (IJETR) 3(1), January 2015. ISSN 2321–0869Google Scholar
  9. 9.
    Usilin, S., Nikolaev, D., Postnikov, V., Schaefer, G.: Visual appearance based document image classification. In: Proceedings of 2010 IEEE ICIP (2010)Google Scholar
  10. 10.
    Girshick, R.:. Fast R-CNN.
  11. 11.
    Microsoft Cognitive Toolkit Documentation. Object detection using Fast R-CNN, August 2017.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Samsung R&D InstituteBangaloreIndia

Personalised recommendations