Abstract
Augmented Reality (AR) technology can overlay digital content over the physical world to enhance the user’s interaction with the real-world. The increasing number of devices for this purpose, such as Microsoft HoloLens, MagicLeap, Google Glass, allows to AR an immensity of applications. A critical task to make the AR devices more useful to users is the scene/environment understanding because this can avoid the device of mapping elements that were previously mapped and customized by the user. In this direction, we propose a scene classification approach for AR devices which has two components: i) an AR device that captures images, and ii) a remote server to perform scene classification. Four methods for scene classification, which utilize convolutional neural networks, support vector machine and transfer learning are proposed and evaluated. Experiments conducted using real data from an indoor office environment and Microsoft HoloLens AR device shows that the proposed AR scene classification approach can reach up to \(99\%\) of accuracy, even with similar texture information across scenes.
This work is partially supported by Sidia institute of science and technology, and Samsung Eletrônica da Amazônia Ltda, under the auspice of the Brazilian informatics law no 8.387/91.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Berger, A., Vokalova, A., Maly, F., Poulova, P.: Google glass used as assistive technology its utilization for blind and visually impaired people. In: Younas, M., Awan, I., Holubova, I. (eds.) MobiWIS 2017. LNCS, vol. 10486, pp. 70–82. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65515-4_6
Andujar, J.M., Mejias, A., Marquez, M.A.: Augmented reality for the improvement of remote laboratories: an augmented remote laboratory. IEEE Trans. Educ. 54(3), 492–500 (2011). https://doi.org/10.1109/TE.2010.2085047
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32
Bichlmeier, C., Ockert, B., Heining, S.M., Ahmadi, A., Navab, N.: Stepping into the operating theater: ARAV - augmented reality aided vertebroplasty. In: 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 165–166, September 2008. https://doi.org/10.1109/ISMAR.2008.4637348
Evans, G., Miller, J., Pena, M.I., MacAllister, A., Winer, E.: Evaluating the Microsoft HoloLens through an augmented reality assembly application. In: Degraded Environments: Sensing, Processing, and Display 2017, vol. 10197, p. 101970V. International Society for Optics and Photonics (2017)
Grubert, J., Langlotz, T., Zollmann, S., Regenbrecht, H.: Towards pervasive augmented reality: context-awareness in augmented reality. IEEE Trans. Visual. Comput. Graph. 23(6), 1706–1724 (2017). https://doi.org/10.1109/TVCG.2016.2543720
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016. https://doi.org/10.1109/CVPR.2016.90
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L.: Large-scale video classification with convolutional neural networks. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1725–1732, June 2014. https://doi.org/10.1109/CVPR.2014.223
Khurshid, A.: Adaptive face tracking based on online learning (2018)
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 225–234, November 2007. https://doi.org/10.1109/ISMAR.2007.4538852
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Liu, S., Tian, G.: An indoor scene classification method for service robot based on CNN feature. J. Robot. 2019, 1–12 (2019). https://doi.org/10.1155/2019/8591035
Mann, S.: Google eye, supplemental material for through the glass, lightly. IEEE Technol. Soc. Mag. 31(3), 10–14 (2012)
Natsume, S.G.: Virtual reality headset. US Patent Application 29/527,040, 07 June 2016
Niu, J., Bu, X., Qian, K., Li, Z.: An indoor scene recognition method combining global and saliency region features. Robot 37(1), 122–128 (2015)
Pai, H.: An imitation of 3D projection mapping using augmented reality and shader effects. In: 2016 International Conference on Applied System Innovation (ICASI), pp. 1–4, May 2016. https://doi.org/10.1109/ICASI.2016.7539879
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010). https://doi.org/10.1109/TKDE.2009.191
Pirri, F.: Indoor environment classification and perceptual matching. In: KR, pp. 73–84 (2004)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Santos, D., Lopez-Lopez, E., Pardo, X.M., Iglesias, R., Barro, S., Fdez-Vidal, X.R.: Robust and fast scene recognition in robotics through the automatic identification of meaningful images. Sensors 19(18), 4024 (2019)
Szegedy, C., et al.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9, June 2015. https://doi.org/10.1109/CVPR.2015.7298594
Wang, L., Guo, S., Huang, W., Xiong, Y., Qiao, Y.: Knowledge guided disambiguation for large-scale scene classification with multi-resolution CNNs. IEEE Trans. Image Proc. 26(4), 2055–2068 (2017)
Weiss, K., Khoshgoftaar, T.M., Wang, D.: A survey of transfer learning. J. Big Data 3(1), 9 (2016). https://doi.org/10.1186/s40537-016-0043-6
Wu, P., Li, Y., Yang, F., Kong, L., Hou, Z.: A CLM-based method of indoor affordance areas classification for service robots. Jiqiren/Robot 40(2), 188–194 (2018)
Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems, pp. 487–495 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Khurshid, A., Cleger, S., Grunitzki, R. (2020). A Scene Classification Approach for Augmented Reality Devices. In: Stephanidis, C., Chen, J.Y.C., Fragomeni, G. (eds) HCI International 2020 – Late Breaking Papers: Virtual and Augmented Reality. HCII 2020. Lecture Notes in Computer Science(), vol 12428. Springer, Cham. https://doi.org/10.1007/978-3-030-59990-4_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-59990-4_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59989-8
Online ISBN: 978-3-030-59990-4
eBook Packages: Computer ScienceComputer Science (R0)