Software to Support Layout and Data Collection for Machine-Learning-Based Real-World Sensors

  • Ayane SaitoEmail author
  • Wataru Kawai
  • Yuta Sugiura
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1033)


There have been many studies of gesture recognition and posture estimation by combining real-world sensor and machine learning. In such situations, it is important to consider the sensor layout because the measurement result varies depending on the layout and the number of sensors as well as the motion to be measured. However, it takes time and effort to prototype devices multiple times in order to find a sensor layout that has high identification accuracy. Also, although it is necessary to acquire learning data for recognizing gestures, it takes time to get the data when the user changes the sensor layout. In this study, we developed software that can arrange real-world sensors. In this time, the software can handle distance-measuring sensors as real-world sensors. The user places these sensors freely in the software. The software measures the distance between the sensors and a mesh created from measurements of real-world deformation recorded by a Kinect. The classifier is generated using the time-series of distance data recorded by the software. In addition, we created a physical device that had the same sensor layout as the one designed with the software. We experimentally confirmed that the software could recognize the gestures on the physical device by using the generated classifier.


Sensor layout Machine learning Distance-measuring sensor 



This work was supported by JST AIP-PRISM JPMJCR18Y2 and JST PRESTO JPMJPR17J4.


  1. 1.
    Sato, M., Poupyrev, I., Harrison, C.: Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2012), pp. 483–492. ACM, New York (2012)Google Scholar
  2. 2.
    Ino, K., Ienaga, N., Sugiura, Y., Saito, H., Miyata, N., Tada, M.: Grasping hand pose estimation from RGB images using digital human model by convolutional neural network. In: 9th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, pp. 154–160. HOMETRICA CONSULTING, Lugano, Switzerland (2018)Google Scholar
  3. 3.
    Kanaya, T., Hiromori, A., Yamaguchi, H., Higashino, T.: Humans: a human mobility sensing simulator. In: 2012 5th International Conference on New Technologies, Mobility and Security (NTMS), pp. 1–4. IEEE, Istanbul, Turkey (2012)Google Scholar
  4. 4.
    Yuan, Y., Kitani, K.: 3D ego-pose estimation via imitation learning. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 763–778. Springer, Cham (2018). Scholar
  5. 5.
    Masai, K., Sugiura, Y., Ogata, M., Kunze, K., Inami, M., Sugimoto, M.: Facial expression recognition in daily life by embedded photo reflective sensors on smart eyewear. In: Proceedings of the 21st International Conference on Intelligent User Interfaces (IUI 2016), pp. 317–326. ACM, New York (2016)Google Scholar
  6. 6.
    Kikuchi, T., Sugiura, Y., Masai, K., Sugimoto, M., Thomas, B.: Eartouch: turning the ear into an input surface. In: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2017), pp. 27:1–27:6. ACM, New York (2017)Google Scholar
  7. 7.
    Ogata, M., Sugiura, Y., Osawa, H., Imai, M.: iRing: intelligent ring using infrared reflection. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST 2012), pp. 131–136. ACM, New York (2012)Google Scholar
  8. 8.
    Miyata, N., Honoki, T., Maeda, Y., Endo, Y., Tada, M., Sugiura, Y.: Wrap sense: grasp capture by a band sensor. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST 2016), pp. 87–89. ACM, New York (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Keio UniversityYokohamaJapan
  2. 2.The University of TokyoBunkyoJapan
  3. 3.JST PRESTOKawaguchiJapan

Personalised recommendations