Skip to main content

Advertisement

SpringerLink
Log in

An acupoint health care system with real-time acupoint localization and visualization in augmented reality

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Acupressure is a noninvasive method in traditional Chinese medicine to increase blood circulation, relieve discomfort, promote health, and prevent disease. This paper presents a multifunctional acupoint application system that not only provides real-time visualization of acupoints on mobile devices for users’ self-acupressure but also provides health care, education, and entertainment through acupressure. The system allows users to obtain acupoint information to relieve specific discomfort or symptoms. It automatically marks the requested acupoints on a real-time camera stream to guide users to locate acupoints so that even users unfamiliar with acupoints can easily find acupoints for self-acupressure. Three core technologies are used: acupoint localization and knowledge, real-time image recognition, and augmented reality. This paper details the system design and implementation. This system uses Google’s machine learning kit for facial detection to extract the user’s facial features and contours in real time on a mobile device. These contour points are used as landmarks with specific displacements to locate acupoints. As there are many acupoints and various positioning methods, the acupoint location is traditionally described in words through specific feature points and corresponding displacements related to each person’s physiological characteristics. This work proposes a novel acupoint positioning and marking mechanism based on the standard acupoint localization, which formalizes various acupoint positioning methods so that positions can be processed and stored in an acupoint database established in this system. This positioning information allows the proposed mechanism to fulfill real-time visualization of acupoints in augmented reality on the camera stream. Experimental results show that the proposed system locates and marks acupoints with low latency and high accuracy. Moreover, the system also provides recommendations for further medical treatment and a mini-game that tests knowledge of acupoints, which enhances users’ experience of acupoint massage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Android Medical—Best medical apps for Android, Available from: https://www.androidmedical.com/apps-acupressure.

  2. J. Petr: Android application for acupressure, master thesis, Department of Computer Science, Czech Technical University in Prague, April 2020

  3. K. Štěpán: iOS application for acupressure, master thesis, Department of Computer Science, Czech Technical University in Prague, May 2020

  4. H. Jiang, J. Starkman, C. H. Kuo, and M. C. Huang, Acu glass: quantifying acupuncture therapy using Google glass, Proceedings of the 10th EAI International Conference on Body Area Networks, 2015, 7–10, https://doi.org/10.4108/eai.28-9-2015.2261520.

  5. M. Chang and Q. Zhu, Automatic location of facial acupuncture-point based on facial feature points positioning, Proceedings of the 2017 5th International Conference on Frontiers of Manufacturing Science and Measuring Technology (FMSMT 2017), 2017, 545–549, https://doi.org/10.2991/fmsmt-17.2017.111.

  6. Lan, K.C., Hu, M.C., Chen, Y.Z., Zhang, J.X.: The application of 3D morphable model (3DMM) for real-time visualization of acupoints on a smartphone. IEEE Sens. J. 21, 3289–3300 (2021). https://doi.org/10.1109/JSEN.2020.3022958

    Article  Google Scholar 

  7. M. Zhang, J. P. Schulze, and D. Zhang, FaceAtlasAR: Atlas of Facial Acupuncture Points in Augmented Reality, 11th International Conference on Computer Science and Information Technology, Vancouver, Canada, May 29–30, 2021, https://doi.org/10.48550/arXiv.2111.14755.

  8. Zhang, M., Schulze, J.P., Zhang, D.: E-faceatlasAR: extend atlas of facial acupuncture points with auricular maps in augmented reality for self-acupressure. Virtual Real. (2022). https://doi.org/10.1007/s10055-022-00663-1

    Article  Google Scholar 

  9. J. X. Zhang: Localization of foot acupoints on a smartphone using augmented reality, M.S. Thesis, Institute of Medical Informatics, National Cheng Kung University, Tainan, Taiwan, R.O.C., 2021

  10. G. R. Huag: On the development of an augmented reality system for palm acupoint, M.S. Thesis, Department of Multimedia Design, National Taichung University of Science and Technology, Taichung, Taiwan, R.O.C., 2016

  11. Gach, M.R.: Acupressure: how to cure common ailments the natural way. Piatkus Books, London, England (1993)

    Google Scholar 

  12. Focks, C.: Atlas of Acupuncture. Elsevier, The Netherlands (2008)

    Google Scholar 

  13. Bleecker, D.: Acupuncture points handbook: a patient’s guide to the locations and functions of over 400 acupuncture points. Draycott Publishing, LLC, Texas, USA (2017)

    Google Scholar 

  14. Tan, H., Tumilty, S., Chapple, C., Liu, L., McDonough, S., Yin, H., et al.: Understanding acupoint sensitization: a narrative review on phenomena, potential mechanism, and clinical application. Evid. Based Complement. Altern. Med. 2019, 9 (2019). https://doi.org/10.1155/2019/6064358. (Article ID 6064358)

    Article  Google Scholar 

  15. Kim, C., Yoon, D., Lee, Y., Jung, W., Kim, J., Chae, Y.: Revealing associations between diagnosis patterns and acupoint prescriptions using medical data extracted from case reports. J. Clin. Med. 8, 10 (2019). https://doi.org/10.3390/jcm8101663. (Article 1663)

    Article  Google Scholar 

  16. Robinson, N., Lorenc, A., Liao, X.: The evidence for Shiatsu: a systematic review of shiatsu and acupressure. BMC Complement. Altern. Med. 11, 15 (2011). https://doi.org/10.1186/1472-6882-11-88. (Article 88)

    Article  Google Scholar 

  17. Napadow, V., Liu, J., Kaptchuk, T.J.: A systematic study of acupuncture practice: acupoint usage in an outpatient setting in Beijing, China. Complement. Ther. Med. 12, 209–216 (2004). https://doi.org/10.1016/j.ctim.2004.10.001

    Article  Google Scholar 

  18. R. Hosseinabadi, K. Noroozi, Z. Poorismaili, M. Karimloo, and S. S. Maddah, Acupoint massage in improving sleep quality of older adults, J. Reha, 9(2008), 8–14, http://rehabilitationj.uswr.ac.ir/article-1-247-en.html

  19. World Health Organization, Who Standard Acupuncture Point Locations in The Western Pacific Region, 2008

  20. Dr. Chen, Hsin-Hao, Attending physician, Department of Family Medicine, Mackay Memorial Hospital, Hsinchu, Taiwan. Available from: https://www.hc.mmh.org.tw/doctor_view.php?depid=49&did=235

  21. Core ML - Integrate machine learning models into your app., Apple Developer. Available from: https://developer.apple.com/documentation/coreml

  22. Create ML - Create machine learning models for use in your app., Apple Developer. Available from: https://developer.apple.com/documentation/createml

  23. Turi Create API Documentation. Available from: https://apple.github.io/turicreate/docs/api/

  24. ML Kit, Google. Available from: https://developers.google.com/ml-kit/

  25. TensorFlow Lite, Google. Available from: https://www.tensorflow.org/lite/

  26. PyTorch Mobile. Available from: https://pytorch.org/mobile/home/

  27. F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C. L. Chang, and M. Grundmann: Mediapipe: A framework for building perception pipelines. Available from: http://arxiv.org/abs/1906.08172 (2019)

  28. ML Kit—Face Detection, Google. Available from: https://developers.google.com/ml-kit/vision/face-detection

  29. Lan, K.C., Litscher, G.: Robot-controlled acupuncture—an innovative step towards modernization of the ancient traditional medical treatment method. Medicines (Basel) 6(3), 87 (2019). https://doi.org/10.3390/medicines6030087

    Article  Google Scholar 

  30. Kamble, M.S., Londhe, V.: 3-D face image identification from video streaming using map reduce (hadoop). Int. Res. J. Eng. Technol. 5, 1613–1616 (2018)

    Google Scholar 

  31. Danish, M., Jiang, Q.: 3D Localization of hand acupoints using hand geometry and landmark points based on RGB-D CNN fusion. Ann. Biomed. Eng. (2022). https://doi.org/10.1007/s10439-022-02986-1

    Article  Google Scholar 

  32. OpenCV. Available from: https://opencv.org/

  33. Android Studio. Available from: https://developer.android.com/studio/intro

  34. Android Developer Guides. Available from: https://developer.android.com/guide

  35. Maria DB. Available from: https://mariadb.org/

  36. SQLite. Available from: https://www.sqlite.org/index.html

  37. Introducing JSON. Available from: https://www.json.org/json-en.html

  38. Android Developer Guides. CameraX. Available from: https://developer.android.com/training/camerax

  39. Beautiful Soup Documentation. Available from: https://www.crummy.com/software/BeautifulSoup/bs4/doc/

  40. National health insurance administration ministry of health and welfare. Available from: https://www.nhi.gov.tw/English/

  41. Canvas, Android Developers. Available from: https://developer.android.com/reference/android/graphics/Canvas

Download references

Acknowledgements

We would like to thank H. W. Lee and W. R. Yang for their assistance in the project development. We also thank H. L. Wu and C. F. Chung for participating in the experiments. Special thanks to the anonymous reviewers for their valuable comments in improving this research. This research was funded in part by the Ministry of Science and Technology, Taiwan, grant numbers 109-2813-C-260-017-H, 109-2221-E-260-011, 110-2221-E-260-001, and 111-2221-E-260-006.

Author information

Authors and Affiliations

  1. Department of Information Management, National Chi Nan University, Puli, Nantou, Taiwan, ROC

    Mei-Ting Su, Mei-Ling Chiang, Chia-Hsuan Tsai, Chi-Wei Lin, Rong-Xuan Liu & Yong-Ting Juang

  2. Department of Family Medicine, Hsinchu MacKay Memorial Hospital, Hsinchu, Taiwan, ROC

    Hsin-Hao Chen

  3. Department of Medicine, MacKay Medical College, New Taipei City, Taiwan, ROC

    Hsin-Hao Chen

Authors
  1. Mei-Ting Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mei-Ling Chiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chia-Hsuan Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi-Wei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rong-Xuan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yong-Ting Juang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hsin-Hao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mei-Ling Chiang performed the project management. Mei-Ting Su and Mei-Ling Chiang wrote the main manuscript text. Mei-Ting Su, Chia-Hsuan Tsai, Chi-Wei Lin, Rong-Xuan Liu, and Yong-Ting Juang cooperated to design and implement the proposed system. Mei-Ting Su performed experiments, and Hsin-Hao Chen verification experimental results. All authors reviewed the manuscript.

Corresponding author

Correspondence to Mei-Ling Chiang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, MT., Chiang, ML., Tsai, CH. et al. An acupoint health care system with real-time acupoint localization and visualization in augmented reality. Multimedia Systems 29, 2217–2238 (2023). https://doi.org/10.1007/s00530-023-01104-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01104-y

Keywords

Search

Navigation