Skip to main content
SpringerLink
Log in

E-faceatlasAR: extend atlas of facial acupuncture points with auricular maps in augmented reality for self-acupressure

  • Original Article
  • Published:
Virtual Reality Aims and scope Submit manuscript

Abstract

Acupuncture is a form of alternative medicine from ancient times and adapted to modern medicine. One typical practice of acupuncture is putting pressure on specific sites (acupuncture points, or acupoints) in the body, called acupressure. Self-acupressure refers to people applying pressure on acupoints using fingers, palms, or special devices to themselves. It has been widely applied because of its convenience and effectiveness. However, localizing the acupoints in any acupuncture practice requires expert knowledge. Novices without any medical background typically find it difficult because of the lack of visual cues. Our previous prototype, FaceAtlasAR, stresses this issue by localizing and overlaying facial acupoints in the augmented reality (AR) context in real-time. This paper will present E-FaceAtlasAR, which extends FaceAtlasAR to show auricular zone maps in the meantime. The system first localizes facial acupoints and auricular zone map in an anatomical yet feasible way. Then, it overlays the requested acupoints and auricular zone maps on the ears in AR. We adopt Mediapipe, a cross-platform machine learning framework, to build the real-time pipeline that runs on desktop and Android phones. Also, we conducted extensive experiments to show its effectiveness and robustness. With this system, users, even not professionals, can position the acupoints quickly for their self-acupressure treatments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Data Availability

The data that support the findings of this study including public datasets AMI dataset and “In-the-wild” dataset, which are available in the public domain. The data citation in the main article has the full URL. Other data are available on request from the corresponding author, M.Z.

References

  • AACP: Evidence Based Acupuncture Training, Acupuncture in Physiotherapy. The Acupuncture Association of Chartered Physiotherapists (AACP)s. The Acupuncture Association of Chartered Physiotherapists (AACP)s

  • Artyfactory THE PROPORTIONS OF THE HEAD. https://www.artyfactory.com/portraits/pencil-portraits/proportions-of-a-head.html

  • Azuma RT (1997) A survey of augmented reality. Presence Teleoperators Virtual Environ 6(4):355–385

    Article  Google Scholar 

  • Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Trans Pattern Analy Mac Intell 35(12):2930–2940

    Article  Google Scholar 

  • Berman BM, Lao L, Langenberg P, Lee WL, Gilpin AM, Hochberg MC (2004) Effectiveness of acupuncture as adjunctive therapy in osteoarthritis of the knee: a randomized, controlled trial. Ann Intern Med 141(12):901–910

    Article  Google Scholar 

  • Bulat A, Tzimiropoulos G (2017) How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1021–1030

  • Chen Y-Z, Maigre C, Hu M-C, Lan K-c (2017) Localization of acupoints using augmented reality. In: Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 239–241

  • Emeršič Ž, Štruc V, Peer P (2017) Ear recognition: More than a survey. Neurocomputing 255:26–39

    Article  Google Scholar 

  • Emeršič Ž, Meden B, Peer P, Štruc V (2020) Evaluation and analysis of ear recognition models: performance, complexity and resource requirements. Neural Comput Appl 32(20):15785–15800

    Article  Google Scholar 

  • Esther Gonzalez LA, Mazorra L (2008) AMI Ear Database. https://ctim.ulpgc.es/research_works/ami_ear_database/

  • Frejlichowski D, Tyszkiewicz N (2010) The west pomeranian university of technology ear database–a tool for testing biometric algorithms. In: International Conference Image Analysis and Recognition, pp. 227–234. Springer

  • Godson DR, Wardle JL (2019) Accuracy and precision in acupuncture point location: a critical systematic review. J Acupunct Meridian Stud 12(2):52–66

    Article  Google Scholar 

  • Gori L, Firenzuoli F (2007) Ear acupuncture in european traditional medicine. Evid-Based Compl Altern Med 4(S1):13–16

    Article  Google Scholar 

  • Hansley EE, Segundo MP, Sarkar S (2018) Employing fusion of learned and handcrafted features for unconstrained ear recognition. IET Biometrics 7(3):215–223

    Article  Google Scholar 

  • Huber P, Hu G, Tena R, Mortazavian P, Koppen P, Christmas WJ, Ratsch M, Kittler J (2016) A multiresolution 3d morphable face model and fitting framework. In: Proceedings of the 11th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

  • Jiang H, Starkman J, Kuo C-H, Huang M-C (2015) Acu glass: quantifying acupuncture therapy using google glass. In: Proceedings of the 10th EAI International Conference on Body Area Networks, pp. 7–10

  • Kanehira R, Shoda A, Yagihashi M, Narita H, Fujimoto H (2008) Development of an acupuncture training system using virtual reality technology. In: 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery, vol. 4, pp. 665–668. IEEE

  • Kaptchuk TJ (2002) Acupuncture: theory, efficacy, and practice. Ann Intern Med 136(5):374–383

    Article  Google Scholar 

  • Kartynnik Y, Ablavatski A, Grishchenko I, Grundmann M (2019) Real-time facial surface geometry from monocular video on mobile gpus. arXiv preprint arXiv:1907.06724

  • Kumar A, Wu C (2012) Automated human identification using ear imaging. Pattern Recognit 45(3):956–968

    Article  Google Scholar 

  • Lan K-C, Li G-S, Zhang J-X (2018) Toward automated acupressure therapy. In: Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, pp. 384–385

  • Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: European Conference on Computer Vision, pp. 679–692. Springer

  • Lee D-T, Schachter BJ (1980) Two algorithms for constructing a delaunay triangulation. Int J Comp Inf Sci 9(3):219–242

    Article  MathSciNet  Google Scholar 

  • Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M, Zhang F, Chang C-L ,Yong MG, Lee J et al (2019) Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172

  • MAI AcuMap. https://www.mai.ai/acumap/

  • Molsberger A, Manickavasagan J, Abholz H, Maixner W, Endres H (2012) Acupuncture points are large fields: the fuzziness of acupuncture point localization by doctors in practice. Europ J Pain 16(9):1264–1270

    Article  Google Scholar 

  • Mount DM (2002) Cmsc 754 computational geometry. Lecture Notes, University of Maryland, US

    Google Scholar 

  • Nogier P (1956) Le pavillon de l’oreille. zones et points réflexes. Bulletin de la Société d’Acupuncture 20:51–57

    Google Scholar 

  • Nogier PM, Cousino A, Graff M (1983) From Auriculotherapy to Auriculomedicine. Maisonneuve, Montréal, QC, Canada

    Google Scholar 

  • Oleson T, Flocco W (1993) Randomized controlled study of premenstrual symptoms treated with ear, hand, and foot reflexology. Obst Gynecol 82:906–911

    Google Scholar 

  • Oleson TD, Kroening RJ, Bresler DE (1980) An experimental evaluation of auricular diagnosis: the somatotopic mapping of musculoskeletal pain at ear acupuncture points. Pain 8(2):217–229

    Article  Google Scholar 

  • Rambau J (2002) TOPCOM: Triangulations of point configurations and oriented matroids. In: Mathematical software, pp. 330–340

  • Raposo R, Hoyle E, Peixinho A, Proença H (2011) Ubear: A dataset of ear images captured on-the-move in uncontrolled conditions. In: 2011 IEEE Workshop on Computational Intelligence in Biometrics and Identity Management (CIBIM), pp. 84–90. IEEE

  • Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semi-automatic methodology for facial landmark annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903

  • Standard C (2008) Nomenclature and location of auricular points. Standard, Chinese Acupuncture Society, Beijing

    Google Scholar 

  • Tkachenka A, Karpiak G, Vakunov A, Kartynnik Y, Ablavatski A, Bazarevsky V, Pisarchyk S (2019) Real-time hair segmentation and recoloring on mobile gpus. arXiv preprint arXiv:1907.06740

  • Unknown (2005) National Standard of the People’s Republic of China, Acupoints. https://musculoskeletalkey.com/ynsa-basic-points/

  • Unknown (2007) Atlas Of Acupuncture Points. www.AcupunctureProducts.com

  • Unknown (2017) YNSA Basic Points. https://musculoskeletalkey.com/ynsa-basic-points/

  • Vickers AJ, Vertosick EA, Lewith G, MacPherson H, Foster NE, Sherman KJ, Irnich D, Witt CM, Linde K, Collaboration AT et al (2018) Acupuncture for chronic pain: update of an individual patient data meta-analysis. J Pain 19(5):455–474

    Article  Google Scholar 

  • White A, Ernst E (2004) A brief history of acupuncture. Rheumatology 43(5):662–663

    Article  Google Scholar 

  • Wu C, Liu P, Fu H, Chen W, Cui S, Lu L, Tang C (2018) Transcutaneous auricular vagus nerve stimulation in treating major depressive disorder: a systematic review and meta-analysis. Medicine 97(52):e13845

    Article  Google Scholar 

  • Yao J, Chen L, Zhang L, Zhou S, Zheng Q, Feng X, You X, Zhang L, Li Y (2019) Effect of auriculotherapy and intervention types on weight control: a systematic review and meta-analysis protocol. Medicine 98(34):e16959

    Article  Google Scholar 

  • Yau E Virtual Reality Helps Chinese Medicine Students Learn Acupuncture and Doctors Treat Cancer. https://www.scmp.com/lifestyle/health-wellness/article/2169092/virtual-reality-helps-chinese-medicine-students-learn

  • Yuan J, Purepong N, Kerr DP, Park J, Bradbury I, McDonough S (2008) Effectiveness of acupuncture for low back pain: a systematic review. Spine 33(23):887–900

    Article  Google Scholar 

  • Zhang M, Schulze J, Zhang D (2021) Faceatlasar: Atlas of facial acupuncture points in augmented reality. arXiv preprint arXiv:2111.14755

  • Zhou Y, Zaferiou S (2017) Deformable models of ears in-the-wild for alignment and recognition. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), pp. 626–633. IEEE

  • Zhu X, Ramanan D (2012) Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. IEEE

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. Computer Science and Engineering, University of California, San Diego, 9500 Gilman Dr, La Jolla, CA, 92093, USA

    Menghe Zhang & Jürgen P. Schulze

  2. Automation, Qilu University of Technology, 3501 Daxue Rd, Jinan, 250353, Shandong, China

    Dong Zhang

Authors
  1. Menghe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen P. Schulze
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Menghe Zhang.

Ethics declarations

Conflict of interest

The authors declare they have no financial or non-financial interests.

Code availability

Code will be available at https://github.com/ZhangMenghe/mediapipe/.

Ethics approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

The authors affirm that human research participants provided informed consent for publication of the images in Figs. 6, 7, 19 and 20.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, M., Schulze, J.P. & Zhang, D. E-faceatlasAR: extend atlas of facial acupuncture points with auricular maps in augmented reality for self-acupressure. Virtual Reality 26, 1763–1776 (2022). https://doi.org/10.1007/s10055-022-00663-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10055-022-00663-1

Keywords

Search

Navigation