, Volume 19, Issue 2, pp 131-149
Date: 17 Aug 2012

Mobile PointMe-based spatial haptic interaction with annotated media for learning purposes

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


In this paper, we propose a framework to facilitate handheld device-based PointMe interaction with annotated media content, where the user points his/her handheld device to the media presentation screen in order to access services/information from the annotated media. Each annotation is encoded into customized learning object metadata (LOM) format that provides access point for available information about a specific media on the screen. By incorporating a fusion of infra-red (IR) motion points and accelerometer data of the handheld device, the system determines the learner’s PointMe location on the media screen. By using the pointed location spatial geometrical query is performed over the displayed annotated media in order to deliver learner’s level-specific interactive learning content to the user’s handheld device. The level customization is achieved by incorporating learner’s age, progress, etc., parameters in the media content delivery process. We design intuitive learning scenarios by leveraging the PointMe interaction scheme, where learners are challenged to answer quiz questions in learning-based games. This real-world interaction technique with the annotated media and seamless virtual learning information acquisition process makes the system transparent to young learners and encourage them to engage and concentrate on their learning activities. We develop a prototype and perform experiments in a technology-augmented learning space to show the suitability of the proposed approach.