Specification and Verification of Complex Location Events with Panoramic

  • Evan Welbourne
  • Magdalena Balazinska
  • Gaetano Borriello
  • James Fogarty
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6030)


We present the design and evaluation of Panoramic, a tool that enables end-users to specify and verify an important family of complex location events. Our approach aims to reduce or eliminate critical barriers to deployment of emerging location-aware business activity monitoring applications in domains like hospitals and office buildings. Panoramic does not require users to write code, understand complex models, perform elaborate demonstrations, generate test location traces, or blindly trust deterministic events. Instead, it allows end-users to specify and edit complex events with a visual language that embodies natural concepts of space and time. It also takes a novel approach to verification, in which events are extracted from historical sensor data traces and then presented with intelligible, hierarchical visualizations that represent uncertainty with probabilities. We build on our existing software for specifying and detecting events while enhancing it in non-trivial ways to facilitate event specification and verification. Our design is guided by a formative study with 12 non-programmers. We also use location traces from a building-scale radio frequency identification (RFID) deployment in a qualitative evaluation of Panoramic with 10 non-programmers. The results show that end-users can both understand and verify the behavior of complex location event specifications using Panoramic.


Sensor Error Syntax Error Sensor Trace Event Query Single Scene 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amelior ORTracker: Orchestrate Patient Flow (2009),
  2. 2.
    Assad, M., et al.: PersonisAD: Distributed, Active, Scrutable Model Framework for Context-Aware Services. In: LaMarca, A., Langheinrich, M., Truong, K.N. (eds.) Pervasive 2007. LNCS, vol. 4480, pp. 55–72. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  3. 3.
    Bardram, J.E.: The Java Context Awareness Framework (JCAF) – A service infrastructure and programming framework for context-aware applications. In: Gellersen, H.-W., Want, R., Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol. 3468, pp. 98–115. Springer, Heidelberg (2005)Google Scholar
  4. 4.
    Bellotti, V., Edwards, K.: Intelligibility and Accountability: Human Considerations in Context Aware Systems. HCI 16, 193–212 (2001)CrossRefGoogle Scholar
  5. 5.
    Chen, J., Weld, D.S.: Recovering from Errors During Programming by Demonstration. In: IUI 2008, pp. 159–168 (2008)Google Scholar
  6. 6.
    Cheverst, K., et al.: Exploring Issues of User Model Transparency and Proactive Behaviour in an Office Environment Control System. User Modeling and User-Adapted Interaction 15(3-4), 235–273 (2005)CrossRefGoogle Scholar
  7. 7.
    Dey, A., Newberger, A.: Support for Context-Aware Intelligibility and Control. In: CHI 2009, pp. 859–868 (2009)Google Scholar
  8. 8.
    Dey, A.K., et al.: A CAPpella: Programming by Demonstration of Context-Aware Applications. In: CHI 2004, vol. 1, pp. 33–40 (2004)Google Scholar
  9. 9.
    Fails, J., Olsen, D.: A Design Tool for Camera-Based Interaction. In: CHI 2003, pp. 449–456 (2003)Google Scholar
  10. 10.
    Garofalakis, M.N., et al.: Probabilistic Data Management for Pervasive Computing: The Data Furnace Project. IEEE Data Eng. Bull. 29(1), 57–63 (2006)Google Scholar
  11. 11.
    Google Maps API - Google Code,
  12. 12.
    Google Web Toolkit - Google Code (2009),
  13. 13.
    Hartmann, B., et al.: Authoring Sensor-Based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. In: CHI 2007, pp. 145–154 (2007)Google Scholar
  14. 14.
    Heer, J., et al.: Liquid: Context-Aware Distributed Queries. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 140–148. Springer, Heidelberg (2003)Google Scholar
  15. 15.
    Real-time locating systems 2009-2019 (2009),
  16. 16.
    Knoll, S., et al.: Viewing Personal Data Over Time. In: CHI 2009 Workshop on Interacting with Temporal Data (April 2009)Google Scholar
  17. 17.
    Lamming, M., Bohm, D.: SPECs: Another Approach to Human Context and Activity Sensing Research, Using Tiny Peer-to-Peer Wireless Computers. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 192–199. Springer, Heidelberg (2003)Google Scholar
  18. 18.
    Lau, T., et al.: Why PBD Systems Fail: Lessons Learned for Usable AI. In: CHI 2008 (2008)Google Scholar
  19. 19.
    Letchner, J., et al.: Challenges for Event Queries over Markovian Streams. IEEE Internet Computing 12(6), 30–36 (2008)CrossRefGoogle Scholar
  20. 20.
    Li, Y., et al.: Topiary: A Tool for Prototyping Location-Enhanced Applications. In: UIST 2004, pp. 217–226 (2004)Google Scholar
  21. 21.
    Lim, B., Dey, A.: Assessing Demand for Intelligibility in Context-Aware Applications. In: Ubicomp 2009, pp. 195–204 (2009)Google Scholar
  22. 22.
    Lymberopoulos, D., et al.: A Sensory Grammar for Inferring Behaviors in Sensor Networks. In: IPSN 2006, pp. 251–259 (2006)Google Scholar
  23. 23.
    Maynes-Aminzade, D., et al.: Eyepatch: Prototyping Camera-Based Interaction Through Examples. In: UIST 2007, pp. 33–42 (2007)Google Scholar
  24. 24.
    McCarthy, J.F., Anagnost, T.D.: EVENTMANAGER: Support for the Peripheral Awareness of Events. In: Thomas, P., Gellersen, H.-W. (eds.) HUC 2000. LNCS, vol. 1927, pp. 227–235. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  25. 25.
    Olston, C., et al.: Generating Example Data for Dataflow Programs. In: SIGMOD 2009, pp. 245–256 (2009)Google Scholar
  26. 26.
    Pascoe, J.: The Stick-e Note Architecture: Extending the Interface Beyond the User. In: IUI 1997, pp. 261–264 (1997)Google Scholar
  27. 27.
    Philly Hospital Uses RTLS to Track Patient Flow, Care and Training (May 2009),
  28. 28.
    Ré, C., et al.: Event Queries on Correlated Probabilistic Streams. In: SIGMOD 2008, June 2008, pp. 715–728 (2008)Google Scholar
  29. 29.
    RTLS Providers Cite Strong Demand From Hospitals (June 2009),
  30. 30.
    Salber, D., et al.: The Context Toolkit: Aiding the Development of Context-Enabled Applications. In: CHI 1999, pp. 434–441 (1999)Google Scholar
  31. 31.
    SIMILE Timeline (2009),
  32. 32.
    Sohn, T., Dey, A.: iCAP: An Informal Tool for Interactive Prototyping of Context-Aware Applications. In: CHI 2003, pp. 974–975 (2003)Google Scholar
  33. 33.
    Truong, K.N., et al.: CAMP: A Magnetic Poetry Interface for End-User Programming of Capture Applications for the Home. In: Davies, N., Mynatt, E.D., Siio, I. (eds.) UbiComp 2004. LNCS, vol. 3205, pp. 143–160. Springer, Heidelberg (2004)Google Scholar
  34. 34.
    Vilamovska, A., et al.: Study on the requirements and options for RFID application in healthcare. Technical report, RAND Corporation (July 2009)Google Scholar
  35. 35.
    Want, R., et al.: An Overview of the PARCTAB Ubiquitous Computing Experiment. IEEE Personal Communications 2(6), 28–33 (1995)CrossRefGoogle Scholar
  36. 36.
    Welbourne, E., et al.: Challenges for Pervasive RFID-based Infrastructures. In: PERTEC 2007, March 2007, pp. 388–394 (2007)Google Scholar
  37. 37.
    Welbourne, E., et al.: Cascadia: A System for Specifying, Detecting, and Managing RFID Events. In: MobiSys 2008, June 2008, pp. 281–294 (2008)Google Scholar
  38. 38.
    Welbourne, E., et al.: Building the Internet of Things Using RFID: The RFID Ecosystem Experience. IEEE Internet Computing (May 2009)Google Scholar
  39. 39.
    Welbourne, E., et al.: Longitudinal Study of a Building-Scale RFID Ecosystem. In: MobiSys 2009 (June 2009)Google Scholar
  40. 40.
    Whitehouse, K., et al.: Automatic Programming with Semantic Streams. In: SenSys 2005, November 2005, pp. 290–291 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Evan Welbourne
    • 1
  • Magdalena Balazinska
    • 1
  • Gaetano Borriello
    • 1
  • James Fogarty
    • 1
  1. 1.Computer Science & EngineeringUniversity of WashingtonSeattleUSA

Personalised recommendations