Skip to main content
Book cover

International Joint Conference on Computer Vision, Imaging and Computer Graphics

VISIGRAPP 2016: Computer Vision, Imaging and Computer Graphics Theory and Applications pp 242–263Cite as

Visual Querying of Semantically Enriched Movement Data

  • 1052 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 693)

Abstract

Visual data exploration is used to reveal unknown patterns that, however, need to be validated, refined, and extracted for a final presentation and reporting. We contribute VESPa, a pattern-based visual query language for event sequences. With VESPa, analysts can formulate hypotheses gained and query the data for matches. In an interative analysis loop the pattern can be altered with further restrictions to narrow down the result set. Our language allows for (1) hypothesis expression and refinement, (2) visual querying, and (3) knowledge externalization. We focus on semantically enrichend movement data, used in law enforcement, consumer, and traffic analysis. To evaluate the applicability we present two case studies as well as a user study consisting of comprehensive and composition tasks.

Keywords

  • Visual query language
  • Semantic movement analysis

This publication is an extended version of our IVAPP 2016 paper [1]

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-64870-5_12
  • Chapter length: 22 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-319-64870-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.
Fig. 12.
Fig. 13.

Notes

  1. 1.

    For this paper, colors were chosen so as to be distinguishable both on colored and greyscale printouts. Conceptually, alternative color schemes can be chosen that are specific to user requirements, for instance, color blindness.

References

  1. Haag, F., Krüger, R., Ertl, T.: VESPa: a pattern-based visual query language for event sequences. In: Proceedings of the 7th International Conference on Information Visualization Theory and Applications (IVAp. 2016), vol. 7 (2016)

    Google Scholar 

  2. Wongsuphasawat, K., Plaisant, C., Taieb-Maimon, M., Shneiderman, B.: Querying event sequences by exact match or similarity search: design and empirical evaluation. Interact. Comput. 24, 55–68 (2012)

    CrossRef  Google Scholar 

  3. Zgraggen, E., Drucker, S.M., Fisher, D., DeLine, R.: (s|qu)eries: Visual regular expressions for querying and exploring event sequences. In: Proceedings of CHI 2015, pp. 2683–2692. ACM (2015)

    Google Scholar 

  4. Pirolli, P., Card, S.: The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of International Conference on Intelligence Analysis, MITRE, pp. 2–4 (2005)

    Google Scholar 

  5. Makanju, A., Zincir-Heywood, A.N., Milios, E.E.: Storage and retrieval of system log events using a structured schema based on message type transformation. In: Proceedings of SAC 2011, pp. 528–533. ACM (2011)

    Google Scholar 

  6. Gaaloul, W., Bhiri, S., Godart, C.: Discovering workflow transactional behavior from event-based log. In: Meersman, R., Tari, Z. (eds.) OTM 2004. LNCS, vol. 3290, pp. 3–18. Springer, Heidelberg (2004). doi:10.1007/978-3-540-30468-5_3

    CrossRef  Google Scholar 

  7. Abela, J., Debeaupuis, T., Consultants, H.S.: Universal format for logger messages (1999). http://tools.ietf.org/html/draft-abela-ulm-05

  8. Do, Q.X., Lu, W., Roth, D.: Joint inference for event timeline construction. In: Proceedings of EMNLP-CoNLL 2012, pp. 677–687. ACL (2012)

    Google Scholar 

  9. Atrey, P., Maddage, M., Kankanhalli, M.: Audio based event detection for multimedia surveillance. In: Proceedings of ICASSP 2006, vol. 5, pp. 813–816. IEEE (2006)

    Google Scholar 

  10. Heydekorn, J., Nitsche, M., Dachselt, R., Nürnberger, A.: On the interactive visualization of a logistics scenario: requirements and possible solutions. In: Proceedings of IWDE 2011, pp. 1–7. Technical report (Internet): Elektronische Zeitschriftenreihe der Fakultät für Informatik der OVGU Magdeburg (2011)

    Google Scholar 

  11. Kim, P.H., Giunchiglia, F.: Life logging practice for human behavior modeling. In: Proceedings of SMC 2012, pp. 2873–2878 (2012)

    Google Scholar 

  12. Atrey, P.K., Kankanhalli, M.S., Jain, R.: Timeline-based information assimilation in multimedia surveillance and monitoring systems. In: Proceedings of VSSN 2005, pp. 103–112. ACM (2005)

    Google Scholar 

  13. Peuquet, D.J., Duan, N.: An event-based spatiotemporal data model (ESTDM) for temporal analysis of geographical data. Int. J. Geogr. Inf. Syst. 9, 7–24 (1995)

    CrossRef  Google Scholar 

  14. Huang, Y., Zhang, L., Zhang, P.: A framework for mining sequential patterns from spatio-temporal event data sets. IEEE Trans. Knowl. Data Eng. 20, 433–448 (2008)

    CrossRef  Google Scholar 

  15. Jiang, F., Yuan, J., Tsaftaris, S.A., Katsaggelos, A.K.: Anomalous video event detection using spatiotemporal context. Comput. Vision Image Underst. 115, 323–333 (2011)

    CrossRef  Google Scholar 

  16. Plaisant, C., Milash, B., Rose, A., Widoff, S., Shneiderman, B.: LifeLines: visualizing personal histories. In: Proceedings of CHI 1996, pp. 221–227. ACM (1996)

    Google Scholar 

  17. Kumar, V., Furuta, R., Allen, R.B.: Metadata visualization for digital libraries: interactive timeline editing and review. In: Proceedings of DL 1998, pp. 126–133. ACM (1998)

    Google Scholar 

  18. Tao, C., Wongsuphasawat, K., Clark, K., Plaisant, C., Shneiderman, B., Chute, C.G.: Towards event sequence representation, reasoning and visualization for EHR data. In: Proceedings of IHI 2012, pp. 801–806. ACM (2012)

    Google Scholar 

  19. Krstajić, M., Bertini, E., Keim, D.: CloudLines: compact display of event episodes in multiple time-series. IEEE TVCG 17, 2432–2439 (2011)

    Google Scholar 

  20. Fischer, F., Mansmann, F., Keim, D.A.: Real-time visual analytics for event data streams. In: Proceedings of SAC 2012, pp. 801–806. ACM (2012)

    Google Scholar 

  21. Havre, S., Hetzler, B., Nowell, L.: ThemeRiver: visualizing theme changes over time. In: Proceedings of InfoVis 2000, pp. 115–123. IEEE (2000)

    Google Scholar 

  22. Guo, X., Li, J., Yang, R., Ma, X.: NEI: a framework for dynamic news event exploration and visualization. In: Proceedings of VINCI 2014, pp. 121–128. ACM (2014)

    Google Scholar 

  23. Marcus, A., Bernstein, M.S., Badar, O., Karger, D.R., Madden, S., Miller, R.C.: Twitinfo: aggregating and visualizing microblogs for event exploration. In: Proceedings of CHI 2011, pp. 227–236. ACM (2011)

    Google Scholar 

  24. Kapler, T., Wright, W.: GeoTime information visualization. Inf. Vis. 4, 136–146 (2005)

    CrossRef  Google Scholar 

  25. Tominski, C., Schumann, H., Andrienko, G., Andrienko, N.: Stacking-based visualization of trajectory attribute data. IEEE TVCG 18, 2565–2574 (2012)

    Google Scholar 

  26. Guo, H., Wang, Z., Yu, B., Zhao, H., Yuan, X.: TripVista: triple perspective visual trajectory analytics and its application on microscopic traffic data at a road intersection. In: Proceedings of PacificVis 2011, pp. 163–170. IEEE (2011)

    Google Scholar 

  27. Sun, G., Liu, Y., Wu, W., Liang, R., Qu, H.: Embedding temporal display into maps for occlusion-free visualization of spatio-temporal data. In: Proceedings of PacificVis 2014, pp. 185–192. IEEE (2014)

    Google Scholar 

  28. Parent, C., Spaccapietra, S., Renso, C., Andrienko, G., Andrienko, N., Bogorny, V., Damiani, M.L., Gkoulalas-Divanis, A., Macedo, J., Pelekis, N., Theodoridis, Y., Yan, Z.: Semantic trajectories modeling and analysis. ACM Comput. Surv. 45, 42:1–42:32 (2013)

    CrossRef  Google Scholar 

  29. Krüger, R., Thom, D., Ertl, T.: Visual analysis of movement behavior using web data for context enrichment. In: Proceedings of PacificVis 2014, pp. 193–200. IEEE (2014)

    Google Scholar 

  30. Zhu, X.Y., Guo, W., Huang, L., Hu, T., Gao, W.X.: Pan-information location map. ISPRS Archives XL–4, 57–62 (2013)

    Google Scholar 

  31. Nguyen, T., Loke, S., Torabi, T.: The community stack: concept and prototype. In: Proceedings of AINAW 2007, vol. 2, pp. 52–58 (2007)

    Google Scholar 

  32. Westermann, U., Jain, R.: Toward a common event model for multimedia applications. IEEE Multimedia 14, 19–29 (2007)

    CrossRef  Google Scholar 

  33. Andrienko, N., Andrienko, G., Fuchs, G.: Towards privacy-preserving semantic mobility analysis. In: EuroVis Workshop on Visual Analytics, pp. 19–23. Eurographics Association (2013)

    Google Scholar 

  34. Shneiderman, B.: Dynamic queries for visual information seeking. IEEE Softw. 11, 70–77 (1994)

    CrossRef  Google Scholar 

  35. Seifert, I.: A pool of queries: interactive multidimensional query visualization for information seeking in digital libraries. Inf. Vis. 10, 97–106 (2011)

    CrossRef  Google Scholar 

  36. Soylu, A., Giese, M., Jimenez-Ruiz, E., Kharlamov, E., Zheleznyakov, D., Horrocks, I.: OptiqueVQS: towards an ontology-based visual query system for big data. In: Proceedings of MEDES 2013, pp. 119–126. ACM (2013)

    Google Scholar 

  37. Russell, A., Smart, P., Braines, D., Shadbolt, N.: NITELIGHT: a graphical tool for semantic query construction. In: Proceedings of SWUI 2008, vol. 543 of CEUR-WS (2008)

    Google Scholar 

  38. Morris, A., Abdelmoty, A., El-Geresy, B., Jones, C.: A filter flow visual querying language and interface for spatial databases. GeoInformatica 8, 107–141 (2004)

    CrossRef  Google Scholar 

  39. Wu, S., Otmane, S., Moreau, G., Servières, M.: Design of a visual query language for geographic information system on a touch screen. In: Kurosu, M. (ed.) HCI 2013. LNCS, vol. 8007, pp. 530–539. Springer, Heidelberg (2013). doi:10.1007/978-3-642-39330-3_57

    CrossRef  Google Scholar 

  40. Kumar, C., Heuten, W., Boll, S.: Geographical queries beyond conventional boundaries: regional search and exploration. In: Proceedings of GIR 2013, pp. 84–85. ACM (2013)

    Google Scholar 

  41. Boyandin, I., Bertini, E., Bak, P., Lalanne, D.: Flowstrates: an approach for visual exploration of temporal origin-destination data. Comput. Graph. Forum 30, 971–980 (2011)

    CrossRef  Google Scholar 

  42. Certo, L., Galvão, T., Borges, J.: Time automaton: a visual mechanism for temporal querying. J. Visual Lang. Comput. 24, 24–36 (2013)

    CrossRef  Google Scholar 

  43. Krüger, R., Thom, D., Wörner, M., Bosch, H., Ertl, T.: TrajectoryLenses - a set-based filtering and exploration technique for long-term trajectory data. Comput. Graph. Forum 2013, 451–460 (2013)

    CrossRef  Google Scholar 

  44. Bonhomme, C., Trépied, C., Aufaure, M.A., Laurini, R.: A visual language for querying spatio-temporal databases. In: Proceedings of GIS 1999, pp. 34–39. ACM (1999)

    Google Scholar 

  45. D’Ulizia, A., Ferri, F., Grifoni, P.: Moving GeoPQL: a pictorial language towards spatio-temporal queries. GeoInformatica 16, 357–389 (2012)

    CrossRef  Google Scholar 

  46. Monroe, M., Lan, R., Morales del Olmo, J., Shneiderman, B., Plaisant, C., Millstein, J.: The challenges of specifying intervals and absences in temporal queries: a graphical language approach. In: Proceedings of CHI 2013, pp. 2349–2358. ACM (2013)

    Google Scholar 

  47. Gotz, D., Stavropoulos, H.: DecisionFlow: visual analytics for high-dimensional temporal event sequence data. IEEE TVCG 20, 1783–1792 (2014)

    Google Scholar 

  48. Fails, J., Karlson, A., Shahamat, L., Shneiderman, B.: A visual interface for multivariate temporal data: finding patterns of events across multiple histories. In: VAST 2006, pp. 167–174 (2006)

    Google Scholar 

  49. Dionisio, J.D., Cárdenas, A.F.: MQuery: a visual query language for multimedia, timeline and simulation data. J. Visual Lang. Comput. 7, 377–401 (1996)

    CrossRef  Google Scholar 

  50. Jin, J., Szekely, P.: QueryMarvel: a visual query language for temporal patterns using comic strips. In: Proc. VL/HCC 2009, pp. 207–214 (2009)

    Google Scholar 

  51. Fegeras, L.: VOODOO: a visual object-oriented database language for ODMG OQL. In: W13. The First ECOOP Workshop on Object-Oriented Databases (1999)

    Google Scholar 

  52. Visual Analytics Community: VAST 2014 Challenge - the Kronos incident (2014). http://vacommunity.org/VAST+Challenge+2014

  53. Krüger, R., Herr, D., Haag, F., Ertl, T.: Inspector gadget: integrating data preprocessing and orchestration in the visual analysis loop. In: EuroVis Workshop on Visual Analytics (EuroVA). The Eurographics Association (2015)

    Google Scholar 

  54. Bracciale, L., Bonola, M., Loreti, P., Bianchi, G., Amici, R., Rabuffi, A.: CRAWDAD data set roma/taxi (v. 2014–07-17) (2014). http://crawdad.org/roma/taxi/

Download references

Acknowledgements

This work was supported by the Horizon 2020 project CIMPLEX, grant no. 641191.

Author information

Authors and Affiliations

  1. Institute for Visualization and Interactive Systems, University of Stuttgart, Universitätsstraße 38, 70569, Stuttgart, Germany

    Florian Haag, Robert Krüger & Thomas Ertl

Authors
  1. Florian Haag
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert Krüger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Ertl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Florian Haag .

Editor information

Editors and Affiliations

  1. Escola Superior de Tecnologia do IPS, Setúbal, Portugal

    José Braz

  2. MiraLab, University of Geneva, Carouge, Switzerland

    Nadia Magnenat-Thalmann

  3. LISA - ISTIA, University of Angers, Angers, France

    Paul Richard

  4. Department of Computer Science and Electrical Engineering, Jacobs University, Bremen, Germany

    Lars Linsen

  5. University of Groningen, Groningen, The Netherlands

    Alexandru Telea

  6. Università di Catania, Catania, Italy

    Sebastiano Battiato

  7. Research Innovation Center, Canon U.S.A. Inc., San Jose, California, USA

    Francisco Imai

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Haag, F., Krüger, R., Ertl, T. (2017). Visual Querying of Semantically Enriched Movement Data. In: , et al. Computer Vision, Imaging and Computer Graphics Theory and Applications. VISIGRAPP 2016. Communications in Computer and Information Science, vol 693. Springer, Cham. https://doi.org/10.1007/978-3-319-64870-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64870-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64869-9

  • Online ISBN: 978-3-319-64870-5

  • eBook Packages: Computer ScienceComputer Science (R0)