Surface Applications for Security Analysis

  • Judith M. Brown
  • Jeff Wilson
  • Peter Simonyi
  • Miran Mirza
  • Robert BiddleEmail author


This chapter relates to human factors in computer security, and how surface technology might support security analysis. This specific domain allowed us to investigate surface application design and development in an established context, and thus learn how the real needs of the domain might best be supported. Throughout, we were fortunate to have partners in industry and government working in the domain who were able to give us advice and feedback. A number of projects were conducted over the span of our research program, each one offering findings that informed later projects. In this chapter, we provide an outline of our work, summarizing each of the main projects and their findings. We cover: (1) a literature review. (2) Ethnographic studies of firstly operators and technicians in seven operations centres, and secondly a team of ten professional analysts working in the security domain; (3) ACH Walkthrough, a collaborative web-based decision-making tool; (4) Ra, a tool that supports rollback, playback and other explorative actions when using web applications like ACH Walkthrough; and (5) Strata, a tool that allows for the annotation of web applications, enabling the work of collaborative teams.


State Object Security Analysis Confirmation Bias Large Display Interaction History 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adeyemi T Interact.js. Accessed 21 Aug 2015
  2. 2.
    Andrews C, Endert A, North C (2010) Space to think: large high-resolution displays for sensemaking. In: Proceedings of the 28th international conference on human factors in computing systems. ACM, New York, NY, USA, CHI ’10, pp 55–64. doi: 10.1145/1753326.1753336
  3. 3.
    Anslow C, Marshall S, Noble J, Biddle R (2013) Sourcevis: collaborative software visualization for co-located environments. In: 2013 first IEEE working conference on software visualization (VISSOFT). IEEE, pp 1–10Google Scholar
  4. 4.
    Becker RA, Cleveland WS (1987) Brushing scatterplots. Technometrics 29(2):127–142.
  5. 5.
    Blackwell A, Green T (2003) Notational systems–the cognitive dimensions of notations framework. HCI models, theories, and frameworks: toward an interdisciplinary scienceGoogle Scholar
  6. 6.
    Blackwell AF, Britton C, Cox AL, Green TRG, Gurr CA, Kadoda GF, Kutar M, Loomes M, Nehaniv CL, Petre M, Roast C, Roe C, Wong A, Young RM (2001) Cognitive dimensions of notations: design tools for cognitive technology. In: 4th international conference on cognitive technology: instruments of mind. Springer, London, UK, UK, CT ’01, pp 325–341.
  7. 7.
    Bostock M Parallel coordinates. Accessed 21 Aug 2015
  8. 8.
    Bostock M, Ogievetsky V, Heer J (2011) D\(^3\) data-driven documents. IEEE Trans Vis Comput Graph 17(12):2301–2309CrossRefGoogle Scholar
  9. 9.
    Brown J, Wilson J, Gossage S, Hack C, Biddle R (2013a) Surface computing and collaborative analysis work. No. 19 in Synthesis lectures on human-centered informatics. Morgan & Claypool. doi: 10.2200/S00492ED1V01Y201303HCI019
  10. 10.
    Brown JM, Wilson J, Gossage S, Hack C, Biddle R (2013b) Surface computing and collaborative analysis work. Synthesis lectures on human-centered informatics. Morgan & ClaypoolGoogle Scholar
  11. 11.
    Brown JM, Wilson J, Biddle R (2014) A study of an intelligence analysis team and their collaborative artifacts. School of Computer Science Technical Report TR-14-04. Carleton UniversityGoogle Scholar
  12. 12.
    Brown JM, Greenspan S, Biddle R (2016) Incident response teams in it operations centers: the t-tocs model of team functionality. Cognit Technol Work 1–22Google Scholar
  13. 13.
    Burton M ACH:A free, open source tool for complex research problems. Accessed 03 Sept 2014
  14. 14.
    Buxton B (2011) Integrating activity theory for context analysis on large display. In: Human input to computer systems: theories, techniques and technology. Available online, chap 4.
  15. 15.
    Chatti MA, Sodhi T, Specht M, Klamma R, Klemke R (2006) u-annotate: an application for user-driven freeform digital ink annotation of e-learning content. In: Sixth international conference on advanced learning technologies, 2006. IEEE, pp 1039–1043Google Scholar
  16. 16.
    Conti G (2007) Security data visualization: graphical techniques for network analysis. No Starch PressGoogle Scholar
  17. 17.
    Denoue L, Vignollet L (2002) Annotations in the wild. In: ECAI 2002 workshop on semantic authoring, annotation and knowledge markupGoogle Scholar
  18. 18.
    Ecma International Draft specification for (ecma-262 edition 6). Accessed 17 March 2015
  19. 19.
    Engeström Y (1992) Interactive expertise: studies in distributed working intelligence. In: University of Helsinki research report, no. Research bulletin 83 in HELDA—The Digital Repository of University of Helsinki.
  20. 20.
    Engeström Y (2000) Activity theory as a framework for analyzing and redesigning work. Ergonomics 43(7):960–974. ISI:000088268400012Google Scholar
  21. 21.
    Engeström Y (2008) From teams to knots: activity-theoretical studies of collaboration and learning at work. Cambridge University Press, Cambridge, UK; New York:, chap Teamwork between adversaries: coordination, cooperation, and communication in a court trial.,,
  22. 22.
    Farah H, Lethbridge TC (2007) Temporal exploration of software models: a tool feature to enhance software understanding. In: Proceedings of the 14th working conference on reverse engineering. IEEE Computer Society, WCRE ’07, pp 41–49. doi: 10.1109/WCRE.2007.49
  23. 23.
    Genest A, Gutwin C (2011) Characterizing deixis over surfaces to improve remote embodiments. ECSCW 2011: Proceedings of the 12th European conference on computer supported cooperative work, 24–28 September 2011. Aarhus Denmark, Springer, London, pp 253–272Google Scholar
  24. 24.
    Globalytica Globalytica software tools: TemACH. Accessed 03 Spt 2014
  25. 25.
    Hackman JR (2011) Collaborative intelligence: Using teams to solve hard problems. Berrett-Koehler PublishersGoogle Scholar
  26. 26.
    Haller M, Leitner J, Seifried T, Wallace JR, Scott SD, Richter C, Brandl P, Gokcezade A, Hunter S (2010) The nice discussion room: integrating paper and digital media to support co-located group meetings. In: Proceedings of the 28th international conference on Human factors in computing systems. ACM, New York, NY, USA, CHI ’10, pp 609–618. doi: 10.1145/1753326.1753418
  27. 27.
    Han JY (2005) Low-cost multi-touch sensing through frustrated total internal reflection. In: Proceedings of the 18th annual ACM symposium on user interface software and technology. ACM, New York, NY, USA, UIST ’05, pp 115–118. doi: 10.1145/1095034.1095054
  28. 28.
    Heer J, Bostock M, Ogievetsky V (2010) A tour through the visualization zoo. Commun ACM 53(6):59–67CrossRefGoogle Scholar
  29. 29.
    Heuer RJ (1999) Psychology of intelligence analysis. Center for the Study of IntelligenceGoogle Scholar
  30. 30.
    Heuer Jr RJ, Pherson R (2010) Structured analytic techniques for intelligence analysis. CQ Press.
  31. 31.
    Heuer Jr RJ, Pherson RH (2010) Structured analytic techniques for intelligence analysis. CQ PressGoogle Scholar
  32. 32.
    Hypothesis Accessed 21 Aug 2015
  33. 33.
    Inselberg A (1997) Multidimensional detective. Information Visualization, 1997. Proceedings, IEEE Symposium on, IEEE, pp 100–107Google Scholar
  34. 34.
    Inselberg A, Dimsdale B (1990) Parallel coordinates: a tool for visualizing multi-dimensional geometry. In: Proceedings of the 1st conference on visualization ’90. IEEE Computer Society Press, Los Alamitos, CA, USA, VIS ’90, pp 361–378.
  35. 35.
    Isenberg P, Fisher D, Morris M, Inkpen K, Czerwinski M (2010) An exploratory study of co-located collaborative visual analytics around a tabletop display. In: 2010 IEEE symposium on visual analytics science and technology (VAST), pp 179–186, doi: 10.1109/VAST.2010.5652880
  36. 36.
    Isenberg P, Fisher D, Paul SA, Morris MR, Inkpen K, Czerwinski M (2012) Co-located collaborative visual analytics around a tabletop display. IEEE Trans Vis Comput Graph 18:689–702. doi: 10.1109/TVCG.2011.287
  37. 37.
    Jacob RJ, Girouard A, Hirshfield LM, Horn MS, Shaer O, Solovey ET, Zigelbaum J (2008) Reality-based interaction: a framework for post-wimp interfaces. In: Proceedings of the twenty-sixth annual SIGCHI conference on human factors in computing systems. ACM, New York, NY, CHI ’08, pp 201–210. doi: 10.1145/1357054.1357089
  38. 38.
    Kammer D, Wojdziak J, Keck M, Groh R, Taranko S (2010) Towards a formalization of multi-touch gestures. In: ACM international conference on interactive tabletops and surfaces. ACM, New York, NY, USA, ITS ’10, pp 49–58. doi: 10.1145/1936652.1936662
  39. 39.
    Kirsh D, Maglio P (1994) On distinguishing epistemic from pragmatic action. Cognit Sci 18(4):513–549. doi: 10.1207/s15516709cog1804_1
  40. 40.
    Krasner GE, Pope ST et al (1988) A description of the model-view-controller user interface paradigm in the smalltalk-80 system. J Object Oriented Program 1(3):26–49Google Scholar
  41. 41.
    Lee B, Isenberg P, Riche N, Carpendale S (2012) Beyond mouse and keyboard: expanding design considerations for information visualization interactions. IEEE Trans Vis Comput Graph 18(12):2689–2698. doi: 10.1109/TVCG.2012.204 CrossRefGoogle Scholar
  42. 42.
    Marquardt N, Kiemer J, Ledo D, Boring S, Greenberg S (2011) Designing user-, hand-, and handpart-aware tabletop interactions with the touchid toolkit. In: Proceedings of the ACM international conference on interactive tabletops and surfaces. ACM, New York, NY, USA, ITS ’11, pp 21–30. doi: 10.1145/2076354.2076358
  43. 43.
    Microsoft (2016) Microsoft surface hub.
  44. 44.
    Mozilla Developer Network ECMAScript 6 support in Mozilla. Accessed 17 March 2015
  45. 45.
    Mozilla Developer Network Proxy. Accessed 17 March 2015
  46. 46.
    Multiple Contributors (2011) Enron email dataset.
  47. 47.
    Noble J, Groves L, Biddle R (1995) Object oriented program visualisation in tarraingim. Austral Comput J 27(4):138–149Google Scholar
  48. 48.
    Owens S (2016) The total economic impact of microsoft surface hub: cost savings and business benefits enabled by surface hub. Forrester Research, IncGoogle Scholar
  49. 49.
    Palo Alto Research Center (2010) Analysis of competing hypotheses software version ach2.0.5.
  50. 50.
    Pirolli P, Card S (2005) The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. Proc Int Conf Intell Anal 5:2–4Google Scholar
  51. 51.
    Plimmer B, Chang SHH, Doshi M, Laycock L, Seneviratne N (2010) iannotate: exploring multi-user ink annotation in web browsers. In: Proceedings of the eleventh australasian conference on user interface, vol 106. Australian Computer Society, Inc., Darlinghurst, Australia, Australia, AUIC ’10, pp 52–60.
  52. 52.
    Russell DM, Stefik MJ, Pirolli P, Card SK (1993) The cost structure of sensemaking. In: Proceedings of the INTERACT’93 and CHI’93 conference on Human factors in computing systems. ACM, pp 269–276Google Scholar
  53. 53.
    Scaife M, Rogers Y (1996) External cognition: how do graphical representations work? Int J Hum Comput Stud 45(2):185–213CrossRefGoogle Scholar
  54. 54.
    Shneiderman B (1981) Direct manipulation: a step beyond programming languages. SIGSOC Bull 13(2–3):143. doi: 10.1145/1015579.810991
  55. 55.
    Simon HA (1956) Rational choice and the structure of the environment. Psychol Rev 63(2):129CrossRefGoogle Scholar
  56. 56.
    Song P, Goh WB, Fu CW, Meng Q, Heng PA (2011) Wysiwyf: exploring and annotating volume data with a tangible handheld device. In: Proceedings of the 2011 annual conference on Human factors in computing systems. ACM, New York, NY, USA, CHI ’11, pp 1333–1342. doi: 10.1145/1978942.1979140
  57. 57.
    Spindler M, Martsch M, Dachselt R (2012) Going beyond the surface: studying multi-layer interaction above the tabletop. In: Proceedings of the 2012 ACM annual conference on human factors in computing systems. ACM, New York, NY, CHI ’12, pp 1277–1286. doi: 10.1145/2208516.2208583
  58. 58.
    Suchman L (1994) Plans and situated action: the problem of human-machine communication, 4th edn. Cambridge University PressGoogle Scholar
  59. 59.
    Thomas JJ, Cook KA (2005) Illuminating the path. IEEE Computer Society PressGoogle Scholar
  60. 60.
    Tuddenham P, Davies I, Robinson P (2009) Websurface: an interface for co-located collaborative information gathering. In: Proceedings of the ACM international conference on interactive tabletops and surfaces. ACM, New York, NY, USA, ITS ’09, pp 181–188. doi: 10.1145/1731903.1731938
  61. 61.
    Wallace JR, Scott SD, Stutz T, Enns T, Inkpen K (2009) Investigating teamwork and taskwork in single- and multi-display groupware systems. Personal Ubiquitous Comput 13:569–581CrossRefGoogle Scholar
  62. 62.
    Wallace JR, Scott SD, Lai E, Jajalla D (2011) Investigating the role of a large, shared display in multi-display environments. Comput Support Cooper Work 20:529–561CrossRefGoogle Scholar
  63. 63.
    Wharton C, Bradford J, Jeffries R, Franzke M (1992) Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations. In: ACM conference on human factors in computing systems (CHI)Google Scholar
  64. 64.
    Wigdor D, Wixon D (2011) Brave NUI world: designing natural user interfaces for touch and gesture, 1st edn. Morgan Kaufmann Publishers, San Francisco, CAGoogle Scholar
  65. 65.
    Wilson J, Brown JM, Biddle R (2014) Interactive parallel coordinates for collaborative intelligence analysis. School of Computer Science Technical Report TR-14-05, Carleton UniGoogle Scholar
  66. 66.
    Wobbrock JO, Morris MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the 27th international conference on human factors in computing systems. ACM, New York, NY, USA, CHI ’09, pp 1083–1092. doi: 10.1145/1518701.1518866
  67. 67.
    Yuill N, Rogers Y (2012) Mechanisms for collaboration: a design and evaluation framework for multi-user interfaces. Trans Hum Comput Inter (TOCHI)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Judith M. Brown
    • 1
  • Jeff Wilson
    • 1
  • Peter Simonyi
    • 1
  • Miran Mirza
    • 1
  • Robert Biddle
    • 1
    Email author
  1. 1.School of Computer Science, Carleton UniversityOttawaCanada

Personalised recommendations