Sketched Graph Drawing: A Lesson in Empirical Studies
This paper reports on a series of three similar graph drawing empirical studies, and describes the results of investigating subtle variations on the experimental method. Its purpose is two-fold: to report the results of the experiments, as well as to illustrate how easy it is to inadvertently make conclusions that may not stand up to scrutiny. While the results of the initial experiment were validated, instances of speculative conclusions and inherent bias were identified. This research highlights the importance of stating the limitations of any experiment, being clear about conclusions that are speculative, and not assuming that (even minor) experimental decisions will not affect the results.
Keywordsgraph sketching empirical studies replication limitations
- 2.Kemmis, S., McTaggart, R.: The Action Research Planner, 3rd edn. Deakin University Press (1988)Google Scholar
- 3.Huang, W.: Using eye-tracking to investigate graph layout effects. In: Hong, S.H., Ma, K.L. (eds.) Proc. Asia Pacific Symp. on Visualisation, pp. 97–100. IEEE (2007)Google Scholar
- 5.Purchase, H.C., Cohen, R.F., James, M.: An experimental study of the basis for graph drawing algorithms. ACM J. Experimental Algorithmics 2(4), 1–17 (1997)Google Scholar
- 9.Plimmer, B., Purchase, H.C., Yang, H.Y.: SketchNode: Intelligent sketching support and formal diagramming. In: Brereton, M., Viller, S., Kraal, B. (eds.) OzChi Conference, Brisbane, Australia, pp. 136–143 (2010)Google Scholar