A Software Tool for the Experimental Investigation of Cognitive Effects in Audiovisual Maps
- 384 Downloads
The increasing importance of cognitively oriented visualization in cartography implies the use of modern approaches of experimental user studies. Such experiments are usually standardized, and researchers try to avoid confounding factors as much as possible. Therefore, modern experimental studies are based on digital, software-based methods and techniques to fulfil the demands. A set of established standard software exists which can partly be adopted to individual research questions in cartography and spatial cognition. However, these tools have not primarily been developed for the field of cartography and, thus, cartographic peculiarities are missing. For example, this can be related to the implementation of multimedia, incl. auditory (map) stimuli to investigate the impact of specific sound elements on cognitive processing in audiovisual cartography. This may also refer to the way (behavioural) user data measured during spatial tasks in maps are presented to the analyst. To make use of a more cartography-focused study software which can be used in map-based studies on spatial memory performance, an individual software tool was programmed. The tool is based on the object-oriented programming language ActionScript 3.0 and allows an easy customization due to its simplicity. The tool can be exported in the runtime system Adobe AIR and is highly suitable for lab studies. This paper presents some functionalities and peculiarities of this tool, including several examples of the written and commented scripting code.
KeywordsAudiovisual cartography Multimedia cartography Empirical cartography User studies Sound Study software
Ein Software-Tool zur Untersuchung kognitiver Effekte audiovisueller Bildschirmkarten
Die zunehmende Bedeutung von kognitiv-gesteuerten Visualisierungen in der Kartographie impliziert die Verwendung moderner Ansätze experimenteller Nutzerstudien. Solche Experimente sind für gewöhnlich standardisiert und es wird versucht, Stör- und Verzerrungsfaktoren so gut wie möglich zu minimieren. Deswegen basieren moderne experimentelle Studien auf digitalen, Software-gestützten Methoden und Techniken, um diese Anforderungen zu erfüllen. Derzeit existiert eine Reihe etablierter Standardprogramme, die jedoch nur teilweise für individuelle Forschungsfragen zur Kartographie und Raumkognitionsforschung geeignet sind. Diese Tools wurden nicht primär für die kartographische Forschung konzipiert, weswegen manche kartographischen Spezifika unberücksichtigt bleiben. Dies bezieht sich bspw. auf die Implementierung multimedialer, und somit auch auditiver Elemente, um den Einfluss von Ton auf die kognitive Verarbeitung im räumlichen Gedächtnis untersuchen zu können. Zudem bezieht es sich auf die Präsentation der erfassten raumbezogenen Verhaltensdaten. Um ein etwas mehr auf Kartographie ausgerichtetes Tool nutzen zu können, wurde eine Spezialanwendung entwickelt. Dieses Tool basiert auf der objektorientierten Programmiersprache ActionScript 3.0. Es ermöglicht wegen seiner Einfachheit eine einfache Anpassung. Es kann als Adobe AIR-File exportiert werden. Diese Laufzeitumgebung ist unabhängig von einer Web-Verbindung und daher sehr geeignet für Laborstudien. In diesem Aufsatz werden einige Funktionalitäten und Besonderheiten dieses Tools vorgestellt, einschließlich mehrerer Beispiele des geschriebenen und kommentierten Skriptcodes.
SchlüsselwörterAudiovisuelle Kartographie Multimediakartographie Empirische Kartographie Nutzerstudien Ton Testsoftware
In cartography and spatial cognition, researchers regularly stress the steadily increasing importance of understanding cognitive structures in map use (e.g. Andrienko et al. 2014; Dickmann 2018; Griffin 2017; Montello et al. 2018; Roth et al. 2017; Touya et al. 2018). If cartographers are familiar with cognitive structures during the user’s processing (e.g. reading and learning) of map content, new cartographic design rules could be considered to improve the efficiency of cartographic communication in (multimedia) maps (Bestgen et al. 2013, Dickmann et al. 2015, 2013).
New proposals of cartographic design rules for ‘better communicating’ maps go back to the outcomes of user studies (e.g. Çöltekin et al. 2017, 2018; Edler et al. 2018a, b; Keil et al. 2018, 2019; Kuchinke et al. 2016). Such user studies are based on study designs that try to minimize interfering factors as much as possible. Following the standards of experimental psychology, the behavioural data of the study participants should only be influenced by the factors and conditions declared and examined (e.g. Myers and Hansen 2014).
To collect-usually quantitative-behavioural data, such as distance estimations or the distance error of recalled or recognized object locations (Bestgen et al. 2017a, b; Okabayashi and Glynn 1984) in cartographic user studies, research groups either use existing ‘standard software’, for e.g. Tobii Studio or OGAMA for eye-tracking analyses (Voßkühler et al. 2008), or develop their own customized software solutions (Edler et al. 2014, 2015a; Popelka et al. 2016; Sasinka et al. 2017). These customized solutions are adjusted to the individual study purpose. However, the inclusion of diversified multimedia elements in such lab software is still rare. Although the relevance of investigating auditory stimuli is regularly emphasized in user-oriented cartography (e.g. Ballatore et al. 2018; Bearman and Lovett 2010; Dodt et al. 2017; Edler et al. 2012, 2018c; Lammert-Siepmann et al. 2011, 2014, 2017; Papadimitriou et al. 2009; Schiewe 2015; Schito and Fabrikant 2018; Reagan and Baldwin 2006), it is no common standard to include auditory stimuli as data elements in study software.
To gather behavioural data on quantitative measures of the performance in memory of object-locations, a software tool for lab studies was developed in the Geography Department at the Ruhr-University Bochum (RUB). The tool is based on ActionScript 3.0, and updated versions of the tool are exported in the cross-platform runtime system (Adobe AIR, formerly Adobe Integrated Runtime). The scripted tool was recently extended to measure the impact of acoustic map elements on spatial memory performance (Lammert-Siepmann et al. 2017).
Related versions of this audio-featured ActionScript-based tool have been used in several studies dealing with cognitive structures in map use. Identified effects, such as the grid effect on map-based object-location memory performance (Edler et al. 2014), have repeatedly been confirmed based on a related recall memory paradigm and a similar (ActionScript-based) software tool (Edler et al. 2015b, 2018b; Dickmann et al. 2017).
This paper is intended to present and discuss some main functionalities of this tool which supports the automatized execution of a recall memory paradigm. The presentation of the tool is based on case examples of written and ‘commented’ scripting code (in the object-oriented programming language ActionScript 3.0). These code-based examples are focussed on important aspects of experimental design. Before these code-examples are presented, the procedure of study design which represents the conceptional background of the tool is briefly introduced.
2 A Recall Memory Paradigm Including Audio Elements
The methodological paradigm used in this study is based on a standard paradigm of experimental and cognitive psychology. It is used to investigate recall performance in memory (Baddeley 1997; Mandler et al. 1969). The method presented in the following is an adaptation of the recall memory paradigm to measure the performance in memory of object locations (see also Postma and De Haan 1996). It includes visual and auditory stimuli linked with topographic maps. The audio files are recordings of spoken place names. They were recorded with a music studio equipment.
The (visual) study materials include digital topographic base maps-an adjustable number, but six in this case. These maps are based on open (ATKIS) data (.shp-files) provided by “Geobasis NRW” (https://open.nrw/). These open data allow to (re-)build topographic maps of North Rhine-Westphalia (NRW), Germany, in the currently defined graphical style. The size (30 cm × 20 cm; in pixels: 1065 × 710), format (landscape) and scale of all maps (1/10,000) were identical in this example. The digital maps are designed for being displayed on a TFT-LCD 24″ screen. They are created and exported with ESRI ArcGIS (v. 10.5). The screen was calibrated to mirror the official colours.
Another graphical layer included six circular (point) symbols which are added to each topographic base map. These points (size: d = 0.5 cm, fill color: #e100c8) marked fictional locations within the represented topography. Each point is given a fictional (German) place name consisting of two syllables and eight characters. Both, location (spatial information) and name (attribute information), are randomly selected for each map. The selection is based on a pool of 40 possible locational options, combined with 48 possible place names. Each place name can only appear in one map.
In the study maps, the place names are presented either purely graphically (written text) or audiovisually (written text, plus played voice recording). The six maps are randomly subdivided into two groups comprising three maps each. The two groups refer to the different conditions of communicating the place names (visual vs. audiovisual). The theoretical background to the study leading to these two study conditions is documented in Lammert-Siepmann et al. (2017).
The procedure of the recall memory study consists of three major phases: (1) memory encoding, (2) filler task, (3) memory recall/retrieval. Memory encoding (60 s) refers to learning the locations of map objects and their semantic identities (place names). The names can be-depending on the study condition: visual vs. audiovisual-(dis-)played by clicking on the point symbols representing the locations. The filler task (45 s) is a visual search task which was independent from the memory-related tasks in the encoding and retrieval phases. Memory retrieval (60 s) involves the recall of encoded object locations and their corresponding semantic object identities (place names). The time values are based on a pre-test that was focused on the definition of suitable time parameters.
3 A Scripted Tool for Experimental Testing of Audiovisual Maps
The study software is based on the object-oriented language ActionScript 3.0 and has been created to run the trial sequences of the user-study automatically and in a standardized way. Using an automatized (digital) approach supports a high comparability of the test results and a minimization of individual confounding factors, such as inaccuracies that might occur if test results are gathered manually or semi-automatized. The created software consists of several modules that could be easily adapted to the requirements of different studies. In other words, its structure was planned to be adaptable without investing too much additional programming/scripting work. In the following, several code-based examples are given to highlight relevant aspects of experimental design. The code blocks are commented so that the functional ideas behind the lines of scripting code are indicated.
3.1 Randomization: Assigning Study Materials to Different Study Conditions
Randomization is a core concept used in many modern experimental studies. When repeating a study sequence several times, the sequence should be changed between the participants. This helps to avoid effects that are linked to a standardized sequence order (e.g. Barker and Milivojevich 2016). The programmed tool supports randomization at different stages. The base maps are selected in a random order and they are also randomly assigned to the two different study conditions (visual vs. audiovisual). A random-based procedure is also used for the selection of (x- and y-) coordinates of the object locations and to the allocation of the corresponding place names.
Approaches of randomization are further used for the allocation of object locations and place names to the (topographic) base maps. Technically, suitable place names for all base maps are stored in a single array, an ActionScript class designed for listing, holding and accessing data. In the initialization of the study software, each map is assigned a specified number of names from this array via the Math.random() method. Similarly, locations on the base map for these place names are chosen from an array that contains multiple previously chosen suitable pairs of coordinates for each map. This selection of locations also takes place during the initialization, managed by a function init ().
3.2 Time Management in the Trial Sequence
3.3 Tracking Behavioural Data of Study Participants
ActionScript 3.0 offers several options for the recording of user input, which is vital for gaining behavioural data in experimental user studies. For example, this refers to the tracking of recalled object locations and object identities (place names). The tracking of user data can also be related to the counting of specific interactions, such as the amount of activating the sound files to learn (encode) the located place names. This automatized analysis may allow conclusions on the comparability of place names used. If there were specific names which have been activated more often, the analyst would have a strong indication for a different level of difficulty, which would be a base for the detection of outliers and, thus, improve the quality of the statistical analysis. If no outliers are identified, the analyst would generate an additional quantity to back up the robustness of the study design and method. Tracking the amount of clicks also allows to check if each participant has played each audio files at least once throughout the encoding phase.
3.4 Visual Map-Based Summary of the Behavioural Data
They include the original (violet) and recalled (yellow) locations and identities of objects. The order of recall (red number above yellow points) and the deviations (black numbers) between original and recalled locations (representing the distortion error in spatial memory) are also included. In addition to a first visual analysis, these screenshots also serve as a tool to give a feedback to the (interested) participants after the test [see a similar approach of visually and spatially summarizing behavioural user data in Edler (2015)].
4 Summary and ‘Flash’
This paper was intended to give a compact overview of a self-scripted study tool that was created to run experimental trials including auditory map stimuli in spatial memory tasks. This tool is characterized by features that help to increase the quality of automatized user testing and the subsequent analysis of collected behavioural data. In addition to the integration of audio files as map elements, this tool includes randomization, time management and a data export based on formatted text files and visual cartographic interpretations of the data. To allow an easy customization of the software to related studies focused on spatial memory tasks, the code was commented and structured into separate units. As ActionScript 3.0 is known (e.g. Huddleston 2009; Muehlenhaus 2013; Richardson and Milbourne 2014) for its simplicity, modifications of the code can be made in relatively easy steps.
In the present literature and online contributions, ActionScript 3.0, which is linked with Adobe Flash, is sometimes no longer considered as a modern, but rather outdated (object-oriented) scripting language (e.g. Fish and Calvert 2016). Such opinions often refer to the forthcoming ‘expiry date’ of the Flash Player at the end of 2020 (Hurtig and Scharlach 2018). The worldwide distribution of interactive and creative contents is more and more associated with open standards, such HTML 5, WebAssembly and WebGL, which has also found its way into cartography (e.g. Crickard III 2014; Derrough 2013; Dickmann 2018). These new standards support modern options of interactivity and visualization and are highly transparent in terms of contents and the programming job.
However, advantages of ActionScript-based air-files are still obvious. A ‘closed system’, like a locally installed air-file, runs without a browser and a connection to the internet. It is not influenced by web-related factors and is easy to script and set up. Therefore, there is still great value of ActionScript-based software for lab studies, which are highly focused on standardization and a reduction of interfering factors.
- Baddeley AM (1997) Human memory. Theory and practice. Psychology Press, HoveGoogle Scholar
- Ballatore A, Gordon D, Boone AP (2018) Sonyfying data uncertainty with sound dimensions. Cartogr Geogr Inf Sci. https://doi.org/10.1080/15230406.2018.1495103
- Barker TB, Milivojevich A (2016) Quality by experimental design, 4th edn. CRC Press, Boca RatonGoogle Scholar
- Bestgen A-K, Edler D, Dickmann F, Kuchinke L (2013) Grid or no grid: distance distortion in recognizing spatial information from complex cartographic maps. In: Proceedings of CogSci 2013—35th Annual Meeting of the Cognitive Science Society, Berlin. https://mindmodeling.org/cogsci2013/papers/0062/paper0062.pdf. Accessed 10 Dec 2018
- Bestgen A-K, Edler D, Ratmer A, Dickmann F (2017b) The perception and estimation of distances in topographic maps are not distorted by grids. Kartographische Nachrichten 67(2):61–67Google Scholar
- Çöltekin A, Brychtová A, Griffin AL, Robinson AC, Imhof M, Pettit C (2017) Perceptual complexity of soil-landscape maps: a user evaluation of color organization in legend designs using eye tracking. Int J Digit Earth 10(6):560–581. https://doi.org/10.1080/17538947.2016.1234007 CrossRefGoogle Scholar
- Crickard P III (2014) Leaflet.js essentials. Packt Publishing, BirminghamGoogle Scholar
- Derrough J (2013) Instant interactive map design with leaflet java script library how-to. Packt Publishing, BirminghamGoogle Scholar
- Dickmann F (2018) Kartographie. Westermann, BraunschweigGoogle Scholar
- Dickmann F, Edler D, Bestgen A-K, Kuchinke L (2013) Spatial distortions in cognitive maps—a chance and challenge to enrich the principles of map design. Kartographische Nachrichten 63:174–181Google Scholar
- Dickmann F, Edler D, Bestgen A-K, Kuchinke L (2015) Auswertung von Heatmaps in der Blickbewegungsmessung am Beispiel einer Untersuchung zum Positionsgedächtnis. Kartographische Nachrichten 65(5):272–280Google Scholar
- Dodt J, Bestgen A-K, Edler D (2017) Ansätze der Erfassung und kartographischen Präsentation der olfaktorischen Dimension. Kartographische Nachrichten 67(5):245–256Google Scholar
- Edler D (2015) The effects of grids in topographic maps on cognitive representations of map information. PhD Thesis, Ruhr University Bochum. http://www-brs.ub.ruhr-uni-bochum.de/netahtml/HSS/Diss/EdlerDennis/diss.pdf. Accessed 10 Dec 2018
- Edler D, Lammert-Siepmann N, Dodt J (2012) Die akustische Dimension in der Kartographie—eine Übersicht. Kartographische Nachrichten 62(4):185–195Google Scholar
- Edler D, Edler S, Dickmann F (2015b) Eine empirische Studie zu Effekten von simulierter Grünblindheit (Deuteranopie) auf das kartenbasierte Positionsgedächtnis. Kartographische Nachrichten 65(4):183–194Google Scholar
- Edler D, Keil J, Tuller M-C, Bestgen A-K, Dickmann F (2018a) Searching for the ‘right’ legend: the impact of legend position on legend decoding in a cartographic memory task. Cartogr J. https://doi.org/10.1080/00087041.2018.1533293
- Edler D, Keil J, Bestgen A-K, Kuchinke L, Dickmann F (2018b) Hexagonal map grids—an experimental study on the performance in memory of object locations. Cartogr Geogr Inf Sci. https://doi.org/10.1080/15230406.2018.1496035
- Edler D, Kühne O, Jenal C, Vetter M, Dickmann F (2018c) Potenziale der Raumvisualisierung in virtual reality (VR) für die sozialkonstruktivistische Landschaftsforschung. Kartographische Nachrichten 68(5):245–254Google Scholar
- Fish CS, Calvert K (2016) Analysis of interactive solar energy web maps for urban energy sustainability. Cartogr Perspect 85:5–22Google Scholar
- Keil J, Edler D, Dickmann F, Kuchinke L (2019) Meaningfulness of landmark pictograms reduces visual salience and recognition performance. Appl Ergon. https://doi.org/10.1016/j.apergo.2018.10.008
- Lammert-Siepmann N, Edler D, Redecker AP, Jürgens C (2011) Designing teaching units via WebGIS: remotely sensed imagery in the language classroom. EARSeL eProc 10(2):149–158Google Scholar
- Lammert-Siepmann N, Edler D, Dickmann F (2014) Die Effekte audiovisueller Informationsvermittlung in Topographischen Karten auf die Behaltensleistung—Das Beispiel geographische Namen. Kartographische Nachrichten 64(6):308–316Google Scholar
- Myers A, Hansen CH (2014) Experimental psychology, 7th edn. Wadsworth, BelmontGoogle Scholar
- Popelka S, Stachon Z, Sasinka C, Dolezalova J (2016) Eyetribe tracker data accuracy evaluation and its interconnection with hypothesis software for cartographic purposes. Comput Intell Neurosci. http://dx.doi.org/10.1155/2016/9172506
- Richardson D, Milbourne P (2014) Foundation action script 3, 2nd edn. Apress, New YorkGoogle Scholar
- Schiewe J (2015) Physiological and cognitive aspects of sound maps for representing quantitative data and changes in data. In: Brus J, Vondrakova A, Vozenilek V (eds) Modern trends in cartography. Selected papers of CARTOCON 2014. Springer, Heidelberg, pp 315–324Google Scholar
- Schito J, Fabrikant SI (2018) Exploring maps by sounds: using parameter mapping sonification to make digital elevation models audible. Int J Geogr Inf Sci. https://doi.org/10.1080/13658816.2017.1420192
- Touya G, Christophe S, Favreau J-M, Ben Rhaiem A (2018) Automatic derivation of on-demand tactile maps for visually impaired people: first experiments and research agenda, Int J Cartogr. https://doi.org/10.1080/23729333.2018.1486784
OpenAccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.