1 Introduction

Designing is a complex activity, in which, apart from creating objects and artefacts, designers are building a whole process of interaction. This process is difficult for young designers because it is necessary to predict how people (users) will perceive, use and interact with the design solution. Observing and understanding users who interact with our solution is vital. It builds empathy and competence and provides us with new perspectives.

Th-at is why user research, with real users, is an important part of designer training. Design students at the Academy of Fine Arts in Katowice take a Usability Research course, the goal of which is to acquaint them with methods in the field of digital product design. Conducting their own research helps them understand real users’ behaviour in the process of interaction with the design solution. It facilitates understanding of the role of research in the process of making informed decisions in the process of creating digital products, e.g. websites and applications.

Due to the complex structure of website interfaces and the amount of data that describe user interaction, the task is difficult. Students encounter problems understanding the structure of the user–website interaction process, comparing the results with the model process, and identifying issues in user interaction and their causes. Thus, the idea of using visualisation of the user–website interaction was introduced as a teaching method. The method focuses on the visualisation of the interaction process. Data gathered during students’ research are presented using schemes that allow for better understanding of the website structure and the process of user interaction with the website. The method supports in-depth analysis and helps to better structure the findings.

The goal of this paper is twofold, first to present the teaching method and the whole process of visualisation use in gaining insights about website design and second to discuss the perception and acceptance of the method among students. We used a modified technology acceptance model (TAM) framework to determine which characteristics of the method influence students' behavioural intention to use it in the future.

2 Data visualisation

The visual representation of a process or phenomenon supports the understanding of its course and the relations between their components and also often becomes a tool that supports decision-making [3]. It requires its author to interpret reality in the form of an abstract image, where the elements of reality and the relations between them are replaced by graphic values. These might be both tangible objects and invisible ones like time, interactions or behaviours. What is crucial for this kind of project is the conscious selection of the elements that will be represented in the scheme and the graphic values that will be assigned to them. It is essential to determine the rules of selection and the choice of graphic values representing real-life elements. When assigning specific graphic values to respective phenomena, we create a kind of a visual language—the applied graphic values of each element and its position in the system determine the meaning. The scheme elements are also often supported with additional descriptions to limit the caption’s content [24]. These decisions will influence how well the main features of the elements and their relation to other ones will be expressed and highlighted [13].

It is particularly important to select graphic values that will facilitate the understanding process of a reader and to make the process of learning the scheme’s rules uncomplicated. Therefore, well-known schemes (e.g. the coordinate system or a timeline) are often applied. The benefits of this type of visualisation cannot be ignored. Understanding relations between data presented as text, digits, tables or lists is challenging for the human mind. Therefore, visualisations facilitate the cognitive processing of data by a human and aid the process of analysis [18].

In the case of complex data sets containing many variables, proper visualisation allows for better understanding of dependencies and relations between phenomena and the creation of the proper mental model. It also reveals the hidden patterns and relations that would not be possible to notice using the traditional forms of data presentation. Modern technological solutions also provide the possibility of presenting information in an interactive, spatial or time-changing form, which serves the construction of the mental model even better [4]. 

Card, Mackinlay and Shneiderman [3] underline that visualisations may aid the thinking/analytical process—using vision to think—through (1) increasing the memory and processing resources available to the users, (2) reducing the search for information, (3) using visual representations to enhance the detection of patterns, (4) enabling perceptual inference operations, (5) using perceptual attention mechanisms for monitoring and (6) encoding information in a medium that can be manipulated. Visualisation is a tool for a better understanding of a wide range of phenomena. It is a medium that makes it possible to efficiently communicate complex data to a wider audience.

2.1 Data visualisation in design education and practice

In design, the visualisation of ideas, processes and concepts in the form of sketches and schemes is embedded in the DNA of the profession. It is a method that supports creative thinking and facilitates prototyping. Therefore, the development of these skills is an important element of designers’ education. This is a particular habit and a manner that should be developed and practised in order to become a natural work tool. Visualisation is a tool that might support designers in many stages of the design process; however, the scientific publications of using visualisation in the research phase to present data are scarce.

Alberto Cairo, the information designer and educator, writes about the significant role of data visualisation, indicating that this is an important tool for extending and acquiring knowledge. But for him, it is also an object around which a discussion can take place [2]. Nevertheless, in order to make this tool effective, it must satisfy specific requirements: reliability, honesty and depth. In the process of transferring the real world into abstract forms and shapes, it is important to balance simplicity and complexity. The designer has to select wisely in order to not skip any essential data and relations, but must also graphically represent the—sometimes dynamic—relations between them to build an appropriate “metaphor” and not an illusion. Visualisation design is also a selection process in which we reject some elements and emphasise the selected ones. Therefore, it is particularly important on which basis designers choose the elements, as an unfortunate or inattentive choice process might lead to distorted, mistaken or even manipulative results. Whether such action is accidental or intentional, whether it is the result of inaccuracy or mistake, it may lead to irreversible consequences. Hence, an ethical approach should always be fundamental to designers’ activities.

Building visual representations is a commonly applied practice nowadays. It is a tool for the analysis of the relations between any given elements, and it also supports decision-making processes. The visualisation of ideas, concepts and processes recorded as sketches and abstract schemes facilitates communication in a team and enables the understanding of complex processes. As design professionals need to understand reality within a broad context and to an extent that allows them to effectively diagnose problems and respond to real needs, teaching students how to visualise complex processes, services and phenomena seems significant. Transferring to the abstract dimension provides the possibility of detaching from literality and the physical aspects of objects and emphasising what is temporary, elusive and which, at the same time, is important and crucial for design implementation. This is an analytical type of exercise, where importance is given to structure—its elements and links that connect the elements. And this way of thinking can be easily applied for other design activities; in the international study of basic design education the development of this type of competencies (syntactical analysis) was pointed by teachers of basic design and design project as one of the most important skills [1].

Krzysztof Lenk, a visual information designer, the founder of the Dynamic Diagrams studio, has prepared a curriculum for training skills in data visualisation for the Rhode Island School of Design in Providence in the USA. As a pedagogue, he conducted a series of exercises with students, using very extensive profiles of the topics: from the Bloody Road of Richard III to the throne, through the analysis of wine types and related customs, finishing with statistical data on social and economic aspects, such as taxes, family finances and the interpretation of the concept of maternity. All the studies were preceded with thorough analysis, supported by step-by-step construction of the visual solution—from sketches depicting a concept to detailed schemes made in graphic programmes [16].

In the context of this paper, it is also worth recalling Lenk’s method of visualising the construction of websites, developed by him in the form of isometric structures. To understand the principles of any website construction, a schema that included every subpage, elements of navigation and all the links between them was prepared. In our educational method, this is one of the first stages of students’ work. They construct by themselves an abstract model of the website that presents all subsites, links and interactions. This task activates the processes leading to a better understanding of the functioning and construction of the analysed website. Lenk puts a lot of emphasis on the step-by-step and handmade iterative process of the construction of the visual form. This process enables students to better understand the specificity of the matter they are dealing with.

Using visualisation as a tool for analysis and a better understanding of a given phenomenon is a widely applied practice. It may be an automatically built or handmade model, in which the shape of elements and their interrelations are defined only by the author. What is most important from the didactic point of view is a deeper understanding of the analysed problem. Nowadays, graphic designers are beginning to use programming languages and other tools for the automatic generation of diagrams or schemes, thus enabling a more flexible approach towards data visualisation. These tools have become more widely available and they are certainly useful because they make it possible to obtain results very quickly. So why, given the development of visualisation technologies, is it worth encouraging students to use traditional techniques, in which they must draw every element of the scheme? A basic value of the designer’s intellectual work is a thorough understanding of the analysed issue in the ideation phase. In this case, the issue is the problem or phenomenon that needs to be transformed into an abstract image readable by others whose knowledge is not as detailed as the designer’s. This kind of thorough study requires tools that allow for the flexible and quick prototyping in exploration of various data interpretations and help to place every element consciously, but most of all, help to capture the whole concept using a scheme.

Programming tools become indispensable with large quantities of data, but they are often used after the completed ideation phase. The project Peak Spotting, implemented by Nand Studio [22] for Deutsche Bahn, a German railway, is an example of this kind of approach. This is a highly complex design of a set of visual tools aiding the management of the railway network transporting a hundred thousand passengers a day. The data are presented in various manners—paths, graphs and maps, with the use of colour coding for the visualisation of traffic intensity and passenger count. The main goal of this tool is to spot and predict the peaks—connected with the high load of certain lines—using machine-learning algorithms. The way of presenting the extensive quantity of data in visual form allows for fast and responsible decision-making. While presenting the process of working on this solution, Thiel [22] emphasised the importance of ideation stage, which was conducted in the form of drawings that also became a communication platform with the client.

Lemons, Carberry, Swan, Jarvin and Rogers [15], in the article The benefits of model building in teaching engineering design, present didactic methodology in which constructing a visual model supports the generation and assessment of designed concepts. The proposed model allows engineering students to track and anticipate the possible behaviours of the users of the proposed designed solution. This method supports the development of creative and analytical thinking (meta-cognitive design skills). From a pedagogical perspective, it also makes the students more aware of the activities they undertake in the design process and their results. Also important in this method is the fact that students build a model manually—they are responsible for selecting each element, they shape every relation in the system—nothing is generated automatically [15].

A similar approach has been presented in a paper by Ranscombe, Bissett-Johnson, Mathias, Eisenbart and Hicks [19] comparing the use of sketches, cardboard models and LEGO sets to facilitate students fluency in idea generation. Although using LEGO was an example of a slightly different visualisation method than described above, the incorporation of LEGO blocks into idea-generating phase instead of high-fidelity CAD sketches resulted in a generation of a larger number of ideas in a more collaborative way. Also, the authors discussed the hindrances of visualisation methods based on using sophisticated CAD software [19].

In usability research, which is part of human–computer interaction design (HCDI), teaching students how to visualise, by drawing or by using artefacts, the behavioural context and dynamic interaction between user and technology is one of the most important tasks of educators [7].

3 Interaction and usability research

Usability research is conducted to verify the usability degree of a designed solution. According to a definition presented in ISO 9241 Ergonomics of human-system interaction—Part 210: Human-centred design for interactive systems, usability is defined as a measure of efficiency, effectiveness and satisfaction with a product used by certain users to achieve specific objectives in a given usage context [11]. This definition refers to two aspects of product usability:

1. Objective and measurable, i.e. effectiveness and efficiency of the accomplishment of specific tasks, in terms of fast realisation time, and a low number of mistakes;

2. Subjective and personal, i.e. emotions and feelings accompanying the usage: the higher the level of satisfaction among users, the higher the degree of usability.

Usability tests are conducted at various stages of the design process (from sketches and lo-fi prototypes to production-ready solutions) in order to verify the design usability level. They make it possible not only to detect design errors but sometimes also lead to new directions in the development and design [9].

A procedure of usability tests is based on observing users while performing various tasks while using a product. Usability testing is often supported by interviews or questionnaires. The effectiveness of a usability test is determined by the choice of tasks. Also, the reliability of results is affected more by the content of tasks than the number of respondents. With five properly selected respondents, 85% of interface errors may be detected [23]. With such small groups, it is especially important to accurately measure, analyse and interpret data. On the other hand, direct contact with users allows for deeper and more detailed analysis of users’ behaviour.

Usability testing is based on the observation of user behaviour, and the various aspects of the interaction between user and product are recorded. As a result, we obtain both quantitative and ready-to-use data (e.g. task realisation time, level of completion, number of errors, mouse movement data [i.e. number of clicks], selected interface elements) as well as a qualitative and subjective data (i.e. the user’s opinions and comments, which require deeper analysis). The most important skill in the process of data analysis is the selection and combination of various types of information and data gained during the whole testing process. The goal is to understand how the user interacts with the interface and to which degree his or her needs and expectations are met.

3.1 Using visualisation to understand website structure and user–website interaction

Websites are complex objects with hundreds of thousands of linked objects—hyperlinks, images, texts, buttons, etc.—forming a complex network of relations. In the process of website development, the user’s behaviour is predicted and designed. Interaction designers create user pathways, in order to control and manage users’ traffic. Still, the process of human-website interaction is a combination of users’ intentional behaviour within the designed interface.

A website is a virtual space, in which navigation is needed in order to help users reach their goals. Interaction design supports a user in exploring a website and achieving his or her intended aims. However, when entering a website and then subsequent subpages, we can see only a more or less static image consisting of text, illustrations, photographs and animations but also interactive elements like menus, buttons and forms. How can we visualise what happens after a website is entered? How does a user use a website? What are his or her goals and how does he or she achieve them? What tools should be used to assess the usability and easiness of using a website objectively?

This challenge was faced by Kahn, Lenk and Kaczmarek [12], who devised a tool, isometric representation of websites structure and content, as well as ways of its communication and interaction with the user. The designer starts to build the visualisation from scratch for a given website, considering its structure and specificity and adjusting the format of presentation. Purposefully designing the map to fit a specific format enables viewing the page structure in whole with a single look and assessing its most important features and relations. Although there is software that generates this type of schemes automatically, the authors claim that it does not allow for viewing the structure in whole. Another reason to do it manually is that an automatically generated page map presents all the elements in an identical, programmed manner, without respecting data hierarchy and specificity.

4 Visualisation of user–website interaction—structure and elements of teaching method

This paper presents a teaching method based on visualisation of user–website interaction. Students in the Usability Research course conduct a website audit and usability test while using advanced, customised visualisation techniques to present its results.

The method involves two stages:

Stage A—usability audit;

Stage B—usability tests with users.

Analysis and evaluation of website functions, navigation, layout and information architecture (IA) in Stage A is combined with findings from Stage B, which consists of a usability test with users. This enables students to identify website errors and problems and then to propose adequate solutions to the diagnosed challenges. The findings from both stages contribute to the final recommendations. A dual-method approach, from an expert (heuristic analysis) and a researcher (works with users) perspective, allows students to compare results achieved in different ways and fill in the gaps. It helps students to see what problems in the website functioning were missed during expert analysis but clearly visible during research (i.e. tests with users). The important part is the comparison of the results, as it helps students learn how two different approaches allow them to see the wider context of user–website interaction. The crucial role in this comparison is visualising the website’s elements and user interactions. The instruction enables students for free choice of graphical representations and elements. Students decide by themselves which means to use, e.g. lines, fonts, or glyphs. By having flexibility in data representation and not sticking to one standard, the visualisations varied but better fit to analyse websites’ specific functions and goals. Students test different solutions and choose the most relevant data representation for both Stages A and B.

Because both stages are complementary, in Stage A the students build their assumptions about the usability level of the website and then verify it in Stage B. This experience provokes double-loop learning—where we look at the same situation but with new data and questioning approach [17], p. 4). It is a very important moment to gain competencies to challenge own assumptions, to control forming premature conclusions and learn how not to stick too much to own beliefs and opinions.

4.1 Stage A

The objective of the first stage is the analysis of the website functions, navigation, layout and information architecture (IA) in the context of its goals. Students design their own way of visualising analysis results. But the visualisation itself allows for a deeper understanding of patterns, relations and structure. Students present user interface elements and structure and analyse their features, i.e. graphics, typography, colours and shape. Visualisation of IA and user scenarios also help identify the advantages and challenges of the website. Below we present examples of visualisations prepared by students during Stage A:

  • different ways of presenting information architecture (Fig. 1). Visualisations 1.1 and 1.2 focus on levels of IA and how its influence user interaction, number 1.3 is focused more on presenting specific areas of IA, where users can perform particular tasks;

  • combined elements and functions (Fig. 2) and layout variations (Fig. 3). Students were analysing the available functions in relation to a specific layout; it helped them to evaluate the consistency of the website design and to distinguish the elements of a visual system;

  • visual features—graphics, colours and typography (Fig. 4). Detailed analysis of website elements (e.g. colours, typography, etc.) helps to identify problems. In this example, we can see that the choice of colours and the number of fonts impaired the legibility and readability of the website but also indicated that the visual system lacked consistency;

  • interaction types (Fig. 5). Identification and analysis of both the number and characteristics of possible interactions with the website allow for further evaluation of the level of website usability and potential problem.

Visualisations help to detect errors and usability problems. All the Stage A findings and insights determine research questions and hypotheses tested in Stage B.

Fig. 1
figure 1

Various ways of visualising IA prepared by students (authors: 1.1. Karolina Chmiel, 1.2. Dominika Bablok, 1.3. Szymon Kuś)

Full size image
Fig. 2
figure 2

Elements and functions of a homepage (author: Klaudia Kadłubiec)

Full size image
Fig. 3
figure 3

Layout variations (author: Magdalena Szumigaj)

Full size image
Fig. 4
figure 4

Visual features of the portal—colours/contrast/typography—for a website lacking a permanent typography system (author: Magdalena Pawlik)

Full size image
Fig. 5
figure 5

Interaction types (authors: Agnieszka Jakimiec, Kinga Pawlik, Magdalena Przybycień, Justyna Wojtaczka)

Full size image

4.2 Stage B

Stage B consists of usability testing with users; the educational goal is to advance students’ skills necessary to test and understand user behaviour. Again, the combination of analysis and visualisation techniques led to better insights. Students base their visualisations on the timeline, IA and layout.

Research questions concerning selected areas of the website are based on the results of Stage A. Then, hypotheses to be tested with users are created and scenarios of specific test and tasks are built. Stage B comprises the following activities:

  1. 1.

    Building the research scenarios to verify hypotheses (recommended tools: questionnaires, interviews, observation and usability tests);

  2. 2.

    Tests and data collection (user behaviour date include recordings of both user and screen, his or her comments, performance time, number of errors, etc.);

  3. 3.

    Results analysis—on user and task level, as well as cross-analysis;

  4. 4.

    Summary—the verification of the hypotheses and recommendations.

An innovative element of this course is the way of visualisation of the user pathway. Students prepare the graphic representation of the user pathway using three methods of presenting data, based on the timeline, IA and layout. Afterward, the elements used in visualisations are summed up (see Table 1) and discussed with students. Each type of presenting user pathway employs the visualisation of different elements/aspects of user behaviour. As a result, students can see differences between pathways; some of them then decide to analyse the pathway further and even create hybrid solutions going beyond the established categories (presented in chapter 4.3 Alternative ways of presenting user-interaction).

Table 1 Three ways of presenting the user pathway
Full size table

The fulfilment of this type of task—visualisation of data gathered in usability tests—requires building a visual code that makes it possible to record the various types of behaviour and compare them, e.g. their frequency, duration and relations. The transformation of observed behaviour into data and then their visualisation requires students to properly choose graphic values, establish the importance and hierarchy of elements, and accurately select and combine various data types collected during tests. In particular, this applies to elements affecting the effectiveness of tasks realised by users, such as time, number of errors, types of problems and user satisfaction. An important part of this activity is also linking data resulting from observation with users’ opinions and comments. Students complete the visualisation of users’ behaviour with this additional data. Frequently, comparing opinions with behaviours makes it possible to diagnose the reasons for failures and problems. Often enough, users are not fully aware of the encountered problems. Therefore, it is worth comparing the results observed by a researcher during tests and opinions declared by users. They may be totally different or supplementary [21].

Three ways of presenting a user pathway, based on timeline, IA and layout, allows for mapping various aspects of the interface use. This makes it possible to differentiate the types of user interaction with the interface, distinguish their interrelations and finally transform them into conclusions and design recommendations. Students choose which type of visualisation would best align with the types of behaviour they analyse. While preparing visualisations, it is important to select the presented data and arrange them in a proper hierarchy—so that the most important elements are most noticeable but can still be related to other data.

The following aspects and elements of interaction are analysed using three ways of presenting the user pathway:

  • In the timeline, the following are analysed: frequency, duration, relation, user activities (clicks, cursor movements, scrolls) and changes in the interface (see Fig. 6). Collected data allow for comparing task performance between users. Students can see the similarities by comparing the route of the line, the number of stages to finish the task and its length. The duration of the performed task indicates the difficulty level—if the task takes much time, it is probably hard to accomplish. They can also analyse which users spend the most time to finish the task and how it reflects their experience level (new vs. regular user) or personal characteristics (e.g. age, profession). Moreover, it helps to detect problems and challenges in the interaction. For example, intervals between subsequent clicks/actions are often indicators of time necessary to make a decision—but when this gap is very long, it often signals difficulties. By highlighting the moments of writing or scrolling, often associated with searching or overviewing the content, students can exclude the activities that may extend the process of task fulfilment;

  • In the visual representation of information architecture (designed during Stage A, which includes interface elements and possible interlinks) users’ strategies of task performance are analysed. Figure 7 shows the results of this part of the analysis. By analysis of the shape of the lines mapping the users’ pathways, students can easily assess the similarities and differences between them. This is also a good tool for comparing users’ behaviour with the desired behaviour pattern—the shortest way to task fulfilment;

  • In the website interface scheme clicked elements of the website are analysed (see Fig. 8). In the presented example, by drawing a user’s pathway on the simplified interface layout, students can see where the user was looking for proper buttons and where he or she clicked incorrectly. In this visualisation method, the locations of longer periods of inactivity in the interactions, indicating moments of hesitation and thinking, may be marked as well. As shown in Fig. 8, the analysis of the user interaction revealed the problems with the choice of proper categories. The reason was both the unclear segregation of categories and not enough differentiated design of categories and subcategories in the interface layout.

Fig. 6
figure 6

Timeline-based user pathway for six users (author: Mateusz Mioduszewski)

Full size image
Fig. 7
figure 7

IA-based user pathway presents in what sequence and which options were chosen by two users (author: Barbara Skrzeczek)

Full size image
Fig. 8
figure 8

User pathway in the context of the website interface scheme-adding a design in the user portfolio (author: Magdalena Pawlik)

Full size image

4.3 Alternative ways of presenting user-interaction

Students start a data analysis from preparing user-interaction visualisation separately for each respondent. They then map data for all the users on one scheme and reduce unnecessary and insignificant information to spot and compare differences and similarities. This identifies users encountering similar problems, choosing the same interface elements or performing differently, thus distinguishing various types of users. In order to present the differences more adequately, students prepare schemes using only selected elements of the user-interaction process.

For instance, in Fig. 9, the analysis focuses on cursor movements (left) and clicks (right) using the simplified interface layout. This graphic interpretation of the users’ behaviour enables fast and efficient identification of most frequently clicked locations. Also, the line of the cursor’s movements shows the search process and indicates clicks locations.

Fig. 9
figure 9

User interface layout with ten user pathways: cursor movements (left) and clicks locations (right) (Author: Karolina Chmiel)

Full size image

Another visualisation (Fig. 10) of users’ behaviour is based on the time and desired user pathway. This enables observation of both sequences of events and task performance time. Additional information is also visible, e.g. moves back to previous stages, skipped stages. We can easily observe, by analysing the line shape, that the user’s pathway does not always correspond with the designed sequence of steps for a process-in this case, the ordering of a product. If the line course changes (goes from the left to right sight and then goes inversely), it indicates that the user is not following the website guidance and needs to go back and go through the process. In the example, user n. 2 reached the step–order and was very close to the end of the process, but something made him go back to the first step and start the task once again. The scheme and users’ comments analysis allow for the identification of incomprehensible, unnecessary and incorrect steps undertaken by users and their consequences. What is an essential detail in this scheme is that the length of the line does not refer to time duration. The line shows which steps the user has passed by or skipped. The time information is provided additionally as a numerical value. This decision was a thoughtful choice of the student, as the main issue was to compare the designed pathway (steps in a specific order) and user performance among several people.

Fig. 10
figure 10

User pathway for three users—a combination of time and user pathway (authors: Magdalena Habarta, Kacper Blajer, Kamil Kowalczyk, Klaudia Korniluk)

Full size image

As the data analysis process continues, students have to carefully select data that will help them identify various types of users. By eliminating the detailed descriptions of interaction and aggregating data from different users and various tasks, the pattern of user behaviour becomes more visible (i.e. typical errors, strategies). Figure 11 presents the process of purchasing a product in two groups of users aged over and under 55. It shows differences in the number of encountered problems between older and younger users and identifies critical elements of the purchasing process, which was difficult for users. In this diagram, the student presents four types of behaviour at each stage of task performance (e.g. completed task, omission, roaming and wandering, uncompleted task and resignation).

Fig. 11
figure 11

User pathway and errors for ten users in two age groups (Author: Magdalena Szumigaj)

Full size image

As shown in Fig. 12, the visualisation was used to answer the question of whether or not the experience level of the user (new or regular) determines the choice of the product search strategy. The various users’ strategies are visible, as well as their differences according to the product type. Furthermore, the inefficiency of banners was confirmed. The conclusions can be easily spotted in the presented example where data are presented in a specific order:

Fig. 12
figure 12

Comparison of product search methods for new and regular users (author: Mateusz Mioduszewski)

Full size image

– the colour indicated the type of user;

– rows in the table refer to the search strategy;

– columns in the table refer to the type of task and products users were looking for.

In each intersection in the table, the number of coloured dots refers to the number of attempts made to look for a specific product with a specific type of search. For example, during usability tests, all users, both new ones and regulars, used a search engine to order photograph prints (blue and orange dots). However, no one uses advertisements and banners, even if they were considered as one of the possibilities to gain the task aim (white dots).

Data analysis and synthesis allow for clear visualisation of differences that are hard to spot by observation alone. Another example (Fig. 13) presents the comparison of the IA of the search engines of two public transport portals (Transport for London, and Polish metropolitan bus service in Silesia). Two portals, looking completely different, were visualised in the scheme using the same set of rules and elements designed by students. Using this procedure, both structures were compared, and the differences were easy to spot and allow for analysis of IA in terms of depth, interconnections and building blocks.

Fig. 13
figure 13

Comparison of the search engine for Transport for London and Polish metropolitan bus service in Silesia (author: Barbara Skrzeczek)

Full size image

The teaching method used in the Usability Research course assumes that during the process of designing the visualisations, students have to transform the results of observation into data, devise a set of rules for both assigning graphic value to them as well as highlighting their intercorrelations. This is a process in which they go from reality to abstract recording, which makes it possible to see what is unnoticeable via indirect observation. Building schemes from scratch, not using any dedicated software, helps students recognise building blocks of interaction, various ways of its graphical representation and the possibilities of combining data gathered in research. Finally, thanks to the whole process, students better understand how users interact with websites and which factors affect the success or failure of this interaction. In human–computer interaction (HCI) design students must draw from multiple knowledge domains; building knowledge using structured processes helps to understand and balance form, function, human needs as well as social context in future projects [7]. Our method will also help students in the future, as they have learned how to present research results and use the knowledge they gain in research to justify necessary changes in design.

5 Evaluation of the method

In this paper, in addition to the presentation of the method itself, the perception and acceptance of the method among students is also discussed. Although there is no commonly accepted definition of a successful educational method, we can point to several factors associated with teaching method success. First, the method should enable students to achieve their goals; second, it should be accepted by students and they should willingly use it; and, finally, the method should be used in a professional or everyday activity. In the case of our method, we were interested how it is perceived by students and if there is an intention to use it in the future. The evaluation was conducted for three consecutive years in a repeated measurement scheme—immediately after finishing the course and one year later. The study has been approved by the Research Ethics Committee at the University of Silesia in Katowice, Poland. The participation in the study was voluntary, students have not been given any credits for taking part in the study, and data collection started after finishing the course. We used an online survey to gather student responses. Informed consent was obtained, as students were entering the survey after being provided with information about the goal of the study, as well as anonymity, privacy and confidentiality procedures.

We had an 83% response rate, as from the total number of 36 students enrolled in two editions of the course, 30 students took part in the survey. The survey consisted of informed consent, basic demographic questions, several open-end questions and TAM questionnaire.

We asked students several open-ended questions to better understand what they think about the method. [25] For the first question, “Would you use the method in the future?”, most students (21 of total number of 30) answered positively, eight with some concerns and only one person negatively. Then, we asked “Which element of the method was most important for effective assessment of the website?”. In answering this question, 14 students indicated the visualisation of the user pathway, while for 13 students specifically, the timeline representation was most effective. Also, for several students, the observation of user behaviour was the source of valuable information about user–website interaction. Finally, we asked students, “Which elements of the method brought the most unexpected results?” and again, students indicated user pathway visualisation. It was pretty surprising, as students experienced a lot of difficulties during the task of user pathway visualisation. It took the longest time and required more support from the teachers. Also, it was hard for students to switch from illustrative use of graphical material to abstract visualisations.

5.1 Students’ behavioural intention to use the methods

In order to understand how students perceive the method and how they intend to use it in the future, we decided to use the technology acceptance model (TAM; [5] as a framework. The technology acceptance model is one of the most popular models in IS research. Based on the theory of reasoned action, TAM explains the behavioural intention (BI) to use the system is the result of perceived usefulness (PU) and perceived ease of use (PEU), see Fig. 14.

Fig. 14
figure 14

Technology acceptance model, TAM [6], p. 985)

Full size image

Pedagogy lacks models allowing for measuring teaching methods’ acceptance. Thus, we decided to adapt TAM for this purpose. The justification was that TAM allows for structuralised control of two critical factors influencing acceptance—easiness of use and usefulness. These elements seem to be compatible with students’ experiences, as they declared both positive and negative aspects of using the method.

We have used the original TAM questionnaire for the study, but we replaced the word “technology” in the original questionnaire with “the method”. Responses could range from 1 (strongly disagree) to 7 (strongly agree).

The survey has been conducted for three consecutive years using an online survey and repeated measurement. The first measurement took place immediately after the class was finished (1st measurement wave); one year later, we asked students to take part in another survey (2nd wave) in order to check how the perception and the acceptance of the method changed over time.

In the first measurement wave, a total of 30 students (87% women) took part in the study, and in the second measurement wave 24 (87,5% women) people repeated the survey. The student sample was predominantly female; such gender imbalance is characteristic for design degrees in Poland (FSP [8]. The average age was 22 (M = 21,93, SD = 1,01). Table 2 presents the results of the survey for the first and second waves.

Table 2 Descriptive statistics for TAM components in 1st and 2nd measurement waves
Full size table

In both waves, perceived ease of use scored lower than perceived usefulness. After one year, the most visible change was lower behavioural intention towards the use of the method, while scores for perceived usefulness and perceived ease of use were rather stable. In the next step, we decided to check if perceived usefulness and perceived ease of use predict behavioural intention to use the method in the future. To assess the influence of the method characteristics (perceived ease of use and usefulness) on students’ intention to use the method, we employed partial least square (PLS) modelling using SmartPLS 3.0 [20]. PLS is a recommended statistical method in case of small samples [10]. Separate models were created for both measurement waves (i.e. 1st wave—right after the class measurement and 2nd wave—one year after).

The results confirmed that perceived usefulness predicts almost 44% of behavioural intention. The results for the 2nd wave measurement were even more explicit, with 62% of behavioural intention predicted by perceived usefulness. The role of perceived ease of use was non-significant in the first measurement, but it significantly influenced perceived usability in the 2nd measurement. Figures 15 and 16 present the model with path coefficients for both measurement waves.

Fig. 15
figure 15

Path coefficients for 1st wave of measurement

Full size image
Fig. 16
figure 16

Path coefficients for 2nd wave of measurement

Full size image

The statistical properties of both models were sufficient; Table 3 presents AVE coefficients for both measurement waves.

Table 3 Intercorrelations between latent variables of TAM in 1st and 2nd measurement waves
Full size table

Behavioural intention to use the methods is mostly determined by perceived usefulness. On the other hand, perceived usefulness itself is influenced by perceived ease of use, which does not directly determine behavioural intention. The effect of capturing the whole influence on the behavioural intention by perceived usefulness, which always lowers the influence of perceived ease of use significantly, has been confirmed in meta-analytic studies [14]. However, in our study, the coefficient is negative, which needs to be reconfirmed in further studies using a bigger sample size. This finding might also be related to the growing experience in using the method—as students become more skilful in using the method, they perceive it to be more useful. For future research, it could be also interesting to investigate further professional development of students in context of intentional use of the competences gained during the course.

6 Discussion and conclusion

For students, understanding the role and the process of visualisation is a difficult and important task. Being accustomed to literality makes their first attempts inadequate and overly focused on the physical features of objects instead of on the rules that constitute the basis of the process being analysed. When students start analysing websites in presentations, they mainly use print screens on which they make comment or underline elements that make up a website. When they are presenting a process of buying goods in an online store, they compile a series of print screens corresponding to respective stages of the process. They need to start to notice relations, structures, patterns and rules and then express this knowledge in the form of abstract forms in order to start visualising. The simplification is an important part of this process, to facilitate the awareness that physical representation is no longer important. Then, students focus on activities—they need to outline occurrences and relations. Among the most valuable results of visualisation are those that support the understanding of the specificity and characteristics of visualised processes.

A task each teacher faces is to build teaching methods and tools that will facilitate the success of their students. The teaching method we have discussed in this paper refers to a very narrow field—usability research. In the process of learning how to visualise user behaviour, IA and pathways, students start to be sensitive to specific details, notice their value and relations and finally are able to arrange them in a context. Combining the research and design is an important characteristic of the curriculum of the Faculty of Design, as nowadays designers are essential partners in the research phase of both digital and physical product development. Designers should know basic elements of a scientific approach to be equal and essential partners in interdisciplinary research teams [17]. Combining practise and theory in usability research educational programmes is an important aspect of designers’ education, as the HCI domain requires multidisciplinarity and drawing the best practices from various disciplines [7].

Our evaluation of the perception of the method indicates that it is appreciated by students. They perceive it as a useful method, a fact that affects their intention to use it. At the same time, there is a change in their perception of the method with time-one year after completing the course, the perceived simplicity of the method is related to its assessed usability and it explains an intention to apply it to a greater degree. This is the confirmation of an assumption that this method is viewed by students as a tool that may be applied in other design activities. Our evaluation of the method also provides more general conclusions related to the understanding and acceptance of teaching methods. The results demonstrate that there is no direct influence of the perceived simplicity of the method on the intention to act; nevertheless, the perceived usefulness turned out to be significant for students. Due to the small sample size, the results should be treated as descriptive for our sample; further research is necessary to confirm such hypotheses for the population. This means that indicating the practical possibilities of using research methods and benefits that may be obtained in this way by students is crucial for acceptance and application in further work. Competencies developed in the environment of usability research for a given website process might be transferred into other design challenges. Skills embracing planning and execution of research, analysis of complex processes and their adequate visualisation, use of specific software, as well as gaining experience in presenting results to various audiences, are universal competencies that allow designers to develop in any field..

Nowadays, design professionals need to combine skills of analysis and synthesis to be able to find answers and solutions for both today’s and tomorrow’s questions, but our method due to thorough data analysis and transformation also supports a reflective practice which is pointed as an important aspect of designers’ education [17]. However, it is also necessary to anticipate future challenges and develop universal competencies which might be useful in various contexts but also will distinguish design alumni on the job market. That is why we should focus on raising awareness among students about its usefulness and a wide range of applications.

We use the presented method in an educational context. However, we also see potential in simplifying the method and standardising it to be used in a professional market. Upon the current experience with using the method, we can now suggest the best ways of visualising various aspects of website structure, but also of usability-related challenges. The method’s most significant advantage is the importance of structure and clarity of visual representation, which support a better assessment of usability. Structuralised sequence of evaluation of different aspects of website usability also has the potential to be a base for the development of heuristics and then automated or even machine learning-based tools for websites and mobile applications analysis. There are several tools on the market which provide UX researchers and designers with automatically generated data. However, the important aspect of our method was teaching students how to choose and adjust the means of graphical visualisation. Therefore, the question of how to preserve this aspect and how to combine this with automated analysis needs to be answered to further develop the method without losing its fundamentals.