Keywords

1 Introduction

The field of Human-Computer Interaction (HCI) is diverse in terms of institutions, disciplines, countries and the people who are or are becoming HCI researchers and practitioners. HCI focuses on the human users with their goals and tasks, needs and requirements, tools and contexts. This is not always be the case for other related disciplines [1]. Learning to do a HCI professional’s work requires learning a specific skillset and qualifications. This knowledge is different from the one required by other designers such as software engineers [2] and system developers [1], and which may even vary across countries [3].

HCI communities vary across the globe. While some countries are small and have only one HCI community, other countries are large and multicultural countries with several different HCI communities with distinct approaches, see e.g. [4]. HCI research and education should thus recognize the cross-cultural nature of the field [3]. It is quite likely that in the future HCI will further grow and get rather more diverse, and so will HCI education with respect to both contents and formats of teaching [1], and with regard to the diversity among students.

Diversity in students thinking and learning styles is important, as cognitive style can predict students’ achievement in design and technology studies [5, 6]. Therefore, it is important to understand how likely students of HCI are to succeed in developing the required HCI skills. The research reported in this paper seeks to contribute to this understanding by exploring the interaction of aspects of the HCI curriculum with students across different institutions, disciplines and countries.

Previous research in higher education of computer science and information technology has established that academic performance is influenced by the cognitive styles of students [7] and by their culture [8]. However, HCI is a growing field increasingly taught in different cultural and professional contexts, not only in computer science but also in design and business, which lends itself to attract students with different cognitive styles. HCI students’ cognitive styles may thus be diverse, and perhaps different from the cognitive styles of traditional computer science and engineering students. There is no universal standard or unified view on HCI education. The idea of a global unified curriculum has been deemed as implausible by a multinational survey [9]. Therefore, the lack of consideration of multicultural aspects in the HCI curriculum presents a challenge for promoting diversity in a field inherently aimed at satisfying human needs and goals [10].

This is why in this paper we try to address the question of cognitive styles and learning HCI in diverse contexts. We report how undergraduate HCI students engaged and performed in equal evaluation and design tasks in workshops conducted in institutions in China, Namibia, India, Mexico and the United Kingdom. This paper develops from a previous one where we reported preliminary findings about cognitive styles and cultural attitudes [11]. In this paper, we report the analysis of the complete dataset with extended findings and discuss implications for the study and promotion of diversity in HCI education through the construct of cognitive styles.

The paper first presents on overview of related work in HCI education and diversity and a review of cognitive style theory as a construct to study learning and cultural diversity. This is then followed by a presentation of the methodological design, the empirical settings, and the workshop structure. Findings related to the cognitive style profiles of HCI students, and how they performed in analytical and creative tasks are presented next. We then move on to a discussion of these findings in terms of related work in diversity in HCI and in terms of the different institutional, disciplinary and cultural contexts where the workshops took place. The paper concludes with a set of recommendations for HCI educators and paves the way for further research reflecting on the limitations and experiences of this project.

2 Studies of HCI Education, Disciplines and Countries

There are many studies regarding HCI Education around the world [12]. Some of them focus on specific contexts, such as how HCI has been taught in different countries or regions like Asia-Pacific [13], Brazil [14,15,16], China [17], Costa Rica [18], India [19], Mexico [20], Namibia [21], New Zealand [22], Romania [23], South Africa [24], and Sweden [25].

Each of these regions has their own challenges when it comes to HCI education, some of which are reported by the above-mentioned authors. For example, Pribenau and Chisǎliţǎ [23] mention how politics, especially the communist period, has influenced the development of higher education in Romania, and the development of HCI as a field being integrated in different universities. In developing countries in the Asia-Pacific region such as Indonesia, Malaysia and Sri Lanka, HCI is not yet considered an essential subject, but an ‘extra’ [13]. This is also the reality in South Africa, where HCI is not considered part of the core of the curriculum for computing [24]. However, in some countries in the Asia-Pacific region, such as Singapore and Australia the opposite is the case. Sari and Wadhwa [13] also note that the promotion of HCI communities is essential to the growth of the field, using Indonesia as an example.

We can see this also in Brazil: one of the challenges faced by HCI was the lack of books in Portuguese to teach [15]. Results from a survey in this country six years later show how books produced by the Brazilian HCI community have been adopted in most undergraduate courses, showing the empowerment that comes from supporting national or regional communities [14].

In a survey about HCI education with New Zealand students and academics it is possible to see the differences on their interests regarding different topics – while students’ most interested subject was Voice Telephony Interaction, this was the least important topic for academics [22]. This is an example of how communities formed by different groups – i.e., academics, students and practitioners – could increase the exchange of knowledge within a region.

Oestreicher and Gulliksen [25] report on the discussions of a national workshop on HCI education focused on curricula and course differences in Sweden. It is possible to see that even within the same country there is a great variety of courses that range from 3 weeks to 1 year of duration, from 6 to 175 students in class. More important than that, the report shows how this exchange and discussions are important for the community members and invaluable for further developing HCI education further in the country.

More recently, a report from the SIGCHI Education Project including several countries shows how important is for the HCI community to have a place where they can share this knowledge, which led to a proposal to develop a living curriculum [9]. This will be a space where members of the HCI community around the world can interact and share their perspectives and resources. The SIGCHI Education Project found that regional and contextual differences should be taken into consideration in the HCI curriculum. Hence, the curriculum should be flexible and diverse, offering locally relevant content to the students.

While the SIGCHI Education project as well as some of the work from authors referred to in this section point towards a clear need for HCI education and its curriculum to be flexible and sensitive to diverse needs, cultures and perspectives, more empirical research on student experience is needed. For instance, Calderon [18] reports her experiences in Costa Rica providing information not only on the HCI courses being taught, but also on the students that took these courses. Students fell in two categories: on one hand, undergraduate students, mostly full-time, not proficient in English, who know their classmates, enjoy working in groups, and do not generate discussions in class; and on the other hand, postgraduate students, part-time, who work during the day, with good in English due to their work, not very familiar with any of their classmates but who enjoy in-class discussions. This knowledge was then applied to the course design and delivery to make the students feel more comfortable and learn the same topics but in different ways – i.e. while the topics of both courses were very similar, the classes’ format and evaluation were different for both classes. The overall results indicate students were satisfied with nearly 70% feeling they have met the learning outcomes comfortably. However, both cohorts indicated the need for more exposure to the practical elements of usability evaluation and interface design. On a similar approach, Day and Foley [26] report on how changing teaching methods by increasing online lectures to allow more co-located hands-on learning activities had a positive effect in the success rates for an HCI course.

We need more studies like that of Calderon’s [18] or Day and Foley’s [26] reporting on student experience and performance. Our research adds to this body of knowledge in HCI education by analyzing students’ performance, taking their cognitive styles and diverse contexts into consideration. In the next section, we introduce the concept of cognitive style and the motivations for using it as construct to study cultural and learning diversity.

3 Relevant Cognitive Style Research

3.1 Culture and Cognitive Styles

The relation between culture, learning and cognitive styles has been well established by [27, 28]. These authors demonstrate how cultural positions influence holistic and analytic reasoning. Holistic reasoning indicates an orientation to the context or field as a whole, and analytical reasoning indicates a tendency to detach the object from its context in the search of causal relations. Using Hofstede’s four dimensions of Power Distance, Uncertainty Avoidance, Individualism-Collectivism and Masculinity-Femininity [29], and the Theorist/Pragmatist scores of Honey and Mumford’s Learning Style Questionnaire [30], Hayes and Allinson [28] identified two dimensions of learning style, Analysis and Action. Further research in this area led to Allinson and Hayes’ Cognitive Style Index (CSI) [31], a compact questionnaire which is designed to test whether individuals tend more towards an intuitivist (right brain dominant) or analyst (left brain dominant) approach. Table 1 below indicates the score range for CSI.

Table 1. CSI Scores Ranges
Full size table

In recent years the validity of measuring cognitive styles as a single analytical-intuitive dimension has been put to question [32], e.g., for studies of technology adoption and use [33]. Nevertheless, CSI as a construct remains as one of the most widely used measures of academic research in management and education [34, 35]. In response to critiques, Alison and Hayes [36] have demonstrated the validity of their scale through independent tests. Thus, there is already a body of cognitive and learning styles research in higher education developed by an active community of academics [34, 37, 38].

3.2 Cognitive Styles in HCI Tasks

HCI may require different skills and thinking styles for different ends of the HCI professionals’ work continuum between user studies/evaluations and sketching/prototyping/persona creation. Bishop-Clark [39] established early that within computer programming certain cognitive styles and personality dimensions affect some phases but not others. Capretz and co-authors have studied the relations between personality and software engineering for years [6, 40, 41]. In their attempt to map job requirements and skills to personality characteristics, they found that the analysis phase aims to establish users’ needs, clients’ requirements, system features and to provide abstract models of the to-be-designed application. These tasks and the related requirements for ‘soft’ people skills go well with personality traits such as being extrovert (likeable, expressive, talkative) and feeling–oriented (emotion-oriented, warm-hearted and have strong interpersonal skills). In contrast, the design phase aims to prototype, elaborate processing functions, and define input and output; these tasks and the related requirements for strong problem solving and innovative skills go well with strong thinking and intuitive personality traits. Further along the system development phase model, system testers need to pay attention to details and should display strong tendencies to be sensing and judging personalities. For HCI, the takeaway is not the precise correlation between the system development phases that may be defined and how they correlate with specific personality traits, but the idea is that predominantly analytical HCI tasks vs creative tasks may require different thinking styles among HCI professionals. So, if diversity among professionals is good in society, problem solving and in software engineering, it should also be good in HCI.

For analytic HCI evaluation tasks - such as usability evaluation tasks - we know that cognitive style has an impact and may be different across cultures. Clemmensen et al. [42] used the literature on cultural cognition, e.g. [43], to argue that classical think aloud usability evaluation assumes that humans are analytic thinkers who tend to separate different subtasks and states of information processing, but that in real life many users and evaluators - and in particular in Asia - tend to think more holistically.

For creative design tasks – such as creating a persona – there are indications from research with other software professionals, e.g. programmers, that designers have increased tendencies toward introversion and intuition preferences [6].

3.3 Normative CSI Data and Where HCI Should Sit

There is a body of research providing CSI normative data for different domains, compiled by Allison and Hayes [36] and shown in Table 2. This data places computing professionals and graduate psychology students as mainly adaptive with mean scores ranging from 43.34 to 38.01. There is no normative data that specifically relates to designers, but some normative data is provided for employees in the creative arts, who have a mean CSI score of 35.13 [44]. The sample size of this study was comparatively small (38 participants), but this score is substantially lower than the profile of the computing professionals and the psychology students presented in Table 2, suggesting a more intuitive approach.

Table 2. CSI Normative Data
Full size table

There is no data supplied specifically for HCI students, but as the subject of HCI generally has some psychology content, it would not be unreasonable to expect some similarity in the profile of HCI students on computing based courses with psychology students and computing professionals. This means that the expectation is that HCI students would tend to show a predominantly adaptive type of cognitive style.

4 Methodology

4.1 Why Explore the Relation Between CSI and HCI Student Performance

The body of research in HCI education we have presented above provides good descriptions of how HCI teaching occurs in different locations. It is reasonable to infer from this literature that HCI education and learning experiences differ because of local academic and professional cultures. However, it is challenging to visualize the cultural and learning preferences of students and how they shape their performance in learning tasks because their work is usually more private and inaccessible. Our research contributes toward addressing that visibility gap. In addition, there is no systematic research mapping the cognitive styles of HCI students with those of other disciplines and professions. Allinson and Hayes’s CSI provides a parsimonious method to profile cultural and learning diversity in HCI students, as well as a substantial empirical work base based on this construct.

4.2 Methodological Design and Workshop Description

The study included visits to five different groups of undergraduate students in UK, India, Namibia, Mexico and China. They represent a broad set of demographics, economy and cultural conditions [12]. Furthermore, cultural usability research done in these countries suggest that while HCI research, practice, and education are growing everywhere, each country faces some unique HCI problems (e.g., Chinese Input Method Improvement [17], Indian design for low literacy [19], United Mexican States’ need for unique state websites [20], or websites that can maintain cross-border connections) [48] and have different traditions for HCI research, practice and education (e.g., Chinese engineering psychology [17], Indian design schools [19], Mexican computer science [20]). However, none of the research so far has formally looked at correlating cultural and psychological profiles to learning performance in HCI education across a set of countries, universities, and disciplines.

The participant universities were Namibian University of Science and Technology (NUST), the Instituto Tecnológico Autónomo de México (ITAM), the Indian Institute of Technology Guwahati (IITG), the Dalian Maritime University in China (DMU), and University of West London (UWL). All 5 institutions are well known within their country. Within these institutions, HCI was a core subject in NUST and UWL, an option in ITAM, embedded within the whole curriculum in IITG and a core subject in the last semesters in DMU’s Computer Science course. The visit also included meetings with lecturers and staff in charge of curriculum design in which we were able to have an understanding of the educational programs and the local context. The educational programs were also different across the surveyed institutions: digital design in IITG, software engineering in UWL and NUST, business engineering and management in ITAM and computer science in DMU. The groups of students who completed the survey were nationals of the same country, except in Namibia where we had two Angolans and one South African, and the UK, where 10 different nationalities or cultural backgrounds were represented, reflecting the cosmopolitan character of the university.

In each university, between 18 and 22 undergraduate HCI students were asked to complete the CSI survey [31] and engage in a workshop, which included HCI analytic and creative tasks for a science education portal. All participants had a similar level of instruction in HCI, and none had taken similar subjects in previous semesters. The activity given to students contained elements focusing on analytical and holistic thinking, i.e. heuristic evaluation as stimulating analytic thinking and persona creation as stimulating intuitive thinking. The performance of students in the workshop was analyzed and correlated in terms of their cognitive styles and task outputs.

The format of the workshop and data collection was planned in consultation with lead tutors in the five participant universities. The authors and partners in each country made sure the content and activities of workshop were of learning value to all participants. The workshop activities and materials were based on those of a real usability evaluation project in which one of the authors of this paper participated. This was done to secure validity, relevance and comparability with HCI industrial practice. The workshop materials and CSI questionnaires were sent to each course leader a week in advance of each visit. The course leader identified a suitable class and scheduled the workshop at a convenient time. In all cases, students completed the CSI questionnaire before the workshop took place.

As mentioned in Sect. 3.1, the CSI survey is a compact questionnaire to identify peoples’ approach when thinking about problems. It includes a set of statements the students should mark as true, false or uncertain about themselves. This way, for each statement the student should choose answer closest to their own opinion. Examples of statements in the survey are: “I am most effective when my work involves a clear sequence of tasks to be performed”, and “I make decisions and get on with things rather than analyse every last detail”. The results CSI survey [3] were used to situate each student in an analytical-intuitive scale.

The workshop was structured according to the following stages: (a) briefing, explanation of heuristics and personas (45 min); (b) introduction to SEED portal and relevant scenarios; individual heuristic Instrument and persona creation tasks for ‘school teacher’ and ‘student’ (45 min); and (c) debriefing (up to 30 min).

After the briefing (stage a), the students were introduced to a learning node in a science education portal sponsored by Schlumberger Excellence in Education Development (SEED) program and relevant use scenarios. The target audience of this portal was schoolchildren aged between 10 and 18. HCI students participating in our study were required to conduct their tasks by considering this context. The SEED portal supported several language options, allowing students to use their preferred language option and concentrate on the task in question. The following is an example of one of the scenarios used:

“Mary, a 14-year-old girl, is ready to complete their science activity on Porosity. She is sitting together with her teacher who is guiding her on the objectives of this activity, which must be completed in the portal. The particular task she needs to do is to experiment with the Porosity Explorer and then to read the two accompanying pages, Hands-on Porosity Explorer and Porosity Explorer: How we built it and why it behaves the way it does. The teacher guides her to the start point of the task within the portal and then lets her complete the task. Once the activity is completed, Mary and her teacher should feel satisfied that the learning objectives have been accomplished and that the usability of the portal has supported this process fully leading to a positive user experience. As a usability consultant in India you should look at this task in the portal and evaluate the extent to which this portal will be usable by local students and teachers attracting a positive user experience.”

With scenarios like this in mind, each HCI student conducted a heuristic evaluation and created personas [49] individually.

In the heuristic evaluation task, the student should consider a set of five heuristics: Visibility of system status; Match between system and the real world; Consistency and standards; Help users recognize, diagnose and recover from errors; and Recognition rather than recall. For each heuristic, they should mark it as Violated, Not violated or Can’t be assessed. In addition, they should explain their reasoning behind their answer, provide examples, and indicate recommendations for redesign if the heuristic was violated. The richness of observations in each student’s heuristic assessment varied, and that was taken into consideration in our analysis. We quantified this richness according to the scale on Table 3. This way, we labeled with a three an answer such as “Missing error messages when the user tries to fill without the beaker. The volume setting works but fill button doesn’t. Either all should be enabled or none”; and with a one an answer such as “Error messages absent. No guidance if student not able to understand what is to be done”.

Table 3. Scale used to code the level of richness for each heuristic assessment done by students
Full size table

In the persona task, each student was asked to produce a persona for a ‘teacher’ using the portal to support their students, and a persona for a ‘student’ using the portal in the school. Both the ‘teacher’ and the ‘student’ were introduced to the HCI students through the scenarios included in the workshop materials, such as the one presented above. In a similar way to the heuristic evaluation task, the persona descriptions varied in depth. For the persona creation, the number of features added to each persona per students was quantified. The features counted were categorized into physical, skills, context and psychological. For example, a student persona was described as “Has many friends. Good in maths, science. Goes to school in class 8. Watches TV. Plays football.”. The text shows one skill feature and four context features, with a total of five features.

The performance of students in the workshop was analyzed and correlated with the findings for CSI profiles. The analysis centered on the distribution of CSI scores across the five groups. These values would then be used to compare with normative CSI data for related professions, and to explore if there is any correlation between CSI and performance in both tasks. The performance of each group was also compared to test for any difference with an analysis of variance (ANOVA). The next section presents the findings from running these analyses.

5 Results

5.1 CSI Score Profiles and Distribution

A total of one hundred HCI students surveyed across the five institutions completed the CSI survey. The total mean for all students was 42.51 falling clearly in the adaptive band (Table 4). This score falls between normative data for students of psychology and computer science provided in Table 2, which meets our expectations for HCI students as discussed in Sect. 2.

Table 4. CSI Mean Scores
Full size table

CSI mean scores for all groups indicate a predominant adaptive cognitive style, except NUST’s, which sits in the upper end of the quasi-intuitive range. An ANOVA test was performed in order to establish if statistically significant differences exist between the country groups’ means. The resulting test shows that variances across all groups can be compared and no significant difference between CSI profiles for each group was found [F(4,95) = 2.037, p = .095].

While no statistical difference in the means can be found, the percentage distribution of CSI profiles for each country group shows the IITG student group differing from the rest. China, Mexico, Namibia and UK have 70%, 73%, 78% and 79% respectively falling in the categories of quasi-intuitive, adaptive and quasi analyst; however, in the case of the Indian students, 95% fell into this range (Table 5). Possible reasons for this will be discussed in the next section when looking at the context of each student group in more detail.

Table 5. CSI Profile Count and Percentage Distribution
Full size table

5.2 Heuristic Evaluation Performance Analysis

Out of the hundred students surveyed, sixty-one attempted and completed the heuristic evaluation and persona creation tasks. It was not always possible for the students who completed the survey to attend the workshop due to scheduling and personal reasons. From all the students who participated on the workshop, the group with the highest relative participation was China with 32.8% (20 students), followed by India with 23% (14 students), Namibia with 21.3% (13 students), UK with 13.1% (8 students) and Mexico with 9.8% (6 students).

Considering the performance of all groups, CSI scores correlate positively with performance in the heuristics evaluation task [r = 0.281, n = 61, p = 0.028]. This means the higher the analytical cognitive profile for a student, the more likely this student will provide a richer analysis. A similar correlation test was applied to performance in the persona creation task but no significant correlation was found [r = 0.190, n = 53, p = 0.890].

Comparing heuristic evaluation performance for each institution some differences can be observed (Table 6). The Indian country group displayed the richest set of heuristic evaluations.

Table 6. Heuristic Task Performance Means per Country Group
Full size table

An ANOVA reveals a statistically significant difference between group means [F(4,56) = 3.535, p = .012]. The Tukey post hoc test (Table 7) shows the performance of India (IITG) to be significantly different from those of UK(UWL) and Namibia (NUST).

Table 7. Tukey HSD Multiple Comparisons Post Hoc Test
Full size table

5.3 Persona Creation Performance Analysis

There was a fall in the number of completed persona creation tasks (53) with respect to the heuristic evaluation task (61) because some Indian students ran out of time or had to leave the workshop earlier. As with the heuristic evaluation task, country group differences, between performance means in completing this task were also observed. Table 8 shows students from DMU in China with the highest mean for persona richness.

Table 8. Persona Creation Performance Means per Country Group
Full size table

An ANOVA reveals a statistically significant difference between group means [F(4,48) = 4.901, p = .002].. The Tukey post hoc test shows this difference lies mainly between the best and worst performers in the task, namely China (DMU) and Namibia (NUST) (Table 9).

Table 9. Analysis of Variance for Persona Creation Task Performance
Full size table

6 Discussion

Our findings merit discussion in relation to the diverse contexts in which the studies took place. There are implications for the nature of HCI as a distinct field in the context of related disciplines, and implications for the study of cognitive styles in the learning of HCI.

6.1 HCI Cognitive Styles: Between Arts and Computer Science

The overall CSI mean distribution for all students surveyed is 42.51 (SD = 9.13), which falls clearly in the adaptive band. This mean falls below scores for students in computer science, software engineering and psychology, and is more analytical than students of arts courses, as reported in Table 2. This reinforces our expectations that HCI professionals need to be able to draw on a balance of intuitive and analytical skills to perform the best possible solutions for users within a good understanding of the affordances and constraints provided by technological systems. Furthermore, these findings are also in line with ongoing cognitive style research on more than 300 HCI professionals (practitioners, educators and those who are both) where 66% of practitioners are clustered in and around the adaptive bands [50]. This study also reports HCI educators to be more analytical and those who have both roles to be the most intuitive of the three professional groups studied.

Our findings on the cognitive styles of HCI students resonate with those of the above mentioned studies on HCI professionals [50] and students [18, 26]: there is a clear need to develop more skills natural to the adaptive-intuitive range of cognitive styles, which are needed to support situated and creative design practice.

6.2 Cognitive Style Congruence with Analytical and Creative Tasks

We found a positive correlation between CSI scores and the performance of the heuristic evaluation, which is fundamentally an analytical task. This echoes findings by Moore [7] when comparing computer science students’ cognitive styles with their academic performance. This finding indicates that analytical HCI tasks can be sensitive to students with congruent cognitive styles. However, when correlating the persona creation task with student performance we did not find the expected negative correlation where students with lower CSI scores, i.e. more intuitive, would be more likely to produce richer personas. In retrospective, in the schools participating in our research with educational programs with a strong analytical orientation and where the relation between tutor and students is very hierarchical, our assessment of the persona creation task might not necessarily have measured HCI students’ level of empathy with the user types presented in the scenarios, but measured the compliance with the tutors’ instructions on the analytic and systematic application of the persona method. This was the case with DMU students as discussed below.

While these correlations are mainly indicative and cognitive styles only explain a percentage of the overall variance between students in their learning, e.g. [51], our results pave the way for more systematic studies of HCI curriculum adoption and development sensitive to diverse cognitive and learning styles.

6.3 Supporting the Case for Quasi-Intuitive and Adaptive Styles When Comparing Task Performance

We consider the following facts in discussing the differences between Namibia (NUST) and China (DMU) found in the ANOVA for the persona creation task. The Namibian group has the lowest CSI mean score, in the upper end of the quasi-intuitive band, and is the only group not in the adaptive band (Table 4), though close to it. This group produced the lowest number of features in the personas they created. In contrast, the Chinese group were the most analytical group, with a CSI score in the upper range of adaptive, and the best performers in the persona creation tasks, i.e. they added more features to their personas. Therefore, following the reflection from the previous section about the nature of the persona creation task, we could see how the group with the more intuitive profile, i.e. the lowest CSI score, might not explicitly include as many features to the persona descriptions due to the high-context nature of the group, which corresponds with the generally collectivistic cultural traits for sub-Saharan Africans documented in cultural [52] as well as HCI research for that region [53,54,55]

In discussing differences in the performance of the HCI heuristic evaluation task, there is also an interesting observation. As indicated in Table 6, 95% of students from India (IITG) were situated in the three central bands (quasi-analytic, adaptive, quasi-intuitive) of the CSI scale, with a mean CSI of 42.14 and the lowest standard deviation (SD = 7.08) of all the groups. One possible reason for the high concentration of profiles around the adaptive band could be due to the unique nature of their educational program. IITG has both a Department of Computer Science and Engineering and a Department of Design. The students who took part in these workshops were design students, but were admitted thorough a competitive process consisting of an admission test based on maths and algorithms. This filter is likely to admit analytical students with an aptitude for design, which in CSI terms means adaptive students. Interestingly, the highest score for heuristic evaluation richness belongs to this group. This theoretically suggests adaptive students are better prepared to engage in usability evaluations with a balance of analytic and intuitive skills.

As an additional validity check we compared the CSI profile count distribution for only those students who completed the design and evaluation tasks and the same proportions were maintained: the Indian group showed the highest concentration in the central bands (quasi-intuitive, adaptive and quasi analytical) with 92.9%; the Namibian was group was the lowest cluster of CSI scores with 46.2% in the intuitive and quasi-intuitive bands; the Chinese group retained the highest concentration of students with CSI scores in the top bands (quasi-analytical and analytical) with 45%.

6.4 Institutional Cultures and the Perceived Value of HCI in the Curriculum

Institutional cultures not only influenced student performance in the workshops, but also their level of participation. China (DMU) and India (IITG) had the highest workshop attendance and completion rate where almost all students who filled the CSI survey attempted and finished the tasks. In these institutions students carefully followed instructions from their tutors and the process of booking classrooms and fitting the workshops into their timetable went smoothly. Organizing the workshops in ITAM was more challenging. ITAM is a highly ranked private technological university in Mexico. Their institutional discourse takes pride in a tradition of forming elites that will manage and lead the country reinforcing a culture of individualism, independence and competitiveness, giving students more control of their study time and choices. This made it difficult to timetable and run the workshops in the same space and time slots, which translated into lower participation when compared to the other country groups. UK (UWL) was also challenging as students are also seen as more independent, with those taking the HCI module were coming from different educational programs, each of which had different patterns of attendance.

The role of HCI as a subject in the curriculum of the five institutions is an important factor to consider. While HCI is core in both IITG and DMU programs, the Indian students seemed more intrinsically motivated and proactive to engage in the tasks than Chinese students, who were mainly reactive and followed the instructions of their tutors. While we cannot establish or isolate the main reasons for this difference, the fact that IITG students were in a design school and DMU students were in an engineering school could be a contributing factor. For the UK (UWL) students, HCI is a core module shared in most of the information technology degree courses, with the students from different nationalities ranging from those with a business specialism to those on more programming focused programs, which may explain the more even spread of the CSI profiles and workshop performance. ITAM students in Mexico took HCI as an option in the later stages of their business educational program, and the value of the HCI subject in their education was not always immediately clear as expressed in comment during the workshops. In London, UWL students in the more technical programs made similar comments about not seeing how HCI can help them ‘build software programs’. This is an important observation where institutions with a techno-centric culture will present bigger challenges to the adoption of HCI in their programs, which coincides with findings from a recent case study of HCI education in Egypt [56].

6.5 Implications for HCI Education Curriculum Development and Delivery

Our findings suggest that HCI education face two major challenges. One is supporting the cognitive styles necessary for learning and doing the creative tasks in HCI. The other is supporting the steadily increasing diversity in delivery, location, and face value of HCI education curriculum.

First, our findings show that HCI students are less analytical than CS but more than art students, and that there is a clear advantage of being adaptive-analytical in the current HCI curricula. This suggests that we should perhaps promote tolerance towards both analytical and creative styles within class and within the individual HCI students. We should think of HCI students as creative designers who should be encouraged to be intuitive and thinking like the programmers studies by Capretz and others, e.g. [6, 39, 40]. It is less clear, however, how to support intuitive thinking in HCI students, or how to make students become more adaptive rather than pure analytical.

One way we observed this could be done is by recruitment. In India the national admission system and technological universities favour and recruit clearly analytical thinkers and then put them into a clearly design oriented study program that requires intuitive thinking [4]. This could also be done the other way around, recruiting mainly intuitive thinkers to more analytical HCI study programs, e.g. more evaluation oriented study programs. However, while we as teachers of HCI perhaps tend to teach the analytical curriculum, or we do it with an analytical approach, there is a clear need to develop a dedicated curriculum for intuitive-adaptive students, and to support the student-as-designer’s cognitive style in HCI education.

Second, we found many differences in the delivery of HCI education between institutions, disciplines, and countries that we found in this study. Such results imply that we should support diversity in delivery, location, and value of HCI curricula. HCI literature already has good awareness and acceptance of such diversity, e.g. pointing to the challenges faced by HCI in Brazil of lacking good HCI books in Portuguese to teach [15], or focusing on emic perspectives in the global south [57]. As pointed out by other authors, e.g. [13], the promotion of HCI communities is really important to the growth of the field. We will add that the promotion of the HCI community should be sensitive to and support diversity in the development of HCI communities, while at the same time keeping the international outlook in the HCI curricula.

Overall, this study helps to increase awareness of diversity in cognitive style and institutions and countries. Such awareness can be used not necessarily to align with students’ cognitive styles, but rather to support and inform student study strategies, making the teacher aware of typical student outcome in HCI courses, and in general support of more metacommunication within and around HCI education.

7 Conclusions

The overall aim of this study is to explore the interactions of the HCI curriculum with students in different types of institutions and disciplines internationally. In doing this, we promote and contribute to an understanding of diversity in HCI education, which is crucial given the main mission of this discipline is to support human interactions from diverse backgrounds with technology. We have focused on the relation between cognitive styles as a cultural and learning construct [28], and how undergraduate students of HCI engage with analytical and creative tasks. Our findings have implications not only for HCI education but also for wider professional field. The findings also contribute to the field of cognitive styles by providing normative data for a hundred students of HCI.

We were able to empirically confirm our expectation that HCI students tend to display adaptive cognitive styles. Compared with normative data for cognitive styles in other disciplines and professions, HCI can be seen as an adaptive field between those of psychologists, and engineers and computer scientists, and above the more intuitive field of art and design. There is also an observed correlation between cognitive styles and performance in heuristic evaluation. Statistically significant differences were also encountered in how groups performed the evaluation and creative tasks.

At a higher level, there are also interesting observations about how student engagement with the workshops was shaped by institutional, disciplinary and national factors. While these factors cannot be empirically isolated in this exploratory study, there are interesting convergences between CSI profiles, cultural behaviors and task performance. For example, we have seen how the group with the highest concentration of adaptive students, delivered the richest set of evaluations; or how the group with the highest concentration of intuitive and quasi-intuitive students produced personas with lowest number of features, fulfilling a cultural high-context expectation for Namibia. We can also see how the relative position and value of HCI in the curricula of the studied universities corresponded with the level of student participation at the workshop stage.

7.1 Key Insights for the Nature of HCI Tasks and Education

What can this mean in practice for teaching HCI in the Global South, and/or introducing HCI as a new subject in strongly technical, design or business driven institutions? The key insights for the nature of HCI tasks and education have already been discussed above, but in short form the takeaways are that we as HCI educators should:

  • Develop a dedicated curriculum for intuitive-adaptive students, and which supports the student-as-designer’s cognitive style in HCI education;

  • Support diversity in delivery, location, and value of HCI curricula, including HCI in different types of school around the world, e.g. business, design, or computer science;

  • Promote the HCI communities with a focus on local institutions, disciplines, and students’ needs, while at the same time keeping the international outlook of HCI;

  • Increase our awareness of diversity in students’ cognitive style in various institutions and countries; and

  • Support more metacommunication within and around HCI education.

This study is one small contribution to enable metacommunication about HCI education.

7.2 Limitations and Further Work

The limitations of this study are plenty since HCI education hardly is a stable easily studied topic. However, we feel that by focusing on well-defined constructs of cognitive styles and measuring these, and then placing the findings in rich and detailed descriptions of the educational, disciplinary and country contexts, we provided a contribution to the HCI community’s knowledge about HCI education. Despite this, and given the value of what we found, we acknowledge that a more balanced and systematic approach combining qualitative and quantitative methods is needed for to the study of culture and context in HCI education. The time and resource constraints under which this project was conducted made it challenging to follow such an approach.

In this study we have highlighted the need for, on one hand, more controlled and extensive studies to establish relation between cognitive styles (and other constructs such as cultural attitudes) and student performance and experience; and, on the other hand, studies aimed at sharing of practice and experiences across different institutional and disciplinary context to understand the opportunities and constraints to develop a more diverse HCI curriculum as suggested by Churchill et al. [9]. An obvious next step is study progression in HCI curricula – some courses should be basic and some advanced – to see how progression and diversity in HCI education can get together across institutions, disciplines and countries. We are also currently engaged in ongoing research focusing on the cognitive styles and preferences of HCI professionals, including practitioners and educators to have a better understanding of the gaps between education and practice [50].