Keywords

1 Introduction

Having a user interface that provides adequate usability and user experience has become a critical factor for success in information systems [1]. To achieve this, a series of best practices and empirical rules are often followed during interface design. These rules, however, seldom take in consideration cultural differences between users [2], and research across several countries and different applications shows that these cultural differences do affect user experience, and often describes how the experience is affected country [3, 4].

In the context of Brazil, on the other hand, we could find no research showing whether or how cultural differences affect user experience. Brazil is a large country, with continental dimensions and a large market of Internet users that is growing very quickly. The country is often conceptually divided in five socio-economical macro-regions, and past research shows that there are indeed significant cultural differences between these five regions [5].

The North is the region with the lowest population density and has an economy based mostly on vegetable, and mineral extraction [6], with one notable exception in Manaus which shows large industrial activity due to fiscal incentives.

The Northeast has approximately 30 % of the country’s population and has an economy based on agribusiness, particularly sugar and cocoa. On the coast there is significant oil extraction and the beaches are also focus of tourism [6].

The Midwest is home to the nation’s capital. The economy has been based on gold and diamonds mining but is currently based in livestock [6].

The Southeast occupies only 10 % of the territory but is home to 40 % of the population, being the region with the highest urban population. The region has a developed and industrialized economy, accounting for 50 % of production in the country [6].

The South is the region with the smallest area in the country (6 %). The economy is based on agriculture and industry and has received a new industrial park in recent years [6].

Systems that observes these differences have a competitive advantage [4] and could offer a better user experience, increasing the user satisfaction [7]. Translations and layout reformatting are not enough to resolve the impact of cultural differences in user experience, it is necessary to reevaluate the design completely [8, 9] and to contemplate the subjective culture of each user [2].

Our goal, then, is to investigate whether these cultural differences between the five socio-economic regions of Brazil are deep enough to affect user experience in web-based interfaces and, if so, how.

This study may indicate that Brazilian companies should prepare their information systems taking these differences into account, since this is an important challenge in a globalized society, where the companies need to keep the right balance between internationalization and localization [10]. In future works, other types of platforms and systems should also be tested, but here we focus in web-based interfaces, particularly for Internet Banking.

“Culture” can be considered one of the most complex words that exist. Initially, its was a material process derivative of agricultural cultivation. But it evolved to a term that denote features “of the soul”, such as moral or intellectual characteristics. The definition of culture shows a profound historic transition, from rural to urban, from strict definitions to wider definitions, and other philosophical questions [11]. Culture can be defined as a collective programming of the mind distinguishing the members of one group or category of people from others [12] and this is possibly the definition used most frequently in HCI [10].

In 1970, Hofstede was invited to run a study to understand why some of IBM’s organizational rules of IBM succeeded in some countries but failed in others. By designing a questionnaire for this purpose and analyzing its answers, Hofstede pro-posed that there are cultural dimensions that differ between countries and created a way to measure and to represent these cultural difference: the Values Survey Module (VSM) [12].

Some authors contradict Hofstede, claiming that his methodology is not correct in an anthropological context [13]. But the concept of culture has been explored in many other points of view and, is a complex concept that does have definitions prior to Hofstede’s that agree with his view [14].

Usability is one of the aspects of a system that affects user experience and, just like this experience, may in turn be affected by these cultural differences. It is defined by ISO as the set of characteristics such as appropriateness, recognizability, learnability, operability, user error protection, user interface aesthetics and accessibility [15]. Usability analysis has always served as a parameter for creating good products, but in these days of extreme competition and greater consumer awareness, it is no longer considered sufficient and it is necessary to also develop and study a user experience (UX) [16].

There are many definitions of UX [17], but one of the most accepted definitions is a person’s perceptions and responses that result from using a product, system or service [18]. Therefore, UX involves usability, but also includes other user feelings, instead taking only product design in consideration [16], and one of the most basic reasons to offer a great UX is that enjoyment is fundamental to many aspects of life [19].

In this work we made use primarily of questionnaires and interviews. But before saying more about the tests, we discuss cultural differences and user experience.

2 Related Work

To evaluate whether cultural differences influence the UX, the most used method was the application and evaluation of questionnaires. As mentioned above questionnaires play a key role in UX evaluation [20] and, in the revised papers, they were used to identify the importance of UX attributes [21–26] and to detail the user experience [27–36]. In second place, Task Performance Measurements were used in some studies [4, 27, 29, 30, 32–34, 37] measuring the rate of success in tasks and the time to finish them. The questionnaire method and task method can evaluate the UX in two distinct moments, during use and afterward, and were combined in several works [27, 29, 30, 32–34]. Three studies conducted only bibliographic analysis [38–40].

Out of the 23, only three studies could not very the impact of cultural differences in UX. All other 20 studies verified and showed differences between users from distinct cultures. The papers that reported a positive result capture what were the main values in each culture, measured user experience or usability and performance on tasks, and then compared results between populations with different cultural roots preserved. On the other hand, two the articles that report a negative results showed problems on the methodology, either using children (who are often considered not to have their culture fully assimilated yet) as the population [36] or failing to measure whether there were actual cultural differences between participants [27]. The last study with negative results [30] says that, while they did not find sufficient evidence to conclude that the tested cultural dimensions affected performance, the performance levels by participants attained suggested that the usability of the interfaces was increased for all users as a result of accommodating high uncertainty avoidance, masculinity, collectivism and high power distance characteristics into the design of the interfaces.

When the interface was culturally adapted in two studies, they reported improvements in performance [4, 37], therefore showing not only a difference in user experience but a real, measurable improvement when users interacts with an interface well-suited to their culture. On both quantitative and qualitative ways, it seems clear that culture influences UX. Users of different cultures value different attributes, react differently to interfaces and have different performances in performing tasks in an identical interface.

EUA and China were the most analyzed countries in these studies. Several countries in Europe and some others in Asia were also analyzed, but Brazil was not contemplated in any study.

3 The Experiment

Five state capitals were chosen to represent their macro-regions: (1) Belém in Para representing the North; (2) Salvador in Bahia representing the Northeast; (3) Goiânia in Goiás representing the Midwest; (4) Sao Paulo in Sao Paulo representing the Southeast; and (5) Porto Alegre in Rio Grande do Sul representing the South.

Our goal was to have at least 20 users from each of these cities, for a total of 100 participants.

Instead of testing for a large number of different web pages or types of pages, we restricted our tests to the specific application of Internet Banking, which is widely used in all regions and complex enough to exhibit more details about the user experience during testing. While other applications, such as e-commerce, are just as popular, or even more so, and just as complex, one of the authors has previous experience with user interfaces for Internet Banking and we decided to take advantage of this experience.

3.1 Population

In order to reduce the influence of other cultural and socio -economics differences, we choose banking employees of Itaú Unibanco Bank aged 18 to 40. In this way, we could guarantee that the population has very similar characteristics with respect to salary range, educational level, educational area, work activities and banking knowledge. Employees of other banks could also present these characteristics, but Itaú was chosen due to the easy access of one of the authors.

Another key requirement was that participants had always lived in the same mac-ro-region they were born, to reduce the chances that the culture from another macro-region could have affected the participant. Our choice of study population brought two risks to the validity of the experiment. First, that the culture in the chosen state capitals was not representative of the entire region and, second, that our particular choice of Itaú Bank employees could constitute a subculture in itself, with more in common among its members due to this common factor than differences due to regional culture.

3.2 Procedure

One author traveled to all five macro-regions of Brazil, and asked for volunteers in bank branches during working hours. The participants were informed about the study’s objectives and the entire procedure, particulary its voluntary character, and then invited to read, fill in and sign Terms of Consent.

The test starts with a warm-up chat, during which the user’s basic information such as age, level of education and hometown were elicited. Then the user fills out Hof-stede’s standardized Values Survey Mode (VSM) questionnaire. Users then perform five tasks in our fictitious Internet Banking System, using a notebook computer (log-ging in and out, checking their account balance, making a payment, a transfer be-tween accounts, and requesting a loan). Finally, after completing the tasks, users fill out a QUIS questionnaire, evaluating several aspects of the interaction, and are inter-viewed, making comments about their experience and preferences. The average time for this entire procedure was approximately 15 min for each user.

3.3 The System

Instead of using an existing banking system, or a mockup of an existing system, which would introduce another variable in the tests (the user’s previous experience with the particular system chosen), we opted to create a fictitious and slightly simplified Internet banking system, called NOVbank. We validated its user interface with heuristic evaluation from other experts in the area that work specifically with the banking application.

Figure 1 illustrates its homepage. All figures showing the banking system show text in Portuguese, since this was the language used for the experiment.

Fig. 1.
figure 1

NOVbank Internet Bank - Home Page

Full size image

After performing the login, the logged-in homepage was displayed. The user can be easily consult account balance, bank statement and manager information (as shown Fig. 2). On the top of the page, the user had a menu with all transaction options: bank statement, savings, payments, transfers, loans, investments, vouchers and contact.

Fig. 2.
figure 2

NOVbank Internet Bank - Logged Home Page

Full size image

All transactions were made up of three steps: filling in data, where users fills the necessary information and chooses options; confirmation, where user confirms all information; and voucher, where the system confirms the transaction.

3.4 Questionnaires

Besides using the VSM to elicit cultural differences, we used the Questionnaire for User Interaction Satisfaction (QUIS). QUIS is a tool developed in the Human-Computer Interaction Lab (HCIL) at the University of Maryland at College Park. It evaluates users’ subjective satisfaction with specific aspects of the human-computer interface [41].

This questionnaire contains 133 questions divided into 12 categories. The two first categories evaluate user knowledge in using computers and the system. In other categories users must evaluate the system with a score (1-9, lowest being negative and highest positive).

To reduce the time of each experiment and focus on issues related to our particular system, we removed some QUIS categories with questions about features not present in the system: past user experience, manuals, tutorials, teleconferencing and software installation.

4 Results

On average 22 volunteers participated the study by region, with more female participants than male, except in São Paulo (see Fig. 3).

Fig. 3.
figure 3

Number of participants by sex

Full size image

The average age of users was 30 years old. All were university graduates and 97 % from humanities courses. 95 % use Internet Banking and, for 71 %, that is the main channel to bank.

VSM results show that there were indeed many cultural differences between participants from these capitals in different Brazilian macro-regions, as seen in Table 1.

Table 1. VSM results
Full size table

PDI results showed that São Paulo and Porto Alegre, with high scores, have a greater acceptance of hierarchy and social inequalities. Belém and Salvador, on the other hand, had a low scores. It is possible see is a pattern in the regions, with the score growing towards the South, while the Midwest had an intermediate score.

IDF results showed São Paulo again with the highest score, which represents a more individualistic culture. Unlike happened with PDI, we did not find a simple geographical pattern to IDV: Salvador and Goiânia showed an intermediate score and Belém and Porto Alegre had a minimal score.

Except for Salvador, MAS results were similar to ITV. This correlation is common, with collectivist societies tending to be collaborative societies. Salvador showed a different result that represents a society with high scores of both collectivism and competition.

UAI results showed Porto Alegre with the highest score, indicates a culture with high uncertainty avoidance that thus accepts more rules, laws and bureaucracy. Salvador and São Paulo had similar scores, and Belém and Goiânia showed low uncertainty avoidance.

Porto Alegre showed a high score on LTO too. This correlation is natural because uncertainty avoidance encourages the society to make long time plans. Goiânia, however, despite having shown a low level of uncertainty avoidance, showed the highest score on long-term orientation.

All five Brazilians macro-regions showed high scores on IVR, showing that Brazilians in general assign a high value to quality of life, especially in São Paulo, which showed the higher score.

These results indicate that the Brazil’s macro-regions, and particularly our experimental population within these regions, do indeed have significant cultural differences.

Figure 5 shows the average times for completion of all tasks, in minutes, for each macro-region. All participants concluded all tasks successfully, but with different time performances (see Fig. 4).

Fig. 4.
figure 4

Time performance on tasks

Full size image

Participants in Belém took the highest mean time to conclude all tasks, with 7.4 min, while São Paulo and Porto Alegre had the lowest with 6 min. In addition to performance differences, the observer noticed that 30 % of users preferred consulting the bank statement from the home page in Goiânia and Porto Alegre, while only 20 % did so in São Paulo and 10 % in Belém and Salvador.

Other noticeable difference was that approximately 60 % of users from Belém and Salvador checked their bank balance before paying a bill during the experiment, which avoided the display of the error message “out of balance”. In Goiânia and Porto Alegre this rate was approximately 50 % and in São Paulo only 20 %, possibly due to the speed in the execution of tasks.

QUIS’ results were analyzed question by question, but are shown in Fig. 5 grouped in 6 categories: Reaction, Screens, Terminology, Learning, Performance and Media. While analyzing individual questions, we noticed a clear pattern. In nearly all questions, Porto Alegre and Goiânia users assigned the highest average scores, Salvador assigned somewhat lower scores, and São Paulo and Belém the lowest, with small but statistically significant differences. We assumed that this variation was either a function of the context of experiment in each city (in Belém, for instance, the experiment coincided with difficulties in the computer system used by the participants at work) or with cultural differences that were of little interest to us in this particular experiment, since they would apply across the board, not to different aspects of the user experience. What we did, then, was to have a closer look at those questions which violated this pattern (for instance, with São Paulo assigning the highest average score), and then correlated the answers to these QUIS questions with certain qualitative comments made during the interviews.

Fig. 5.
figure 5

QUIS’ Results

Full size image

The question “Does the system have enough features?” showed a violation of the pattern in Salvador, getting the lowest average rating there. Many users in Salvador wanted additional features, such as masks in form fields (i.e. automatically showing dots, commas or dashes in account and ID numbers, monetary values etc.) or the option to use a bar code reader when making payments. Users in other regions in general did not complain about a lack of features, even though in São Paulo some users gave suggestions for more features, such as search for operations and highlighting certain information.

Salvador also broke with the pattern in the question “Does the error messages clarify the problem?”, getting a low score. As mentioned above, 60 % of Salvador’s user checked the balance before making a payment and, because of this, did not see any error messages (this was the only error likely to occur during the test).

In the question “Is the organization of elements on the screen is useful or useless?”, Porto Alegre’s score was well above average. 70 % of the users in that city said during the interviews that they approved of the system’s layout. In addition, users in the South praised that balance information and manager contact information showed right on the home page.

In questions related to security, the pattern was maintained but all scores were lower. Many users in all regions said they believed the system should require an additional security check, since the system required only one password and they were all used to having to enter at least two passwords (one to log in, one to confirm most operations).

Regarding design matters, Belém and Porto Alegre had completely opposite scores. Porto Alegre’s users considered the design adequate, with clear colors and basic information, while Belém’s users wanted more and vibrant colors, larger and rounded fonts and more information presented on the screen. This phenomenon also occurs with nearby macro-regions, Salvador had a score and opinions similar to Belém and Goiânia and São Paulo was similar to Porto Alegre. This was the aspect that, in this experiment, showed the greatest impact of the design on user experience. It is also interesting to notice that the correlation between wanting more information and more colors makes some sense (with more information shown all at once, more colors might help in telling the information apart).

5 Conclusion

Our goal was to investigate whether these cultural differences between the five Brazilian regions are deep enough to affect user experience in web-based interfaces. Our results showed that, even in a somewhat homogenous group (in our case, bank workers), the cultural differences between regions are notable. These differences not only existed, but can also be correlated with differences in user experience when using the same information system. The five Brazilians macro-regions had very different VSM results and showed differences in task performance, when evaluating the aspects of the system with QUIS and when voicing their opinions and suggestions in the inter-views. The most important difference between regions was regarding the amount of information and colors shown in the interface, with North, Northwest and Midwest favoring more of both and Southwest and South preferring less. Users in the North-east were also affected more negatively by the lack of certain optional features.

Our work suggests several interesting possibilities for further investigation. The first one is to analyze others platforms, such as mobile, to see if and what other important patterns emerge. Other types of system, such as e-commerce systems, are also of interest for further analyses, especially to compare those results with ours and see if the patterns we found extend to such systems (we believe they do, since the questions and the differences we noted are more general, not specific to Internet Banking like the problem with security and two checks was, but this belief should be investigated). Other sources of cultural differences, such as age, genre and income, and how they affect user experience could also be investigated. Finally, even for web-based inter-faces and Internet Banking in particular, these differences could be further investigated, either analyzing aspect by aspect in more detail or measuring performance and satisfaction when using culturally adapted interfaces (for instance, one with more colors and information presented and one with less) across different regions.

Our culture is a large part of who we are and it must be represented in the products we buy, the services we consume and the experiences we enjoy. We believe that, with this work, we have collaborated with the understanding of cultural differences in Brazil and that, with this understanding, we can begin making better, more pleasant systems and interfaces to our users.