Keywords

1 Introduction

Today, the need for usable website design is taken for granted. At the same time the individual, and as a result the internet user, demonstrates an increasing preference for the aesthetically pleasing and not the merely functional; a need which is steadily being recognized by the usability specialists. The paper’s scope is to investigate the relation between website usability and aesthetics. The case of architectural websites was initially chosen, since their owners and their users, who are considered design experts, have quite different aesthetic responses from those of typical web users. After going through the existing literature, where reference to the tradition of Aesthetics is rather restricted, five heuristic rules relating to basic aesthetic parameters are proposed and documented hereafter. The heuristics are based on existing aesthetic theories that have been around and validated for years and also can be used as guidelines during the website GUI design process. In this study, two methods were used to evaluate three indicatively chosen architectural websites: the first used heuristic evaluation that also included the five new heuristic rules and the second two questionnaires, one for general usability evaluation (SUS) and the other for evaluating website attractiveness (AttrakDiff).

The analysis of the results indicates that when the design expert user cannot identify with the aesthetics of the website, this adversely affects negatively the perceived usability (which can be considered as a hind towards “what is not beautiful is not usable”). Moreover, when the design expert user cannot identify with the aesthetics of the website, and also he/she is intensely dissatisfied by its perceived usability, this eventually leads to a negative image of the overall impression of the website. Also, the more aesthetically sensitive and critical is the eye of the user the more demands it imposes on perceived images and the visual perception of any object and that directly affects in a catalytic manner the perceived website usability.

The rest of the paper is structured as follows: the 2nd paragraph provides an introduction to the relation between aesthetics and usability and discusses the first methodological step, which refers to the architectural websites survey conducted. The 3rd paragraph presents the proposed aesthetic heuristics and the performed heuristic evaluation extended with the new rules. Paragraph 4 presents the evaluation of the three architectural websites by the architects, using the SUS and the AttrakDiff questionnaires and summarizes the results, while paragraph 5 draws the main conclusions and discusses directions of future work.

2 Aesthetics and Perceived Usability

The term ‘Aesthetics’ addresses the overall perception of beauty or the perception of what is beauty by a single person or a group of people. Respectively, something is characterized as aesthetic when it is associated with the senses and our perception through them.

Aesthetics are mainly expressed through art and design, with design ranging from designing a city to designing the smallest everyday object. When we are working on web design, the role of aesthetics is crucial for the way a webpage is perceived. The human mind only needs 50 ms to create positive or negative impressions for a website. This is an aesthetic decision, which can influence our judgment for the whole website [1]. The conflict between form and function has long been observed in architecture and design. On the one hand, emphasizing on function stresses the importance of the object’s usability and usefulness but on the other hand, emphasis on form indulges mostly the aesthetic and social needs of designers and users.

The basic principles of Human Computer Interaction emphasize in effectiveness rather than aesthetics. Generally, the criterion of aesthetic design is an integral part of the effective interaction design. The pursuit for the prominence of usability in the field of Human Computer Interaction sometimes reaches to a point that the measurement of a system’s usability, determines the success in a GUI design [2]. The concepts of aesthetics and usability represent in a way two vertical dimensions of Human Computer Interaction. The aesthetics refers mostly to non-quantifiable, subjective and affected-based user system experience. On the other hand usability is usually measured by rather objective means and sets effectiveness as the most important criterion.

The marginalization of the aesthetic aspects of HCI is not a constructive position for various reasons. Firstly, it reveals a gap between the research objectives of HCI and the practice of many computer related industries on the topic of design criteria. Secondly, it ignores important needs of computer users, who are also consumers of other products and services and might also evaluate aesthetics and contemporary design in addition to usability. Thirdly studies have shown that “the aesthetic perceptions of an interface are highly correlated with perceptions of the interface’s ease of use” [2], or in other words, “what is beautiful is usable”.

2.1 Architectural Websites Survey

Websites of architectural interest combine features of various website types. They aim at attracting new customers and maintaining existing ones, while posing as a kind of portfolio or monography. For the first objective it is important that these sites comply with usability principles but for the second, they must prioritize high aesthetics and free expression of creativity.

Architectural website types include:

  • websites of architects or architectural offices

  • websites of architectural case studies or architectural contests

  • news websites with content related to architecture and design

This study focuses exclusively on the first category of websites as the main interest is set on examining the relation between aesthetics and usability in websites with users that have high demands on both features.

The first methodological step was to conduct a survey. For this purpose the Usability and Aesthetics Questionnaire (UAQ) was designed and distributed to owners of architectural websites. The UAQ comprised 26 questions grouped into five sections namely: General website info, About requirements, About usability, Usability evaluation, About Aesthetics and usability. The purpose of the questionnaire was to collect general information about the website and the way it was designed, the expected and actual users, the degree to which usability principles were applied, whether the website had been evaluated for its usability and how, the degree to which aesthetics affected the design of the website and whether aesthetics or usability had to be compromised (one to the expense of the other). For this investigation 300 websites of international renowned architects were recorded. All these websites were chosen on the basis of interesting design. A questionnaire was dispatched to 285 of the websites (15 websites did not provide a way of contacting the owner) and response was received by 55 of them. 51 % of the responses were from architects with headquarters in Europe, 13 % in Asian countries, 31 % in countries of North and South America and 5 % in Australia). The questionnaires were distributed in the period between January and February 2011.

Responses can be summarized as follows: The primary reasons for constructing and maintaining a website is attracting new customers and creating a recognizable profile in the architectural community. When designing the website the anticipated visitors were the potential new customers and the existing ones, as well as architects and designers. In practice though the majority of visitors are architects and designers followed by students in the respective fields. In all cases the architect or the group of architects had an active participation in the design process of the website. One of the most complex and interesting features of architectural websites is the presence of the architectural stigma in the website design. Also, as expected, the role of aesthetics in the design of these websites is extremely important.

3 Heuristic Evaluation Extended with Aesthetic Heuristics

3.1 Aesthetic Heuristics

Heuristic evaluation was performed by five experienced usability evaluators a number that according to [3] is adequate number for identifying around 75 % of usability problems. Initially evaluators used the typical set of Nielsen’s heuristics:

  1. 1.

    Visibility of system status

  2. 2.

    Match between system and the real world

  3. 3.

    User control and freedom

  4. 4.

    Consistency and standards

  5. 5.

    Error prevention

  6. 6.

    Recognition rather than recall

  7. 7.

    Flexibility and efficiency of use

  8. 8.

    Aesthetic and minimalist design

  9. 9.

    Help users recognize, diagnose, and recover from errors

  10. 10.

    Help and documentation

Moreover evaluators were reminded of the eight more important website usability problems as identified by Nielsen and Lorangel [4] that still occur even today in many websites, despite the extensive and long use the internet. Evaluators were asked to classify those problems under the respective listed heuristic.

Briefly these commonly observed problems comprise:

  • Visited links that do not change color

  • Unavailability of the Back button

  • Links that open new browser windows

  • Pop-up windows

  • Design elements that look like advertisements

  • Violation of web conventions

  • Vaporous content and empty hype

  • Dense content and text that cannot be scanned by the eye

In addition, evaluators also used a newly introduced set of aesthetic heuristics defined as follows (their numbering continues from the core 10 heuristic rules of Nielsen):

  1. 11.

    Successful use of colors: it is a very important parameter of the overall impression and attractiveness of a website

  2. 12.

    Symmetry: it is connected to beauty and is a characteristic present essentially in every form in nature. Three basic forms of symmetry are distinguished, namely reflection symmetry, rotation symmetry and translation symmetry. Apart from its aesthetic value symmetrical forms poses some qualities that are important for design purposes, such as their ease of recognizing and recalling.

  3. 13.

    Harmonic layout: the layout of a webpage is the composition (arrangement) of its elements and the harmony of this composition refers to ratios. The golden ratio is the analogy observed quite often in nature and is considered the ideal of beauty and harmony. The omnipresent Fibonacci sequence is another example of high aesthetics, used in art, music and architecture. Using a grid for the synthesis and alignment of the elements of a composition creates a sense of order in the design.

  4. 14.

    Balance: it is of great importance in the design and the aesthetic impression of a website. Balance in a composition is analogous to balance of physical elements. A composition is balanced when the visual weight of design elements in each side in almost the same.

  5. 15.

    Grouping and hierarchy: using visual qualities such as shape, size, position and color, even hue or saturation, the partial elements of a composition can be grouped or hierarchically organized.

3.2 Architectural Web Sites and Scenarios

Two methods were combined for the usability and aesthetics evaluation of the architectural websites: (a) heuristic evaluation by experts and (b) user questionnaires. Users in this study were architects, as they are the majority of those visiting the websites of this type and due to their attitude towards aesthetics.

Both methods were applied for the evaluation referred of three selected architectural websitesFootnote 1 owned by international award winning architectural firms (i.e. their studies or projects have excelled in architectural competitions, or their architects have received honors in the field of architecture). Also the selected three websites were among those that responded to the UAQ, so as to have a view of their initial attitude towards usability and a picture of their users and user needs regarding the website.

For the heuristic evaluation of the websites the experts were advised to use a full-screen browser window. They were given 3 equivalent navigation scenarios, one for each website corresponding to typical tasks visitors undertake when navigating. The aim was (by accessing various content categories and pages and while searching for specific information) to evaluate whether the website serves the needs of its indented users, as these were stated in the UAQ by the website owner. Evaluators were first asked to navigate freely through the websites and then to locate information on specific projects based on different types of information (location and progress stage or location and project award). Then they were asked to locate all awarded projects, a publication, a scheduled lecture, contact information, architects working at the firm and open position, if any.

3.3 Evaluation Results

For the heuristic evaluation, the evaluators were given:

  • The list of heuristics with their analysis along with a list of commonly observed website usability problems.

  • The proposed navigation scenarios for each website.

  • The assessment form for reporting problems, their relevance to the heuristic rules violated and their estimated severity (significance).

Evaluations were conducted by five experienced usability evaluators, one of which has studies in Architecture and another in Graphic Arts. After collecting the evaluations and analyzing the data, a single list was created for each site containing the problems and heuristic rules violations, the estimated severity of each problem identified (on a scale of 0–4), and the number of reviewers who reported the problem.

The following charts reflect the violations and the heuristics identified for all three architectural sites. We observe that the heuristic rules in which all 3 architectural sites have the most important compliance problem is “consistency and standards” (rule 4). Other commonly observed violations are: “flexibility and efficiency of use” (rule 7), the “visibility of system status” (rule 1) and “user control and freedom” (rule 3) (Figs. 1 and 2).

Fig. 1.
figure 1

Overall chart with the rules’ violations in the three architectural websites

Full size image
Fig. 2.
figure 2

Comparative char to heuristic rules’ violations in the three architectural websites

Full size image

The severity of the majority of problems identified in all 3 sites was level 3 (“major problem with high priority for correction”) and follow problems in level 4 (“catastrophic problems with urgent need for correction”) (Fig. 3).

Fig. 3.
figure 3

Comparative chart of problems distribution according to their severity in the three architectural websites

Full size image

In the heuristic evaluation of the three sites, the problems identified, regardless the level of their severity and the priority that should be given to correct them, don’t have the same degree of importance for the operation of the website. For example, the lack of a search feature at the first website, is rated with the severity score of 3.5 (on average) and detected by four evaluators and the lack of geographical categorization at the same site was identified by three evaluators and evaluated with a severity score of 4 (on average). The lack of the above features (separately or in combination) is the most essential problem encountered and identified by the evaluators.

4 Architects’ Questionnaire

4.1 Participants

The questionnaires were distributed to 40 architects and a separate questionnaire was designed for each one of the 3 websites. Most of the architect website visitors were architects, designers and students of respective departments, as it was derived from the UAQ. The user sample used complies with the Hassenzahl’s conditions [5] for the methodological approach of studies that refer to the relationship of aesthetics and usability. This stands for both the research participants and the products evaluated, as we made a thorough selection of a homogenous sample of both the participants and the evaluated websites. The websites were selected from a broad set of candidate distinguished architectural websites, as it has been analytically described in Sect. 3.2 and the participants were architects who had participated in large-scale studies for public and private projects, architectural competitions, exhibitions, publications etc. and demonstrated an active interest in quality, aesthetics and education.

The answers collected, reached the number of 30 out of 40 for the first and the third website (70 %) and 28 out of 40 for the second one (65 %).

4.2 Questionnaire Design

The questionnaire used for this evaluation is composed of two parts. The first part is based on the System Usability Scale (SUS), developed by John Brooke, while he was working for the Digital Equipment Corporation and is known as the «easy and quick» scale [6]. SUS consists of 10 statements where users declare their degree of agreement, on a 5-value scale. Half of the statements are phrased in a positive manner and the rest in a negative one. At the end, the result is calculated as a percentage. Studies have shown that a SUS result smaller than 60 % is considered low, while a result higher than 80 % is considered very good [7].

The choice of SUS versus another questionnaire such as the Questionnaire for User Interaction Satisfaction (QUIS) adapted for the Internet, or the Computer System Usability Questionnaire (CSUQ), is due to the higher precision of this questionnaire in analyzing even small samples. As it is apparent from the research of Tullis and Stetson [8] 100 % precision can be achieved using SUS with a sample of 12 users or more, while the other questionnaires require larger samples. Also, SUS is freely available for usability evaluations, for both research and commercial purposes, provided that any published report contains a clear reference to it.

The second part of the questionnaire aims to assess aesthetics in relation to the websites’ usability and for this purpose a set of questions from the AttrakDiff9 questionnaire were used [5]. AttrakDiff is designed as a tool for measuring the attractiveness of interactive products from User Interface Design GmbH and Marc Hassenzahl. With the help of opposing pairs of adjectives, users can determine exactly how they perceive the evaluated product. From these pairs of opposite adjectives derives the overall composition of the evaluated dimensions which according to Hassenzahl are:

  • Pragmatic Quality (PQ), which describes a product’s usability. In the case of a website it shows how successfully users can achieve their goals while navigating this website.

  • Hedonic Quality – Stimulation (HQ-S), which describes the extent to which the website can meet one of the primordial human needs, the need to develop and progress, in terms of innovation, interest and stimulus in functions, contents, interaction and presentation style.

  • Hedonic Quality – Identity (HQ-I), which shows to what extent the evaluated product allows the user to match identify with it.

  • Attractiveness (ATT), which describes the overall value of the evaluated product based on its perceived quality.

The Hedonic and the Pragmatic Qualities are independent and contribute equally to the evaluation of the Attractiveness. The platform used for the online distribution and management of the questionnaire was SurveyGizmo 3.0 (www.surveygizmo.com), in its free version. Architects were given specific scenarios to guide them through the examination of the websites.

4.3 Discussion and Main Findings

The comparative ratings for the 3 websites evaluated by the same users in the System Usability Scale (SUS), are shown in the following chart (Fig. 4).

Fig. 4.
figure 4

Comparative ratings in SUS scale.

Full size image

The SUS rating for the first website is the lowest, just 50.09 %, which as it is lower than 60 % is considered “very low”. The rating of the third website is close to 60 (59.02 %), but it is still low for this scale. The highest rating was obtained the second website (78.84 %), which is close to 80 and is considered very good.

Observing the results of the questionnaires of the 3 websites, we can derive a first verification between the results of the System Usability Scale (SUS) and the results of the Pragmatic Quality of AttrakDiff. Both the usability rating of each website and the relationship between the ratings of the three websites evaluated agree. So, initially, we have verified the users’ reliability and the reliability of the two types of questionnaires used.

In the comparative chart with the mean values of Hassenzahl’s constructs for the 3 architectural websites evaluated (Fig. 5), we observe the following:

Fig. 5.
figure 5

Comparative chart of the mean values of the 4 Hassenzahl’s constructs

Full size image

The first website, having the lowest rating of the Pragmatic Quality and the Hedonic Quality in terms of identity, it resulted in the lowest rating in the overall perception of the website’s Attractiveness, although it achieved the highest rating of the Hedonic Quality in terms of stimulation.

The second website has exactly the opposite picture of the first one in the attractiveness evaluation (AttrakDiff). It has the highest ranking in the Pragmatic and the Hedonic Quality in terms of identity, the lowest rating in the Hedonic Quality in terms of stimulation and the overall perception of the website’s Attractiveness is at the highest levels of both the constructs (qualities) evaluated for the same website. It is a case where the Pragmatic Quality and the Hedonic Quality in terms of identity, influenced the final overall perceived picture of the product in a high degree despite the lack of a big stimulus in the Hedonic Quality.

The third website is an average case in comparison with the other two websites. The mean values of the constructs evaluated are steadily halfway between the highest and the lowest values of the other two websites. The very low rating in the Pragmatic Quality has little impact on the overall perception of the website’s attractiveness and in this case the contribution of the Pragmatic and the Hedonic quality cannot be regarded as equal.

5 Conclusions

In the literature survey conducted in this work concerning the relationship between aesthetics and usability, it was observed that in most cases that there are references to aesthetic criteria, they are presented ad hoc without proper justification based on established aesthetic theories. In this study it was attempted to introduce a set of heuristic rules with aesthetic parameters, exploring and using widely accepted aesthetic rules and theories. These heuristics can be applied in designing the graphical interface of websites assuring its compliance with well established aesthetic principles. The proposed rules that are to be used as an extension to the core ten heuristics of Nielsen are: successful use of color, symmetry, harmonic arrangement, balance and grouping data and hierarchy. The newly introduced rules were used in the heuristic evaluation of three architectural sites and it is a remarkable observation that in all three sites evaluators detected violations to all the aesthetic heuristics. These first results are positive regarding the new set of heuristics but further research is required in order to prove their general liability and their aesthetic coverage.

Of particular interest is the correlation between the results of the usability evaluations from design expert users with high aesthetic demands and the findings of the expert usability evaluators. The opinion of architects as far as the three sites’ usability is concerned is reflected in two different types of questionnaires (SUS and the first part of AttrakDiff, which refers to the Realistic Quality) and thereby the accuracy of the result was verified. What we observed considering the three reviews of AttrakDiff, is that when you ensure user satisfaction in 2 of the 3 constructs (‘realistic’ quality, ‘hedonic’ quality-identification, ‘hedonic’ quality-stimulus), the product attractiveness is close to the good levels of these 2 constructs. But it seems that the opposite also holds; when the two of the three constructs are very low, the product attractiveness is in the same low levels.

For the future it would also be interesting to use the same methodology (websites, questionnaires, etc.) but with a different user sample, i.e. users with typical/low aesthetic demands, non-specialized or not familiar with aesthetics, who could also be considered as potential customers of the architectural sites. This way it could be investigated whether there are differences in the perceived quality of those sites. As part of this research, we could further widen the sample both at user level but also at the size and type of the evaluated website sample.