Keywords

1 Introduction

From a Human Computer Interaction (HCI) perspective, the availability of personal computers from the late 1970s was quite a disruptive technological innovation from the time when the only users who interacted with computers were ICT experts and professionals. Personal software and personal computer platforms made very basic literate human being a potential computer user. As non ICT professionals started to use personal computers and systems, usability and user experience (UX) challenges surfaced and became major barriers to user adoption and improved uptake [5].

According to the ISO 9241-11 standard, “usability is the degree to which software can be used by specified consumers to achieve quantified objectives with effectiveness, efficiency, and satisfaction in a quantified context of use”. Usability is a subjective measure because levels of usability depend on the user and the context of use [2].

Customized experimental design, defining usability measure and usability measurement techniques for complex products, systems and services is difficult [15].

Although usability is a well-established concept with proven measure and measurement methods, HCI researcher and practitioners have different level of understanding of UX operational definition, metrics and measurement methods [13].

The international standard on ergonomics of human system interaction, ISO 9241-210, defines user experience as “a person’s perceptions and responses that result from the use or anticipated use of a product, system or service” [10].

Beside the broad range of proposed definitions for UX, there is a consensus of opinion among HCI researchers and practitioners that UX as a concept is the result of the interaction between three elements: the user, the system and the context [13]. No one can predict for certain how a system will be perceived by the end users. Informed decisions can be made about UX during each phase of the overall system development lifecycle by using a mix or relevant user research and UX design methods and tools [7].

The level of interest for UX designers and researchers within organizations is constantly growing in most developed countries across the globe. Hence, competitive organizations and specifically multinational corporations compete for recruiting skilled UX consultants and experts that would be able to have a holistic impact.

The “2017 UX and User Research Industry Survey Report” published by UserTesting [1], indicates a positive trend toward early, frequent research positive trend toward early, frequent research positive trend toward early, frequent research throughout the design, development, and optimization of digital products. The majority of 2238 surveyed UX professionals reported that their testing frequency has increased moderately or significantly.. The number of UX professionals who do user research on their competitors has more than doubled year-over-year [1].

A strategy can be defined as approach or plan of action to achieve a major goal resulting in advantageous conditions. UX strategy applies this definition to UX design [4]. A UX strategy can be used as a basis for UX design to predict the future. A UX strategy builds a common vision in terms of user standpoint and based on organization business and product strategy.

This research tried to analyze, understand and evaluate the UX of spatial web-based projects by considering Tehran online mapFootnote 1 as a case study. The Web-based map of Tehran is subset of Tehran Spatial Data Information (SDI) project which provides spatial services for users such as citizens, scholars, public sector and private partners. The main objective of this research is to answer the following research questions:

  • How do existing and new users interact and engage with Tehran web-based map?

  • What is the role of UX in Tehran web-based map?

  • Which models and methods can be used to evaluate it’s UX strategy maturity?

  • How mature is Tehran web-based map in terms of UX strategy?

  • What key recommendations can be formulated to improve the level of Tehran web-based map UX strategy maturity?

2 Review of Literature

While performing extensive review of literature on topics related to the UX strategy maturity of Web-based map projects, we realized that much research has been performed on the connected topic of GISFootnote 2 and online maps usability.

Harding (2009) provides a research methodology which established a reference base of qualitative data on user needs for geographic information with respect to context of use. The key elements which contribute to geographical information systems usability are namely: (1) information content, (2) information quality, (3) information structure and interoperability, (4) information file format, (5) information volume, (6) information presentation, (7) information cost, (8) information delivery and (9) information selectability [8].

Nivala et al. (2007) studied the level of usability methods’ familiarity among map application and systems developers. The results showed that although usability engineering is slowly being incorporated into design of map applications, knowledge on how to execute the methods is still very low or almost non-existent. Many companies are actually interested to implement such methods, but one key problem is the lack of resources and applied knowledge on how to implement an approach [19].

Ingensand and Golay (2010) evaluated the task-oriented usability of a Web GIS. They realized there are different strategies for interacting with Web GIS and that users perform differently according to selected personal strategies. These differences in performance are related to the users’ experience and expertise with respect to similar systems usage and to specific user interface key functions and features [9].

Different models to design a UX strategy and assess its maturity within organizations have been proposed in the past few years. Jaime Levy (2015) suggests a quadruple tenet to design a UX strategy framework. Levy’s UX strategy framework is composed of: (1) Business Strategy, (2) Value Innovation, (3) Validated User Research and (4) Killer UX Design [14].

Stern (2014) has also proposed the CUBI model for achieving improved UX competitiveness. The CUBI model includes 4 main layers: (1) Content, (2) User goals, (3) Business Goals and (4) Interaction [20]. Each layer has dedicated sub-layers.

Kalbach (2014) has designed the UX Strategy Blueprint which is a simple tool to help define a given UX strategy. The elements within the UX Strategy Blueprint are: (1) Challenges, (2) Aspirations, (3) Focus Areas, (4) Guiding Principles, (5) Activities and (6) Measurements [11].

Nielson (2006) also defined an eight stages process for achieving Corporate UX Maturity [16]. In the initial stages developers and corporations show hostility towards usability. Most companies realize the value of making designs easier for human end users. In the next stages a fixed budget is dedicated and there shall be an official usability group, led by a UX manager in the company. Then, there should be an integrated user-centered design team incorporated within the corporation.

And finally the last stage should be a user-driven corporation. At this final maturity level, the usability methods will affect corporate strategy and activities [17].

Chapman and Plewes (2014) have developed a UX Maturity Model composed of 5 stages [6]. This model can help understanding any organization’s UX level of maturity. The model provides six key indicators of UX maturity as follow: (1) the timing of UX (2), the UX expertise and resources, (3) the usage of appropriate techniques, (4) the leadership and culture in the company, (5) the degree of integration of UX with other corporate processes and finally (6) applying design.

The five stages toward improved maturity are illustrated in Fig. 1 namely:

Fig. 1.
figure 1

Five Stages of UX Maturity [6]

Full size image

(1) Beginning, (2) Awareness, (3) Adopting, (4) Realizing and (5) Exceptional [6].

3 Research Methods

To answer our research questions, we used selected complementary research methods. Initially, for 30 days, quantitative data related to users of Tehran map portal were logged and analyzed. We also used heatmap tools. Then based on the analysis of user behaviors, we did proper experimental design towards planning usability tests. Figure 2 provides a visual insight about how data sources were used to inform the experimental design towards defining relevant usability testing tasks.

Fig. 2.
figure 2

Collecting quantitative and qualitative data via multiple user research methods

Full size image

3.1 Quantitative Data

Online real-time and real-world user’s behavior quantitative data was collected by using professional Web Analytics tools for a period of 30 days. We identified users to analyze and understand the way they interacted with Tehran online map portalFootnote 3. Our analysis included segmentation of users based on key factors such as visitors, time, frequency, actions, content and engagement. This part of research, also known as web analytics, was performed over one month period. We used Google Analytics to generate specific Java Script tracking codes and embedded the codes in the proper source code with the assistance of a partner company in charge of Tehran Web-based map.

3.2 Qualitative Data

Online real-time and real-world user’s behavior qualitative data was collected by using professional remote heatmapping tools, provided by SIGCHI Iran Chapter [21], for a period of 30 days. Heatmaps represent where the visitors concentrated their clicks and how frequently they clicked on a given screen zone. Generally, a color scale moving from blue to red indicates the frequency of clicks. Thus, a red spot over an area of a webpage might indicate that a participants clicked on this part of a webpage much frequently than other areas [18]. In the qualitative part of the research we analyzed selected screen heatmaps for specific visual elements to analyze and understand usage behaviors and possible design challenges as barriers to usage.

3.3 Mixed Method

Based on the data analysis performed on quantitative and qualitative data, we identified selected tasks to be performed in given context of use by selected users in a lab based environment on a desktop computer screen. The usability testing tasks were selected based on the newly available online sculpture database of Tehran Municipality. Table 1 provides a description of five selected user tasks to be performed by users.

Table 1. List of usability testing tasks
Full size table

The usability test session were conducted in a customized office of the Tehran Beautification Organization transformed into a usability lab for one full day. All usability test participants were staff members of Tehran Beautification Organization including 8 women and 10 men. A specific participant selection procedure and protocol was used to selected relevant participants based on previous knowledge and experience with Tehran online map portal and sculpture database widget. Informed consent forms, pre-test questionnaire, observer recording sheet, system usability scale (SUS) questionnaire and professional time and video recording software were used.

4 Results

4.1 Existing and New User Interactions and Engagement with Tehran Web-Based Map

To understand how users interact with Tehran online web-based map, quantitative data was collected and analyzed with Google Analytics.

Quantitative Data Analysis (Using Web Analytics).

Table 2 displays Tehran online web-based map key data about visitors during a period of 30 days (10 April 2015 – 10 May 2015).

Table 2. Tehran map portal visitors’ information (in a month)
Full size table

Visitors accessed Tehran online web-based map via different sources. Namely, 59.1% of visitors came via search engine, 33.9% of visitors came by directly typing the <http://map.tehran.ir> URL and 7% came from links. In term of geographical origin of users, 94.2% of visitors were from Iran, 3% from the USA, 0.7% from the UK, 0.4% from The Netherland, 0.3% from Germany and the rest from other countries.

One of our quantitative metrics was the engagement rate of visitors with the portal. This metric is measured in terms of user’s actions or clicks and visit duration.

Most visitors spent less than 10 min in Tehran map portal. A total of 19.78% of visitors engaged between 9 and 10 min and 19.44% engaged less than one minute. So, the engagement time for 12% of visitors is less than one minute which can be specified by two groups of users. Regarding bounce rate of 10–30% for service sites [12], a 6% bounce rate indicates the most visitors are in need of information and very much willing to engage with the map.

Qualitative Data Analysis (Using Heatmaps).

Given the technical architecture of Tehran online map is single screen and all services are shown on the main screen, it was only possible to produce a heatmap for that screen. Figures 3 and 4 display the main homepage of Tehran online Web-based map and it’s associated heatmap.

Fig. 3.
figure 3

Tehran online Web-based map main homepage (Color figure online)

Full size image
Fig. 4.
figure 4

Tehran online Web-based map main homepage generated heatmap (30 days data) (Color figure online)

Full size image

Figure 4, generated over 30 days or visitor heatmapping, displays red (hot) areas that are visitors “focus zones” on the main screen and colder colors (yellow, green) display a lower level visitor interaction and engagement. The “routing” and “Tehran key places” widgets located on the left side toolbar and the “search” and “more information” widgets located on the top toolbar got very high level of user interactions. Figure 4 provides a key insight about visitors not willing to access or read the “user manual” via the “help” section and to “register” with the map located on the top bar.

Mixed Data Analysis (Using Lab Based Usability Testing).

To determine to what extent the usability test participants were familiar with online map service, a dedicated questionnaire was distributed between them before the usability test. Among a total of 18 usability test participants, 8 were women and 10 were men.

Figure 5 displays a very low level of GIS and online maps usage by the participants for business purposes over a week period. In addition, participants were more familiar with Google map rather than Tehran map. A total of 45.8% of participants used online maps to find streets and public places. Routing features, urban traffic monitoring and street view functions were also mostly used by participants. A total of 72% of participants used a personal computer and 22% a smart phone to access online maps.

Fig. 5.
figure 5

Usability test participants pretest results (N = 18) (Color figure online)

Full size image

Table 3 displays the average time on task (task duration) as performed by participants for each of the five tasks presented in Table 1.

Table 3. Average time of task (task duration) for usability test participants (N = 18)
Full size table

Based on the usability testing protocol the maximum time on task was 180 s. Figure 6 displays how users performed on each task. Half of the participants well performed on task T1. One-third of participants could get T1 completed with help and two participants didn’t manage with T1 and failed. Task T2 got the highest success rate among all five tasks during the usability testing sessions. Quick observation of the sculpture’s picture is one major explanation for the high speed of T2 completion.

Fig. 6.
figure 6

Tasks T1 to T5 completion status by usability test participants (N = 18) (Color figure online)

Full size image

Task T3 was the most failed task by participants with an average completion time of 121.17 s as the longest recorded time among all five tasks. One underlying explanation is that as most participants didn’t manage to find and use the «Tehran city neighborhoods» selector so they would need to zoom out and in to identify the neighborhoods.

Almost 28% of the usability test participants were able to successfully perform task T4 that required using the horizontal toolbar at top of the screen as well as using the «distance calculation» widget. One of the reasons participants’ failed completing task T4 is that they were didn’t understand how to use and operate this widget.

A total of 8 participants managed to successfully complete task T5 that required using «Street view» widget. One of the participants did a shortcut by using the picture of sculpture could to complete this task. Two participants also didn’t manage to figure out and understand how to use and operate the widget.

Recurrent improvement suggestion for this map designers made by most of the usability test participants was to add simple and short usage guidelines, to use different color codes to identify the widgets and use simple unambitious icons on toolbars.

The usability test results confirm that although Tehran online Web-based map is full of advanced features and widgets, users have a hard time understanding how to usage and operate them. In addition, one of our participants looked into the available “help” section to learn how to use the widgets. Some usability test participants reported that the waring messages and text used inside the dialogue box were confusing.

Figure 7 displays the setting of one of the offices at Tehran Beautification Organization that was converted to a usability testing lab for one full day. The usability test participant is looking at the computer screen, the moderator is sitting on the side, the first observer is looking at the notebook screen (remote observation using recoding software) and the second observer (direct observation) is taking notes as well.

Fig. 7.
figure 7

Usability testing lab setting used to record participants interactions (N = 18)

Full size image

4.2 What Is the Role of User Experience in Tehran Web-Based Map?

A standard System Usability Scale (SUS) questionnaire was distributed to each participant right after the end of the usability test sessions. The Tehran online Web-based map obtained a SUS final score of 66.5. Based on Bangor (2009) adjective rating and acceptability ranges [3], 66.5 is an acceptable SUS score than is positioned in between “ok” and “good” categories. Therefore, based on our usability test, Tehran online Web-based map portal usability has still much room for usability improvements.

Apart from the usability tests we performed with the participants, we also reviewed and analyzed the initial Request for Proposal (RFP) written for Tehran online Web-based map by TMICTO. The RFP we analyzed didn’t include any minimum requirements for key human factors such as accessibility, usability and user experience factors. Our understanding is that TMICTO is either not aware of a user centered system development lifecycle method or no resources are dedicated to human factors.

4.3 Which Models and Methods Can Be Used to Evaluate It’s UX Strategy Maturity?

Different methods and patterns have been proposed for evaluating the UX strategy maturity within organizations. In this research, Chapman & Plewes (2014) model was used to evaluate the UX strategy maturity of Tehran online Web-based map portal. As discussed in literature review section, Chapman and Plewes (2014) proposed a 5 stages model that drives an organization from an “immature” toward a mature “stage” [6]. This model can help organization with defining each stage indicators quite distinctively, so that they would be able to evaluate each maturity stage based on clear metrics.

4.4 How Mature Is Tehran Web-Based Map in Terms of UX Strategy?

Based on our findings we can affirm that the level of UX literacy and UX strategy maturity at TMICTO is very low. TMICTO would definitely benefit from learning and applying a user-centered system development lifecycle methodology. It also appears that there is a very low level of applied knowledge about usability, UX and human factors at TMICTO. Based on chapman & Plewes (2014) model, this organization is at the very beginning stage. This first stage is where the organization just thinks of UX as end-user product add-on or a service visual design enhancement.

4.5 What Key Recommendations Can Be Formulated to Improve the Level of Tehran Web-Based Map UX Strategy Maturity?

Improving the level of UX literacy and institutionalization within organization requires driving through each and every stage of the maturity model. According to Nielson (2006) and based on the literature, reaching the ultimate stage of maturity might take as long as 40 years [17]. Therefore, organization must have short term, medium term and long term planning with regard to understanding and continuously improving the level of UX strategy maturity. Based on our research findings, key recommendations are proposed as follows for all the team members of Tehran online Web-based map based at TMICTO:

  • Taking advantage of professional user research experts and UX consultants outside the organization,

  • Learning and applying the foundations of Human-Computer Interaction (HCI) including User-Centered Design (UCD), usability and UX by all researchers, designers and managers working at TMICTO,

  • Understanding the importance of incorporating key human factors and metrics to each and very future Request for Proposal (RFP) related to Tehran online web-Based map and other information systems

  • Performing continuous rapid and rigorous user research with a wide mix of analytical and empirical methods

The following key recommendations are proposed for the medium term:

  • Defining and incorporating usability requirements and UX targets to each and every project at TMICTO,

  • Ensure that all users and those who will be affected by an existing or future system have their needs clearly articulated during the assessment and design of requirements; later the system must be validated to meet requirements via usability testing

Long term recommendations are as follows:

  • Prioritizing UX as a key success and competitiveness factor in each new project,

  • Sponsoring user research and UX design for the very first day of each new project,

  • Partnering and networking with local world-class professional communities of practice such as the AM SICHI Iran Chapter [21] for actionable professional training, applied and basic combined experience sharing towards better interactions.

5 Conclusion

In this research we aimed at investigating the UX of spatial web based projects and the level of UX strategy maturity within projects and more specifically organizations in charge of those projects. The Tehran online Web-based map owned by TMICTO was taken as our case study.

First, we tracked real-time and real-world usage of Tehran online Web-based map by collecting quantitative and qualitative data during 30 days. Quantitative data was also collected and analyzed with dedicated web analytics tools such as Google Analytics during the same 30 days. Qualitative data was collective via dedicated remote user research tools such as heatmap tools. The “routing” and “Tehran key places” widgets located on the left side toolbar and the “search” and “more information” widgets located on the top toolbar got very high level of user interactions. Based on the collected and analyzed data from 275320 unique visitors, we defined specific tasks for a specific context of use and conducted usability testing sessions with 18 participants including 8 women and 10 men from Tehran Beautification Organization.

Based on the results of the usability testing session, the Tehran online Web-based map obtained a SUS final score of 66.5. Based on our research findings we can affirm that the level of UX literacy and UX strategy maturity at TMICTO is very low.

The comparative survey of TMICTO and Tehran online Web-based map portal with respect to Chapman & Plewes (2014) UX Maturity Model’s indicators showed the TMICTO is still at the very beginning stage in terms of UX literacy, UX strategy and UX maturity. Much has to be done to incorporate UX from the overall lifecycle.

We proposed short term, medium term and long term practical recommendations for improving the current state of TMICTO UX literacy, maturity and Tehran online web-Based map UX strategy maturity. TMICTO should prioritize UX as a key success and competitiveness factor in each new project towards improved UX maturity.

Our research findings also highlight the negative outcomes of low usability and user experience as very strong barriers to improved end-user engagement. Sponsoring user research and UX design for the very first day of each new project and programs can be key driver towards overcoming such barriers. Finally, we strongly recommend partnering and networking with local world-class professional communities of practice such as the AM SICHI Iran Chapter <http://www.sigchi.ir> for actionable professional training, applied and basic combined experience sharing towards better interactions.

6 Limitations

One of the limitations of this research is the time constraint of 30 days (one month) during which the remote user research with web analytics and heatmap tools were performed. Extending this period of time to 6 or 12 months would provide us with richer real-time and real-world data towards improved research insights and findings. The authors also acknowledge that the context of use, selected tasks and participants with respect to the usability tests are domain specific and linked to Tehran Beautification Organization.