Keywords

1 Introduction

Alibaba.com is the world’s largest online business to business (B2B) trading platform, allowing global suppliers to wholesale their products to buyers all over the world. The Alibaba.com office is headquartered in China, where almost everyone involved in creating the platform including business development, product management, user experience and engineering teams are located. However, a significant portion of the Alibaba.com users, particularly those who use the site to purchase goods, are located in the United States. This poses a challenge for the stakeholders when trying to understand their users, due to geographical separation, communication barriers and knowledge gap between the stakeholders and the users.

1.1 Time Zone Differences

Stakeholders have difficulty finding a convenient time to talk with users. Because of 12 to 15 h in time difference between the stakeholders in China and the users in the US, they can only communicate in real time during one another’s after business hours. This reduces the number of users available and willing to participate in studies or interviews. Certainly the stakeholders can travel to meet the users in person, but it would become cost prohibitive from a budget and time resource standpoint to require overseas travelling for every project.

1.2 Communication Barrier

Language differences pose a barrier for the stakeholders when trying to communicate with the English speaking users located in the US. Even though generally most of the stakeholders have competency in written English, their verbal capacity varies. As such, it is difficult for them to carry on an in-depth conversation with the users to discuss their experience in using Alibaba.com or wholesale trading in general.

1.3 Knowledge Gap in User Needs

Unable to easily access the core users for insights, most of the stakeholders initially did not have an easy way to identify user needs, understand their pain points and implement solutions to address them. As a result, many of the stakeholders had to rely on their own assumptions when developing products and creating feature designs. However, the problem is that the process of B2B global trading is highly complex with specific import and export laws and regulations unique to each country. It is complicated to understand how users navigate these laws. The stakeholders’ assumptions often do not correctly match the users’ actual needs, resulting in products and features that did not support the users.

2 Approach

The author is a user experience researcher at Alibaba.com based in a satellite office in the United States, and therefore had better access to the majority of the Alibaba.com users for research. Being closer to the users allowed for more flexibility when conducting research with them and larger available pool of users to participate in research. Because of this, the author was able to utilize different methods to help the Alibaba.com stakeholders to better understand how the users normally use the platform and be able to discover their true needs. This paper is a case study describing the approach and methods used by the author to help bridge the gap between the stakeholders and its product users.

2.1 Using Focus Groups to Gather User Feedback on Key Product Concept

Initially the business and product management leaders at Alibaba.com did not have a clear understanding of the capability of user experience research. Due to limited exposure to research methods in the past, they misconceived user experience research as solely usability testing and nothing else. Therefore, when there was an opportunity that these top business and product management team leaders were travelling to the US for a business meeting, a focus group study was organized specifically for them to attend in person. The focus group served multiple purposes:

  • Expose the top decision makers to additional methods to break the misconception that user research only involves user testing

  • Provide influence and impact on product strategy early on during the concept development stage

  • Allow these top decision makers to receive first hand user feedback on key product initiatives

  • Gather feedback from a group of targeted users in a shorter amount of time

  • Provide opportunity for stakeholders to ask users questions and discuss product experience issues face to face with users

Two focus group sessions were conducted over two days with eight participants in each session. The participants were composed of one group of B2B buyers who were existing Alibaba.com buyers and another group of B2B buyers who were not. Other employees from the US office attended the session to provide interpretation to the stakeholders during focus group discussions.

Topics discussed in the focus group were specifically chosen to help the stakeholders, who rarely have the opportunity to come face to face with their users, to build a general profile of them based on actual user feedback rather than assumptions. Participants were asked about their motivations for and barriers from using Alibaba.com, as well as key benefits and top concerns of using the platform, just to name a few examples.

From the focus group, the stakeholders were able to hear the users’ explanations of the commonly heard issues on the platform. For instance, the stakeholders had previously learnt from customer support that it was difficult for users to find products on the platform, but did not receive a concrete example of how and clear explanation of why. The participants during the focus group, with proper moderation by the researcher, were able to provide more details to bring clarity to the stakeholders.

The key objective of the focus group was to gather feedback on a new product concept. This portion of the research was particularly impactful as user’s reaction and feedback revealed aspects in the new product concept that required significant refinement for it to be considered useful by the users. Without these insights, the product would have launched and likely resulted in negative impact to the business.

The focus group sessions were recorded and a summary report was created to present the insights with the product development teams in China. Overall, the focus groups enabled the stakeholders to better understand the user experience issues on the platform and the contributing factors. But most importantly, after they attended the focus groups, the stakeholders realized that the user experience research is more than just design testing as they previously believed. They now realized that user research could provide significant impact to the product direction in the very early stage of the process.

2.2 Contextual Inquiry to Understand the User Process

After the product concept was refined based on user feedback from the focus groups, the product entered the next stage of the development cycle. A prototype version of the product was created and an initial round of remote user testing commenced with Alibaba.com users in the US to assess the interactions and designs of the product. However the results gathered from the remote user testing were mostly negative because the page design did not match what the users would normally do. Apparently, important steps within the trading process were missing in the product which led to user confusion. After discussing with the product managers and the designers, it was realized that additional information was needed to fully understand the specific details in the complex process of B2B trading on and off the Alibaba platform.

To gain better understanding of the trading process conducted by the targeted users, a contextual inquiry study was planned with 10 online B2B buyers. As a semi-structured interview method, contextual inquiry can be used to gather information about the context of use, where users are first asked a set of standard questions and then observed and questioned while they work in their own environments [1]. The results from contextual inquiry can be used to define requirements, learn what is important to users, and discover information about a domain to inform future projects [1].

Typically, contextual inquiry takes place at the users’ home or work environment for realistic and direct observation. However, time constraints as well as the lack of available qualified participants in the vicinity limited the possibility of visiting all the participants at their locations. Instead, half of the contextual inquiry sessions took place remotely with the users across different regions in the United States. Those remote sessions occurred with the users sharing their computer screen to show online steps and any important documents created during the B2B trading process.

The buyers were asked to complete the steps of the most recent business purchase they made with a supplier found online, while the team members observed with minimal interruption. The researcher observed the users explaining the steps and asked questions along the way to better understand the details.

The results provided clarity on certain details in the B2B trading process, which helped to refine the missing flow in the new product. In addition, it provided a more complete picture of the typical user work flow, as well as identified aspects in the process where new features on the Alibaba.com platform could further support the users when conducting B2B trading.

2.3 Using Customer Journey Map to Document the Process

One of the goals of the contextual inquiry was to identify the user process to help the product managers and designers to create site features and page flow. It was important to present it in a visual medium to illustrate the steps. Therefore the customer journey map was utilized as a tool to show the steps the users go through when on and off the Alibaba website when conducting B2B trading. As shown in Fig. 1, a customer journey map illustrates a visual representation of a user’s needs, the step-by-step flow of interactions required to fulfill those needs, and the resulting emotional states a user experiences throughout the process [2].

Fig. 1.
figure 1

Example of a customer journey map [3]

Full size image
Fig. 2.
figure 2

An early version of the customer journey map

Full size image

By showing how customers feel throughout their journey, customer journey maps invite stakeholders to enter the world of customers and share in their experience. In turn, stakeholders are better able to convey their story to management, fellow colleagues, and the teams who are responsible for improving the service and product experience [2].

As shown in Fig. 2, the map organized the B2B process into several phases to represent the different goals of the users. It illustrated the key pain points in each phase of the process to highlight areas of opportunity for improvement during product design. The map was presented to various product teams. It was used as a reference tool for product managers during new product planning and a guide for designers during website design. It served as a living document that the product development teams in China could use to understand the complex B2B trading process and identify issues in the user experience. Once new significant changes in the user process are found, the map will be updated and continue to serve as a living document.

2.4 Collecting Quantitative Data to Identify the Users

Results gathered from the focus groups and contextual inquiry provided qualitative insights on the product and the users’ work process. But one missing facet of data for the stakeholders was quantitative information on the users, both on those who currently use the Alibaba.com platform and those who do not. Such data helps the stakeholders to identify who the users are, their purchase activities, their motivations for using Alibaba.com, and barriers to using the platform.

A survey was designed and launched to gather data from general B2B buyers in the United States regarding their demographics, company information, buying needs, buying frequency, reasons for current users to use the platform for B2B trading and concerns that non-Alibaba.com B2B buyers have for the platform. Because this data was previously not available to the stakeholders, many of them made assumptions regarding these aspects based on user feedback from qualitative research, external research report or educated guess. The availability of these quantitative data helped to validate their assumptions and was used as a reference for planning business, product and marketing strategies.

3 Discussion

The use of these previously described methods are standard practice within the user experience industry. However, the combination of these four methods and the sequence in which they were used provided timely and appropriate insights to help the stakeholders to understand the users.

For instance, the focus groups impacted the product strategic direction during the early stage of the product development cycle. It also established credibility of user experience research and broadened the scope of influence research was capable of in the perspective of the stakeholders. It helped to open up their trust and their understanding of the capability of user experience research. The successful outcome of this first phase of research led to assigning additional resources to support the next research initiative, the contextual inquiry.

The contextual inquiry was instrumental in identifying the details of the core user experience - a complete picture of the user process. The information gathered from the contextual inquiry was valuable in helping the stakeholders to understand the typical online and offline processes of trading with suppliers, their points of interaction within the Alibaba.com website, and the problems they encountered when using the website. They were fundamental to the product development and user experience teams in creating useful and usable features.

The customer journey map was a visual way to present information on the user experience process by illustrating the process step by step and highlighting barriers that were presented along the way. The product and design teams were able to use this to properly plan product designs. After all, information that is not usable to the audience is the same as the absence of information.

4 Building up the Research Participant Database

As mentioned before, the stakeholders in China faced the problem of insufficient number of users in the US willing to participate in paid research studies during afterhours. One strategy that the research team has been using to help quickly find and schedule qualifying participants is to retain the names of those who have previously participated in research. Even though limited in numbers, they tend to be willing to participate in research again if the previous experience was worthwhile to them. Also, in every project there are a handful of users who expressed interest in participating in research but did not meet the required screening criteria. These users are another source of participant leads available to quickly contact for future research.

To store their contact information, an internal spreadsheet accessible only to the user research team was created. Their contact information, along with the project name, last research participation date and last contacted date were recorded. It is important to not contact someone who has attended a research study in the past 3 months or in similar projects in order to avoid sampling bias.

5 Guiding Principles

The methods and approaches described here were effective in providing user insights and data to the stakeholders. Every organization is structured differently and has different needs so they might not be applicable to all. However, based on this experience, the author has come up with some guidelines to help those researchers based in the satellite office to work effectively and provide more impact in their research work.

  1. 1.

    Identify the needs of the stakeholders: The stakeholders are internal customers to the research team. Understand what projects or initiatives they are working on is imperative to creating meaningful impact in your research. The stakeholders will not always know what they need. Be proactive in determining what information they could use to help answer the questions that they have in those projects.

  2. 2.

    Discover your strengths and play to your advantages: Working in satellite offices has many challenges, such as delayed or lack of updates in business critical information, as well as difficulty in providing impact in an isolated environment. However, there are advantages to being in a satellite office such as ability to work independently and access to different external resources not available elsewhere. Identify the unique advantages of being in remote office and use it to support the team.

  3. 3.

    Show the value of research: Not everyone understands what user experience research is about and what it can do. Demonstrate what research is capable of by inviting stakeholders to participate or observe. Provide insights that they did not expect but can fulfill their needs.

  4. 4.

    Create data artifacts to show the results: A picture is worth a thousand words. PowerPoint is a great presentation tool, but is not as effective for reporting. Often times it is more meaningful to document insights by illustrating it in a visual medium. It helps the teams to remember the findings that you have discovered and help them to create products because it can be displayed in front of them during product planning or discussions.