Keywords

1 Introduction

Over the past few years, we have seen that analytics is becoming known for providing unprecedented opportunities to companies. Identifying customer needs, changing market segments, enhancing user experiences, optimizing processes, and getting competitive advantages are among a few potential domains where analytics can offer promising advantages to grow in the market [4, 6]. However, despite the hype and potential benefits of analytics, mentioned in both scientific and practitioner-oriented literature (e.g. [5, 12]), companies are not seen utilizing the full potential of it. For instance, according to survey results presented in a recent study [4], only 33% of established companies, 19% of medium companies, and 10% of small companies, from Europe, are using analytics. These statistics show that analytics has yet to become mainstream, particularly, for small companies.

Analytics practices inside small companies like software startups are imperceptible. However, the uncertain nature of startups and their focus on innovation within limited resources make them promising candidates to practice it. Prior research concludes that startups realize the benefits of analytics [11] and analytics practices should be present as a toolkit to support startup operations in cutting operational costs, supporting decisions, and getting insights [7]. On the contrary, the existing literature sheds no light on the analytics practices currently employed by these companies. For example, a recent study [10] argues that best practices of applying analytics in a startup context are yet a mystery. Going in the same vein, when we particularly focus on software startup literature, we identify the challenges to analytics adoption faced by software startups [8] and the perception of startups about analytics [9]. In particular, the latest work on analytics in software startups [9], comprehends how analytics is understood in the startup context and what are the possible usage scenarios. Therefore, nevertheless, we lack in getting the key analytics practices applied by software startups while carrying out analytics-related operations. This is the research gap that our current study aims to address. Thus, the overall guiding question is:

RQ: How are software startups applying analytics?

Applying a case study research method [3], we gathered the data from three software startups at different startup life-cycle stages [1]. We primarily used semi-structured interviews to collect the data. Later, we utilized thematic analysis to analyze the collected data. The findings contribute in terms of providing insights for software startup companies to introduce or enhance analytics in their specialized context to unlock opportunities. The use of these practices will help them to avoid numerous pitfalls.

2 Analytics and Software Startups

Analytics has brought a significant transformation for several companies [16]. However, most companies are still striving to figure out the fundamentals of applying analytics in their environments [18]. While there is no consensus regarding the definition of analytics, most of the literature we came across (e.g. [5, 18]) refers to the definition offered by Davenport, Harris, and Morison [16]. According to the authors, analytics can be defined as “the use of analysis, data, and systematic reasoning”. In the startup context, we came across the definition provided by Croll and Yoskovitz [17], who think that analytics is to identify, measure, and track the riskiest parts of the startup. Thus, it measures and enlightens the way to product and market growth. The immediate outcome of practicing analytics is on the decision-making of the companies, which is transformed by insights generated from data [5]. Thus, analytics includes a set of tools, techniques [4] and human sense-making [5] to create value from data. In addition, utilizing analytics is neither a mechanical process [4] nor all the analytics practices can fit well in all sorts of companies [15].

Software startups are young and innovative companies that are characterized by limited resources, and dynamic markets and have to face multiple influences [14]. These companies offer software-intensive products or services and work under extreme conditions of uncertainty [13]. While the research on software startups is evolving, numerous studies are focused on supporting engineering activities in these companies. Therefore, there are relatively few studies reporting on analytics for software startups. For example, two recent studies [8, 9] discuss analytics in software startups. Together these studies report challenges that software startups face and comprehend how software startups understand analytics. In contrast, Berge et al. [11] studied hardware startups and explored benefits and challenges while utilizing analytics.

3 Research Method

We followed a multiple case study design in our study. According to Yin [3], case studies are suitable to address exploratory studies. In the same way, we studied multiple cases for a rich understanding of the key analytics practices. We approached 10 startups through our personal network, LinkedInFootnote 1 , and CrunchbaseFootnote 2 . Finally, three software startups agreed to participate in the research. We classified these startup companies based on different startup life-cycle stages. We followed the classification proposed by Klotins et al. [1]. Authors categorize startups into four stages e.g. inception, stabilization, growth, and maturity. Our studied startups belong to the last three stages i.e. stabilization, growth, and maturity.

We collected data primarily through semi-structured interviews, however, we considered additional public information about the companies as well. We conducted interviews remotely through Zoom and MS TEAMS. Each interview lasted from 60 to 90 min. The first author led the interviews together with the second author. Throughout the interviews, we followed a pre-designed interview protocol in which we defined open-ended questions regarding the introduction of the startup, product, team, measurement activities, and analytics practices undertaken. Afterward, we transcribed the interviews and analyzed the text data using open coding technique [2]. The coding process has resulted in renaming a few of the codes and adding more codes as the data analysis evolved. We carried out the data analysis process using NvivoFootnote 3, a qualitative research software. According to the agreement with the respondents in the informed consent form, we report our findings anonymously.

Table 1. Case Overview
Full size table

Case (A) is a startup company established in 2020 and delivers software-intensive services in the marketplace industry. It has already found its product-market fit and has a fair number of paying customers. Thus, we consider it in the stabilization phase. The team comprises six members, including the CEO and CTO of the startup. We interviewed the CEO of the startup.

Case (B) is a software startup that produces both B2B and B2C solutions in the business domain of sales and marketing. The startup was bootstrapped in the year 2018 and since then it has acquired several thousand paying customers. The company consists of a three-member team, headed by the CEO. Our data analysis suggests that the startup is in the growth phase as the efforts of the company are primarily shifted toward marketing and sales directions. We interviewed the CEO of the startup to understand their analytics practices (Table 1).

Case (C) is a unicorn startup providing transportation services through its software-intensive platform. The startup runs its analytics operations using its seven-member team and it has already acquired a very large customer base since its inception. It is currently focused on optimizing its operations and lies in the maturity phase according to the startup and product evolution stages. We interviewed the User Experience (UX) manager of the startup.

Lastly, it should be noted that the team size is not the same as all employees of the company. For the first two cases i.e. Case A and B, team size also refers to the overall employees of the company. However, for the third case, which is a unicorn startup, comprising hundreds of employees, we only considered the team mainly responsible for conducting analytics-related operations.

4 Findings

We identified nine key practices and classified them into four categories. We followed the analytics framework provided by Kayser et al. [19] to guide our classification of practices. This analytics process is structured in an attempt to succeed with analytics while moving from data to value. These high-level categories are business need, data collection, data exploration, and data communication. On the other hand, the identified individual key practices include focusing on a few KPIs, use of tools, communicating with the team, hiring people with an analytics background, blending data and intuition, understanding goals, collecting data early, sharing customer insights, and reaching out to customers. Figure 1 depicts an overall view of the identified analytics practices according to the analytics framework.

Fig. 1.
figure 1

Key Analytics Practices Mapped with the Analytics Framework, based on [19]

Full size image

P1: Focusing on a Few Key Performance Indicators- We noticed that focusing on a few Key Performance Indicators (KPIs) is a principally visible key analytics practice in our data. Our data analysis also reveals that the tracked KPIs are associated with the goals that startups desire to achieve through analytics. In addition, the domain in which startups work also plays a significant role in deciding which KPI is important to monitor. For instance, the respondent from CASE C states: “we understand the NPS and we have CSAT that we are monitoring the user in different moments of the journey to understand the CSAT”. Here, NPS refers to net promoter score while CSAT stands for customer satisfaction score, two key KPIs to measure customer experience. It is pertinent to state that startup C had a goal to improve customer satisfaction. Similarly, the founder of startup A stated that the number of customer applications is important to them. The founder indicated this as his startup is a marketplace and he thinks it is important to measure the network effect in this domain. The respondent continues expressing the need to collect the data that matters the most. He expressed it in the following words: “Actually capture data that matters...that matters most”. “We are getting better and better”, he further added.

P2: Use of Tools- All respondents from the studied startups reported the use of tools to support analytics setup in their companies regarding data collection, exploration, and communication. However, according to the interview data, startup A is only using Google Analytics while startup B is using Google Analytics as well as Facebook Analytics. Overall, respondents from both startups expressed that they are satisfied with the use of these tools and that these tools are satisfactory to achieve their analytical goals. A key determinant in selecting these tools is the “freemium feature”. On the other hand, as one might expect, startup C is using a number of tools to collect and explore data. It is worth noting that as the startup is mature, therefore, it heavily relies on internal tools to support the process. When it comes to collecting qualitative data, the startup is found using third-party tools like Qualtrics and Tableau.

P3: Communication with Team- Our data analysis reveals another common practice applied by startups in terms of communicating the data and insights with the team. All the respondents reported that they share and talk about data and interesting insights with other team members. For startup C, the communication happens in a more formal way i.e. communicating insights through documents and using the language of data while other startups do it in an ad-hoc manner. The founder of startup A expressed this in the following words: ”We do talk to each other, not like groups, focus groups,s or things like that... But we talk about analytics and interesting Data”.

P4: Hire People with Analytics Background- All studied startups believe in hiring trained resources for collecting data and turning it into insights. Only startup B has not hired someone with skills, however, the founder expressed his desire to do so to utilize the full potential of analytics. It is expressed as:“ There is so much interesting data you will get from this... for our company, because there is no person, like really knowledgeable about this analytics... we don’t really implement it like 100% but just 10%, and with just 10% we get like I think many insights”. The CEO continued: “ If there’s a person that knows this thing, I think they should do this”.

Startup B has hired a resource to collect data, visualize data, and communicate it to the team. Similarly, startup C has three small yet dedicated sub-teams to manage analytics.

P5: Blend Data and Intuition- The respondents claimed that they use both data and intuition to act and make decisions. While one can expect it from startup C, however, interestingly, CEOs of other studied startups also highlighted this practice. For instance, the CEO of startup A alluded to his practice: “maybe we should be a little more rational ...capture data. But if we care about data, we wouldn’t start a company because...95% fails”. The CEO of startup B holds the same belief: “(decisions) based on this analytics only...No...I’m, I never like this”.

P6: Understanding Goals- Our data analysis shows that it is a common practice to first understand the goals of the analytics. We also find that goal identification has a relation with defining KPIs to track. Therefore, in all cases, the informants explicitly reported the use of this practice. However, the goals varied in all our studied cases. For example, for startup A, user research and increasing network effect remain key goals. Startup B is involved in user research as well as having an increase in customer acquisition. On the contrary, startup C has a lot of goals e.g. improving efficiency, sustainability, testing hypotheses, and conducting A/B tests. The startup reported it: “we have the goal to achieve it as a startup and to be self-sustainable as a startup”. Interestingly, the startup also indicated their prior analytics goals and we find that those goals are now acting as primary goals for our other studied startups at earlier stages in comparison. It is apparent from these words: ‘Two years ago, we had the goal of having more users, but now the goal we have, we can keep growing, but not investing a lot in new users, but most trying to be more efficient”. This shows the evolution in terms of establishing analytical goals as the startup grows.

P7: Collect Data Early- The findings also show collecting data early is another key practice. Respondents from startups A and C explicitly indicated this. The respondent from startup A advised in the following words: ”Capture data early on so you understand”. He also reflected on his mistake: “the mistake we made is we implemented. We implemented Capturing the Data that matters most very late”. The respondent from startup C also alluded to the same opinion of relying on (already collected) internal data when they are not so sure about solutions.

P8: Share Customer Insights- We observed that both startups A and B are implementing this analytics practice. The practice refers to promoting insights and numbers to public forums and using them for publicity and marketing purposes. For instance, the respondent from startup A articulated: “Now we have data and we say...to clients, these are the data for you... And it helps us”. A similar practice was stated by startup B. The respondent expressed: “I just posted it like this”. The core purpose was to “to build the trust of the people like, oh, this application is working... many people [are] using it or something like that”, the startup added.

P9: Reach Out to Customers- All the studied startups are found reaching out to customers to understand what users expect, discuss new proposals, take feedback, collect opinions, or add/remove product features. This is rather surprising because we expected mature startups to understand the qualitative data, however, startups at earlier stages have also been found practicing it. For instance, the CEO of startup A indicated:“Customers’ input is very important and we do reach out to our customers and talk to them”. He added further:“Still we want to hear from the clients and the customers what they perceive as value.”. The same practice has been echoed by startup B in the following excerpt: “ they can request their likes, discuss with my team and then with the other members, and then if we have like... let’s say a conclusion like all this”.

5 Discussion

Our research question was focused on determining how software startups are applying analytics. In this regard, we tried to identify the analytics practices. Our results depict nine analytics practices that are employed by software startups while applying analytics. These nine practices are classified into four categories in accordance with the analytics framework provided by Kayser et al. [19].

Contrary to our expectations, the study results have presented a handful of analytics practices that startups use. It might be related to the fact that we confirmed the use of analytics from our potential startup cases before proceeding with the data collection. Correspondingly, most of the practices are associated with the data collection phase, followed by business needs and data communication. This also accords with the findings of an earlier study where it is ascertained that most analytics challenges occur within the data collection phase [8]. On the other hand, the data exploration phase contains the least number of reported analytics practices. Similarly, our results also support previous research (e.g. [4, 5]) where it is indicated that analytics should be based on tools, techniques, and human sense-making.

While we report only on practices applied by startups at various stages of evolution, the frequency and intensity of implementing these practices vary from case to case. As one might expect, the startup at a mature stage is found applying practices intensively. The respondents from other startup companies have expressed their desire to strengthen analytics practices while highlighting the scarcity of resources. Consequently, their analytics process is somewhat ad-hoc in its current form. In addition, it is interesting to relate that, according to our data, startups at earlier stages learned from their mistakes. For example, collecting data early and hiring people with analytics backgrounds are example practices that have been adopted by startups after making mistakes. Likewise, understanding goals and focusing on a few KPIs are practices that have been highly recommended by the practitioner-oriented literature [17]. Our study empirically confirms the use of these practices by software startups.

Regarding threats to validity, our study has a number of limitations. For instance, we studied three software startups and interviewed one respondent from each case. Nevertheless, finding software startups with existent analytics set up and covering different startup stages are hard to manage. In the same way, although, we came across a number of other analytics practices reported by respondents in their startups, however, we did not report them in the findings as we had a limited number of agreements among the respondents.

6 Next Steps

Our findings enhance the scientific knowledge in disseminating analytics practices carried out inside software startups. However, considerably more work is needed to identify the analytics practices that are suitable for each stage of a software startup. Therefore, future research might put efforts into exploring key analytics practices that would be a good fit for a startup at a particular startup stage. In the future, we plan including more cases for each startup stage as well as interviewing multiple respondents from each case.