Keywords

1 Introduction

Software startups are significantly contributing to making the world a better place. Today’s most influential software businesses initiated their journey as a startup. Netflix, Airbnb, Uber, LinkedIn, Canva, and Slack are only a handful of instances. These small yet innovative companies are witnessed driving the economy of today’s contemporary world [10]. Innovation, uncertainty, scarcity of resources, high reactivity, and time pressure are some notable characteristics that distinguish these companies from other software businesses [6]. The proliferation of startups across the globe is continuously booming. Nevertheless, more than 90% of the startups completely fail and only 15% of those that sustain themselves get a successful exit [4, 10]. This high failure alludes to how much money startups have wasted and may continue to waste. The significant reasons identified after studying thousands of startups are actually related to each other i.e. no market need and running out of cash [10].

On the other hand, analytics has become more and more prevalent for a wide range of companies including software companies(e.g. software analytics [12]). For these established companies, evidence indicates that analytics can play a pivotal role in maximizing the productivity of companies, reducing costs, helping to identify trends, and maintaining competitor advantage [11]. However, when it comes to startups, there is a lack of a comprehensive understanding of what constitutes analytics for startups and how startups can utilize it to drive success and growth. Therefore, this study fills the gap in the academic literature by attempting to understand how startups can benefit from analytics in terms of raising the odds of success, reducing uncertainties, coping with dynamic markets, and learning. Thus, the following Research Questions(RQs) are guiding our study: What benefits, related to analytics, do software startups ascertain?(RQ1) and What are the key practices to define analytics inside startups?(RQ2)

We performed a Gray Literature (GL) review [8] and collected videos as GL data source to address our RQs. We identified 16 relevant videos and then used thematic analysis and quasi-statistic to synthesize findings. Therefore, we identify and present ten opportunities that analytics can bring to startups along with two analytics practices. These results aim to help startup companies in defining the analytics setup.

2 Related Work

Despite the ever-growing significance of analytics, there is a lack of knowledge regarding what constitutes analytics for software startups and how can these companies utilize it.

A few recent studies [3,4,5] develop our earlier understanding of analytics for startups in terms of role of analytics in startup companies, analytics challenges for startups, and perception of startups regarding analytics. Much of the related work, in the field of software engineering, is focused on analytics about software and its associated artifacts [12]. Therefore, it still remains a challenge to translate many of existing research insights into actionable steps, especially within the unique environment of startups.

3 Research Method

We conducted a Gray Literature(GL) review [8] due to the lack of existing scholarly research and limited access to primary data. We also aimed to elicit knowledge from startup practitioners who are directly influencing novice entrepreneurs [6]. The use of GL is not a new development in the field of Software Engineering(SE). Several studies in SE and startups (e.g. [1, 2, 6, 7]) have utilized GL, particularly selecting, web pages, blogs, videos, books, technical reports, or white papers, as data sources.

We utilized YouTube to collect GL data. Our eight search strings included “software startup”, and “analytics” and its associated terms. We expanded our search to include the first 50 search results for each search string. After applying inclusion/exclusion criteria to 400 videos, we identified 16 potentially relevant videos addressing the RQs. The final version of the dataset contained 415 min of videos (seven hours) [14], and 81574 words (181 pages).

We started our data analysis by extracting metadata and demographics of practitioners. Later, we performed thematic analysis [9] to synthesize the data, focusing on identifying recurring themes within the data. In conjunction with thematic analysis, we also applied quasi-statistics [13] method that advocates to identify the most frequently occurred analytics benefits and practices.

4 RQ1: Benefits of Analytics for Software Startups

B1: Data-Driven Decision Making

Facilitating startups to make data-driven decisions appeared as one of the key advantages characterized by several practitioners. Smart decisions, quick decisions, and informed decisions are the possible outcomes startups can achieve by utilizing analytics. For example, in the instance of GL8, the practitioner reported:’By understanding these metrics, data-driven business decisions can be made”. Decisions cover a wide range of tasks in which startup founders must be interested. It includes decisions, for instance, identifying best-performing acquisition channels or identifying the type of interested users.

B2: Improving Efficiency and Focus

Startups can certainly improve their business efficiency and start focusing on things that really matter. A practitioner from [GL3] alluded: ‘you want to start using data to drive your focus”. It is complemented by another practitioner in [GL9] in the following words:“[Analytics] helps you really keep it there, like figure out where to start, where to focus...your efforts when you’re thinking about your product. and what to do next”.

B3: Visibility and Realism

B3 promises comprehensive visibility of the startup as a business, and, more importantly, brings founders closer to reality. According to [GL1], visibility means “what’s going on across our business in the corner of our eye...knowing that if something big happens we’re not going to miss it.”. On the other hand, startup founders are always in love with their ideas [4]. Here, “analytics helps you [to] be real with yourself. Do customers actually want this?”, added by practitioner from [GL13].

B4: Enhancing User Experience

Startups can achieve user experience enhancement by using analytics in several ways. For example, by getting in-depth user insights, improving user engagement, and maximizing user retention. The practitioner[GL3] encouraged this in the following words: “ [understand] who is the user and what are some characteristics of this user”. Another practitioner from [GL9] goes deeper into this and explains the user understanding process: “[Identify] what are the demographics, behavioral details, what are their needs, obstacles...you likely might have already some sort of profile of your users...”.

B5: Fostering Data-Driven Culture

Analytics can foster a data-driven culture inside a startup. Eventually, data becomes the language that everyone speaks in the company. It is reported at length, for instance, in [GL12] in the following words: “Want to have a culture at your startup that believes in data...that looks at the metrics all the time and that starts at the top, the CEO, and the VPs... the people who watch these numbers, who measure these numbers... And who talk about them in group meetings, who talk about them in their emails”.

B6: Understanding and Insights

B6 promises comprehensive real-time insights to understand various actions and outcomes for a startup. It covers aspects like “what’s happening right now”, as the practitioner [GL1] reported. The practitioner continued explaining this in the following excerpt: “something great, maybe we’re featured in a blog post that we didn’t expect to get a huge influx of traffic”. A similar indication about real-time insights is furnished by [GL11] in the following words: “it is important because obviously, you should know what state your business is in at all times”.

B7: Detecting Growth Challenges

Analytics helps startups to detect all the possible user growth issues as well. Startups might get some customers early on but then the user growth, retention, or engagement decreases. According to the practitioner from [GL3], one apparent reason is the product-market fit. He mentions this in the following words: “the products that have no product market, the engagement over time, for all cohorts, will go to zero”.

B8: Team Alignment

Another noteworthy benefit that analytics can offer is team alignment. The insights obtained through analytics can make everyone on the same page. This is supported by a practitioner from [GL3], who expressed his opinion in the following excerpt: ‘you want to motivate your team... use this data... So what you’re gonna do is you’re gonna set [shared] goals”. Adding on top of that, while explaining the questions that analytics can help out with, the practitioner from [GL9] commented:“...and so this helps to create alignment on your team”.

B9: Improving Product Usability

Startups, usually in the early stages, need to launch their products. They can assess with the help of analytics how usable the product is, how users are using it, do users understand the product, and which features are getting popular. The practitioner at [GL2] thinks that “almost every product that’s launched is unusable or highly unusable for the first three months”. That is the time to improve product usability through analytics.

B10: Supporting Product Development and Enhancement

This theme reports two perspectives. The first one is related to testing the product market fit, a fundamental activity for startups. The second one is accelerating product development through analytics. Both perspectives insist on a feedback mechanism to elicit user behavior. A practitioner from [GL13] reported:“analytics is incredibly important... it helps you test product-market fit”? Another practitioner from [GL11] agrees and states its use in ”building new features, launching new features, and so on” (Table 1).

Table 1. Overview of the Identified Benefits of Analytics for Software Startups
Full size table

5 RQ2: Practices to Define Analytics in Software Startups

5.1 Prioritize Key Metrics

The most prominent advice reported by practitioners is to identify top-level KPIs first. It is explicitly highlighted in 11 videos. While there exists a lot of definitions of KPI, the practitioner from [GL11] defines it as a “set of quantitative metrics that indicate how healthy your business is doing”. There are a plethora of metrics available to startups. However, like others, practitioner in [GL3], indicated to select one. He expressed it in the following words: “there is usually almost only one metric that represents value for each company”.

Thereafter, in eight videos, there are guidelines on selecting and defining the KPI from a variety of metrics. The practitioner in [GL1] guided in the following words:“the one metric that matters is the metric that you choose to focus on, so that’s the metric that you’ve decided will have the biggest impact on your growth”. Going into more details and while guiding how startups can selecting top-level KPIs, a practitioner from [GL16] commented:“what is a number that you’re willing to bet the company on? If that number goes south. You deserve to die. And if that number goes up. You will like...you will have made a huge difference in the universe”. Our data analysis also reveals that the business domain of a startup is an important factor in deciding the top-level KPI. It will vary from domain to domain and thus there is no silver bullet.

Later, adding supporting metrics to top-level KPI is considered an essential step. It is found in four videos. Some practitioners like [GL1] referred to it as “nuance” metrics while others, such as, [GL9] referred to it as secondary metrics. However, the purpose remains the same. As an example, if the selected KPI for an e-commerce startup is the number of sales then average sales or a unique number of customers will help to present the full picture with top-level KPI[GL1].

Lastly, we come across the indication of regular monitoring of selected KPI. Practitioners consider monitoring and taking action based on monitoring as essential as the selection itself. Commenting on this, the practitioner in [GL1] mentioned:“if we pick KPIs and then ignore them... we’re also in trouble...if we pick and monitor our KPIs diligently but we don’t assess... everything we do and everything a whole team does around o...at the end of the day, we’re still screwed”.

5.2 Keep Analytics Simple

This theme classifies and presents high-level codes that strive to educate startups on the basics of setting analytics in their companies. The first lesson practitioners communicate here is to learn that ’less is more’. Our data analysis, based on instances found in seven videos, highlighted that some founders become overwhelmed with analytics as they attempt to model every aspect of their startup. It is apparent from the following excerpt of a practitioner [GL8]“The point is not to track everything because eventually if you do try to track everything, you’re just going to be... ended up in a [situation] where you’re just tracking things without actually making it... decisions without actions”. Another practitioner[GL16] expressed:“Don’t boil the ocean...”.“less is more”, he added further.

Next, we have a very similar but critical issue, labeled as “analysis paralysis”. This situation occurs when a startup starts over-complicating analytics stuff e.g. selecting the best analytics tool, building a tool from scratch, thinking too much about selecting the right metrics, and putting a lot of time into looking at the data. The issue is referred to as analysis paralysis. One of the practitioners[GL1] warns startups by pointing out how to know if they are doing analysis paralysis. The practitioner reported:“when are you spending too much time looking at the numbers? versus actually action stuff”.

Along the same lines, accurate estimates are not required when a startup is using analytics. It was highlighted in four videos in different instances. For instance, the practitioner[GL5] advised it in the following excerpt: “you’re a startup. You’re not going to have a lot of data to be able to do like fine-grained analysis... You may have some data, you may have other people’s data, you can still draw a box. around. products”.

The last category in this theme refers to the adoption and focus regarding analytics. This was presented in five videos. It states that with the passage of time, focus on KPIs and metrics change, tools change and business segments change as well when startups pivot. As an example, the practitioner[GL10] clearly emphasized:“companies mature and grow, they start to shift their attention from the metrics that they used in the beginning stages of their business to metrics that are important later on in their business”.

6 Conclusions and Future Work

Our research presented ten analytics benefits and two practices for software startups, drawing on experiences of startup practitioners. Primarily, our findings are particularly relevant for early-stage startups, as these companies are often hesitant to practice analytics. On the other hand, we conclude that while there is no silver bullet solution to define the top-level KPI, answering a few questions and the business domain of a startup might contribute to define it. Likewise, our results also highlight areas directly influenced by analytics. For example, the immediate impact of using analytics produces product design decisions, product engagement strategies, and enhancement of user experiences. At same time, analytics is found offering a supporting role to solve fundamental pain points of startups. It includes identifying the target customers, target market, or testing product market fit.

In the current study, we fell short of utilizing snowballing techniques to figure out more related videos such as YouTube recommendations and indications of other sources in our data. Therefore, this remains an important addition for future work. Moreover, additional work is needed to include blog posts and website data to draw a full picture of analytics inside startup companies. Therefore, we intend to take these variables into account for our immediate future work.