Keywords

1 Introduction

Application Programming Interfaces (APIs) are an important component in software development. They allow for code modularity, reuse, and separation of concerns. With the advancement of web-based applications, web-based APIs have an even more prominent role, and have even been described as “the glue of the web”. While traditionally API usability has been studied in the context of development environments (IDEs) for desktop computers, research on web-based API usability is, to date, rather scarce. Moreover, APIs for Educational Technology (EdTech APIs) are even less often the subject of research with regard to their usability.

Web-based APIs are different to traditional APIs, not necessarily in structure, but primarily in the way they are offered. Dedicated portals are built to offer access to and document an API for developers. Online documentation and offering of APIs present unique usability challenges.

In addition, EdTech APIs present challenges specific to the domain of Education. For example, the variable utilisation of the hosting servers of an API is different in the case of EdTech as typically peak load will occur during a brief period of time, such as an examinations period. Other challenges include the increased need for transparency and data privacy. Another important aspect is that educational software applications are usually being developed by a combination of software and content developers; the case of subject-matter experts participating in the development process is not common in many fields, but in education content developers have to either prototype an application of verify its pedagogical appropriateness.

For the aforementioned reasons, it is necessary to treat EdTech API usability specially, as the domain-specific usability issues in a web context call for separate investigation.

2 EdTech APIs

In the Learnovate Centre, we partner with around 50 well-established EdTech organisationsFootnote 1. With regard to APIs that are EdTech domain-specific, we have identified the following recurring issues:

  • High scalability for short periods of time. Some of our partners need to scale their API usage for a brief period of time, usually during state exams. In some occasions the server utilisation has reportedly reached \(200*n\) of the required number of servers n for a period of only two days, while the utilisation during the rest of the year was ranging from n to \(2*n\).

  • Geo-location of cloud servers. Other Learnovate partners have a legal obligation to locate student data within the country they reside, which is not always possible with large sparsely located cloud server farms.

  • IP Protection. Other requirements concerning Intellectual Property and Data Protection may consist of a combination of institutional policies and legal obligations. Example of such requirements are the following:

    • — Universities require restriction on cloud usage.

    • — Requirements for ISO 27001 and other security certification.

    • — U.S. Safe Harbor agreement.

Overall, the aforementioned requirements consist barriers to API adoption. Moreover, the often requirement of a combination of two or all three of the above in EdTech enhances the domain-specific barriers to EdTech web-based API adoption.

3 Traditional API Usability

This section presents previous work around the usability of APIs. Since web-based APIs are a more recent advancement, most previous work assumes that developers have typically downloaded a Software Development Kit (SDK) and are using an IDE to write their code. Thus, this section describes usability issues around the use and documentation of APIs in IDE environments and not in interactive web consoles. Interactive web consoles will be discussed in the following section.

3.1 Identifying Usability Issues

Many different approaches have been taken in the study of API usability. One study [28] has correlated posts on bug-tracking systems with different API usability factors; the percentage of posts for each factor is presented below:

  1. 1.

    Missing feature — 43.5 %

  2. 2.

    Correctness — 31.1 %

  3. 3.

    Documentation — 27.3 %

  4. 4.

    Exposure of elements — 10.3 %

  5. 5.

    Memory management \({<}9\,\%\)

  6. 6.

    Function parameter and return \({<}9\,\%\)

  7. 7.

    Technical mismatch \({<}8\,\%\)

  8. 8.

    ...similar smaller issues include Naming, Callers perspective, constructor parameter.

While the percentage of each category of bugs does not necessarily imply its importance, it is shown sufficiently that API usability affects the final software product. Thus, it is of concern to the developer and the end user alike. Apart from requests for missing features, the high percentages of bugs around the correctness and the documentation of an API show that features that are poorly communicated are causing poor development despite their usefulness.

A different approach exists in [11]. The team of this research project conducted interviews with developers about the barriers to adopting an API, in order to identify any usability issues that may occur. Since reported usability issues may differ from actual ones, workshops were conducted with developers in order to validate the reported issues while developing a piece of software. Below is an ordered list with encountered usability problems:

  1. 1.

    Documentation

  2. 2.

    Conceptual correctness

  3. 3.

    Callers perspective

  4. 4.

    Complexity

  5. 5.

    Data types

  6. 6.

    Leftovers for client code

  7. 7.

    Error handling/Exceptions

  8. 8.

    Consistency and conventions

  9. 9.

    Method parameters and return type

  10. 10.

    Factory pattern.

While results are similar in both the case of workshops and interviews, some differences exist. Specifically, interviews also mentioned Naming, but it was not encountered in workshops.

Another finding was that consistent error messages across various error types can be beneficial for developers and enhance greatly the usability of an API. Eight error types have been grouped and identified in [1, 2], e.g. Structure error, Consistency error, etc.

Apart from identifying API usability factors, previous work has also investigated ways to improve the usability of APIs, and is presented below.

3.2 Improving API Usability

While the best way to design a usable API is to produce and follow guidelines that ensure its usability [12]Footnote 2, it is also important to provide to developers the appropriate tools that will help them understand and use an API.

A number of API classes and methods usability factors, which focus on API improvement, has been identified in [19] and is listed below.

  • High number of classes is negative, re-structuring in sub-packages is beneficial

  • Constructors are easier than factory methods

  • Distinctive names should be used

  • Parameters per method should be as low as possible

  • Knowledge of domain and programming exp equally useful

  • Few concepts (classes/methods) lead to high learnability.

One proposed way to improve API usability is to evaluate it automatically using Complexity Metrics and Visualizations [7]. To achieve this, a tool called Metrix has been built in order to evaluate Bandi et al. metrics, like the sum of Bandis complexity metrics for a class and the number of classes with a particular complexity.

Other approaches use text analytics to improve API usage within IDEs; for example, Jadeite uses a wordcloud-like experience to improve documentation by showing which calls are more frequent [22, 25]. This helps identify calls that are otherwise similar. Consider for example the call /system/student and the call /system/class/student: it is highly probable that a new developer would want to use the one that has already been used the most by other developers.

Finally, other ways to make APIs more developer-friendly are colour coding, tooltips in code editors/IDEs [8, 20], or diagrams [27] that help developers understand the mental model of the API.

However, while these tools improve greatly the usability of APIs that are typically part of an SDK (so that metadata to produce tooltips or diagrams are readily available), web APIs have additional challenges that need to be addressed.

4 Findings

4.1 API Usability Survey

Literature Review Findings. The table below presents a triangulated list of usability factors, compiled by combining the factors listed in the literature of the previous section. We anticipate that these issues are not specific to EdTech. Documentation, correctness, and low complexity, are the three factors that were persistent throughout (Table 1).

Table 1. This table shows the difference between the perceived API usability issues, as described in interviews, and the actual ones, identified in a workshop.
Full size table

Developer Survey Results. In order to specify EdTech-specific usability issues and factors, we conducted a survey across CTOs and developers of our industry partners \((n=11,k=49)\). The ease of integration into an organisation’s existing technology stack, the API performance, and its availability, were the technical factors that were described as essential (see Fig. 1). Concerning non-technical factors (see Fig. 2), as most important were described the API meeting the organisation’s needs, transparency as how data are stored and processed, and the data privacy policy of the service.

Fig. 1.
figure 1figure 1

The importance of technical factors that affect the adoption of APIs according to replies from CTOs and developers of EdTech industry partners of Learnovate. The survey was sent to Learnovate’s k partners, and n replied \((n=11, k=49)\).

Full size image
Fig. 2.
figure 2figure 2

The importance of non-technical factors that affect the adoption of APIs according to replies from CTOs and developers of EdTech industry partners of Learnovate. The survey was sent to Learnovate’s k partners, and n replied \((n=11, k=49)\). Policy issues around data protection are especially important in Educational Technology.

Full size image

From the results of this survey, we conclude that policy issues around Data Protection are a sine qua non for EdTech API adoption. As such, these policy issues should be considered as tightly coupled with API usability and investigated under this prism during the design and development of an API. In addition, it is important that portals offering EdTech APIs are transparent in describing these policies—this and other portal-related usability issues are described in the following section.

4.2 EdTech Portal Survey

The adoption and use of new software can be positively influenced through early experimentation with the software, the internalising of structures and cultures surrounding its use, and progress validation as it is being used [5]. In scenarios where the software is not already in use within an organisation the additional factors of software discoverability and clarity in software capabilities need to be addressed. In the case of web-based APIs these challenges are addressed in part through discovery services such as the API Directory provided by ProgrammableWebFootnote 3. However, increasingly there is a growth in API or developer portals to enhance the enrolment and on-boarding of developers.

Although API portals do exist for educational technology it was found that the majority of them were either simple documentation pages (EngradeFootnote 4, Khan AcademyFootnote 5) or were only open to select partners and affiliates (TurnItInFootnote 6, KnewtonFootnote 7). In light of this in order to understand common trends across API portals 18 portals were identified through web searches that offered web-based APIs (Table 2).

Table 2. API Portals Surveyed.
Full size table

Within these portals we assessed how they facilitated early experimentation and the internalisation of structures. The assessment of progress validation has more relevance within large software teams where mentors with prior experience exist [5]. Its consideration in the context of API portals is a future direction for this research.

Early experimentation with web-based APIs requires readily available access to the functioning APIs and access to documentation. The following factors were identified as potential barriers to early experimentation:

  • Access to documentation without signup

  • Complexity of the signup process

  • Availability of a cost-free pricing tier

  • Availability of interactive documentation.

Of the portals surveyed all provided access to API documentation allowing developers to begin planning integrations without committing to a particular API. Although documentation goes part of the way to enable experimentation it still relies on the developer technically integrating with the API before realistic queries can be made and responses received. In order for this to happen in all cases except one (Clever) the developer has to signup to the portal in question.

Fig. 3.
figure 3figure 3

Usability metrics of the surveyed API portals. Above, interactive documentation is increasingly common. Below, the number of required fields to sign up varies greatly; complexity can be a barrier to adoption.

Full size image

The use of interactive documentation is increasingly common in API portals (see Fig. 3). Such documentation allows developers to make live API calls from within the portal where the submitted fields and responses are documented. By allowing developers to test real world queries to an API without the need to write any code, early experimentation is encouraged through interactive documentation. Examples of common interactive API documentation are Swagger UI, Mashery IO Docs, 3Scale Active Docs, and Mulesoft RAML API Console.

The complexity of the signup process is seen as a barrier to experimentation and it is desirable to simplify the process. As a simple indicator, the complexity of signup was determined by the number of form elements (e.g. text fields, check boxes, drop downs, etc.) that had to be read and completed (see Fig. 3). The median number of form elements was nine with the minimum being four as required by Mendeley (first name, last name, email, password).

Four portals also provided OAuth sign-in for using other services, removing the need to complete a registration form and further easing signup. However, only one portal (Marvel) offered several service logins (Facebook, Google, Yahoo, Twitter, Marvel).

Two of the portals did not allow online signup as they only provided access to approved partners or affiliates (Nike+, Yellow Pages (Canada)).

5 EdTech API Usability Requirements and Guidelines

As an output of our research we produced the following guidelines, and designed a portal which incorporates these guidelines.

API Packaging Related. These guidelines are related to API Packaging. API designers and developers are encouraged to use them; please note that guidelines marked with an asterisk \(\{*\}\) are EdTech domain specific (Fig. 4).

  • * Domain knowledge balances lack of programming experience

  • * Data Processing should be transparent

  • APIs should be generic enough, that is decoupled from specific applications

  • Sub-packaging is better than large categories

  • Fewer method parameters is better

  • Categorise error messages consistently.

API Portal Related. Other usability aspects that affect API usage and are related to the portal that offers the API are listed below. As above, guidelines marked with an asterisk \(\{*\}\) are EdTech domain specific (Fig. 5).

  • * Explain the API’s Data Protection policy (or how it may facilitate an application’s policy)

  • Provide an interactive console documentation system

  • Visualise the popularity of a method call

  • Colour code to improve learnability

  • Represent visually the API’s structure.

Fig. 4.
figure 4figure 4

(a) Colour-coding the structure of a call at an interactive console can improve the legibility of the documentation and, thus, the learnability of an API. (b) A bar that visualises the amount of calls on that method helps distinguish it from similar ones (Color figure online).

Full size image
Fig. 5.
figure 5figure 5

A diagram that represents the structure of an API helps developers understand its mental model. Visual learners benefit especially from this representation, as API documentation is typically in text form.

Full size image

6 Conclusions

In conclusion, after reviewing the API usability literature, surveying EdTech CTOs and developers, and surveying API portals, we created a concrete set of eleven guidelines that facilitate usable APIs for EdTech applications. While most of the guidelines hold for each and every API, some are specific to the EdTech field.

Moreover, we have designed and implemented a portal that offers EdTech APIs and implements the above guidelines. However, other aspects of the portal are still under development, thus public access is still not available (contact us for details).