Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Cloud computing has transformed the Information Technology (IT) industry [1] by hosting applications and providing resources (e.g., CPU and storage) as services on-demand over the Internet [15]. Cloud providers, such as Google Cloud Platform and OpenStack, usually offer these services in the form of REST (REpresentational State Transfer) [4] APIs, the de facto standard adopted by many software organisations for publishing their services.

However, although cloud computing offers huge opportunities for the IT industry and has gained maturity, there are still many issues that must be addressed [15]. In particular, we observe that cloud providers, such as Google Cloud Platform, present their own proprietary APIs. Other cloud APIs, although proprietary such as OpenStack, provide open implementations of cloud services. Conversely, open and standard cloud APIs have been proposed, such as the Open Cloud Computing Interface (OCCI) [7], which is a neutral-vendor cloud standard.

Consequently, there exists a wide variety of cloud APIs that might be difficult to understand and use by developers, especially within a complex and technical context as cloud computing. Moreover, well-designed REST APIs may attract client developers to use them more than poorly designed ones, particularly in the current open market, where Web services are competing against one another [6]. Indeed, client developers must understand the providers’ APIs while designing and developing their systems that use these APIs. Therefore, in the design and development of REST APIs, their understandability and reusability are two major quality characteristics, which are reachable when best practices for REST APIs design [6] are followed.

Several practices were proposed or identified in the literature [2, 6, 8, 9, 12] as CRUD function names should not be used in URIs or Lowercase letters should be preferred in URI paths. In particular, a valuable contribution is the one of Massé [6], who compiles several design practices about REST APIs. Yet, despite proposing a large list of 65 practices, Massé [6] did not propose a complete list.

Consequently, we propose two contributions. For our first contribution, we review the literature extensively and compile a catalog of 73 best practices in the design of REST APIs making APIs more understandable and reusable.

For our second contribution, we evaluate and compare the design of the cloud computing REST APIs using best practices of this catalog. Compared to previous works, including some of ours [2, 8, 9], we study the conformance with best practices of REST APIs from the perspective of cloud providers.

After identifying and analysing 73 best practices, our results show that Google Cloud Platform follows 66 % (48/73), OpenStack follows 62 % (45/73), and OCCI 1.2 follows 56 % (41/73) of the best practices. Second, although these numbers are not necessarily high, partly because of the strict and precise specification of best practices, we showed that cloud APIs reach an acceptable level of maturity.

The reminder of the paper is organised as follows. Section 2 presents a survey about best practices on REST APIs to support our evaluation. Section 3 describes the study performed on the three cloud computing REST APIs. Section 4 presents and discusses our results and the threats to their validity. Section 5 presents some related work. Finally, Sect. 6 concludes the paper with future work.

2 Best Practices on REST API Design

REST APIs are hard to design [6] because they are not often based on precise and documented specifications but only on an architectural style [4]. Thus, we now present a catalog of REST API best practices, pertaining to understandability and reusability, extracted from the literature and organised to support our analysis on cloud REST APIs.

To build our catalog, we surveyed several studies of REST APIs elaborated by Massé [6], Rodrigues et al. [2], Palma et al. [8, 9], Vinoski [13], Stowe [12], and Richardson and Ruby [11]. In particular, Massé [6] provides a concise catalog of practices organised by categories. We analysed all the cited studies to identify good practices and organise them by categories inspired from Massé’s work.

Table 1. Numbers of practices by category
Full size table

Our literature review produced a catalog of 73 best practices to design understandable and reusable REST APIs, grouped into five categories. Table 1 lists the categories and the numbers of practices per category. Tables 2, 3, 4, 5 and 6 describe the identified practices in each category with a short description, relevant references, and the results of our analysis on the three Cloud REST APIs. The analysis of each API is further discussed in Sect. 4.

The first category, URI practices, describes how URIs are exposed by services (Table 2). The second category, Request Methods, describes how HTTP methods must be used by REST APIs (Table 3). Error Handling practices specify how HTTP messages must be used as a response of a HTTP request method (Table 4). HTTP Header practices describe how must be used HTTP headers to complete requests with metadata or complementary data (Table 5). Finally, Others is the category for grouping different and various practices as Media Types, Message Body Format, Versioning, Security, Response Representation Composition, Documentation and Hypermedia Representation (Table 6).

Table 2. URI design best practices
Full size table
Table 3. Request methods best practices
Full size table
Table 4. Error handling best practices
Full size table
Table 5. HTTP header best practices
Full size table
Table 6. Other best practices
Full size table

3 Study Design

This section presents the design of our study, which aims to address the following four research questions:

 

RQ1:

What are the main services provided by cloud REST APIs?

RQ2:

How many best practices are followed by cloud REST APIs?

RQ3:

What best practices are adopted by all APIs?

RQ4:

What best practices are adopted by none of the APIs?

 

3.1 Objects

The objects of our study are three different cloud REST APIs including the proprietary cloud API of Google Cloud Platform, the open source API of OpenStack, and the standard OCCI. We specifically target these APIs because they represent the range of the different types of cloud APIs available: commercial offer, open source implementation, and open standard.

Here is a short description of each of the three studied APIs:

Google Cloud Platform is a proprietary cloud platform that consists of a set of physical assets (e.g., computers and hard disk drives) and virtual resources (e.g., virtual machines, a.k.a. VMs) hosted in Google’s data centers around the globe. Google Cloud documentation is available at https://cloud.google.com/docs.

OpenStack is an open source cloud platform that controls large pools of compute, storage, and networking resources throughout a datacenter. OpenStack documentation is available at http://docs.openstack.org.

Open Cloud Computing Interface (OCCI) is a cloud computing standard that comprises a set of open community-lead specifications delivered through the Open Grid Forum. OCCI is a protocol and API for all kinds of management tasks. OCCI 1.2 documentation is available at http://occi-wg.org/about/specification.

3.2 Procedure

We investigated and analysed manually in details the documentation of each of the three APIs studied. More precisely, we identified the services provided by each API and extracted the list of URIs. Then, we compared them with each best practice as defined in our compiled catalog of best practices. The analysis of OCCI has been cross-validated by two contributors of the standard. We performed an analysis of the three APIs and reported the results of this analysis in several tables given in the next section dedicated to the results.

4 Results

We now report the results of our analysis to answer our four research questions.

4.1 RQ1 What Are the Main Services Provided by Cloud REST APIs?

We performed a manual analysis on each REST API documentation to identify the services provided. This first analysis allows us to identify 11 services as listed in Table 7. Our results show that Google Cloud Platform and OpenStack provide all identified services (11/11) while OCCI describes only four services in its specifications (4/11). We conclude that Google and OpenStack API have a good support for several services while OCCI has yet some lacks.

Table 7. Service analysis comparing Google Cloud, OpenStack, and OCCI
Full size table

Indeed, current OCCI 1.2 official specifications only cover 4 of the 11 services listed in Table 7: VM Managing, Storage, Networking, and Tagging. However, OCCI-based services were proposed for Container Managing [10] and Monitoring [3]. Image Managing, Scaling, Access Control, Data Processing and Machine Learning would be addressed in future as there are key services for building open standard cloud platforms.

4.2 RQ2 How Many Best Practices Are Followed by Cloud REST APIs?

For each practice, we analysed the documentation of the corresponding API to assess whether this provider follows or not the practice. Tables 2-6 present the detailed results of this assessment for each API and the 73 best practices by category. Table 8 presents a summary of this assessment by category of practices and shows that on average 61 % (44/73) of the practices are followed by the three APIs. Google Cloud Platform follows 66 % (48/73), OpenStack follows 62 % (45/73), and OCCI follows 56 % (41/73) of the best practices.

Moreover, OCCI follows only 35 % (7/20) of URI practices, while Google Cloud Platform and OpenStack follow URI practices in 90 % (18/20) and 85 % (17/20), respectively. OCCI 1.2 fails to support URI design best practices, but future OCCI releases could improve this as the OCCI REST API is automatically synthetised from a metamodel instead of designed by hand as in Google Cloud Platform and OpenStack.

All APIs strongly apply Request Methods best practices listed in Table 3 with 87 % on average. Each API requires just one improvement on PUT method for Google Cloud Platform and OpenStack, and on HEAD method for OCCI. For the latter, OCCI implementations such as erocciFootnote 1 and rOCCIFootnote 2 already support HEAD method best practice, so a consensus in the OCCI community should be easily attainable to include this best practice into OCCI specifications.

Finally, Error Handling, HTTP Headers, and the other categories are the less followed with only 52 %, 46 %, and 56 %.

Error handling in Google Cloud Platform and OpenStack (6/16 or 37,5 % in Table 8) requires to be strongly improved with a better documentation in the error messages.

Regarding HTTP header best practices, all the three cloud REST APIs can be improved but especially OCCI 1.2, which only supports 1 practice, i.e., 10 % in Table 5. Here the OCCI 1.2 HTTP Protocol specification must be extended to support more HTTP header best practices. For instance, this specification must explicitly state that Content-Type must be used, Content-Length should be used, Last-Modified should be used in responses, ETag should be used in responses, Cache-Control, Expires, and Date response headers should be used to encourage caching, Cache-Control, Expires, and Pragma response headers may be used to discourage caching, and Caching should be encouraged. Let’s note that OCCI implementations such as erocci and rOCCI already implement most of these best practices, so the consensus into the OCCI community should be reasonably attainable. In contrast, both stores must support conditional PUT requests and location must be used to specify the URI of a newly created resource are best practices that no API seems to want to support.

Table 8. Followed practices by category and API
Full size table

4.3 RQ3 What Best Practices Are Adopted by All APIs?

In Table 9, we identified from our results the set of practices that all APIs follow, forming a “consensus”. We found that only 32 % (24/73) of practices were followed by all APIs. This means that the cloud API providers are not yet in agreement on the main good practices to prioritise and might be guided by technical decisions. However, it should be pointed out that the practices are strict and detailed. This explains why this number is very low. Moreover, we could identify that the majority of practices are adopted at least by one API. Overall, the APIs, even if do not follow strictly all best practices, implement relatively well all practices and are thus well-designed.

Table 9. Practices followed by All APIs
Full size table

4.4 RQ4 What Best Practices Are Adopted by None of the APIs?

An opposite analysis allows us to identify the set of practices that none API follows, forming a negative “consensus”. We found that only ten best practices (14 %) are applied by none of the three APIs analysed. Table 10 lists the practices followed by no API. This list could be analysed to understand why these practices are not followed by more APIs. For example, as any cloud API provider performs long running actions, e.g. starting a virtual machine takes some minutes, then all cloud APIs must return 202 (“Accepted”) HTTP status to indicate successful start of an asynchronous action. Another example is that 304 (“Not Modified”) should be used by all cloud APIs to preserve network bandwidth.

Table 10. Practices followed by No API
Full size table

4.5 Threats to Validity

As with any such empirical study, threats exist that reduce its validity, which we attempted to mitigate or had to accept. We now discuss these threats and the measures that we took with respect to them.

Threats to the construct validity of our study concern the relationship between theory and observations. We assumed (1) that good practices can be codified and shared among developers and (2) that these good practices improve the quality of the REST APIs of the cloud providers that follow them [15]. Although these assumptions are legitimate and have been withheld by many researchers and works before for example that of Zhang and Budgen [14], future work should study whether these good practices apply universally to all cloud services.

Threats to internal validity concern confounding factors that can affect our dependent variables. Although we did not carry any statistical analysis on the characteristics of the studied REST APIs, we assumed that the good practices were representative characteristics of the REST APIs. However, there may be other characteristics that describe more accurately these REST APIs, in particular their understandability and reusability. Future work include analysing and contrasting more APIs with more practices to uncover possible other characteristics. We also related the APIs and the practices manually thanks to the information provided in the literature and by the APIs documentations. Yet, other researchers should perform similar analysis to confirm/infirm ours.

Threats to conclusion validity deal with the relation between the treatment and the outcome. Again, we did not carry any statistical analysis between the REST APIs and the identified good practices and the characteristics of understandability and reusability. Yet, we argued that comparing these REST APIs according to their use of the practices is sensible because they belong to the same design space. We also accepted that external characteristics could influence their design and, hence, their quality. We accepted this threat and future work could uncover more novel characteristics and measures of REST APIs.

Threats to external validity concern the generalisability of our results. Although we presented, to the best of our knowledge, the largest study on the design of REST APIs based on an catalog of good practices from the literature, we cannot generalise our results to all REST APIs. Future work is necessary to analyse more REST APIs, from other cloud providers, to confirm and–or infirm our observations on their design quality characteristics.

5 Related Work

To the best of our knowledge, few work studied the evaluation of REST APIs.

Rodríguez et al. [2] evaluated the conformance of good and bad practices in REST APIs from the perspective of mobile applications. They analysed large data logs of HTTP calls collected from the Internet traffic of mobile applications, identified usage patterns from logs, and compared these patterns with design best practices. Zhou et al. [16] showed how to fix design problems related to the use of REST services in existing Northbound networking APIs in a Software Defined Network and how to design a REST Northbound API in the context of OpenStack. Both of these two previous work made contributions to the design evaluation of REST APIs for two specific domains, mobile and networking, while we consider the domain of cloud services.

Maleshkova et al. [5] analysed a set of 220 publicly-available Web APIs, including RPC, REST, and hybrid (a mix of RPC and REST) styles. They investigated six characteristics of the APIs: general information, types of Web APIs, input parameters, output formats, invocation details, and complementary documentation. This work provides a view on how Web APIs are developed and exposed. In particular, it shows that Web APIs are not necessarily REST and that they suffer from under-specification because important information such as data-type and HTTP methods are missing. This work supports our paper on the need to study the design of REST APIs.

In our previous works [8, 9], we evaluated the design of several REST APIs based on REST patterns and antipatterns, which correspond to good and bad practices in the design of REST services. However, the APIs evaluated were selected from different and general domains. They included Facebook, Twitter, Dropbox, and Bestbuy. So, it was not possible to compare and discuss the results among the APIs. Moreover, the list of patterns and antipatterns was really limited compared to the catalog of best practices presented in this paper.

6 Conclusion and Future Work

In this paper, we claimed that well designing REST APIs is difficult for cloud providers although there exist best practices pertaining to understandability and reusability. We supported our claim by performing, to the best of our knowledge, the first study evaluating and comparing the designs of the REST APIS of several cloud providers. We included in our study the REST APIs provided by Google Cloud Platform, OpenStack, and OCCI. We presented thus two contributions, a catalog of best practices from the literature and an evaluation of the use of these practices in three sets of APIs.

For our first contribution, we reviewed the literature extensively and compiled a catalog of 73 best practices in the design of REST APIs making APIs more understandable and reusable. We believe that our catalog is exhaustive at the time of writing. We hope that this catalog can be used by other researchers to study some quality characteristics of REST APIs as well as practitioners when designing, implementing, and assessing their own REST APIs.

For our second contribution, we evaluated and compared the design of the REST APIs using best practices from the literature and showed that Google Cloud follows 66 % (48/73), OpenStack follows 62 % (45/73), and OCCI 1.2 follows 56 % (41/73) of the best practices.

Thus, we showed that best practices can help evaluate REST APIs and design better REST APIs in terms of understandability and reusability. Moreover, in opposition of a recent assessment [2], we also showed that cloud APIs reach an acceptable level of maturity when considering good practices pertaining to understandability and reusability.

Future work includes studying whether these good practices apply universally to all cloud APIs. In particular, we planned to analyse and contrast more APIs, especially other commercial offers like Amazon Web Services and other open source cloud stacks like Apache’s CloudStack, with more practices to uncover possible other characteristics. Finally according to the results of this study, we will contribute to the improvement of OCCI specifications in order to make them more visible as OCCI is the only open standard addressing the management of any cloud computing resource.