From Requirements to a RESTful Web Service: Engineering Content Oriented Web Services with REST

Chapter

Abstract

This chapter presents an approach for proceeding from a set of requirements to an implemented RESTful Web service for content oriented systems. The requirements are captured into a simple domain model and then refined into a resource model. The resource model re-organizes the domain concepts into addressable entities: resources and interconnecting links, hypermedia representations, URIs and default HTTP operations and status codes. The approach has emerged from the experiences gained during developing RESTful Web services at Nokia Research Center.

Introduction

REST and Resource Oriented Architecture (ROA) (Richardson and Ruby 2007) are particularly well suited for content oriented Web services whose core value are in storing, retrieving and managing interlinked content through a uniform interface. While REST has gained significant popularity as the architecture for Web services, there is a notable lack of methods and modeling notations for developing RESTful services from requirements. There is a need for communicating the requirements and design intent to the different stakeholders, as well as to map the requirements and new features to the existing implementation in a way that preserves consistency of the API and supports service evolution over time.

Probably the best known formulation of how to design RESTful Web services has been presented by Richardson and Ruby which involves finding resources and their relationships, selecting uniform operations and respective response codes for each resource, and defining their data formats. The formulation is too abstract to be followed as a method and does not facilitate communication between service architects and other stakeholders. Even with a priori content oriented services, it is often a non-trivial exercise to refine a functional specification to resource-oriented, descriptive state information content. According to our experiences, many developers find it hard to make a paradigm shift from object oriented design that emphasizes hiding state-related data behind task-specific operations of an instructive interface. Such API centric thinking has a tendency to make resulting Web services look more like a collection of unrelated APIs instead of a set of interlinked content accessible using a uniform interface.

This chapter presents a systematic but light-weight approach for proceeding from a set of requirements to an implemented RESTful Web service and integrating new requirements with an existing system for content oriented services. The requirements are first collected into a domain model – expressed with UML class diagrams – which is essentially a structural model describing the domain concepts, their properties and relationships, and annotated with information about required searching, filtering and other retrieval functionality, and constraints. The domain model is then gradually refined into a resource model that is used to derive the resources and interconnecting links, representations, assigned URIs, and default HTTP operations and response codes of the Web service. The concepts of a resource model can be further mapped to implementation level concepts in service specifications, database schema and source code.

The approach has emerged during the development of a RESTful Web service for building Mixed Reality services at Nokia Research Center and reported in Selonen et al. (2010a,b). The gained experiences suggest it supports architects’ communication with the service clients, and perhaps more importantly, with software engineers not familiar with REST and ROA. The presented approach helps bringing in new features to the system in a non-intrusive way in a rapid pace while supporting traceability of requirements and actual implementation.

The chapter is organized as follows. In “Overview of the RESTifying Approach,” we introduce the “RESTifying approach” with an overview of the process, domain models, resource models and discussion on how to refine the former into the latter. “Example Web Service: Social Points of Interest” gives a step-by-step example on how to apply the approach for a simple Web service starting with requirements and domain model, and refining the domain model into resource models and respective components of a RESTful Web service. “From Resource Model to Implementation” discusses how to implement the service defined by the resource model. Finally, “Concluding Remarks” gives concluding remarks.

Overview of the RESTifying Approach

While representing arbitrary functionality as uniform resource manipulations is hard, content oriented systems already exhibit resource oriented characteristics with some additional filtering and querying capabilities. With this notion, we claim that for such systems, a domain model can essentially capture most of the system requirements. In order to refine a domain model to a resource model, we identify a minimal set of modeling concepts that can then be mapped to implementation level concepts and hypermedia content offered through a RESTful interface.

Figure 11.1 shows the artifacts produced by the approach. The service requirements are captured into a domain model, expressed as UML class diagrams. The domain model is then refined into a resource model. A resource model organizes the concepts of the domain model to addressable entities that can be mapped to elements of a RESTful Web service interface, service implementation and database schema. We use UML profiles to constrain the elements in the domain model and information model.
Fig. 11.1

The artifacts produced by the approach

Domain Model

The domain model describes concepts of a system, the attributes and attribute types related to the concepts, sub-concepts of the concepts and filtering criteria related to the concepts. As it represents the key concepts of the system and their attributes using the vocabulary of the problem domain, it can be used to communicate the system requirements among the system stakeholders, and as a starting point for software development. It is in principle a subset of a vanilla class diagram with a few additional annotations.

Since content oriented Web services have by definition their value in storing, retrieving and managing content, we assume that a domain model with additional constraints for queries and attribute values can capture essentially enough information to be used as the main source for building the service.

Figure 11.2 illustrates a simple domain model. It has concepts, concept attributes and associations between the concepts. Attributes can have types; associations can have multiplicities (cardinalities) and they can be directed and composite (whole–part relationships). The domain model profile package shown in Fig. 11.1 simply defines the allowed UML elements: classes, associations, attributes, generalizations and comments.
Fig. 11.2

An example domain model

Looking at the domain model, one can outline a respective RESTful Web service: classes look like candidates for resources, attributes as constituents of resource representations, and associations as links. However, it is not obvious how exactly the links are represented, what URIs are assigned to exposed resources, what HTTP operations are allowed per resource and how creation, retrieval, update and deletion of resources is to be allowed.

Resource Model

A resource model re-organizes the elements of a domain model to addressable entities that can be more easily mapped to resources of a RESTful Web service. The concepts of a domain model become resources; depending on their association multiplicities, they either become Items or Containers containing Items. Compositions become resource–subresource hierarchies that are reflected by the URI paths while normal associations become hypermedia references between resources. Attributes are used to generate resource representations and candidates for hypermedia content types. The resource model concept is adapted from Laitkorpi et al. (2009).

Figure 11.3 shows the concepts of a resource model as a resource model profile. Items represent individual resources having a state that can be retrieved, created, modified and deleted. Containers can be used for retrieving collections of items and creating new ones. Projections are filtered views to containers. Resources can have sub-resources and links to other resources. In addition to the resources and interconnecting links, a resource model instance can be used to infer the other components of a RESTful Web service: resource representations, assigned URIs, and default uniform HTTP operations and response codes. The resource model is formalized into a UML profile, where each element becomes a stereotype.
Fig. 11.3

Simple resource model profile

Representation for Resource r is as follows:

In the proposed model, Containers and Projections do not have properties and thus neither do their representations. Each Item i of type T has exactly one id attribute that defines a unique name (among other items of type T). For Containers, id attribute is the name of the container. A relative URI for Resource r is defined as follows:

The default HTTP operations and response codes for resources are as follows:

The default status codes for each resource and request can be selected from the HTTP status codes: 200 OK for successful operation, 201 Created for successfully creating a new resource, 400 Bad Request for requests with malformed representations, 404 Not Found for non-existing resources, 405 Method Not Allowed for unsupported methods and so forth. The implemented service can decide a finer level of granularity and select more refined communication patterns at will. For example, if a resource contains read-only properties or properties that can only be incremented, attempting to modify them might result in 403 Forbidden or 409 Conflict status codes.

Refining a Domain Model to a Resource Model

In the domain model, each concept represents an aspect of the system that will become an addressable resource. Concepts can link to other concepts. Links will become links in the hypertext representations. Concepts can also have subconcepts which are parts whose existence is tied to the parent concept. Subconcepts that do not warrant individually addressable resources can be marked as data types (UML  ≪ dataType ≫  stereotype).

Concepts can have attributes that define their state and representation. An  ≪ id ≫  attribute represents a textual identifier that will be used when constructing the URI of the concept. For this presentation, we define two additional types of attributes: a  ≪ readOnly ≫  attribute is a read only attribute that will be set by the system and that the user cannot modify, and an  ≪ appendOnly ≫  counter attribute whose value can only be incremented. The domain model elements can be mapped to resource model elements roughly as follows:

  1. 1.

    Domain classes represent the domain concepts: the content the service is to manage. If not defined otherwise, classes will become  ≪ item ≫  resources: addressable resources of their own right with a URI and representation.

     
  2. 2.

    Attributes of classes as well as data types belonging to classes will become attributes of respective  ≪ item ≫  resources and manifest themselves in the resource representations.

     
  3. 3.

    Associations represent relationships between the concepts and they will become  ≪ ref ≫  associations between resources that appear as links in the representations. Bi-directional associations (i.e. associations navigable to both directions) are represented as two directed  ≪ ref ≫  associations.

     
  4. 4.

    Composite associations represent whole-part relationships between resources and subresources, and become  ≪ sub ≫  associations between resources.

     
  5. 5.

    Associations representing collection of elements – i.e. associations having upper multiplicity bound greater than 1 – will manifest themselves as  ≪ container ≫  resources containing  ≪ item ≫  resources.

     
  6. 6.

    Notes attached to classes informally describing queries (searches, filtering) are mapped to  ≪ projection ≫  resources with attributes for each query criterion. Query attributes that refer to resource attributes are marked with  ≪ index ≫ .

     
  7. 7.

    Attributes constrained informally in notes are marked with respective constraint stereotypes. For example, if an attribute is read only (e.g. whose value is to be set by the system), one can use  ≪ readOnly ≫ . There can be common constraints, but it is usually up to the service architect to identify and pre-define such constraints and how they are mapped to implementation level concepts.

     

In “Example Web Service: Social Points of Interest” the above approach is applied to a small but non-trivial Web service for social Points of Interest.

Content Negotiation, Inlining and Verbosity

A resource model is independent of any particular content type. In what follows, we will give examples using plain XML. However, it is possible to support any structured representation format for data interchange, including JSON and Atom Publishing Format.

Our framework allows clients to control the number of requests and amount of transferred data through inlining and verbosity. With inlining, the client chooses the interlinked resources to be included to a request response instead of having to fetch each linked resource using separate requests. With verbosity, the client can avoid consuming full representations and get only the most important properties per resource. Both inlining and verbosity are communicated using a custom HTTP headers: x-rest-inline: linkname, linkname and x-rest-verbosity: level. Inlining and verbosity have proved to be valuable concepts when developing RESTful Web services for constrained clients like mobile devices.

Example Web Service: Social Points of Interest

To exemplify the approach, we develop a simple service that allows users to create and share personal points of interests. Consider the following high-level service description:

A user can create points of interests (POIs) that can have a title and a description. Users can assign a location (as coordinates) and tags to a POI. Other users can comment POIs and rate them with a thumb up or down vote. Users can search for POIs based on their location (radius and bounded box search), their tags and their popularity (view count). All created content is public, but only the authors of any particular content element can modify and delete them; other users can only read content made by somebody else.

The requirements clearly describe a content oriented Web service: its value is in creating, retrieving, modifying and deleting content, with additional requirements on searching for content and enforcing simple constraints.

Same requirements can yield several similar alternative domain models. Design decisions like whether one resource is a subresource of another one or just linking to each other, or whether a particular property of a resource is promoted to be exposed as a subresource with its own URI, will lead to slightly varying service descriptions. Regardless of the stucture of the particular domain model, however, we can derive a RESTful Web service interface exposing the information content present in the model. If the resulting API turns out to have unwanted or missing features – for example, resources that are always retrieved at the same time but not connected in the domain model, resulting in unnecessary client–server round-trips – the domain model can be adjusted accordingly.

Figure 11.4 shows the domain model for the POI Web service derived from the service description given above. While it is obviously not the only possible model, it nevertheless should be a reasonably good approximation of the requirements.
Fig. 11.4

Domain model for social points of interest Web service

The domain model defines following structural features:

  • User of a system has a unique username and associations to owned POIs, Comments and Ratings.

  • POI is owned by a User. It has a title and a description, (preparing for the inevitable future request) creation and modification date, and total number of thumbs up and down. POIs have subelements Viewcount, Ratings, Tags and at most one Location. POI can link to arbitrary amount of Comments.

  • Viewcount is modeled as a separate class to emphasize increasing view count being (an implicit) requirement.

  • Location has a longitude, latitude and altitude.

  • Tag is a simple textual element.

  • Comment has a text body and an association to the commented POI.

  • Rating has a thumb attribute for thumbs up or down. Each Rating points to exactly one User and one POI.

In addition, informal features and constraints defined in notes are as follows:

  • POIs can be searched based on bounded box, radius, tags and popularity (view count),

  • attributes for thumbs up and thumbs down are read only attributes, based on the ratings given for the POI, and

  • view count can only be incremented (implicit requirement).

There is no explicit requirement to model Viewcount, Location and Tag as classes; however, a good rule of thumb is to model each concept that might either represent a key concept in the system that we might want to expose as an addressable entity, or that is used as part of a query or whose usage is otherwise constrained.

The use of a composition association (black diamond) denotes that both Location and Tag are parts of a POI and not first class citizens of the Service. This also implies a lifetime constraint: if a POI is removed from the Service, its Location and Tags are removed as well. Further, the multiplicities indicate that any individual Location or Tag element must always point to exactly one POI. There can be at most one Location element associated with any POI, but arbitrary amount of Tags.

In what follows, subsets of the domain model related to specific requirements are looked at in an iterative manner and refined into resource model fragments and RESTful Web service descriptions. The final resource representations, links, and URIs are incrementally merged from partial descriptions.

Root Level Resources

Domain model concepts that are not subconcepts of others but defined as first class entities become root level resources. Resource model for the root level containers is shown in Fig. 11.5. Each class that does not have an owning composite class becomes a root level  ≪ item ≫  : POI, User and Comment. Unless defined otherwise, we assume there can be an arbitrary amount of each root element and thus create  ≪ container ≫  elements for the items. As a convention, we use a plural form of the contained item name for the containers: POIs, Users and Comments. The  ≪ root ≫  element is for illustration only, representing the Web service to be built and defining its root URI path. Resources have URIs and thus need an identifier: for containers, the identifier is the container name; for items, we can define one explicitly by an attribute with stereotype  ≪ id ≫  (User.username) or omit it and get one generated by the system.
Fig. 11.5

Root element resource model

Following the approach defined in “Resource Model,” the implied URIs and partial resource representations are as follows (with the “http://example.com prefix omitted):

The operations and HTTP status codes follow from the default operations defined for containers and items. Note that following the above definition, a User is just another resource in the service. In practice, we probably want to restrict creation of new Users to be done through some specific administrative interface. In principle, though, there is no fundamental need why User resources cannot be accessed through the same unified interface. The resource model further implies the following representation fragments:

Representations for Users and Comments are derived in a similar fashion.

Points of Interest Resource Model

Resource model for a Point of Interest is shown in Fig. 11.6. The POI, Viewcount, Location and Tag elements of the domain model are refined into respective resource model concepts. POI, Viewcount, Location and Tag elements become  ≪ item ≫  elements with respective attributes, with the former being a composite class containing the three latter using a  ≪ sub ≫  association. We decide not to expose view count or tags as first class elements with URIs, so we define them as  ≪ property ≫  elements. We use two special but generic stereotypes to represent the attribute constraints:  ≪ readOnly ≫  for read only attributes (created and modified dates are set automatically by the system) and  ≪ appendOnly ≫  for attributes whose value can only be increased (viewcount).
Fig. 11.6

Point of interest resource model and subresources

The resource model in Fig. 11.6 implies the following resources:

The resource model further implies the following resource representation with location subresource inlined (x-rest-inline: location, see 11):

The decision on whether parts of a resource should be promoted as addressable subresources to be individually retrieved and updated is up to the service architect: is a particular aspect of a resource worth exposing independently instead of being just a property of the main resource. In the example, we could choose to expose a  ≪ container ≫  Tags containing  ≪ item ≫  Tag elements, as well as a  ≪ item ≫  Viewcount.

Queries and Filtering Resource Model

Following from the domain model fragment shown in Fig. 11.4 and the previous resource models, we further derive the projections corresponding to the different queries. Figure 11.7 shows a resource model for POI related projections. We build a  ≪ projection ≫  for each identified query: search by radius, search by bounded box (“area”), search by a tag and order by popularity (“most_viewed”). Each projection is effectively offering a projection to the resources contained by the POIs container, making them are  ≪ sub ≫  subresources of the POIs container. As they do not own the resources of their parent container, they instead produce a set of  ≪ ref ≫  resource references depending on the query parameters provided by the user. The projection attributes define the search parameters; the ones referring directly to POI attributes can be marked with  ≪ index ≫  stereotype to hint later in the implementation phase that the property is used for indexing. Search functionality is obviously critical for the performance of a Web service. Details beyond simple searching over indexed attributes are beyond the scope of the chapter.
Fig. 11.7

Point of interest resource model for projections: search by radius, area, tags, and view count

The resource model implies the following new projection resources:

One can name the queries as above or alternatively, if a query is uniquely identifiable by its parameters, omit the name. It is possible to combine the queries and have queries with several parameters like:

An example of a POI projection representation is as follows:

A client can have design time knowledge of the supported query templates, but this couples the client to the service. The URI templates implied by the projection resources can also be mapped to a service description that a client can access at run time. In the above representation, however, we have chosen to use the OpenSearch content type. This way the client can dynamically retrieve descriptions of the currently supported projections for a container.

Comments Resource Model

From the domain model in Fig. 11.4, we can infer that User has a collection of links to POI elements and Comment elements, POI has a collection of links to Comment elements, and Comment has a link to POI and a User. This is formalized in the resource model shown in Fig. 11.8. The collections of links are  ≪ projection ≫  elements containing  ≪ ref ≫  links to the items. POI and Comment has a  ≪ ref ≫  reference pointing back to a User. The bi-directional association between POI and Comment has been broken down to two relationships: a  ≪ ref ≫  relationship from a Comment to POI and a Comments  ≪ projection ≫  containing a collection of links from POI to Comments.
Fig. 11.8

Comments resource model for user and POIs

The implied new resources are as follows:

Effectively, the new POI and Comment  ≪ projection ≫  elements are convenience URIs to the respective containers with the username and POI id as context. For example, the first URI implies a query /pois?user=user.username.

The implied new resource representation fragments are as follows. For User:

For POI:

For Comments:

And finally, for a Comment:

Ratings for Points of Interest

Ratings are analogous to Comments described in previous section with a few additional constraints. First, there always exists exactly one link from a Rating to a User and POI, and there can only exist one Rating for each POI by a particular User. Both of these are resource state constraints that have to be enforced by the implementation. Second, only the User can see the individual Ratings he has made, change them and revoke them. Other Users can only see the total amount of thumbs up and down in the POIs. Details regarding user management and access control are Web service specific and beyond the scope of this chapter.

The Rating resource model implied the following additional resource:

The representation for a Rating is as follows:

Service Requirements Revisited

After the domain model has been refined into a resource model, we want to trace back the service requirements and see how the implied RESTful interface fulfills them. Figure 11.9 shows four example sequence diagrams showing the HTTP level interaction between a client and the service: creating a POI, commenting a POI, attempting to remove a POI made by a different user and finally searching for POIs based on a tag with comments inlined.
Fig. 11.9

Service requirements revisited

From Resource Model to Implementation

Resource Oriented Architecture enforces a uniform interface across the Web service implementation. Most of the functionality related to REST HTTP interface, representations, models and persistence should be implemented as a cross-cutting concern over the Web service implementation and then applied to the concepts and their relationships defined by a resource model. With proper implementation, it should be relatively easy to extend the service with new concepts in an almost declarative manner.

A concrete binding between the domain model, resource model, and the implementation is done by mapping resource model concepts to the concepts of the selected technology implementation architecture. In our previous work (Selonen et al. 2010b) we have used Java EE, Hibernate, MySQL, and Restlet framework for implementation. The POI service can be implemented in a similar fashion. The binding is summarized in Table 11.1.
Table 11.1

Example implementation binding to Java EE, MySQL, Hibernate, and Restlet

 

API (Restlet)

Representation (XML/JSON)

Model (Hibernate, Java EE)

Persistence (MySQL)

 

Item

Restlet resource bound to the URI. Supported default operations are GET, PUT and DELETE.

Representation parsing/generation based on the item attributes. Subresources inlined per request basis.

A native Java object (POJO) generated for each item with a Hibernate Data Access Object and binding to database elements.

Items are rows in respective database table with columns specified by item attributes. References map to foreign keys.

 

Container

Restlet resource bound to the URI. Supported default operations are GET and POST.

Representation parsing/generation delegated to Items.

Basic retrievals to database, using item mappings.

Containers are database tables.

 

Projection

Implemented on top of respective Containers.

Representation generation delegated to Container.

Extended retrievals to database, using item mappings.

Stored procedures for more advanced database queries. Tables implied by Container.

 

Concluding Remarks

According to our experiences (Selonen et al. 2010a,b) engineering RESTful Web services can be difficult for service architects lacking prior experience of REST. The RESTifying approach attempts to systematize the process of moving from service requirements to an implemented service. It applies customary UML domain models to capture domain concepts and together with additional constraints uses them to derive a Web service interface description. It further facilitates capturing the service requirements and communicating them to different stakeholders in a consistent manner with standard software engineering artifacts instead of ad hoc representation examples. The presented approach has emerged from the experiences gained during development of several RESTful Web services. Our goal has been to move away from designing individual APIs to bringing providing access to all content through a uniform programmable Web interface.

The approach has been successfully applied to several service domains that have their core value in storing, retrieving, and managing interlinked content. We argue it is possible to transform any domain model conforming to the domain model profile – i.e. containing only classes, attributes, associations, and generalizations – to a resource model and therefore to an implemented RESTful Web service. Consequently, the approach should be applicable to any service whose requirements can be captured with a domain model and simple constraints on how to access and retrieve content. How to refine a service whose value is in algorithms, processes and complex transactions to resource-oriented, descriptive state content is a valid design problem on its own right.

The RESTifying approach is currently applied manually for designing new services and integrating them with existing Web service platforms. As a next step, we are hoping to experiment with building tool support for the approach: to proceed from a domain model to a resource model and further to a RESTful service description, and integrating the approach to our existing modeling environment. As future research, with proper Web service infrastructure, we hope to be able to generate most of the server side code and client side stubs directly based on the domain model, allowing new resources be added in a generative and declarative way for a consistent Web service interface.

Notes

Acknowledgements

The author would like to thank Markku Laitkorpi, Petros Belimpasakis, Arto Nikupaavola, Yu You and Vlad Stirbu, the former NRC Service Software Platform team and NRC Mixed Reality Solutions program for their valuable contribution.

References

  1. M. Laitkorpi, P. Selonen, and T. Systä. Towards a model-driven process for designing restful web services. In IEEE International Conference on Web Services, ICWS 2009, ICWS, Los Angeles, CA, USA, 6–10 July 2009, pages 173–180.Google Scholar
  2. L. Richardson and S. Ruby. ReSTful Web Services. O’Reilly Media, 2007. pages 108–136.Google Scholar
  3. P. Selonen, P. Belimpasakis, and Y. You. Developing a restful mixed reality web service platform. In Proceedings of the First International Workshop on RESTful Design, WS-REST ’10, pages 54–61, ACM, New York, NY, USA, 2010.Google Scholar
  4. P. Selonen, P. Belimpasakis, and Y. You. Experiences in building a restful mixed reality web service platform. In B. Benatallah, F. Casati, G. Kappel, and G. Rossi, editors, Web Engineering, volume 6189 of Lecture Notes in Computer Science, pages 400–414. Springer, Berlin, Heidelberg, New York, 2010.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Nokia Research CenterTampereFinland

Personalised recommendations