The REST API
This document describes important concepts relating to the design of the NeuroScale Platform API.
NeuroScale's RESTful Design¶
NeuroScale exposes its services through a RESTful interface. Representational State Transfer (REST) is a widely adopted, resource-oriented model of web services that emphasizes stateless design using HTTP-explicit methods on hierarchical URIs. It is much easier to use compared to alternatives such as SOAP- or WSDL-based web services and is the predominant web services paradigm deployed today. REST as originally described, captures a formal, abstract architectural style of design of services. RESTful web services can be considered then, an instantiation of REST using HTTP, URI and specific guidelines regarding their interactions. More specifically, a RESTful web service will have the following features:
- Uses HTTP verbs only
- Resources are identified using a directory-structure style URI
- Communication is stateless
- Standard formats such as JSON or XML are used for data
The basic idea is that the standard interface supported by HTTP should be used in a way that adheres with the definition of the protocol (RFC 2616). These methods are then used to create, read, update and delete (CRUD) identified resources on the system. The basic philosophy in NeuroScale is that we use HTTP GET to invoke resources that require no input while we use HTTP POST to create new resources. HTTP PATCH is used to update an existing resource while HTTP DELETE is used to remove a resource from the system. Resources themselves are "things" in the system that merit abstraction. We can examine both these concepts in light of the key abstractions relevant to the NeuroScale service. To reiterate, so far the key abstractions we have identified are:
- Pipelines
- Instances
- Streaming Endpoints
Each of these "things" is represented as a resource that is identified uniquely using a global naming scheme. On the web, the URI concept fulfils this role and therefore every pipeline, instance and streaming endpoint is assigned a URI in NeuroScale. As an example, we could identify the "thing" that is the set of all available pipelines with the following URI:
A sensible URI for a specific pipeline (for example, a focus pipeline) might be:
where we use a directory-structure approach to identify related resources that have a hierarchical relationship with the parent node (in this case, pl_n4KZiCJjtivd9o3oGVQK8M is a globally unique identifier for our hypothetical focus pipeline - more on this later). REST emphasizes such directory-style naming within URI in an attempt to make the resources intuitive and self-documenting to some extent. The decomposition of a URI should reflect the logical makeup of the resource and its constituent components. Not surprisingly then, a specific instance on NeuroScale might be identified as follows:
Through the use of these URI and appropriate use of HTTP methods you can retrieve information on available pipelines, instantiate them and delete them when they are no longer needed. In the next section we can take these ideas and look at putting them together to discover how the NeuroScale API can be used to create functionality for your application.
Putting it all together¶
We can now examine the process of instantiating a pipeline in NeuroScale using these basic concepts of HTTP verbs and URI identifiers. We will leave out some detail regarding access tokens, etc. to preserve clarity (and which will be covered in detail in the tutorials) but will still reveal further detail regarding the RESTful principles involved in the design of the NeuroScale API.
To create a new instance of our hypothetical focus pipeline we use the basic form below:
POST /v1/instances HTTP/1.1
Host: api.neuroscale.io
Accept: */*
Content-Type: application/json
Content-Length: 200
{
"key_1": "value_1",
"key_2": "value_2",
...,
"pipeline": "pl_n4KZiCJjtivd9o3oGVQK8M"
}
Note we do not show the full body of the request for now. The HTTP POST method here is used to create a new instance of a specific pipeline. The body of the request is encoded in JSON in the form of key-value pairs and ordered lists. One can see in this example, that the requested pipeline (pl_n4KZiCJjtivd9o3oGVQK8M) is a specific key-value pair in the JSON-encoded body of the request.
The requested resource that is created has a representation that is returned in the HTTP response. Again this representation is JSON-encoded.
JSON
JSON is a very popular alternative to XML. It is a lightweight, human-readable, data interchange format which unlike XML, maps simply on to data structures used in common programming languages. This use of JSON represents the penultimate RESTful feature of NeuroScale to emphasize. While REST does not specify any particular message format for resource exchange over HTTP it is expected that common formats that a client might be expected to handle should be used. JSON is one such format and is also the default format used for real-time communication in NeuroScale during development. A binary form of JSON called MessagePack is recommended for production. We will not discuss MessagePack in this document.
The normal response to the request above, assuming that the resource (i.e. the instance) can be created, is a JSON-encoded message which describes the instance in parameters relevant to the user. For the sake of flow and to gradually move towards real examples we will reproduce some typical output. Be warned it looks "busy" at first glance, however a calm examination will see this is simply JSON, i.e. a bunch of key-value pairs and ordered lists.
HTTP/1.1 201 CREATED
Allow: POST, OPTIONS, GET
Content-Type: application/json
Date: Wed, 16 Sep 2015 17:48:21 GMT
Server: nginx/1.1.19
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
transfer-encoding: chunked
Connection: keep-alive
{
"created_at": 1442425700,
"description": "",
"encoding": "json",
"endpoints": {
"count": 2,
"data": [
{
"created_at": 1442425700,
"id": "ep_uTM23uQVC8ecTj6AcoPbUA",
"instance": "in_DnDFUSgWCJXTSwGU3YNCM4",
"mode": "write",
"object": "endpoint",
"updated_at": 1442425700,
"url": "mqtt://streaming.neuroscale.io:443/maiYbX8nrBdWPStj82G4Xb/in"
},
{
"created_at": 1442425700,
"id": "ep_BLstXqCnt27BVS7agdou8i",
"instance": "in_DnDFUSgWCJXTSwGU3YNCM4",
"mode": "read",
"object": "endpoint",
"updated_at": 1442425700,
"url": "mqtt://streaming.neuroscale.io:443/maiYbX8nrBdWPStj82G4Xb/out"
}
],
"object": "list",
"url": "/v1/instances/in_DnDFUSgWCJXTSwGU3YNCM4/endpoints"
},
"id": "in_DnDFUSgWCJXTSwGU3YNCM4",
"metadata": {
"nodes": {
"in": [],
"out": []
}
},
"mode": "predict",
"object": "instance",
"parameters": {},
"pipeline": "pl_n4KZiCJjtivd9o3oGVQK8M",
"properties": {},
"replace_nans": true,
"replace_nans_with": 0.0,
"state": "configuring",
"time_to_live": 21600,
"updated_at": 1442425701,
"user": "usr_ZTNc2sEjoq6JoEgPNFUg6g"
}
One of the most important JSON-encoded values returned from this request is the ID of the instance (in_DnDFUSgWCJXTSwGU3YNCM4) which can be appended programmatically to the base URI to create the full URI of this running instance. With this URI we can retrieve the state of this specific instance resource through the use of the HTTP GET method as follows:
The HTTP 200 (OK) response to this request is identical to that produced under the original POST unless some updates (via PATCH) have been applied.
Another important JSON structure returned is that encapsulating the parameters describing the streaming endpoints (the "endpoints" key in the JSON shown above). Within that structure we see descriptions of the URI for streaming from and to the instance as follows:
mqtt://streaming.neuroscale.io:443/maiYbX8nrBdWPStj82G4Xb/in
mqtt://streaming.neuroscale.io:443/maiYbX8nrBdWPStj82G4Xb/out
These are complete URI and consequently include protocol and port numbers. We will see how to use these later.
Stateless Server Service
At this point it is worth noting that the communication with the service is stateless (another of the tenets of RESTful design). By stateless here we mean that the request contains all the information necessary for the server to complete the request. Each request is complete and independent as far as NeuroScale is concerned. Now obviously your application has state so the complete interaction with NeuroScale is not stateless in that sense. What we mean here is that the NeuroScale service does not remember the state of your application. NeuroScale will not use information from previous requests you have made because these have not been stored anywhere on the server. It is up to the client side to look after relevant state for the application. This stateless approach improves reliability as a failed request has no influence on other requests. It also improves scalability as NeuroScale does not have to look up application state on every request and so can respond faster. Finally as each request is complete it is easier to debug and visualise what is going on.
At this point armed with the URI of the instance (by appending the ID to the base API URI) and those of the streaming endpoints we have an eager NeuroScale service ready to push out computation results on any data sent to it. The next step then is how exactly does one send data to this instance? As mentioned in the Conceptual Overview we use a message-driven publish/subscribe approach in NeuroScale and in particular we use a protocol called MQTT to achieve this. MQTT is the topic of the next document.