MQTT Message Format

This document explains the structure of data messages consumed and produced by NeuroScale instances. Before continuing, please make sure you have read the Getting Started and NeuroScale Platform guides for an introduction to the NeuroScale Platform, and the page on Instance Metadata.

NeuroScale Message Format

All data messages consumed and produced by a running instance on the NeuroScale Platform must adhere to the message format described in this section. Therefore, it is imperative that developers wishing to communicate with their running instances read this section carefully in order to understand how to correctly structure the input data they send and how to correctly parse the output data they receive.

Important!

Instances created on the NeuroScale Platform can be configured to accept messages encoded using either JSON or MessagePack (see the API reference documentation for more information). Regardless of the encoding used, the underlying format of the messages remains the same (i.e. the exact same structure is transmitted over the network, it just happens to be encoded differently depending on the method chosen).

For simplicity, we assume the use of JSON throughout this document. In production, however, it is recommended to use MessagePack to help minimise bandwidth usage and reduce latency.

A message contains one or more streams, where each stream contains a name, an array of multi-channel data samples and an array of timestamps. Continuing our example from the previous sections, a message containing 5 independently sampled 3-dimensional acceleration vectors is represented as a JSON-encoded dictionary (or object) of the following form:

{
    "streams": [
        {
            "name": "accelerometer",
            "samples": [
                [ 100, 200, 300 ],
                [ 200, 300, 400 ],
                [ 300, 400, 500 ],
                [ 400, 500, 600 ],
                [ 500, 600, 700 ]
            ],
            "timestamps": [
                1445271230.10,
                1445271230.12,
                1445271230.14,
                1445271230.16,
                1445271230.18
            ]
        }
    ]
}

Recall from the previous section that messages are delivered to an instance's input nodes by publishing on specific MQTT topics. In the case of our example, by publishing the above message on /pJBQgaFBXvfQJvEjntSjwB/in/default it will be delivered to the input node named "default", whereby the names of the streams contained in the message are matched to those defined by the corresponding instance metadata for that input node (non-matching streams are silently ignored and discarded).

Similarly, by subscribing to /pJBQgaFBXvfQJvEjntSjwB/out/default, the results of calculating the magnitude of each vector will be returned in an identical format (with the obvious exception that each data sample contains only a single value or channel as defined by the instance metadata for the output node named "default"):

{
    "streams": [
        {
            "name": "magnitude",
            "samples": [
                [ 374.16573867739413 ],
                [ 538.5164807134504 ],
                [ 707.1067811865476 ],
                [ 877.4964387392122 ],
                [ 1048.8088481701516 ]
            ],
            "timestamps": [
                1445271230.10,
                1445271230.12,
                1445271230.14,
                1445271230.16,
                1445271230.18
            ]
        }
    ]
}

JSON Schema

The following is a JSON Schema that defines the message format accepted by NeuroScale. Developers can use this to validate that their messages are correctly formatted prior to transmission.

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "title": "NeuroScale Message Format",
    "description": "Message format for the NeuroScale Platform.",
    "type": "object",
    "properties": {
        "streams": {
            "description": "List of streams.",
            "type": "array",
            "items": {
                "$ref": "#/definitions/stream"
            }
        }
    },
    "required": [
        "streams"
    ],
    "definitions": {
        "stream": {
            "description": "Multi-channel stream of time series data (or event markers).",
            "type": "object",
            "properties": {
                "name": {
                    "description": "Name of the stream (must match instance metadata for corresponding input/output node).",
                    "type": "string",
                    "minLength": 1
                },
                "samples": {
                    "description": "Array of data samples.",
                    "type": "array",
                    "items": {
                        "anyOf": [ {
                            "description": "Multi-channel sample (array of numbers).",
                            "type": "array",
                            "items": {
                                "type": "number"
                            }
                        }, {
                            "description": "Multi-channel sample (array of strings, e.g. event markers).",
                            "type": "array",
                            "items": {
                                "type": "string"
                            }
                        } ]
                    }
                },
                "timestamps": {
                    "description": "Array of double-precision floating-point timestamps (in seconds).",
                    "type": "array",
                    "items": {
                        "type": "number"
                    }
                }
            },
            "required": [
                "name",
                "samples",
                "timestamps"
            ]
        }
    }
}