NeuroScale Platform Concepts
This page is a brief but important introduction to the key concepts on which the NeuroScale Platform is built and the two different services NeuroScale provides: Streaming and Insights.
NeuroScale Concepts
As you read through the documentation and become more familiar with the NeuroScale Platform you will learn of several more important abstractions that conceptually capture issues related to authorization, resource usage, etc.
On this page we explain the following concepts:
- Pipelines
- Streaming Instances
- Streaming Endpoints
- Batch Tasks
Pipelines¶
Pipelines in NeuroScale are conceptually similar to a pipeline in everyday life: a route, channel, or process along which something flows through-- in NeuroScale, data flows through a sequence of processing operations. In order to enter, pipelines require input data to have a predefined structure, as the data flows through the pipeline the data is processed to produce an output, again structured in a predefined way.
Pipelines are made up of processing nodes that the signal data flows through, with each node performing an action (algorithm, programmatic function, etc.) on the data, manipulating it until the desired format is reached at the end of the pipeline.
A pipeline can have a single or multiple inputs, and a single or multiple outputs, and these inputs and outputs can be applications, hardware devices, file systems, or any combination of the three.
NeuroScale provides a collection of pipelines suitable for processing of raw EEG data for a variety of purposes, including identifying brain state, brain-computer interfaces, event related potential analysis, and many more. Pipelines can also include dynamic properties that allow the user to customize and tune pipelines parameters when deploying in order to meet their specific needs.
In addition to these, users who license the NeuroPype Suite desktop application can also create their own pipelines which they can then upload to their NeuroScale account and run in the same manner as the pipelines made available on NeuroScale. This provides users with significant flexibility and power in signal processing in the cloud.
Streaming Overview and Key Abstractions¶
The NeuroScale Streaming service is a web-based platform that translates raw signals into meaningful information in real-time. The platform performs this processing feat by establishing streaming endpoints, which your application can connect to and then both 1) publish data and 2) retrieve processed results at high throughput and low latencies. What happens in between these endpoints behind the scenes is rather complex, but can be abstracted and modeled in terms of pipelines and streaming instances.
Streaming Instances¶
Streaming Instances in NeuroScale represent a running pipeline and can be considered an instantiation of a specific pipeline that may be manipulated/interrogated in a number of ways. For example, once a chosen pipeline has been instantiated and is running, details like its unique set of streaming endpoints can be extracted. Streaming Instance information such as this can be used by your application to interact with NeuroScale (e.g. processing brain data). These basic concepts are illustrated in Figure 1 and depict, at this level of abstraction, an architecturally simple computational resource that can be integrated easily into third-party applications — a usability feature further enhanced by NeuroScale's embrace of REST principles. Streaming pipelines are particularly well suited to brain-computer interface (BCI) applications.
Streaming Endpoints¶
Streaming endpoints in NeuroScale are set up when a Streaming Instance is created from a pipeline, and are considered unique and fixed for the duration of your processing session (while the Streaming Instance exists). In this context, the streaming endpoints can be considered as a publish/subscribe messaging pattern, which allows great flexibility in building applications.
Streaming Pipelines¶
Streaming pipelines take one or several data streams, for example from an EEG headset, heart-rate monitor, or other kind of sensor device, and output one or several streams at the above-mentioned endpoints. Additionally, some pipelines can be configured to accept a file of recorded signal data as an input which is then streamed and handled as a stream (this can be useful for demo purposes).
Insights Overview and Key Abstractions¶
NeuroScale's Insights service processes recorded biosignal files into informative reports and processed datasets. This contrasts NeuroScale's streaming service, which is designed to return continuous real-time results. Like Streaming, Insights runs pipelines on the Neuropype processing engine, but takes as files as inputs (rather than live data streams), and generates files as outputs (processed data, html reports, etc.) instead of a live stream with an endpoint.
Tasks¶
Insights "Tasks", like Streaming "Instances", represent a running pipeline and can be considered an instantiation of a specific pipeline that may be manipulated and interrogated in a number of ways. However, Tasks read in one or more input files and produce one or more output files containing the results. A Task ends when the file is finished processing, whereas an Streaming Instance will continue to run until it is stopped. For example, a Task is started by running a report pipeline and ends with a report generated of peer reviewed article HTML format in your Storage.
Pipelines¶
Insights pipelines are conceptually the same as those used for Streaming, except they accept and produce different inputs (files rather than live data streams) and outputs (datasets and reports rather than live streams with endpoints). Most Insights pipelines generate a graphical analysis report with figures in HTML format. Such pipelines are indicated as such in the pipeline name or description.