Input

Intro

This section describes how to get data into Vision 42. Under the input tab you can do this manually:

Automatic input can be configured under the IO tab, by administrators. The standard URL syntax is used to describe input locations. A readable password or other secret can be avoided by filling in the dedicated secret field. Errors will be indicated by a clickable exclamation mark. They will be cleared automatically, once solved.

A network protocol method can either push or pull data. Let's have a closer look at both. Live examples can be found in the demo app.

Note: all TLS connections (Transport Layer Security) use a certificate that is signed by a universal trusted CA (certificate authority), domain validated (DV) and valid for 90 days.

Push data

These kinds of protocol methods push data directly into Vision 42. They function real-time, efficiently, and are as such recommended. The polling interval in the input table can be left empty.

HTTP POST

Hypertext Transfer Protocol

Data or media files can easily be posted to this endpoint: https://vision42.net/{app}/post

Administrators can grant access in the input table, based on:

The first two require a standard Authorization header, respectively Basic and Bearer. The post endpoint is part of the REST API. Examples using curl:


curl \

--data '{}' \

--user demo:demo \

https://vision42.net/demo/post


curl \

--data @file \

--header 'Authorization: Bearer aGFybWxlc3M=' \

https://vision42.net/demo/post

Note: this method is being used by Rail 1435.

MQTT subscribe

Message Queuing Telemetry Transport

Vision 42 can subscribe to a data stream on a broker. URLs will typically look like this:

Available variants, including TLS and WebSocket: MQTT, MQTTS, WS, and WSS. Default, QoS 1 (Quality of Service) and standard version 5 will be used. If needed, exceptions can be made for versions 3.1.1 and 3.1. All topic wildcards are allowed.

To avoid message loss while offline, a persistent session with a 1-day-expiry is being used. However, it is up to the broker to respect this.

Examples and extensions:

mqtts://{token}@api.allthingstalk.io#device/{device}/#

wss://_@mqtt.dataplatform.knmi.nl#dataplatform/file/v1/Actuele10mindataKNMIstations/2/#

mqtts://{application@tenant}@{region}.cloud.thethings.network#v3/+/devices/+/up

MQTT publish

Message Queuing Telemetry Transport

Vision 42 comes with an integrated broker, on which a data stream can be published:

Identical examples using Mosquitto and MQTT.js:


mosquitto_pub \

--url mqtts://demo:demo@vision42.net/demo/example \

--message '{}'


mqtt publish \

--protocol mqtts \

--username demo \

--password demo \

--hostname vision42.net \

--topic demo/example \

--message '{}'

In the input table, the default URL mqtt:{app}/# subscribes to all of your topics. This URL may be disabled, split up, or narrowed down.

AMQP consume

Advanced Message Queuing Protocol

Vision 42 can consume a data stream from a broker queue. A typical URL will have this form: amqps://user@broker/vhost#queue

Available variants, including TLS: AMQP and AMQPS. Standard 0-9-1 is supported.

Extension:

SFTP PUT

SSH File Transfer Protocol

Data and media files can be transferred to the integrated server:

There are two authorization methods available:

FTP PUT

File Transfer Protocol

Similar to SFTP, data and media files can be uploaded to the integrated server:

FTP is not recommended because it is insecure and it requires you to share the credentials. However, we will keep supporting FTP PUT, as long as there are devices in use which have no better way to communicate.

Pull data

These protocol methods pull data periodically into Vision 42. They are inefficient and not real-time, and are only to be used when no push mechanism is available. The polling interval in the input table has to be filled in.

HTTP GET or POST

Hypertext Transfer Protocol

URLs can have the following base forms, depending:

Available variants, including TLS: HTTP and HTTPS. To POST parameters as a JSON object, make use of a hash fragment: https://server/endpoint#{JSON}

Examples and extensions:

https://hicws.vlaanderen.be/KiWIS/KiWIS?timeseriesgroup_id={groups}&period=PT20M&format=csv&metadata=true&datasource=4&request=getTimeseriesValues

https://opendata.meteo.be/service/ows?typenames=aws:aws_10min&service=WFS&request=GetFeature&outputformat=json


https://opendata.meteo.be/service/ows?typenames=synop:synop_data&service=WFS&request=GetFeature&outputformat=json

Private static APIs are also supported.

http://{server}:8000/v1/projects/{project}/pointsJSON?typesIds={0|100}&apiKey=SECRET

This URL will fetch data of all points in the given project. The typesIds parameter is optional; 0 for measurements and 100 for adjustments.

https://{user}@{portal}.water-munisense.net/webservices/v2/groundwaterwells/query?property=water_level


https://{user}@noise.munisense.net/webservices/v2/soundmeasurementpoints/query?property=daily_overview_lden_day

Indoor climate is also supported. Any desired property can be included.

https://obscape.com/portal/api/v3/api?parameters=turbidity&username={user}&key=SECRET

This URL will fetch data from all stations. Any desired parameter can be included.

https://api.profoundportal.online/v1/external/project/

This URL will fetch frequencies and velocities of all active projects.

https://waterwebservices.rijkswaterstaat.nl/ONLINEWAARNEMINGENSERVICES_DBO/OphalenLaatsteWaarnemingen#{JSON-body}

https://{user}@senceive.io/api/consumers/current

This URL will automatically insert all available AMQP queues in the input table.

https://download.waterinfo.be/tsmdownload/KiWIS/KiWIS?timeseriesgroup_id={groups}&period=PT20M&format=csv&metadata=true&datasource=1&request=getTimeseriesValues

SFTP GET

SSH File Transfer Protocol

URLs come in two flavors:

While a password may be used, it is recommended to authorize our public key:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0skCWi+E3HMQgeuINQPYnO24K96JIGMVrca9xUzrtnbws31DeDkec0y+DH4ilUpZbEb2I+6krcQYEZnn92/IHBADJj7+iRBtmZJhPvguloaNygyBLF4E0yk6YB2PBdGXuoY/trGWj4h3h9NPKXpbkQwZtfmj8v4cQ7QGxbatmn62/piHfgY9tnL7ZpN9oxO4AQdSUisU4knPZViS7Go4Rj2bPDSVssqI+vCeNONjnvdSHXibwlIb/AiHMXJfSLEZytCnC1Ran9z9fPUWP0xCh+pHVKVv7hVwW6jc0R6081e4amBZOWsMiskoY/0CQGAfek9E6ON8yz7d+k7dKu2QZ Vision 42

Notes:

FTP GET

File Transfer Protocol

URLs are similar to SFTP:

Available variants, including TLS: FTP and FTPS. The former is insecure and thus not recommended. Only extended passive mode is supported (EPSV and EPRT).

Notes:

FILE read

The file: URL is used internally to process received data. It allows administrators to pause or slow down all network input.

S3 GET

Simple Storage Service

Currently under evaluation