Table of Contents

Progressive Web App

Vision 42 ® is a progressive web app. This means it can be installed on your pc, smartphone, and tablet. Installed apps have these benefits:

• They run separately from the browser.

• Links, in messages for example, can be opened with the installed app. This avoids the clutter of many open browser tabs.

• Files can be directly opened with your app through the operating system.

• The app icons can have badges indicating progress or errors.

• Shortcuts can be used, through the app icon.

• The window title bar can function as the header menu, thus saving screen estate.

State engine

Vision 42 keeps track of its internal state and even unlocks the state through its URL. This results in some powerful features:

• Go back and forth using the history buttons.

• Share something with another user, using any email or chat application.

• Make bookmarks inside the app.

• Reload without having to navigate, click, and type back to where you were.

Always up-to-date

When necessary, any visible information in the app will be automatically refreshed, after 15 seconds at most. This refresh will be delayed for elements the user is interacting with. For most elements, the focus will be used to determine this. For instance, a cursor in an editable input field will pause refreshing. For maps, this is an open plot window. Mind that it is not easy to keep all information up-to-date and to never disturb the user in the process.

Sign in

QA environment

If you do not want to pollute the production environment with test data, you can use the QA environment at https://vision42.net/qa/your-organization/ (Quality Assurance). Note that data in QA will be overwritten daily by production data.

In QA, you can try things out with impunity. This is especially useful for new project files or for trying out new data formats. Large reports can be run in this QA-environment, without impact on production performance.

Password managers

Vision 42 deliberately integrates well with password managers. We strongly advise to fully make use of a password manager. It can generate a strong random password, which it will remember and even fill in for you. Many password managers can even function across devices. A quick start is to use the Google Password Manager that is built into our recommended browser, Google Chrome.

Remark: In the rare case where you have to access Vision 42 instances of two different customers, make use of different domains in the URLs - for example vision42.net and vision42.eu. If not, a password manager can get confused, because it only can handle a single cryptographically encoded password per domain (and per account).

Persistence

Signing in will persist until the local storage is cleared by the browser. This is a deliberate choice of a large convenience gain over a tiny security loss.

We strongly advise against signing in on a public or untrusted device. Should you do so anyhow, make sure to sign out. Closing the Vision 42 tab or window will not sign you out.

Identity and access management

The authentication and authorization mechanisms of Vision 42 can be replaced by an OAuth2 integration with an external identity and access management (IAM) system. This may be your own IAM-platform or that of a third party like Google, LinkedIn, Facebook, and countless others. User properties and even roles can be synchronized.

The following limitations apply:

• Only a single OAuth2 provider can be integrated at once.

• The OAuth2 endpoints have to be reachable from our servers without the need for VPN or equivalent.

• Integration requires some professional services which are not included in the monthly subscription.

User section

Your password

Enter a new password twice, to change it. The strength has to be 50% at least, as indicated by the gauge.

The minimum length is six. Passwords that have a mixed case, are longer, or contain a special character will be rated stronger. Trivial text will be rated weak.

Your picture

Vision 42 supports Gravatar (globally recognized avatar) for profile pictures:

• One avatar for everything, everywhere.

• Gravatar powers your public profile, visible wherever you post, comment, and interact online.

Alternatively, you may paste the URL of a profile picture on the Internet.

Your print

Set your preferred page size, orientation, and margin here. The default is an A4 portrait with a 10 mm margin. These settings will be remembered per device.

Your notifications

Using this table, you can subscribe to notifications on a per-project basis. There is a choice for receiving outage alerts or not. Message subjects will be prefixed with [alert] or [outage], to facilitate classification rules.

Vision 42 aims to send you as few messages as possible:

• Alert statuses of individual sensors will be aggregated per instrument.

• Only a change in status will trigger an alert message.

• Status changes will be bundled into messages per project.

The body of an alert message includes:

• Values and thresholds, for alerts.

• Time lag, for outages.

• A handy link to plot all mentioned instruments with a single click or tap.

Subject: [alert] Example project

BH-11: green -> red (level = -1.96 > -2 m)

BH-42: green -> yellow + outage (level = -1.8 > -2 m & 63 min lag)

BH-50: yellow -> green

https://vision42.net/your-organization/#section=plot&instruments=11~42~50

Email will be sent by blind carbon copy (bcc), out of privacy considerations. Only an explicit opt-in mechanism has been made available. Subscribing someone other than yourself, is not easy on purpose.

Map section

In reading order, you will find the following controls:

Tip: The search function of the browser can be used throughout Vision 42. For most computers, its keyboard shortcut is Ctrl + F. For Apple computers, this is ⌘ F. In the Map section, this can be especially useful to quickly select a project by part of its name or code.

Edit

The edit mode can be used by administrators to position instruments visually on the map.

Field navigation

This button toggles the navigation mode, which locks the map to your device location. This is especially useful for field engineers. The button icon can indicate several conditions:

• Gray: navigation is inactive.

• Blue: navigation is active.

• Filled: location data is available.

• Strike-through: Vision 42 does not have permission to use your location.

GPX

This button lets you export the data within the map boundaries to a gpx-file (GPS Exchange Format), which is automatically or easily opened with a navigation app.

North

This arrow always points north. It is the only map control that will be included on prints. Clicking this button will reset the heading and tilt of the map.

Google Maps

Clicking the Google logo in the bottom-left corner, will take you to the same location in Google Maps. This comes in handy for navigation / route purposes: just right click and choose route to this point.

Street View

The Pegman icon can be dragged onto roads. This will open Street View with an integrated instrument layer. You can even open plot windows, just like on the map. On smartphones and tablets, this integration functions as augmented reality.

Zoom

The + and - buttons will zoom the map in or out, in discrete steps. Using a scroll wheel (mouse) or a pinch gesture (touch), will zoom fractionally.

Instrument markers

The markers on the map indicate instrument locations and names. Their color denotes their alert status: green, yellow, or red. Keeping severity in mind, red will take precedence over yellow.

A stop symbol ■ stands for an outage; measurements were expected, but did not arrive. A blank white marker indicates an idle instrument, either one without data or one excluded from import.

By default, outage detection is performed by artificial intelligence (AI). New sensors will be trained briefly. The outage detection is adaptive, taking both changed behavior and recent outages into account.

Per instrument, a manual override of automatic detection is available. This can be useful for intermittent measurements, like a total station in foggy conditions.

Businesses are hidden

In urbanized areas, like city centers, the map could become quite cluttered. Of course you could turn off the labels and end up with satellite imagery and nothing else. Granular hiding of businesses (bars, restaurants, retail, ...) has made the map much less crowded, while retaining street names and other useful information such as metro stations, schools (traffic congestion) and the like.

Note: Due to a bug in Google Maps, a hybrid map may still display businesses.

KML layers

The Vision 42 team can add your KML or KMZ layers to the map. Please create a ticket to do so.

Plot section

Open data as spreadsheet

All plots and most tables have an Open spreadsheet button. It lets you export the current table or the data of the current plot to a csv-file (comma-separated value), which is automatically or easily opened with any spreadsheet application, like Microsoft Excel.

Alert zones

When a plot contains alert zones, a ! button will appear. Pressing that exclamation mark will automatically scale the plot, so all alert zones are in view.

Hiding series

By clicking on a legend entry, the corresponding series will be plotted alone. Add more series by clicking more entries. Click any colored entry again, to display all series.

Plot menu

Left click or tap any value point to open the plot menu. To close it, click outside the menu or press the escape key. Dependent on the context, the plot menu may contain these actions:

• ! Median time

  • Plot time has been aggregated at this zoom level and some fine-grained actions are not available.

• Plot versus time

• Plot versus [axis]

• (Un)pin profile

• Download file(s)

  • Only when imported from an original data file, for the moment.

  • For convenience, this will also download the file of the instrument datum, if any.

• Set instrument datum

• Clear instrument datum

• Mark as invalid

• Delete

• Delete series

  • This includes invalid values.

  • This will only delete values. The sensor and its metadata will not be touched. This way, you still can exclude the sensor from importing, if needed.

• Delete file

  • This includes invalid values.

  • In addition to values, the file itself will be deleted.

Some actions can be applied on time ranges, using the following menu options:

• Select from here

• Zoom up to here

• Invalid up to here

• Delete up to here

• Cancel selection

Vertical measurements

By default, Vision 42 will plot a profile in the conventional way; with the x-axis pointing right and the y-axis pointing up. Vertical measurements will be plotted vertically, with an optional ground level indication. The orientations of the x-axis and the multiple y-axes will be set automatically, based on the metadata information received. Here are some examples, with the name of the vertical axis mentioned between brackets:

• Measurand SAA (Z)

• Geokon in-place inclinometer (z)

• RST Instruments in-place inclinometer (z)

• RST Instruments inclinometer (Depth)

• Glötzl inclinometer (Step)

• SAA via Leica GeoMoS (Sensor_Z_)

Vision 42 was designed to function fully automatically and in real time. Usually, the loggers or instruments send their data directly to us, or the app will fetch them from somewhere. Next the cumulative profiles ​​of the above inclinometers will be calculated, for example. Plotting of values relative to a dynamic instrument datum is embedded in the application, as well is recalculation to absolute levels in mASL.

Inclinometer checksums will not be plotted by default, nor will their depths show up. This behavior can be controlled by the do-not-plot flag.

In-place inclinometers

For the sake of traceability, Vision 42 will make use of the original vertex numbers in the IPI file. The (excluded) angles and the temperatures are being measured on those vertices. As is the case for inclinometers, the top angle measurement will have a relative depth of zero. This way, the relative depths will correspond to the depth values in the data files, if any.

The calculated cumulative profile contains an extra value, the zero origin. It begins half a gauge deeper and ends half a gauge higher, while the intermediate vertices fall between the angles. These interlaced vertices will be numbered higher, following the original vertices.

Invalid measurements

Invalid measurements:

• can be marked using a graphical selection of the beginning and the end.

• will not be plotted.

• are excluded from tables with manual water level measurements.

• will not be exported.

• will not be sent through the REST API.

• will not be used to calculate formulas.

• will not be used when compensating with air pressure.

Marking measurements as invalid has the advantage over deleting, in that they do not return when one reimports data. Invalid measurements will only become deleted using Delete file or Delete series in the plot menu.

For profile data, marking a value of the first vertex (Z1, Depth1, ...) as invalid will exclude that time point from the slider.

Measurement interval

The measurement interval is defined within Vision 42 as the time between the last two measurements. This allows dynamic measurement frequencies.

Media section

In this section, you will find uploaded and imported media files. The metadata can be edited by regular users, but not by guests:

• A project for allowing guest access, filtering, and local coordinates.

• A date and time for ordering.

• A name for identifying.

• Coordinates for map filtering.

The location will be shown on the inset map. Media files can be deleted, after confirmation. When a media file cannot be embedded by your browser, a download button will appear. At the time of writing, this was the case for PDFs on Android Chrome.

The orbital camera can be positioned interactively:

• Control the azimuthal and polar angles by touch or click dragging.

• Zoom by pinch gesture or mouse wheel.

• Pan by using two-finger touch, or dragging with right-click or modifier keys.

• Center the tapped point on the model.

• Reset the view by tapping outside of the model.

Input section

Delimiter-separated values

Most data formats will be imported in a generic fashion. This will automatically handle:

• All known byte order marks

• Detection of the character encoding

• Common value delimiters

• Cell trimming

• Date-time and timezone notation

• Partial or missing date-time information

• Multiple date and/or time columns

• Analysis of metadata schemata

• Metadata gathering, including serial numbers and firmware / hardware / software versions

• Analysis of data structures

• Concatenated data, for consistent delimiter and time column(s)

• (Live) coordinate transformation

Note this includes comma-separated values (CSV) and RFC 4180.

These column designations might come in handy (case and white-space insensitive):

• CHANNEL [ UNIT ]

• DEVICE | CHANNEL | UNIT

• EASTING, NORTHING, ELEVATION, and/or SURFACE

At least these delimiter-separated data formats are supported:

• AE Sensors LoRa

• ARGUS

• ARGUS (Iv)

• Azimut Monitoring Greenbee

• BARTEC SYSCOM, format versions 1, 3, and 4 at least

• Brem optical fiber, with and without extensive header

• Campbell Scientific (CSV)

• Campbell Scientific (TOACI1)

• CAP (English)

• CoMeT

• Ekopower iBOX

• ElliTrack

• Empty files

• FBGS FBG-scan

• Fluvius

• GeoCSV, with a date-time column

• GeoMetiri

• Geosense Linx

• Geosense Linx Connect

• Geosense Wi-SOS, including GEOSTRING

• Inventec

• Keller ADT

• Keller Datamanager

• KISTERS WISKI, CSV format with metadata

• MEET HET, ambiguous profile data excluded

• Microsoft Excel (CSV)

• OSMOS

• Profound VIBRA(+) (CSV)

• RST Instruments data logger

• rsync.net

• Senceive (CSV), including in-place inclinometers

• Smet Group

• Soil Instruments VWLog8 GPRS

• Speedtest

• SVANTEK (CSV)

• SVANTEK (SvanPC++)

• Sylex

• Villari (CSV and XLSX)

• Vision 42 (export & legacy)

• Vista Data Vision

Tip: Feel free to use our demo to try out (other) data formats.

Binary formats

• Apache Parquet, the benefits of delimiter-separated values apply.

• Hierarchical Data Format (HDF)

• Microsoft Excel (XLSX), a template can be downloaded from the Input section.

• miniSEED

• Network Common Data Form (netCDF)

• Profound VIBRA(+)

• Protocol Buffers based on your proto definitions.

• Senceive.io

Other binary formats will be skipped.

JSON family

• JavaScript Object Notation (JSON)

• JSON Lines (JSONL), for streaming

• Newline delimited JSON (NDJSON), for streaming

• Prefixed JSONL

• Prefixed NDJSON

• Basetime Locator One, timezone = UTC

• Buildwise fiber

• Munisense

  • • Ground water wells

    • Sound measurement points

    • Indoor climates

• The Things Network Storage Integration

The benefits of delimiter-separated values apply.

Inclinometers

• DGSI Digitilt Uniform eXchange

• Geosense

• GLÖTZL

  • Including German and legacy variants.

  • The legacy year 2.000 problem will be corrected automatically. 

• RST Instruments

• Soil Instruments Vertical System

The benefits of delimiter-separated values apply.

Other formats

• ARGUS (export)

• Campbell Scientific (TOA5)

  • Geokon in-place inclinometer

  • RST Instruments in-place inclinometer

• Diver

  • Including CTD-Divers.

  • Including LDM EAE- and VEI-edition software.

  • Including EnviroMon software.

  • Including Dutch variants.

• Endress+Hauser

• Geosys

• Inducon

  • Exceptionally, the recognition of this data format is based solely on the file name, because no more metadata is available, unfortunately.

  • Both the instrument and sensor names will be taken from the file name.

• Leica GeoMoS, including automatic coordinate transformation and update of location.

• Measurand

• network Common Data form Language (netCDL)

• Profound

• Solinst Levelogger software

Media files

• Exchangeable image file format (Exif)

• Joint Photographic Experts Group (JPEG)

• Graphics Language Transmission Format (glTF)

  • Self-contained binary .glb and JSON/ASCII .gltf are supported.

  • Large 3D models require a decent graphics card, at least.

  • Note the OpenGL axis convention (x,z,-y) versus typical GIS (x,y,z).

• Portable Document Format (PDF)

• Portable Network Graphics (PNG)

In absence of more specific information, these and all other formats will make use of:

• The date and time of the file itself.

• The location of the upload device, when allowed.

Export formats

Please note that files from AE Sensors, ARGUS (export and Iv Infra), KISTERS WISKI, Leica GeoMoS, and Senceive are currently not always stored within Vision 42. This happens when they cannot be linked with a single data logger or a single instrument. After all, these files contain a complete project or site. Only original data files are stored by Vision 42, for the moment.

For this type of data format, instrument identifications must be unique across projects. A compelling reason for this, is the lack of other (sufficiently) unique information such as serial numbers. A simple solution is to use a unique prefix per project. After the initial import, you can remove that prefix from the instrument names, as you see fit.

An error will be raised as long as there are duplicatie sensors.

Continuous input

Vision 42 can import project data continuously from multiple MQTT, MQTTS, AMQP, AMQPS, SFTP, FTP, FTPS, HTTP, HTTPS, (OPeN)DAP, (OPeN)DAPS, FDSN, FDSNS, and KNMI locations. The standard URL syntax must be used to configure import locations. We advise making use of the separate password field, to avoid a readable password in the URL.

A location may be a growing file, or a directory in which new files arrive. Possible errors can be found under the Setup section.

Older values will be discarded, except when "Delete files" is set. Note that the latter implicates the built-in SFTP and FTP servers.

Continuous input from directories, with the "Delete files" flag off, has a fairly strict requirement: the natural order of filenames must be chronological. Vision 42 keeps track of which file was last imported, and will continue from there on the next time. This is necessary because directories can contain millions of files and obtaining modification times is on a per-file basis. In combination with often under-performing FTP servers, this would at least undermine the quasi-real-time nature of the Vision 42 app, and often become undesirably slow and cause heavy system load.

For SFTP, public key authentication is supported, so passwords can be avoided. To configure, add this public key to .ssh/authorized_keys:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0skCWi+E3HMQgeuINQPYnO24K96JIGMVrca9xUzrtnbws31DeDkec0y+DH4ilUpZbEb2I+6krcQYEZnn92/IHBADJj7+iRBtmZJhPvguloaNygyBLF4E0yk6YB2PBdGXuoY/trGWj4h3h9NPKXpbkQwZtfmj8v4cQ7QGxbatmn62/piHfgY9tnL7ZpN9oxO4AQdSUisU4knPZViS7Go4Rj2bPDSVssqI+vCeNONjnvdSHXibwlIb/AiHMXJfSLEZytCnC1Ran9z9fPUWP0xCh+pHVKVv7hVwW6jc0R6081e4amBZOWsMiskoY/0CQGAfek9E6ON8yz7d+k7dKu2QZ Vision 42

FTP(S) servers and in-between firewalls must support extended passive mode. If not, connections will fail. Technical: your FTP software and your NAT firewall (Network Address Translation) must be able to handle EPSV and EPRT commands.

Built-in SFTP and FTP servers

From within the application, an administrator can ask for access to the built-in SFTP and FTP servers. This is a one-time step. From then on, all received files will automatically be imported. You may pass the credentials to your trusted integrators. For SFTP, public key authentication is supported, so passwords can be avoided.

Only files that are at least 5 seconds old will be imported via the built-in SFTP and FTP servers. This is so to prevent data loss. After all, while a file is being written to, an incomplete import and a subsequent deletion could otherwise occur.

In case of external SFTP or FTP(S) connections, this check does not happen because there is no guarantee that the clocks of both servers will run synchronously.

Identification of files

Vision 42 aims to function fully automatically, using a data chain that is as short as possible while avoiding Excel processing and intermediate servers.

The app will automatically link files to instruments based on data in the file, usually the header. In absence of this, the last alphanumeric word from the filename (for input by upload) and/or the full path (for continuous input) will be used instead. An underscore will usually be considered part of the identifier. This to match BH_14 and not BH in a filename, for example.

In case of a new file with previously unknown instruments / sensors, the application will populate the system with the correct instruments and their position.

Note that within an instrument, sensors will be matched by column number, if applicable.

Caution: As an administrator, you may change the identification of an instrument. This can break things, so only do this if you know exactly what you are doing.

Metadata

Barring exceptions, metadata such as units is only read the first time. This is consciously so, to make changes in the application possible. If not, the data would be overwritten every time, by metadata in files over which you may not have control.

Calculations and reimporting

Calculations are performed during import. A notable exception is the dynamic calculation of values relative to a given instrument datum. This method of calculating allows for parameters that vary over time. Here are some examples:

• A defective sensor has been replaced with a new one. The calibration formula needs to be changed and will be automatically applied to subsequent values.

• After a settlement, the instrument elevation has changed. Subsequent values will be calculated with the appropriately changed elevation parameter.

This method is simple and avoids having to define the full time course of each calculation parameter.

Reimporting requires little effort:

1. Plot menu > Download file(s)

2. Input > Upload > Overwrite previous values

3. Input > Upload > Choose files or just drag and drop

Coordinate transformations

Regional coordinates will be transformed and updated in bulk, using the project's given coordinate system. This means instrument locations will will be updated accordingly, even live.

• For coordinates relative to an instrument datum, the corresponding value in the file will be used. An example of this is difference in topography.

• In case of absolute coordinates, the most recent value from the database will be used. This allows for application of formulas on coordinates. For example, some tunnel boring machines only send the last digits. A simple addition can get you complete coordinates.

• Coordinate updates will adhere to the do-not-import flag.

About time

The timezone field, in the instruments and data loggers table, dictates how times should be interpreted on input.

Vision 42 can handle milliseconds. For the time format, fractional seconds of arbitrary precision are being read. For the database, this is rounded to milliseconds, because the hard limit of the plots is 1 millisecond.

Editing in Excel

For example, editing in Excel can be useful for correcting depth errors in inclinometer measurements. After missing a "click," the remaining column can be slid down and the missing value of the previous measurement can be inserted. Using an ordinary editor, this is all too laborious.

Inclinometers

On import, cumulative profiles will be calculated automatically. The deepest value is the origin, inherited from the previous, non-invalid profile. Usually, the origin will be zero, except for shortened inclinometers. The inclinometer length may vary over time.

A cumulative profile begins half a gauge deeper and ends half a gauge higher, while the intermediate values fall between the angles. These interlaced vertices will be numbered higher, following the original vertices.

Note that the top angle measurement has a relative depth of zero and that even depths may vary over time.

Internet of Things (IoT)

The Vision 42 app has extensive provisions for the Internet of Things. The following technologies have been fully integrated and tested:

• LoRa - Long Range

• LoRaWAN - Long Range Wide-Area Network

• Sigfox

• The Things Network, including the Storage Integration.

• AllThingsTalk

• MQTT - Message Queuing Telemetry Transport

The Things Network

Add a continuous input like this:

• URL: mqtts://your-application-id@your-tenant-id@your-region.cloud.thethings.network/v3/+/devices/+/up

• Password: your MQTT API key.

Reference documentation can be found here. The Storage Integration is also supported. Vision 42 can import the (NDJSON) stream of stored messages.

AllThingsTalk

Add continuous inputs like this:

• URL: mqtts://your-device-token@api.allthingstalk.io/device/your-device-id/#

Reference documentation can be found here.

We do not recommended AllThingsTalk for a large numbers of sensors.

Private LoRa networks

There is a good change that Vision 42 will be able to import data from your private LoRa network. Add a continuous input with a custom MQTT URL. Please see the documentation of your private LoRa server / gateway.

However, your private MQTT broker has to be reachable from our servers without the need for VPN or equivalent.

Sensors section

Behavior of tables

The column header is like a search bar, with ...

• Dynamic search on text fields.

• Filtering on selected items.

• Sorting on all columns.

Some more features:

• It has input functionality, depending on given rights.

• You can delete entries, again depending on given rights.

• It’s smart if you want to see more lines, but fast because it does not show all data. It doubles the number of extra entries shown each time the icon in the caption area is clicked.

Virtual and manual sensors

Instruments and sensors will be automatically created on import. In case of virtual or manual sensors, you will want to create these yourself, since there is no data available to import.

New sensors can be created under the SENSORS tab. Make use of the plus sign + at the bottom of the sensor table. An instrument (whether virtual or not) must be entered. Next, fill in the formula of the virtual sensor or set the manual flag to mark the sensor as such.

Transferring sensors

You can easily transfer a sensor to another instrument, using the drop-down list in the first column of the sensors table. This will not be a problem for new sensor data. Vision 42 will remember where the data was coming from. This is visible in the origin column.

Time lag

Per sensor, the average time lag is displayed in the sensors table. Its unit denotes either milliseconds (ms), seconds (s), minutes (min), hours (h), days (D), weeks (W), months (M), or years (Y).

Do-it-yourself profile plot

In case you need a profile plot, but your data does not contain vertex positions, do the following:

• Enter the vertex positions in the sensors table.

• Enter a name for the axis in the instruments and data loggers table.

• (Re)import your data.

Formulas

Introduction

Let's start off with a simple example. Given are two instrument sensors, BH1-level and BH2-level. Create a new sensor difference at the bottom of the sensor table, for example under instrument BH1. For the sensor difference, enter the formula [123]-[345], using the corresponding keys to the right of the sensor table.

In the tooltip of the formula you will find a summary example in terms of syntax:

[key1]>1,2 ? 3.4E-5**6 : log([key2];10)+elevation²-position³

Note that:

• A formula may be split over multiple lines and white space can be added, for readability.

• A leading hash sign ( # ) will disable a formula.

• There is no leading equal sign ( = ), like Microsoft Excel requires.

Numbers

• Both point ( . ) and comma ( , ) may be used as the decimal separator.

• Numbers can be written in exponential notation.

Identifiers

• The lowercase letter x is shorthand for the own sensor [key].

• The words elevation, surface, and position will be replaced by their corresponding values.

• The word null may be used to omit a value.

• These predefined constants can be used.

Operators

• The power operators are **, ^, ², and ³.

• The ternary operator ( ? : ) is allowed and provides if-then-else functionality.

• More operators and their precedence can be found here.

Out-of-band filter example:

-42<x && x<42

? x

: null

Functions

• Function parameters must be separated by a semicolon ( ; ).

• These math functions can be used.

Error handling

Broken formulas will automatically be disabled on calculation, accompanied by a helpful error message. Example:

# Call to undefined function ln()

ln(x)

Additionally, both upload and continuous input will show the error.

Cyclic references will also be detected and disabled.

Calculation

Whenever a source sensor changes, the formula will be calculated. An exception to this behavior is a (calibration) formula which depends on its own sensor. The calculations can also be done for the past. Re-importing is the trigger for this.

You can clone a single sensor using the simple formula [123].

The measurement freshness in formula calculations is asymmetrical:

• 210% of the interval for earlier values, keeping late arrivals in mind.

• 110% of the interval for later values, to allow for small interval variations.

Calibration

Calibrated sensor data can be achieved by adding a calibration formula to a sensor.

In case this formula depends on the own sensor, the original value will be replaced by the calibrated value. Such a formula will only be calculated for the points in time of the original sensor.

Alternatively, a virtual sensor can be used to store the calibrated data alongside the original data.

Air pressure compensation

The automatic air pressure compensation only uses linear interpolation between a time up to 110% of the interval before and a time up to 110% after. This provides the most correct value. The formulas do the same - using 210% / 110%, supplemented with substitution with the previous or the next measurement within 210% / 110% of the interval. Extrapolation is never considered because it produces incorrect values ​​near inflection points. If no near measured values ​​are found, previously calculated values ​​are deleted by both mechanisms - except for upload without the overwrite check mark. When importing, all relationships between values ​​and calculations are taken into account. This ensures, for example, that all air pressures are read first and then the dependent calculations are made.

The formula mechanism is therefore more recent, better and more suitable for continuous measurements and alarms. The intention is to replace the automatic air pressure compensation and all other implicit calculations with formulas, with or without a formula builder or the like.

In summary, the formulas will currently do much better than the automatic air pressure compensation. A difference between the measurement and the send interval can still get in the way here, but to a lesser extent. We do not wish to go much further than 125% of the interval, because we strive for a certain balance of correctness and usability.

Repurpose instruments or data loggers

Administrators can clear an instrument id. On arrival of new data, a new instrument will be created automatically.

Override outages

The automatic detection of outages can be overridden. To do so, fill in the outage field in the instruments and data loggers table, by specifying the desired time lapse in minutes.

Confirm alerts and outages

Vision 42 can repeat alerts and outages every 15 minutes until they have been confirmed. To do so, make use of the confirm flag in the instruments and dataloggers table. The messages will indicate which instruments have to be confirmed:

Subject: [alert] Example project

BH-11: yellow -> green

BH-42: green -> red (level = -1.96 > -2 m & confirm)

https://vision42.net/your-organization/#section=confirmed

https://vision42.net/your-organization/#section=plot&instruments=42

Click or tap that first link to confirm the alert or the outage. A confirmation message will automatically be sent:

Subject: [confirmed] Example project

To: you and relevant colleges

Reply-To: your@address

BH-42 -> Your Name

https://vision42.net/your-organization/

Without project

This table lets you easily manage instruments which are not linked to a project:

• Change their names.

• Set or change their instrument types.

• Link them to a project.

• Completely delete an instrument, its values regardless of validity, its sensors, and its files.

Cleaning up files

Through a background process, where unreferenced files of at least one week old are removed; this way there will be multiple backups. References without a file are also continuously checked.

Projects section

Projects and guests

The checkbox controls the project visibility. Disabled projects will be hidden where possible, resulting in a less cluttered and faster app.

A coordinate system and a unit of elevation can be assigned per project. Among other things, reporting and importing will make use of this.

Periodic reports

In this table, you can add receivers, adjust the title, and set the period. The column last denotes when the report did last run. When empty, the past period will be reported on. Changing the period will clear last and thus resend the past period.

The subject of report messages will be prefixed with [report], to facilitate classification rules.

Currently, the available periods are:

• Daily, from midnight up to midnight.

• Weekly, from Monday up to Monday.

• Biweekly, from Monday up to the second Monday.

• Monthly, from the first day to the last day of the month.

No PDF file will be attached to report emails, because:

• It is often larger than the size limit of a typical email server, which would result in failed delivery.

• Sending large PDF files periodically, will sooner or later fill up the receivers mailbox. This would result in all email to the receiver being discarded, until space is freed up or added.

• A PDF file is not interactive.

• Correction of data after sending a report, would not be reflected in the PDF file.

Deleting old values

Old values of a project, be it measured or calculated, can easily be deleted in bulk up to a given date. This operation may take a while to complete, dependent on the size of your database. Closing or reloading the app window will break off the delete operation. Values marked invalid will be left behind, to aid future re-importing.

Report section

Using this section, you can easily and rapidly create nice looking reports. Just start out by selecting the instruments you want to report on. Note you are not limited to a single project.

Anatomy

From top to bottom, a report consists of:

• A title, which is editable.

• The from (inclusive) and to (exclusive) dates. When left blank, the last-used period will be plotted.

• The names and clickable contact addresses of the project administrators.

• A map showing the location of all selected instruments. The markers will contain the instrument type letter or symbol.

• A legend clarifying the instrument types present.

• A free-text information field.

Next up are all selected instruments:

• The type and name of the instrument.

• Relevant metadata:

  • The project code and name.

  • The contact address of the project administrator.

  • The coordinate system and the coordinates.

  • The elevation of the instrument.

  • The elevation of the surface.

  • The period in which this instrument has been measuring.

• The plot showing the desired period, which has been made exempt from refreshing.

• A table containing all manual measurements, if any.

Saving a periodic report

In order to create a periodic report, you must save it to the database first. That is the function of the save button. After successfully having stored the current report, you will be redirected to the projects section. There you can finish defining your periodic report.

The following report metadata will be saved to the database:

• The report title.

• The map boundaries.

• The map layer.

• The free-text information field.

• The selection of instruments.

Limited by RAM memory

While generating large reports, a browser may become sluggish for a longer time. After a while, some browsers will offer the choice to close the app or to keep waiting on it. This is a indication that the RAM memory has been depleted and the report is better split into smaller parts.

Review section

This section allows for rapid reviewing of specific data. Currently, reviewing of inclinometers and manual measurements is provided, with more on the way.

Manual measurements

An optimized table will present your measurements next to any derived virtual sensors. Changing a measurement will visually recalculate the corresponding virtual value. Changing a date/time will do so for both manual and virtual sensors.

Invalid values will be clearly marked by a red strike-through. Deleting a row will include any invalid values.

Setup section

Instrument types

This table is meant for managing instrument types and corresponding letters or symbols for their marker. The latter is used in the report section, for instance.

Instrument properties

In this table, you can define properties per instrument type. Leave the instrument type blank for generic properties, ie. all instrument types at once. The report flag toggles inclusion on reports. The information field may be used as a manual for the property. It will show up when users fill in the property values - the specifications of an instrument.

The input field is meant for advanced users. A little explanation about the syntax is in order:

1. If you do not specify anything, Vision 42 will automatically choose between a text or a numeric field, depending on whether the unit field is filled in.

2. A list of choices can be defined, by separating the options with a | sign (vertical bar). Example: full grouted | push in | sand pocket

3. Some standard HTML input types and unquoted attributes may be specified: Example: number min=0 max=10 step=2

Errors

This table contains errors, if any, encountered by your background jobs like continuous import or export.

Forward

Vision 42 can forward all received data and corresponding events of a given project to multiple destinations. The standard URL syntax must be used to configure forwarding. We advise making use of the separate secret field, to avoid a readable password in the URL.

Possible errors will be indicated by a clickable exclamation mark. They will be cleared automatically, once solved.

Forwarded data will remain in the send queue, until acknowledged. This guarantees delivery and makes outage recovery fully automatic.

The data payload looks like this and has generic and channel properties:

{

"device_id": "58A0CB000011D9C3",

"latitude": 50.964592,

"longitude": 3.72257827,

"altitude": 21,

"easting": 104607.24027609851,

"northing": 184011.27231100108,

"time": "2023-11-13T12:16:39.432Z",

"CO₂ [ppm]": 1067,

"Indoor air quality": 74,

"Relative humidity [%]": 68,

"Temperature [°C]": 19,

"Volatile organic compounds [ppm]": 1

}

• Format: JavaScript Object Notation (JSON), implying UTF-8 character encoding

• Coordinates: World Geodetic System (WGS 84) and local easting / northing

• Date/time: ISO 8601 with milliseconds in Coordinated Universal Time (UTC)

• Values: Relative to their instrument datum, if any

The event payload is similar. Outage events will be repeated, to distinguish between sensor(s) and system down.

{

"device_id": "58A0CB000011D9C3",

"latitude": 50.964592,

"longitude": 3.72257827,

"altitude": 21,

"easting": 104607.24027609851,

"northing": 184011.27231100108,

"time": "2023-11-13T12:31:39.588Z",

"channel": "CO₂ [ppm]",

"type": "!",

"event": "1005 > 994 ppm"

}

Specifications per network protocol:

• AMQP and AMQPS
  Append the publish /queue to the URL.
  0-9-1

• MQTT and MQTTS
  The path component will be used as the topic, suffixed by a device name.
  The Quality of Service (QoS) level is 1 for data and 2 for events.
  The default is version 5.

For TLS forwarding, a CA-issued certificate is being used. Brokers may use this for authentication, like Microsoft Azure Event Grid.

Export

Vision 42 can export project data to multiple SFTP, FTP, and FTPS locations. The standard URL syntax must be used to configure export locations. We strongly advise making use of the separate password field, to avoid a readable password in the URL. The locations must be directories, in which files will be exported in an orderly fashion. Existing files will be overwritten. Possible errors can be found in the Errors table.

The exported files contain times, instrument (or location) names, sensor (or quantity) names, units, and values. The latter will be relative to their instrument datum, if any. Invalid values will be excluded. Note that this structure is identical to the Senceive CSV format. Additionally, the Vision 42 files adhere to several standards:

• CSV format: RFC 4180.

• Date/time notation: ISO 8601 with milliseconds in UTC.

• Character set: UTF-8.

With usability in mind, exported files will not exceed 200.000 lines. This method of exporting is one-way only. It can not be used to keep data in the target system fully synchronized. After all history has been exported, data older than 25 hours will be exported continuously. This way, instruments have a day to upload their data.

Caution: Importing into the originating instance will usually not match the original instruments. If so, any instrument datums and calibration formulas will have to be reset.

For SFTP, public key authentication is supported, so passwords can be avoided. To configure, add this public key to .ssh/authorized_keys:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0skCWi+E3HMQgeuINQPYnO24K96JIGMVrca9xUzrtnbws31DeDkec0y+DH4ilUpZbEb2I+6krcQYEZnn92/IHBADJj7+iRBtmZJhPvguloaNygyBLF4E0yk6YB2PBdGXuoY/trGWj4h3h9NPKXpbkQwZtfmj8v4cQ7QGxbatmn62/piHfgY9tnL7ZpN9oxO4AQdSUisU4knPZViS7Go4Rj2bPDSVssqI+vCeNONjnvdSHXibwlIb/AiHMXJfSLEZytCnC1Ran9z9fPUWP0xCh+pHVKVv7hVwW6jc0R6081e4amBZOWsMiskoY/0CQGAfek9E6ON8yz7d+k7dKu2QZ Vision 42

FTP(S) servers and in-between firewalls must support extended passive mode. If not, connections will fail. Technical: your FTP software and your NAT firewall (Network Address Translation) must be able to handle EPSV and EPRT commands.

WMS layers

Web Map Service is a standard for serving map images over the Internet. All embedded maps in the Vision 42 app, like those in the map and report sections, can be enhanced with WMS layers. However, some basic WMS knowledge is required to do so. Please create a ticket, should your need help.

SCADA integration

Vision 42 can serve data, alerts, and outages to SCADA clients (Supervisory control and data acquisition). Please create a ticket to add your project to our shared SCADA servers. Mention for whom we need to create a username and password.

• Protocol: OPC-UA (Open Platform Communications Unified Architecture)

• Hostname: vision42.net

• Port for production: 4843

• Port for quality assurance: 4844

• Port for development: 4845

• Server certificate: signed by a universal trusted CA (certificate authority), domain validated (DV) and valid for 90 days

• Anonymous access: not allowed

A channel alert indicates its level, corresponding to the four thresholds: -2,-1,0,1,2. Aggregations per device and per project equal the maximum absolute level: 0,1,2.

A channel outage is a simple boolean: false,true. Aggregations per device and project indicate the outage percentage.

Organization section

Overview

This section lets you manage the organization structure and the users of the app. It is geared towards administrators, while regular users can view and browse the information. Out of privacy considerations, this section is not available for guests.

It is divided into three parts:

• A representation of the organization structure.

• A Users section.

• A Departments section.

In the structure graph you can select a department.

• The users linked to this department will be shown in the Users section.

• The administrators of a department are shown when hovering over the department in the graph.

• If you are an administrator in this department, you can link existing users to it.

Creating new users

New users are first created without a department:

• Go to the empty department, depicted by a empty circle.

• Add the user info at the bottom:

  • Name.

  • Identification; preferably the email address.

  • Administrator flag.

  • Create the user by clicking the plus icon.

• Next select the department the user belongs to and link the user to it.

Email, notifications, channels, and SMS

The identification field controls how messages will be sent to a user:

• A regular email address is recommended, for most use cases. Email can have attachments, which the upload message and the inclinometer review message make use of. Note that email from Vision 42 is being monitored and protected from eavesdropping and impersonation.

• In case (alert) messages might get lost in the noise of other emails, a notification address can help out. A good and easy example of this is Pushover. Their do-not-disturb and quiet-hours settings are nice to have.

• Messages can be sent directly to Slack and Teams channels.

• An SMS gateway address can be used. However, SMS is insecure and severely limited regarding message length and is missing a subject, attachments, and line breaks in some cases. The dedicated email address your-organization@vision42.net may be used to authenticate the sender at the gateway of your choice.

• A non-address will exclude the user from any messages.

Combinations of the above can also be realized, by creating more than one account per person.

Forgotten passwords

Password management is done by clicking the reset-password link, next to the name of the user. If the identification of the user is a contact address, the new login credentials will be sent automatically. If not, the password will be displayed on screen, in order to convey it to the user by other means.

If desired, the user can change the reset password using her or his profile page.

Organization structure

This third section handles the organization structure. New departments can be created by providing:

• A name.

• The optional parent organizational entity.

 Add the new entity by clicking the plus icon.

Access rights

Members of your main department, typically denoted by your company name, have the right to change things within Vision 42. Administrators have more rights, like adding users and organizational entities.

Users who are not members of that main department are considered guests. They can only view their assigned projects and edit their own data.

Print / PDF

Invoking

Make use of the menu or the dedicated buttons within the Vision 42 app. The Ctrl + P or ⌘P key combinations will also work as expected. Avoid using any print option in the browser's menus, since those will not resize maps correctly. The latter can also occur when printing from Android.

Note: Google Maps is here to blame.

Customized header

Vision 42 will add a nice header to prints and PDFs, containing customer information:

• Your logo.

• Your street address.

• Your website URL, email address, and phone number.

• A QR code, typically pointing to your contact page.

Please create a ticket, should this information have to be updated.

Typically, a browser will add its own header and footer, over which Vision 42 has no control. You can simply turn these off once, in the print dialog.

Quality

Care has been taken to achieve the best possible print / PDF quality. Plots, for example, will remain scalable vector graphics (SVG). Only maps are bit-mapped, due to the very nature of aerial and satellite photography.

Share

Use this menu option if you want to share a plot, a report, a review, a map, etc. with a user. The app state URL will be shared by chat, email, clipboard, or any other suitable app. In absence of sharing facilities, the URL will be copied to the clipboard.

About section

Create a ticket

This link allows you to create a ticket, directly in our software management system. Most fields will be filled in automatically. Please enter a one-line summary and a detailed description of the problem.

 Tickets can cover:

• Code defects

• Documentation

• Feature requests

• Service requests

Once submitted, you will receive an automatic message whenever your ticket changes. In those messages, you will find links to edit or append the ticket and to add an attachment. The security code is there to stop spambots. Please ignore the login page, which is meant for staff members taking care of your ticket.

Tip: Instead of attaching a screenshot, you may attach a single-file web page. Just use the keyboard shortcut Ctrl + S or ⌘S and choose the .mhtml option, if needed. Avoid using any save option in the browser's menus, since those will not include dynamic content. This method has the following advantages:

• It will capture the full page, not just a visible part.

• Text can still be copied and pasted.

• The resulting file contains more debug information.

• No need for a screenshot application.

Statistics

This section contains interesting statistics about your data in the Vision 42 application. Note that the number of values will only be counted once a day, for performance reasons.

Changes

A history of all changes made to your instance of Vision 42 can be found here. In the QA-environment this list is supplemented by available changes not yet promoted to production.

Sign out

This menu option will delete your stored credentials and sign you out of the application.

REST API