High-Speed, Reliable Data Collection and Logging

Data accuracy is highly dependent on how IoT systems collect, transport and deliver data. It is critical that information systems receive data from data acquisition solutions that can run, on-premise, as an edge solution and can store and forward data on communication failures. Edge IoT systems like Open Automation Software eliminate the dependency on network reliability that cloud solutions require, and can deliver data with high precision and greater reliability.

Open Automation Software is deployed as an edge solution to collect and process data on-premise utilizing a first-in-first-out (FIFO) data method to deliver all samples of received data over the network to user interfaces, applications, remote OAS Engines, and databases.

Example solution gathering data in real time from a PLC to an edge or on-premise OAS instance, passing the data to a remote OAS instance which is logging data to an open database such as MS SQL Server. (click image to enlarge)
Watch the video to see how OAS transports 1,000,000 samples to a Raspberry Pi device, processes it at a rate of 100 nanoseconds per sample, and inserts 1,000,000 records into MS SQL Server in one database transaction.

By running as an edge computing solution Open Automation Software provides the reliability to retain the data in all possible failure scenarios. OAS is able to achieve the highest data accuracy possible through optimized communications with controllers and applications utilizing light network transport that is both secure and has a delivery confirmation, providing high throughput performance to databases with its store and forward capability.

Controller and Application Communications Failover

OAS supports buffered communications to controllers and applications to perform bulk polling of multiple samples in one call.  This provides a sampling speed matching the controller or application time cycle. It also provides store and forward capabilities directly on the controller so that there is zero data loss in the case of temporary communication failures.

OAS also supports communications redundancy failover to a backup controller providing an automated switch from primary to secondary controller or application.

OAS can also receive data from .NET applications and REST interfaces to update tag values.  See the Real Time Data Access WriteTags method to see how timestamps can be included with values to transfer multiple values in one simple call.

Upon failure to communicate with a data source, OAS can be configured to automatically fail over to a backup source or network address. (click image to enlarge)

Handling Remote Network Communications Failures

OAS runs as an edge solution collecting and processing data on-premise enabling it to reliably process all data in the event of a network failure. When data is transferred over the network, all value changes since the last successful transfer are transmitted, not just the latest value. This is important for building accurate data models from the data source. Network packets from client to server and server to server are encrypted and compressed to achieve the smallest packet size available.

The real-time trend and alarm information is processed locally and cached on the OAS Engine so when the remote user interfaces are able to reestablish a connection all trend and alarm data is provided to 100 nanosecond accuracy.   OAS also has client and server watchdogs to notify applications when there is a network failure.  If real-time access is critical, there can be an automated client switchover from primary to backup OAS Engine within the client application. Each OAS engine can also support multiple client interfaces.

Store and Forward functionally is also built in for data logging and alarm logging so the highest possible retention of data is available during temporary network failures.

Deployed as an edge solution, if remote systems cannot be reached, the edge instance of OAS will continue to capture source data, ensuring data integrity when communications are reestablished. (click image to enlarge)

Database Performance and Data Buffering

OAS has built in store and forward functionality to account for database engines failures as well.  In the event of a communications failure attempting to write data to a database, OAS can be buffer data to a local disk. When communication is reestablished, OAS will write all data out to the database, restored as first received first sent updates to the database in bulk calls.

The database engine can be located on the same system as the OAS Engine or on a remote system, as multiple records are inserted with just one call. This eliminates any possibility for data to be dropped or lag behind in update to the database. OAS can log up to 2,000,000 values per table to SQL Server.

In the event of a failure to log data to a database, OAS will buffer the data locally and write it back out when the database is back online with zero data loss. (click image to enlarge)

Download a Fully Functional 30-Day Trial of the Open Automation Software Platform