Knowledge of common failure points helps guide data logging system design

In selecting a data logging and alarm logging solution it is best to consider what happens to the data when everything is not performing perfectly.  In a system’s lifetime at least one of the following if not all five will occur multiple times:

  1. Database Engine Failure
  2. Network Failure
  3. Missing High Speed Data
  4. Inaccurate Manual Setup
  5. Defective Controller Handshaking

Database Engine Failure

When the connection to the database engine fails during database backup, maintenance, or network failure to remote engines data will be lost if the data logging and alarm logging solution does not provide store and forward functionality.For systems that do have store and forward capabilities it is important to select one that can maintain the data for long periods of time.  This requires that the data to be inserted or updated to the database must be stored to disk instead of just buffered to RAM.

Steps to replicate condition:

  1. Stop database engine during data logging and alarm logging.
  2. Verify data buffered to disk.
  3. Start database engine.
  4. Verify all data has been archived before, during, and after database engine shutdown.

Note: An advanced test to account for a system restart during data buffering is to shutdown and restart the logging server between steps 2 and 3.  Most store and forward solutions will lose data if the server PC is shutdown.

Network Failure

When the network connection between the data source server and data logging server is down, data will be lost if the solution does not implement a distributed network design with store and forward at the data source. By relying on cloud solutions for data and alarm archiving data will be lost when the connection to the data source is broken and data cannot be sent to the cloud solution.  Also there are many tunneling solutions that provide network transport, but do not maintain the data during network outages.

Steps to replicate condition:

  1. Disconnect network between data server and cloud system or remote data logging server.
  2. Verify data buffered to disk at the source.
  3. Reconnect network.
  4. Verify all data has been archived before, during, and after network outage.

Missing High Speed Data

It is important for a logging solution to provide data processing right at the source and process all values received from the data source.  Data and alarms can be missed if data transitions quickly and the solution processes only the current value sampled.Consider a sensor that receives a sharp spike in data for a very brief time interval.  If sampling occurs before and after the spike, no alarm would be recorded and the high data value would be missed in the data archiving.  Some systems can sample at a much faster rate, but then have problems moving the large amount of data to the database engine efficiently and then data begins to backup in the system.  It is important to select a system that can handle high bursts of data during critical events.

Also data that is processed directly within the controller can be queued to be handed over to a data logging system.  It is important that the data logging solution have the ability to provide handshaking to the controller to inform when the data is received and processed and ready for the next record.

Steps to replicate condition:

  1. Use a data source of a .NET application or high speed communications that can provide time stamp and data at high rate, microsecond samples if possible.
  2. Have the data change cycle below and above alarm conditions a series of times.
  3. Verify that all data samples are recorded to the database, and all alarm events have been captured and recorded.

Inaccurate Manual Setup

Automated or programmatic setup is important in system setup so human error does not affect the system accuracy of what data is logged and alarmed upon.  With large amounts of data to be collected and processed it is very easy to point to the wrong variable address, alarm limit, or database field.Steps to verify setup accuracy:

  1. Insure there is automated or programmatic setup of the data source, data logging, and alarm logging configuration.
  2. Use a CSV Export to spreadsheet and verify each row for data addresses and field names match.

Defective Controller Handshaking

Some systems will queue data in the controller for best data accuracy and assurance of no data loss on communication failure to the controller.  This is typically done with a queue or array within the controller where the data is buffered and then passed to the data logging engine when it is available to record the data.  It is important to enable a handshaking technique that can validate data has been successfully archived.Steps to verify accuracy:

  1. Start the controller cycles processing records to be logged
  2. Stop database engine.
  3. Have controller execute several cycles of records to be logged.
  4. Start database engine.
  5. Verify that no records are missing from the cycles performed before, during, and after the database engine shutdown.

How Open Automation Software solves Data Logging Issues

Open Automaton Software addresses all of these scenarios with the following features.

OAS Data Logging

  1. Store and Forward
  2. Distributed Network Architecture
  3. High Speed Data
  4. Automated Setup
  5. Programmatic Setup
  6. Alarm Notification
  7. Controller Handshaking

1. Store and Forward

Store and Forward capabilities that can buffer data and alarms to disk in a small binary file both on database engine and network failure. And data is maintained even after a server reboot.
View the following video of a demonstration of OAS Store and Forward performance.

2. Distributed Network Architecture

Distributed Network Architecture with data, alarm, and trend processing at the data source.
View the following video demonstration of data being transferred from Australia to the US, and then accessed anywhere in the world to be logged and displayed.


3. High Speed Data

High Speed Data processing of queued data with proper indexing of time samples and efficient database bulk insert to pass up to 1,000,000 records per call.
View the following video demonstrating data logging data changing at the rate of 100 nanoseconds.
View the following video demonstrating data logging of data from an Allen Bradley ControlLogix controller at the rate of 20 milliseconds.

4. Automated Setup

Automated Setup to eliminate human error in setup. OAS has automated setup for both data sources and data logging.

One Click OPC feature is used for automated setup of OPC Items.

One Click Allen Bradley for automated setup of AB controller communications.

One Click Database feature is used to setup data logging of all data points.

5. Programmatic Setup

Programmatic Setup is fully supported for data sources and logging configurations through both .NET and REST API.
View the following guide for .NET programmatic setup.
View the following guide for REST API setup.

6. Alarm Notification

Alarm Notification on data logging or alarm logging failure. OAS can call personnel via phone or send a text message and email when data buffering is active.  Even though OAS can store many years worth of data to disk to restore the database engine it is best to be notified immediately when there is a problem.
View the following video on how to setup email notifications.

7. Controller Handshaking

Controller Handshaking to communicate with the controller to provide confirmation that the data is received and ready for the next record. Within OAS logging there are confirmation and error feedback status that can be updated to live tag values for both alarm notification and to be sent directly back to a controller for closed loop data handshaking.
View the following steps on how to setup handshaking with a controller.

Download a Fully Functional 30-Day Trial of the Open Automation Software Platform