Knowledge of common failure points helps guide data logging system design
In selecting a data logging and alarm logging solution it is best to consider what happens to the data when everything is not performing perfectly. In a system’s lifetime at least one of the following if not all five will occur multiple times:
- Database Engine Failure
- Network Failure
- Missing High Speed Data
- Inaccurate Manual Setup
- Defective Controller Handshaking
Database Engine Failure
 |
When the connection to the database engine fails during database backup, maintenance, or network failure to remote engines data will be lost if the data logging and alarm logging solution does not provide store and forward functionality.For systems that do have store and forward capabilities it is important to select one that can maintain the data for long periods of time. This requires that the data to be inserted or updated to the database must be stored to disk instead of just buffered to RAM.
Steps to replicate condition:
Note: An advanced test to account for a system restart during data buffering is to shutdown and restart the logging server between steps 2 and 3. Most store and forward solutions will lose data if the server PC is shutdown. |
Network Failure
 |
When the network connection between the data source server and data logging server is down, data will be lost if the solution does not implement a distributed network design with store and forward at the data source. By relying on cloud solutions for data and alarm archiving data will be lost when the connection to the data source is broken and data cannot be sent to the cloud solution. Also there are many tunneling solutions that provide network transport, but do not maintain the data during network outages.
Steps to replicate condition:
|
Missing High Speed Data
 |
It is important for a logging solution to provide data processing right at the source and process all values received from the data source. Data and alarms can be missed if data transitions quickly and the solution processes only the current value sampled.Consider a sensor that receives a sharp spike in data for a very brief time interval. If sampling occurs before and after the spike, no alarm would be recorded and the high data value would be missed in the data archiving. Some systems can sample at a much faster rate, but then have problems moving the large amount of data to the database engine efficiently and then data begins to backup in the system. It is important to select a system that can handle high bursts of data during critical events.
Also data that is processed directly within the controller can be queued to be handed over to a data logging system. It is important that the data logging solution have the ability to provide handshaking to the controller to inform when the data is received and processed and ready for the next record. Steps to replicate condition:
|
Inaccurate Manual Setup
 |
Automated or programmatic setup is important in system setup so human error does not affect the system accuracy of what data is logged and alarmed upon. With large amounts of data to be collected and processed it is very easy to point to the wrong variable address, alarm limit, or database field.Steps to verify setup accuracy:
|
Defective Controller Handshaking
 |
Some systems will queue data in the controller for best data accuracy and assurance of no data loss on communication failure to the controller. This is typically done with a queue or array within the controller where the data is buffered and then passed to the data logging engine when it is available to record the data. It is important to enable a handshaking technique that can validate data has been successfully archived.Steps to verify accuracy:
|
How Open Automation Software solves Data Logging Issues
Open Automaton Software addresses all of these scenarios with the following features.
- Store and Forward
- Distributed Network Architecture
- High Speed Data
- Automated Setup
- Programmatic Setup
- Alarm Notification
- Controller Handshaking
1. Store and Forward
Store and Forward capabilities that can buffer data and alarms to disk in a small binary file both on database engine and network failure. And data is maintained even after a server reboot.
View the following video of a demonstration of OAS Store and Forward performance.
/videos/#data-buffering
2. Distributed Network Architecture
Distributed Network Architecture with data, alarm, and trend processing at the data source.
View the following video demonstration of data being transferred from Australia to the US, and then accessed anywhere in the world to be logged and displayed.
3. High Speed Data
High Speed Data processing of queued data with proper indexing of time samples and efficient database bulk insert to pass up to 1,000,000 records per call.
View the following video demonstrating data logging data changing at the rate of 100 nanoseconds.
/videos/#high-speed-data-logging
View the following video demonstrating data logging of data from an Allen Bradley ControlLogix controller at the rate of 20 milliseconds.
/videos/#logging-data-from-controllogix-at-high-speed
4. Automated Setup
Automated Setup to eliminate human error in setup. OAS has automated setup for both data sources and data logging.
One Click OPC feature is used for automated setup of OPC Items.
- Guide: /knowledge-base/one-click-opc/
- Video: /videos/#one-click-opc
One Click Allen Bradley for automated setup of AB controller communications.
One Click Database feature is used to setup data logging of all data points.
5. Programmatic Setup
Programmatic Setup is fully supported for data sources and logging configurations through both .NET and REST API.
View the following guide for .NET programmatic setup.
/knowledge-base/net-programmatic-configuration/
View the following guide for REST API setup.
/knowledge-base/getting-started-rest-api/
6. Alarm Notification
Alarm Notification on data logging or alarm logging failure. OAS can call personnel via phone or send a text message and email when data buffering is active. Even though OAS can store many years worth of data to disk to restore the database engine it is best to be notified immediately when there is a problem.
View the following video on how to setup email notifications.
/videos/#send-alarm-emails
7. Controller Handshaking
Controller Handshaking to communicate with the controller to provide confirmation that the data is received and ready for the next record. Within OAS logging there are confirmation and error feedback status that can be updated to live tag values for both alarm notification and to be sent directly back to a controller for closed loop data handshaking.
View the following steps on how to setup handshaking with a controller.
/knowledge-base/log-buffered-data-from-a-plc-or-controller/