A comparison of logging Industry 4.0 data in an open format vs. a proprietary format
Logging industrial automation data in an open format provides users and automated systems the flexibility to utilize reporting tools, business analytics, and learning engines to harvest data with 100% transparency and accuracy. An open format allows engineers to select the best tools to match the application requirements without being locked into proprietary systems to access their data.
Have you been locked out of data access due to proprietary formats and cumbersome development tools?
Open vs Proprietary
There are several features an open database format can support that are not achievable with a proprietary format.
Open Database
Easy Access
With an open format, data sets can be returned from simple SQL query statements like “SELECT * FROM myTable”.
Data for Learning Engines and Business Analytics
Data can be consumed by any learning engine or business analytics tool for maximum flexibility in processing data in meaningful way. It is important to have all raw data including timestamps directly from the source for accurate data models. Using an open format, you control what gets logged and how it is stored.
No Cost To Access
No need to rely on one specific vendor to access your data. Choose the best tools to meet the requirements. Whether it be off-the-shelf reporting tools or your own custom solution, you get to decide the best solution for your requirements.
Fastest Results
Data results are returned quickly without the need for post processing or need rely on one specific vendor to access your data. No requirement for data processing components. You are even free to perform offline data transformations and optimizations.
Easy Management
Data archives can be backed up or mirrored in incremental steps. Use standard database administration tools for easy management.
Update Existing Records
Updates to existing records can be performed easily.
Smaller Datasets
Return any number of records or columns of data that is needed. This results in faster queries and lighter network traffic.
Smaller Memory Usage
By logging the raw value only, the native value in bytes is used for disk storage.
Insert Multiple Records in One Call
OAS can log up to one million records per network call, this eliminates the possibility of data getting backed up within the data flow.
Open Developer Support
All database developers and administrators can understand and work with an open format. Using an open format on industry standard databases, there is no need for database administrators to learn any specialized skills.
Third Party Reports
Use with any reporting tool to generate dynamic or automated reports.
Proprietary Database
Difficult Access
Data cannot be accessed and viewed directly. Custom or proprietary tools must be used to access your data.
Inconsistent Number of Records
Proprietary systems sometimes use compression algorithms to overcome bloated data representation, leaving missing or uncertain data gaps within the data set.
Limited to Specific Platform
Data can often only be consumed by the originating platform. Variable fees are sometimes charged in order to access the data. Why pay additional fees to see your own data?
Slow Response
Data that is obtained needs to be processed through additional routines, slowing down the history results.
Proprietary tools and user interfaces
In order to set up archiving routines or mirroring, the tools from the originating platform must be used, limiting the options for archiving.
Records Cannot be Updated
In some industries like pharmaceutical applications this is desired, but in manufacturing and material handing it is often beneficial to update existing records based on the step product creation or distribution.
Bloated Data
In order to log data in a proprietary format additional data needs to be recorded for indexing, status, quality, source of origin.
This results in larger data sets returned and larger disk capacity to archive the data.
Constrained Delivery
If the format of the data does not allow bulk inserts to the database, data can often get obstructed if the network or database engine is slow to process, which then results in either high memory usage or data loss.
Specialists Required for Support
Working with proprietary data will require specialized skills to work with datasets. Selection of third party tools will be limited, and sometimes not able to process data results. Time required to develop solutions to providing meaningful results can become costly.
Limited Reports
The choices of reporting tools is limited to what a vendor makes available to you, sometimes only available from the originating platform.
Open Automation Software Solution
Open Automation Software Performance
One challenge of logging data in an open format is data throughput. OAS overcomes this by archiving data at a rate of over one million values per second, up to two million values using MS SQL Server. Open Automation Software has worked directly with database providers Microsoft, Oracle, and others to achieve the best throughput to insert new records in a database while still maintaining a format that can be directly queried. This is achieved by processing data in the OAS Engine with a first-in-first-out value processing, carrying the data quality and timestamp directly from the data source.
Data Historian Performance Benchmarks
Open Data Sources
The OAS Data Historian can log data from several sources, the most commonly used are Industry 4.0 like Modbus devices, Siemens S7 controllers, Allen Bradley PLCs, OPC and OPC UA Servers, MQTT, MTConnect, Azure IoT, and AWS IoT. OAS also supports programmatic interfaces for .NET applications and REST API clients and custom driver support though the Universal Driver Interface, allowing you to log data from virtually any source including your own real time application.
High Scalability
Each OAS Engine instance can log to over 10,000 separate tables. The design can be a narrow format, mapping each data point to one or more table, or wide format with all data points in one table. OAS can log to multiple database engines of multiple providers at the same time. Combined with the built in store-and-forward feature data can be mirrored easily from the data source to multiple destinations maintaining complete accuracy on redundant storage.
Secure Networking
OAS transports data from service to service securely using custom encryption with light-weight transport with compressed packets.
Reliability
With OAS, there is no fear of data loss since data even if the database engine cannot be reached. In the event of a network failure, database engine failure, or if communications is lost for any amount of time, the store-and-forward feature will queue data to disk. OAS can also log to multiple remote database engines reliably with the same bulk insert performance mentioned above for mirroring and backup.
Quick Setup
Configuration can be implemented in 4 possible ways, manually, CSV Import, automated setup, or programmatic setup.
If you’d like to learn more or try the OAS Data Historian please submit the form below to receive a fully-functional trial of the OAS Platform today.