IoT Edge Computing with UDI: Maximum Flexibility and Reliability

IoT Edge computing sets the stage for running anything anywhere, and with OAS this allows you to move data from anywhere to anywhere. This is ultimately the most flexible distribution or deployment platform. OAS brings even more to the party with the Store-And-Forward feature, ensuring zero data loss upon disruptions in connectivity. This feature is also available in custom drivers developed with the Universal Driver Interface (UDI), truly making it possible to connect OAS to ANY data source.

“Life is a traveling to the edge of knowledge, then a leap taken.”
– D.H. Lawrence

OAS and Edge Computing

Edge v Cloud Computing
Click to enlarge

So what is Edge Computing? See this previous article that covers the basics and describes the differences between Cloud Computing and Edge Computing. OAS is a true Edge Computing platform as it allows you to deploy instances of the OAS server close to the source of your data, and connecting it to other instances of OAS services on both local and wide area networks. With OAS you can even share or aggregate your data with OAS instances on the internet. How you choose to deploy OAS is up to you and your requirements.

By deploying an instance of OAS next to your data source, you can perform data logging, data transformations and calculations at the source, thereby distributing the load of the entire system and allowing each component to continue working in the face of network interruptions.

 

In a typical cloud based system, a controller will communicate and push data up to a cloud system for easy connectivity. However, there is a big flaw in this design as you don’t know if the control is communicating with the cloud unless safeguards like watch dogs and confirmations of delivery are written into the code. In these circumstances, data can be lost because the data is not resident at the source.

Benefits of Edge Computing over Cloud Computing include:

  • Data accuracy and reliability
    An event may have occurred during communications information which affects the data accuracy. Cloud computing results in lost data. Edge Computing with OAS retains data at the source until it can be passed along.
  • Reduced operating costs
    In a typical cloud system, customers (you) are charged for all data used on a monthly basis and if data is lost due to a network failure, you are still charged for that data. Compare this with OAS where data is distributed and are not charged for any data usage as it is hosted on your own hardware.
  • Increased efficiency and flexibility
    In a cloud system, all data access is bottlenecked and distributed through a central cloud server. With OAS and Edge Computing, each data source or location can be configured as an independent point of processing or access. Local data can be used and managed without relying upon a remote connection. OAS also delivers data at very high speeds. Every single value that occurs from the data source is delivered – first in, first out – processing down to 100 nanoseconds.
  • Security

    Open Automation Software communications is very light weight, it’s a binary communications system – compressed and encrypted so that it can’t be hacked for terrorist activity. Edge Computing also reduces single points of failure in your system, reducing inherent vulnerabilities in a central cloud solution.

Store and Forward

One of the key features of the OAS Platform is Store-and-Forward, which caches captured data at the data source and transmits it to other systems when network communications resume. Typically, cloud computing solutions cannot do this. If a system is dependent upon cloud storage and processing, data is lost when communications are interrupted.

Store-and-forward enables your solution to self-heal and your data remains intact. If the database engine or data source to data logging isn’t available, that data is buffered locally and written onto a hard disk.

UDI with Store and Forward

The OAS Universal Driver Interface is a natural extension of the OAS operating philosophy – moving data from anywhere to anywhere. The UDI allows developers to create custom communication drivers for ANY data source, whether it be a proprietary database, 3rd party API, previously unsupported devices, file systems, or literally anything that can generate data. The UDI architecture also allows you to deploy a driver on any system, including Windows, Mac, Linux and even Raspberry Pi devices without requiring a full OAS Platform installation at the data source. And the UDI fully supports the Store-and-Forward capabilities of OAS.

This means you can create a simple driver that behaves just like a built-in OAS device driver with all of the Edge Computing benefits built in!

Would you like to learn more?

See the following resources for more information on Edge Computing, The Universal Driver Interface, OAS Data Historian, and more:

 

OAS Takes IoT Automation to the Next Level

Automated Set-Up & Deployment

What if you could connect and read data from any device (Windows, Mac, Linux, Android and iOS) from anywhere in the world? What if you could spin up a device and it could automatically set up data points and start transmitting data as soon as it came online?

With Open Automation Software (OAS) Universal Driver Interface SDK, or UDI you can build a custom driver that automates configuration on startup. Once created, the driver can be deployed to multiple systems with no additional coding, simplifying new installations and dramatically reducing the cost of scaling your operations. The UDI also opens up new connectivity options by making it possible to connect to any other data source, device, or external API.  

Eliminate human error, capture all your data and save time by automating processes previously manually entered across your enterprise.

The Open Automation Software Platform is already incredibly cost effective and flexible with unlimited connections and a decentralized network. While the platform supports MOST Data Sources and Endpoints, it does not support all….until now with the integration of the Universal Driver Interface.  

Introducing The Universal Driver Interface SDK (UDI)

The OAS Platform has always been highly configurable and supported automated configuration through the use of the .NET Connector. But OAS has embraced the auto set up even further with the introduction of the Universal Driver Interface SDK. The UDI allows you to create communication drivers between devices, data, and APIs to integrate with the Open Automation Software IIoT Framework.

The Universal Driver Interface is also designed for cross-platform communications, it can be run remotely on different operating systems (PC’s – Linux, Mac OS, Android, iOS, even Raspberry Pi).

Simply run the Universal Driver Interface and it can automatically set itself up with the data model and individual data points you defined within the driver.  The set-up of the communication points between different machinery is then automated.

How Does It Work?

In a typical platform communication, Open Automation Software communicates with controllers in a manufacturing plant, e.g .Allen Bradley, Siemens, Modbus and OPC servers and Clients.

This is all done on a Windows Server PC. Below, you can see a typical platform working on a windows server as well as other cloud-based technology communications.

The above diagram illustrates the existing Open Automation Software platform without the new driver interface.  Each OAS Platform installation communicates with data sources and destinations using built-in drivers. Once data is within the OAS Platform, it can be shared with other instances of OAS over wide or local networks.

By implementing a driver using the UDI SDK, you can now open up many more opportunities to connect between devices, data, and APIs. A UDI implementation bridges the gap between previously unsupported or proprietary data sources and the OAS Platform.

OAS Universal Driver Interface Overview


Some of the advantages of the Universal Driver Interface are:

  • flexible networking
  • remote connectivity
  • support for multiple operating systems
  • one-click deployment
  • no touch, automated set up
  • one-click data logging
  • speed – develop the code, run it and it’s done
  • scalability – auto set up of multiple data points across manufacturing plants and remote locations eradicating human error and is far more time efficient.

Universal Driver Interface Set-Up

The Universal Driver Interface is built upon the .NET Standard 2.0 Framework, allowing it to be deployed and run on Windows, Mac, Linux, Android, iOS, and even Raspberry Pi platforms.  (See all supported platforms)

It’s simple to implement into your existing Open Automation Software platform.  The online working code examples are installed with the software.

Your Developer can then use the custom code supplied in VB.Net or C# to create your own custom communication drivers, data models and have those data points automatically set themselves up in the OAS Platform. Within the same code set, Developers can add default properties for the new set up to occur. 

Your Developer can define your own data model, names, data types and parameters. Those Tags can then auto set up in the Open Automation Software server to begin the communication process. Then, when a user starts their device, it will automatically add a driver interface with a specific set of values across all data points.

How do you currently enter data into your Open Automation Software platform?

If it is plant-by-plant or process-by-process through manual entry, there is room for human error as well as inaccurate data being introduced into your systems.

With automated entry, you do take some time to figure out what you want the code to do, but it is well worth the effort. Auto setup eliminates human error by 100% based on the code routines and saves a lot of time since incremental deployments require minimal effort.

Driver set-up

Within the Open Automation Software, you can specify driver types and specific properties of the driver interface as well as manually set up Tags and other features. While the manual process works, it is not as effective or efficient, especially if you have many points of data to enter. Instead of multiple manual entry points, you can run the Universal Driver Interface which automatically tells the Open Automation Software its data model for the driver and data points and automatically adds them to the programming.

The Universal Driver Interface has been designed for cross platform communication. In other words, it allows you even greater flexibility to create your own basic code to communicate with the Open Automation Software platform and run on different operating systems. The Universal Driver Interface has a flexible networking and operating system to define your data models integrating them into the Open Automation Software platform.

Related articles/further reading:

Open Visualizations with OAS

Are you constrained by your current HMI builder tools?
Are you facing burdensome licensing costs with your current solution?
Is it difficult to customize your visualizations to meet your specific needs?

These are but a few problems you may be faced with when you’re locked into using a proprietary or “canned” HMI solution. Of course, every product has its benefits, and your current solution is no exception. Proprietary solutions do a great job of getting you up and running quickly, and are geared towards developing visualizations with little or no programming skills. But these benefits come with a cost: lock-in.

In order to develop these drag-and-drop or non-programmatic HMIs, vendors supply proprietary tools and deployment platforms. You’re often restricted on what type of visualization you can create within their sandbox, limited on what applications you can share data with, and you can rarely integrate with your own custom graphics and applications.

The solution? An open architecture such as the OAS Platform.

When a product uses an open architecture, you can use commodity servers, development tools, and they are built from the ground up for integration. The OAS Platform even exposes multiple APIs for automation and visualization development in ANY application platform and language.

Benefits of Open Visualization

  • Ubiquitous or common skills needed to develop a visualization
    With an open platform, you can build a team from the millions of developers and consultants with .NET and web application development experience. Common languages and popular development tools are all supported such as C#, VB, Javascript, Python, Visual Studio and your favorite code editor.
  • Licensing and distribution costs are dramatically lower
    Incremental costs can typically be capped when adding new clients or screens. The OAS Platform allows for unlimited client connections with no additional fees or per-seat charges, so you’re free to expand without breaking the bank.
  • No limitations on the type of visualizations possible
    Using common tools for developing your own symbols and screens, or using any 3rd party set of graphics, you can now create the screen you need and not just settle for what your HMI builder supports. Create images and symbols in popular graphics editing suites, 3D modeling packages, or anything that can produce standard files such as JPG and PNG to be used in Windows, web, and native applications.
  • No vendor lock-in
    You are free to integrate existing products for the best solution. Don’t get rid of what you already have, enhance and improve it by integrating an Open Visualization.

How-to Develop an Open Visualization

Step 1: Connect your data to the OAS Platform

The OAS platform supports connectivity to virtually any data source through options such as native device drivers, database access, or API integrations

Step 2: Organize your data structures

Configure, aggregate, and organize your data points within the OAS Platform allowing you to expose them through APIs and Developer Tools for .NET, web, and REST API clients on any platform.

Step 3: Connect your visualizations

Display dynamic, real time and historical data in your custom visualizations. Build native apps for Windows as well as iOS and Android mobile devices in .NET and Xamarin. Build cross-platform web visualizations using any web application environment and any browser. Build custom integrations and visualizations on any platform using the REST API.

If you’d like to learn more about the OAS Platform and how it can transform your Industrial Automation or IoT solution, please contact us for a guided demonstration.

The Power of an Open Architecture

A comparison of features and benefits of an open architecture platform vs. an off-the-shelf solution for Industrial Automation and IoT

In IT it has become almost axiomatic that it’s cheaper to purchase an off-the-shelf solution instead of building your own application from scratch. A custom piece of software can very expensive because it is being built for a single purpose and with one customer in mind. It usually takes much longer to get up and running and to work out all of the bugs.  But developing custom software is still embarked upon because it comes with some significant benefits, especially if you are an OEM, interested in reselling the solution.

What if there was a software platform that provided the benefits custom development, but also mitigated the risks of rolling out your own solution, and even dramatically reduced costs over time?

So let’s explore this a bit further, breaking down the pros and cons of both custom development  and off-the-shelf products. We’ll then demonstrate how the OAS Platform is unique in the IIoT space and what an open architecture can do for you.

Off-The-Shelf Products

The pros and cons of off-the-shelf (OTS) or “canned solutions”, which may still include some tools for designing HMI screens and logic, but within proprietary environments.

PROS

Rapid configuration

One of the most attractive features is the relative ease at which OTS products can be configured and deployed since the feature set and tools are well-defined.

Built-in tools and applications

In order to speed up and simplify deployment, most OTS products include many tools for creating things like HMIs, and often have full-blown applications ready to be used in production.

Generally well-supported

Coming from many well-established companies, these products usually come with a team of support personnel and documentation, which usually comes at an additional cost.

Great for demos or proofs-of-concept

When an OTS platform is narrowly focused on a solution domain, and when tools and applications are proprietary, these products are impressive during tightly controlled demos. Little or no programming is required to get up and running.

CONS

Difficult (or impossible) to customize

Once you start using these products, you’ll quickly find that you need some functionality that they do not provide within the basic toolset. So then you must resort to using APIs with some custom development to extend them. Other times, customization is not even possible.

Strict licensing, difficult for OEMs

To protect their intellectual property and branding, software vendors tend to limit OTS platforms by the number of users connected to them, and it can become very expensive and restrictive very quickly for OEMs, making it difficult to fit these products into your business model.

Proprietary data format

Another way license restrictions are enforced is to lock down the data within the system. OTS vendors provide tools for importing or exporting data, but the data is usually stored in a proprietary format.

Proprietary tools and user interfaces

If your requirements are to generate an application with a specific user interface, you may find an OTS solution limiting. Many of them provide a large library of symbols and objects, but your application will look like many others, and will unlikely include your own company branding. You’re also required to use the vendors tools for configuration and modification.

Specialists are expensive and scarce

OTS vendors generally sell to a niche market, so while the install base and investments may be large, the actual number of individuals with the skills to use the software is a small pool relative to the global software and IT community. Few consultants have the necessary skill to work with each OTS product, and if so, come at a high price even when they are available.

You’re paying for features you don’t need

Most of these products have wide variety of features, but since they’re developed for most situations and not just yours, you end up paying for features you will never use. During an evaluation of an OTS product, the number and depth of features is a selling point, but are rarely ever used in production.

Custom Development

The classic alternative to off-the-shelf products is a fully customized application developed from the ground up. Like a well-tailored suit, this can be the best fit for your needs, but comes with its own challenges. And just like a well-tailored suit, custom software development doesn’t always respond well to size and scope changes.

PROS

Fully Customizable

Since the solution is developed for your specific needs, it can contain any feature you require and nothing else

Open data formats

You can choose your data store, and are not locked into proprietary tools for import/export, or license restrictions on your own data.

Affordable and highly available resources

Choosing your own development languages and tools also means that you can create your custom solution using ubiquitous technologies. Software engineers and consultants are abundantly available at reasonable prices.

No licensing restrictions

If the solution is yours, your only licensing restrictions would come from any contracts established with your development team and any underlying technologies utilized in the development of the architecture.

Use any data source and data store

Utilizing APIs and communication protocols within custom development means that almost any device and data source can be integrated. Also, every open data store can be used and even mixed within a custom solution. This makes the underlying data available at any time for real time monitoring, historical analysis, and reporting using the tools of your choice.

Can host any user interface

Since the data is open, this also means any possible user interface can be developed, with your own company branding, which is necessary for OEMs and software resellers.

CONS

Upfront development may be required

With a custom solution, there’s no avoiding it. The software needs to be developed and tested before it can be rolled out into production.

No built-in user interface or apps

Because you’re starting with a blank slate, you can have any interface you would like, but that means none exist. All of the interfaces and tools need to be created.

Requires customization when requirements change

If your company grows, your requirements change, or you realize that your original assumptions were not entirely correct, the solution must be modified. Whenever a custom solution needs modification, that usually means more custom development, unless the modifications were planned upfront.

 

A Third Way: The Open Architecture

The OAS Platform is a robust alternative for building an Industrial Automation or IoT solution. The Platform mitigates many of the risks of pure custom development, while still offering all of the advantages of a custom solution. How is this possible? OAS provides a robust and scalable networked platform that can be connected to data sources and data stores, all through a set of configuration applications. At the same time, the OAS platform exposes a series of APIs and developer tools for the creation of custom applications and the complete automation of the entire system. The following are just a few of the features OAS offers:

Universal Data Connector

The UDC is a server platform that manages connectivity to devices, databases, file systems and more. It also includes native drivers for many standard PLCs and IoT components that can be connected and configured with no programming required. The UDC is able to be networked with other instances, distributing and aggregating data as needed at the fastest speeds possible.

Learn more about the Universal Data Connector

Flexible licensing for end users and OEMs alike

The OAS Platform offers flexible licensing. Whether you are an automation engineer, systems integrator, or software developer looking to use the platform as a component of your own products, OAS licensing can be adjusted to meet your needs. Licenses are based on the number and type of data sources being connected, not the number of clients connected.

Learn more about OAS Platform Pricing 

Open data model

Your data is your data, so when the OAS platform moves data between source and destination, or when the data is stored in a database, it is always available in open formats. No proprietary tools are required, and no additional licenses are needed just to read or even move that data using other tools that you prefer.

Learn more about Device and Data Connectors 
Learn more about the Data Historian 

Automated configuration and deployment

Every aspect of the server configuration can be automated using the various APIs that are exposed for .NET and REST clients. This means an OAS Platform can be programmatically configured and deployed, dramatically reducing costs for rolling out new installations when your organization or install base grows.

Learn more about OAS Developer Tools and APIs 

Infinitely customizable using open development tools and APIs

Access to the OAS APIs and development components is available using open tools and with popular development environments and languages. Desktop applications can be written in any .NET language using Visual Studio. Many visual components are provided for both WinForms and WPF for real time data access and control, as well as for Trend and Alarm visualization. Web application developers can use the Web HMI, Web Trend, and Web Alarm products to deploy web-based visualizations on any web platform and any web browser, desktop or mobile. And using the PCL components, native iOS and Android applications can be integrated into your solution.

The UDC also exposes several APIs for automation and customization. Software developers are free to build their own user interfaces using completely custom code, or they can use the many tools and controls provided for desktop, web, and mobile applications.

Learn more about OAS Developer Tools and APIs 

Highly available and affordable resources

Because the OAS Platform can be configured and automated using open development tools and popular languages, it becomes much more affordable to apply resources to your next project. The pool of .NET and web developers available is one of the largest in the IT market, so ongoing maintenance risks are greatly reduced as well.

If you’d like to learn more about the OAS Platform and how it can transform your Industrial Automation or IoT solution, please contact us for a guided demonstration.

Fix IoT Cloud Leaks

What’s not to like about Cloud based IoT Platforms?

Nothing, if you are comfortable with relying on other services and infrastructure to manage, maintain and secure your data. If lost or missing data is OK, if someone misses a notification of a song download or about a new product release, or maybe even if you are low on milk. But if you rely on up-to-date real-time data for critical decisions, then current cloud-based IoT platforms may not be the best choice.  When seconds count, are you sure of your IoT infrastructure provider?

A few of the major questions or concerns with the current cloud-based IoT platforms are:

  1. Connectivity. How do I get my data from my devices to the Cloud based IoT environment? A lot of industrial devices do not support IoT protocols. Do I have to replace all of these devices that I have invested in just to get my data to Cloud? What if my data source is a spread sheet, a database or an existing application? What if I want to perform calculations on the data prior to transmission?
  2. IoT Protocol. There are more than a few. Which do I choose? Since it will be a large investment, what happens in a few years if there is a new protocol required by my IoT provider? Will the old protocol still work or will I have to reinvest in converting all my devices to the new protocol. If I decide to switch to a different cloud-based IoT provider, will my existing devices / system still work?
  3. Security. Is my data going to be secure from hackers, and who is controlling my data? What 3rd party partners will have access? Is SSL secure? How about TLS? Which one will work with my IoT Provider? How do I manage the certificates required by my cloud-based IoT provider? How secure are the cloud-based servers? The major players in the IoT arena are a prime target for hacking because of the wealth of data available and the visibility of their servers and infrastructure, so a single breach could expose thousands of customers including you.
  4. Reliability. What happens when the Internet connection or the cloud-based servers go down? Will my organization be blind for decision making? What happens when my cloud based IoT provider is under a DDOS attack? Will I be losing data that can’t get transferred to the cloud?
  5. Pricing. What happens if my provider decides to raise its subscription price or worse yet decides to get out of the IoT market? For the big players, IoT is just a side business.
  6. Tools. What tools can I use to access, display, analyze, and store my data? Only the tools provided by my subscription?

What if there existed a platform that provides you all of the advantages of connecting not only to IoT specific devices and platforms, but literally 10s of thousands of existing real time and legacy devices? What if this platform could rapidly, securely and cost-effectively transfer millions of data points per second and insure that you have all of the data maintained even when the communications between the source and destination is down, while providing N levels of redundancy?  And better yet, what if it didn’t rely on 3rd parties services to handle and disseminate your data, allowing the platform to be completely managed by your own IT department or hosted in the cloud? What about a platform that offered interfaces to SQL Server, Oracle, MySQL database and the ability to generate real-time and historical applications based on .Net, HTML, JS, and iPhone and Android OS?

Open Automation Software Solution

OAS has been satisfying all of these requirements for decades and is in use in a number global fortune 100 companies, military and nuclear applications and as small as single site locations. For Reliable data transmission, Saving historical data, Analysis, Big Data, Dashboards, Reporting, Data Analysis,  Trending, Alarm and Events, Alarm Notification, SCADA/HMI, OEE and many more.

OAS is based on DNA (Distributed Network Architecture)

Connectivity:

Built in protocols for major brands of Industrial PLCs and devices.

  • Allen Bradley, Siemens, Modbus and more.
  • OPC UA, DA, AE
  • MQTT, AMQP
  • MS Excel
  • SCADA / HMI applications
  • Databases
  • .Net or Web Applications, REST and others

What Protocol to choose for my devices:

No need to decide on what protocol to use or if you have to convert all your devices. OAS supports all of the major device protocols and if your device is not supported OAS has a universal driver interface.

Security:

  • With OAS only you control your data. OAS also provides built in user security to N levels.
  • OAS uses secure compressed and encrypted protocol, accepted by not only the nuclear industry but also the US military.
  • You can host OAS on your own servers or on any cloud base service provider of your choice.
  • You can also have multiple servers in different locations or by different providers. Providing N levels of redundancy.
  • Need secure one way communications (Data Diode) OAS has that covered too.

Reliability:

OAS does not assume a consistent connection. In fact a lot of our customers choose OAS for their data transmission for this reason specifically. For instance if you are logging data to a remote database using OAS and the network connection is lost all data is buffered at the source. Not just for minutes but for days or months. Once the connection is restored, the buffered data is automatically transferred to the remote database. OAS can also move data to multiple destinations through multiple networks simultaneously.  Redundancy is easy using OAS.

IoT Service providers:

With OAS you own the software. No worries about prices going up or Cloud based IoT service providers leaving you high and dry.

Tools:

With the OAS Platform you have open and easy access to your data using a variety of software and development tools.

  • Easily log and retrieve data with Microsoft SQL, SQL Express, Access, Oracle or MySQL.
  • Easily create interactive applications using VB.Net or C# in Visual Studio with live or historical data. Applications that run securely over the LAN, WAN or internet.
  • Easily create secure Web applications using HTML and/or JS to interact with Live or historical data. Web applications that are not dependent on a specific browser or development platform so you can view and interact with the web application from a verity of different platforms such as iPhone and Android devices.
  • Interact with live and historical data with OAS REST interface.
  • Interface with existing software applications through Web Services, APIs or Databases.
  • Read and write data to existing SCADA/HMI application via APIs, OPC Client or Server.
  • Read and Write data to Excel Spread sheets.
  • No per seat licensing. (Unlimited Client Applications)
  • Programmatically read and write live or historical data using VB.Net C# or JS.
  • Programmatically configure OAS for rapid deployment of your system changes, updates and features
  • Create business or analytical application to interact with live or historical data.
  • Add Live and historical data to your existing applications
  • Trend live or historical data in desktop or web applications.
  • Create IOS or Android specific applications using OAS PCL and Xamarin forms.
  • View, acknowledge, log and preform notifications (Email, SMS or Voice) on any alarm or event
  • Automatically generate reports based on events or time and automatically email the report to any number of recipients.

 

Want to secure your IoT Data? Download OAS for a 30 day trial.

What is Big Data?

Buzzword or not, Big Data is definitely not a passing fad or something that forward thinking businesses can ignore.

At first glance the Big Data concept seems self-explanatory. Big Data, as the name suggests, is surely all about very large quantities of data.

Big Data certainly involves vast quantities of data, usually in the region of petabytes and exabytes. But the size of the data set is just one facet of Big Data.

Characteristics of Big Data

 

In 2001 Douglas Laney began the process of defining the emerging concept of Big Data. He did so by identifying three defining characteristics of Big Data:

  • Volume
  • Velocity
  • Variety
  • Veracity*
What is big data? Big data concept Volume, Variety, Velocity and Veracity

*Laney originally posited 3 characteristics but over time other ‘Vs’ have been added. Here we include the extra characteristic of Veracity

Volume

Volume refers to the quantity of data generated and stored by a Big Data system.

Here lies the essential value of Big Data sets – with so much data available there is huge potential for analysis and pattern finding to an extent unavailable to human analysis or traditional computing techniques.

Given the size of Big data sets, analysis cannot be performed by traditional computing resources. Specialized Big Data processing, storage and analytical tools are needed. To this end, Big Data has underpinned the growth of cloud computing, distributed computing and edge computing platforms,  as well as driving the emerging fields of machine learning and artificial intelligence.

In dealing with these large volumes of data, the big data computing system needs also to be able to reliably detect critical and/or missing data – see Veracity below.

Variety

The Internet of Things is characterized by a huge variety of data types. Data varies in its format and the degree to which it is structured and ready for processing.

With data typically accessed from multiple sources and systems, the ability to deal with variability in data is an essential feature of Big Data solutions. Because Big Data is often unstructured or, at best, semi-structured one of the key challenges is the task of standardizing and streamlining data.

Products like Open Automation Software specialise in smoothing out your big data by rendering data in an open format ready for consumption by other systems.

Velocity

The growth of global networks and the spread of the Internet of Things in particular means that data is being generated and transmitted at an ever increasing pace.

Much of this data needs to be analyzed in real time so it is critical that systems are able to cope with the speed and volume of data generated.

Systems must be robust and scalable and employ technologies specifically designed to protect the integrity of high speed and realtime data. handle the rate such as advanced caching and buffering technologies.

Big Data systems rely on networking features that can handle huge data throughputs while maintaining the integrity of real time and historical data.

Veracity

Data quality and validity are essential to effective Big Data projects. If the data is not accurate or reliable than the expected benefits of the Big Data initiative will be lost. This is especially true when dealing with realtime data. Ensuring the veracity of data requires checks and balances at all points along the Big Data collection and processing stages.

Complexity can be reduced through automated systems. An example of this is Open Automation Software’s One Click automated setup feature that can quickly scan a server and automatically configure huge numbers of data tags without the risk of human error.

Accurate queuing and buffering of data, timestamping and and the use of the most efficient communications protocols go a long way to ensuring the veracity of data.

 

Like to know how OAS meets your Big Data needs? Give it a try today!

What is Google IoT Core?

what is google iot core

Industry 4.0 Data with Google IoT Core

Google IoT Core is an Internet of Things (IoT) service from Google that allows businesses to manage their IoT infrastructure and data. Google IoT Core is comparable to Amazon’s IoT Gateway and Microsoft’s Azure IoT Hub.

The key elements of Google IoT Core include:

  • Device management – an interface for managing and authenticating ‘things’ or devices
  • Protocol bridge – provides the connection endpoints and management services needed to provide secure and reliable telemetry.

OAS Google IoT Core Interface

Always on the leading edge of Industrial IoT technology, OAS now includes support for Google IoT Core. Moving data from your local or remote data sources to (or from) Google IoT Core is straightforward with OAS. It doesn’t matter what the data source – with Open Automation Software you can move it to and from Google’s IoT Core framework.

 

Use the OAS Google IoT Core Interface to quickly, efficiently and securely move data to and from Google IoT Core and other systems. There’s no limit to the type and quantity of data you can move with OAS. Production data, energy monitoring, inventory, GPS data and more can all be automated with OAS. Harness your data to improve overall equipment effectiveness and improve supply chain efficiency.

What is Google iot core

Data quality is a high priority for IIoT Applications

For Industrial Internet of Things applications, data transfer has to be secure, accurate and fast. With OAS, data quality is verified via timestamp using MQTT. Take live local, remote or plant floor data and easily use it in big data analytics.

Write live data to devices within Google IoT Core using OAS’s built in MQTT protocol driver. This enables you to take full advantage of any or all of the other big data and IoT services provided by Google IoT Core. You can complement Google IoT Core with other OAS products; our Data Historian product is an easy and reliable way to move data to Google IoT Core.

MQTT – A Highly Efficient Protocol for Remote Devices

OAS Google IoT Core interface utilizes MQTT, a lightweight protocol that implements a client/broker architecture. Originally developed as a machine to machine (M2M) protocol by IBM, MQTT is now implemented in several open protocols.

Due to its client/server architecture and its small footprint MQTT is particularly well suited for small devices with limited power or network connectivity. A common application is to use this protocol on hundreds or thousands of remote devices that may have battery or solar power and be utilizing cellular or other limited or expensive communications.

Advantages of MQTT

  • Small packet size = low overall bandwidth usage
  • Great choice for limited connectivity – remote device connects to the broker when possible and publishes its updated state
  • Able to both subscribe and publish to brokers making it easy to take remote device data and View, Historize, Alarm, Trend and Notify via Web, .NET, Email, Voice and SMS.

Share non machine to machine (M2M) data and share it with M2M devices and clients utilizing the analytics available on Google IoT platform.

OAS with Google IoT Core

Use Open Automation Software’s MQTT Connector to interface your industrial data with Google IoT Core.

Industrial Internet of Big Things

Open Automation Software

IIoT Software for Big Things

Open Automation Software is designed for scalable design, deployment, and performance.

Automated Setup

Automated Deployment

High Performance Data Transport

Learn how OAS is used to monitor Big Things on a Big Scale for Big Data solutions.

  • Submarines
  • Aircraft Carriers
  • Airports
  • Manufacturing Plants
  • Nuclear Power Plants
  • Volcanoes
  • …anything with electronic data

Visit https://www.openautomationsoftware.com to download a free 30 day trial.

5 Common Failures with Data Logging and Alarm Logging

Knowledge of common failure points helps guide data logging system design

In selecting a data logging and alarm logging solution it is best to consider what happens to the data when everything is not performing perfectly.  In a system’s lifetime at least one of the following if not all five will occur multiple times:

 

1. Database Engine Failure
2. Network Failure
3. Missing High Speed Data
4. Inaccurate Manual Setup
5. Defective Controller Handshaking

Database Engine Failure

When the connection to the database engine fails during database backup, maintenance, or network failure to remote engines data will be lost if the data logging and alarm logging solution does not provide store and forward functionality.For systems that do have store and forward capabilities it is important to select one that can maintain the data for long periods of time.  This requires that the data to be inserted or updated to the database must be stored to disk instead of just buffered to RAM.

Steps to replicate condition:

  1. Stop database engine during data logging and alarm logging.
  2. Verify data buffered to disk.
  3. Start database engine.
  4. Verify all data has been archived before, during, and after database engine shutdown.

Note: An advanced test to account for a system restart during data buffering is to shutdown and restart the logging server between steps 2 and 3.  Most store and forward solutions will lose data if the server PC is shutdown.

Network Failure

When the network connection between the data source server and data logging server is down, data will be lost if the solution does not implement a distributed network design with store and forward at the data source. By relying on cloud solutions for data and alarm archiving data will be lost when the connection to the data source is broken and data cannot be sent to the cloud solution.  Also there are many tunneling solutions that provide network transport, but do not maintain the data during network outages.

Steps to replicate condition:

  1. Disconnect network between data server and cloud system or remote data logging server.
  2. Verify data buffered to disk at the source.
  3. Reconnect network.
  4. Verify all data has been archived before, during, and after network outage.

Missing High Speed Data

It is important for a logging solution to provide data processing right at the source and process all values received from the data source.  Data and alarms can be missed if data transitions quickly and the solution processes only the current value sampled.Consider a sensor that receives a sharp spike in data for a very brief time interval.  If sampling occurs before and after the spike, no alarm would be recorded and the high data value would be missed in the data archiving.  Some systems can sample at a much faster rate, but then have problems moving the large amount of data to the database engine efficiently and then data begins to backup in the system.  It is important to select a system that can handle high bursts of data during critical events.

Also data that is processed directly within the controller can be queued to be handed over to a data logging system.  It is important that the data logging solution have the ability to provide handshaking to the controller to inform when the data is received and processed and ready for the next record.

Steps to replicate condition:

  1. Use a data source of a .NET application or high speed communications that can provide time stamp and data at high rate, microsecond samples if possible.
  2. Have the data change cycle below and above alarm conditions a series of times.
  3. Verify that all data samples are recorded to the database, and all alarm events have been captured and recorded.

Inaccurate Manual Setup

Automated or programmatic setup is important in system setup so human error does not affect the system accuracy of what data is logged and alarmed upon.  With large amounts of data to be collected and processed it is very easy to point to the wrong variable address, alarm limit, or database field.Steps to verify setup accuracy:

  1. Insure there is automated or programmatic setup of the data source, data logging, and alarm logging configuration.
  2. Use a CSV Export to spreadsheet and verify each row for data addresses and field names match.

Defective Controller Handshaking

Some systems will queue data in the controller for best data accuracy and assurance of no data loss on communication failure to the controller.  This is typically done with a queue or array within the controller where the data is buffered and then passed to the data logging engine when it is available to record the data.  It is important to enable a handshaking technique that can validate data has been successfully archived.Steps to verify accuracy:

  1. Start the controller cycles processing records to be logged
  2. Stop database engine.
  3. Have controller execute several cycles of records to be logged.
  4. Start database engine.
  5. Verify that no records are missing from the cycles performed before, during, and after the database engine shutdown.

How Open Automation Software solves Data Logging Issues

Open Automaton Software addresses all of these scenarios with the following features.

OAS Data Logging

  1. Store and Forward
  2. Distributed Network Architecture
  3. High Speed Data
  4. Automated Setup
  5. Programmatic Setup
  6. Alarm Notification
  7. Controller Handshaking

 

 

1. Store and Forward

Store and Forward capabilities that can buffer data and alarms to disk in a small binary file both on database engine and network failure. And data is maintained even after a server reboot.
View the following video of a demonstration of OAS Store and Forward performance.
https://www.openautomationsoftware.com/videos/#data-buffering

2. Distributed Network Architecture

Distributed Network Architecture with data, alarm, and trend processing at the data source.
View the following video demonstration of data being transferred from Australia to the US, and then accessed anywhere in the world to be logged and displayed.

https://www.openautomationsoftware.com/videos/#internet-of-things

3. High Speed Data

High Speed Data processing of queued data with proper indexing of time samples and efficient database bulk insert to pass up to 1,000,000 records per call.
View the following video demonstrating data logging data changing at the rate of 100 nanoseconds.
https://www.openautomationsoftware.com/videos/#high-speed-data-logging
View the following video demonstrating data logging of data from an Allen Bradley ControlLogix controller at the rate of 20 milliseconds.
https://www.openautomationsoftware.com/videos/#logging-data-from-controllogix-at-high-speed

4. Automated Setup

Automated Setup to eliminate human error in setup. OAS has automated setup for both data sources and data logging.

One Click OPC feature is used for automated setup of OPC Items.

One Click Allen Bradley for automated setup of AB controller communications.

One Click Database feature is used to setup data logging of all data points.

5. Programmatic Setup

Programmatic Setup is fully supported for data sources and logging configurations through both .NET and REST API.
View the following guide for .NET programmatic setup.
https://www.openautomationsoftware.com/knowledge-base/net-programmatic-configuration/
View the following guide for REST API setup.
https://www.openautomationsoftware.com/knowledge-base/getting-started-rest-api/

6. Alarm Notification

Alarm Notification on data logging or alarm logging failure. OAS can call personnel via phone or send a text message and email when data buffering is active.  Even though OAS can store many years worth of data to disk to restore the database engine it is best to be notified immediately when there is a problem.
View the following video on how to setup email notifications.
https://www.openautomationsoftware.com/videos/#send-alarm-emails

7. Controller Handshaking

Controller Handshaking to communicate with the controller to provide confirmation that the data is received and ready for the next record. Within OAS logging there are confirmation and error feedback status that can be updated to live tag values for both alarm notification and to be sent directly back to a controller for closed loop data handshaking.
View the following steps on how to setup handshaking with a controller.
https://www.openautomationsoftware.com/knowledge-base/log-buffered-data-from-a-plc-or-controller/

 

Visit https://www.openautomationsoftware.com to download a free 30 day trial.

What is an IoT Gateway?

IoT Gateways – Powering the Industrial Internet of Things

As the Internet of Things expands, businesses find themselves faced with the challenge of integrating a multitude of devices and protocols, many with varying  power and connectivity requirements. This mix may also include legacy technologies.

IoT Gateways are emerging as an essential component in building a robust IoT and for delivering computational power in edge computing scenarios.

What is an IoT Gateway?

An IoT Gateway is a solution for enabling IoT communication, usually device -to-device communications or device-to-cloud communications. The gateway is typically a hardware device housing application software that performs essential tasks. At its most basic level, the gateway facilitates the connections between different data sources and destinations.

A simple way to conceive of an IoT Gateway is to compare it to your home or office network router or gateway. Such a gateway facilitates communication between your devices, maintains security and provides an admin interface where you can perform basic functions. An IoT Gateway does this and much more.

 

 

What functions does an IoT gateway perform?

IoT Gateways have evolved to perform many tasks, from simple data filtering to visualization and complex analytics. These smart devices are helping power the current wave of IoT expansion.

IoT Gateway feature set

A versatile IoT Gateway may perform any of the following :

  • Facilitating communication with legacy or non-internet connected devices.
  • Data caching, buffering and streaming
  • Data pre-processing, cleansing, filtering and optimization
  • Some data aggregation
  • Device to Device communications/M2M
  • Networking features and hosting live data
  • Data visualization and basic data analytics via IoT Gateway applications
  • Short term data historian features
  • Security – manage user access and network security features
  • Device configuration management
  • System diagnostics

IoT Gateways and Edge Computing

In edge computing, critical data processing occurs at the data source rather than in a centralized cloud-based location. A versatile IoT Gateway is the essential link in delivering edge computing power to technicians in the field or at the plant floor. IoT Gateways that come equipped with these capabilities are referred to as ’Smart’ Gateways.