Getting Started Kafka Consumer
OAS is both a consumer and producer of Kafka messages. If you want to send data to Kafka please see the Getting Started Kafka Producer guide.
You can view the Getting Started with Kafka Video to familiarize yourself with the following steps to setup OAS as a producer and consumer of Kafka.
- 00:00 – Introduction
- 01:20 – Kafka Producer
- 09:07 – Kafka Consumer
- 10:20 – Save Configuration
- 11:00 – Transaction Logging
- 11:45 – Publish 100,000 Values per Second
- 15:50 – Contact Us
Configure a Kafka Driver
The following steps can be used to setup direct communications to receive messages from local or remote a Kafka Broker.
Step 1
Start Configure OAS application from the program group Open Automation Software.
Step 2
Select Configure-License and verify that Kafka is one of the available Drivers in the lower left of the form. If you do not see the Kafka driver available contact support@oasiot.com to update your license.
Note: You will need to be running Open Automation Software Version 18.0.0.4 or greater to support Kafka communications. You can download the latest version at www.openautomationsoftware.com/downloads/open-automation-software/
Step 3
Select Configure-Drivers.
Step 4
Select localhost or the remote service you wish to modify with the Select button to the right of the Network Node list.
Note: Optionally select the Live Data Cloud node if you are hosting data over the Internet with a standard Internet connection.
Step 5
Enter a meaningful Driver Interface Name that you will refer to this connection and define the properties.
Set the driver to Kafka. Set the Bootstrap Servers of the brokers you wish to publish to.
If sending data to the cloud set the appropriate Security Protocol.
When connecting with Confluent Cloud set the SASL Username with the provided API Key and the SASL Password with the Secret Key.
Step 6
Select the Add Driver button in the lower part of the form to add the Driver Interface.
Note: If you need to define several Driver Interfaces you can use the CSV Export and CSV Import on the toolbar in the upper right together with Microsoft Excel.
Reading Data from Kafka Broker
Step 1
Select Configure-Tags.
Select localhost or the remote service you wish to modify with the Select button to the right of the Network Node list.
Note: Optionally select the Live Data Cloud node if you are hosting data over the Internet with a standard Internet connection.
Step 2
Select to Add a Tag.
Note: You can also add organizational Groups as many levels deep as you prefer and add tags to groups. To do this first add a Group to Tags Group at the root level, then right click on the Group in the right window to add additional Groups or Tags.
Step 3
Change the Data Type property to String or JSON.
Change the Data Source Tag property to Kafka.
Step 4
Select the correct Driver Interface from the Driver Interface pull down list.
Step 5
Enter the topic.
Step 6
Select Apply Changes to begin monitoring the topic from the Kafka Broker.
Step 7
To define multiple tags use the CSV Export and CSV Import on the toolbar in the upper right together with Microsoft Excel.
Note: You can also programmatically define Tags using the free to use OASConfig .NET Standard assembly, OPCSystems .NET Framework 4 assembly, or REST API.
Step 8
Select the Save button on the toolbar at the top.
Step 9
Create a directory on the local C: drive with the name OASDemo.
Save the file DemoTags.tags in the directory C:\OASDemo or the correct path for Linux.
Step 10
Under Configure – Options set the Default Tag Configuration File so when the computer restarts the tag file will automatically be loaded.