How To Articles > How to Transfer Data from Modbus to Kafka

How to Transfer Data from Modbus to Kafka

Modbus Kafka


Open Automation Software can be used to transfer data from Modbus TCP, Modbus RTU, and Modbus ASCII devices to local Kafka brokers or cloud hosted Kafka brokers.  Use the following guide to setup data transfer from Modbus to Kafka. This tutorial walks you through downloading and installing OAS, defining Modbus tags, and publishing data to Kafka in a JSON format. The message topic to publish is automatically formatted to include tag id, value, quality, and timestamp. All Modbus and Kafka parameters are adjustable to deliver the message in the desired format.

Step 1. Download and Install the Open Automation Software and Start the OAS Service

If you have not already done so, you will need to download and install the OAS platform.  Fully functional trial versions of the software are available for Windows, Windows IoT Core, Linux, Raspberry Pi and Docker on our downloads page.

On Windows run the downloaded Setup.exe file to install one or more of the Open Automation Software features. Select the default Typical installation if you are not sure what features to use or the Custom installation if you want to save disk space on the target system.  When prompted agree to the End User License Agreement to continue the installation.

For more detailed instructions and video tutorials, visit the installation guide for your system:
Windows Installation | Linux Installation | Raspberry Pi Installation | Dockers Installation

When the installation is finished the OAS Service Control application will appear.  Use this application to start the 4 Services. If this is the first time installing the software it will automatically enter Runtime with an example Tag Configuration.

Step 2. Configure Your Modbus Data Source

  1. First, you will need to open the Configure OAS application from the program group Open Automation Software.

  2. Select Configure >> License from the top menu and verify that Modbus is one of the available Drivers in the lower left of the form. The demo license will have this by default. If you do not see Modbus available, contact support@openautomationsoftware.com to update your license.

  3. Select Configure >> Drivers from the top menu.


  4. Select localhost or the remote service you wish to modify with the Select button to the right of the Network Node list.


  5. The Configure Drivers Screen will appear. Select Modbus from the Driver dropdown box.


  6. Enter a meaningful Driver Interface Name that you will refer to this physical connection when defining Tags with a Modbus Data Source.

  7. Specify the Connection as Ethernet or Serial.

  8. Specify the Modbus Type as Master or Slave. Master will be used when communicating to a Modbus device. Slave will be used when other Modbus masters will be communicating to OAS.

  9. When setting up a Slave interface over Ethernet set the IP Address to the computer IPv4 IP address or network node name if the master is on a remote PC. You can also use 127.0.0.1 or localhost if the Modbus master will be on the same computer.

For more detailed instructions on configuring your Modbus data source, click here to see our Getting Started Modbus tutorial or watch the video tutorial below:

Step 3. Configure Your Modbus Tags

OAS provides multiple ways to add and define tags:

To add a Tag manually:

  1. In the OAS Configure Application, select Configure >> Tags from the top menu.


  2. Select localhost or the remote service you wish to modify with the Select button to the right of the Network Node list.


  3. Click on the Add Tag button located at the top of the Tag browser on the left portion of the screen.


  4. A dialog box will appear. Enter a name for your new tag and click ok.

  5. A configuration screen will appear for your new tag. Select your data source type in in the Data Source dropdown box.


  6. Specify the correct data type in the Data Type dropdown box.

  7. Click Apply Changes at the bottom right of the window.

For more detailed instructions on configuring your tags, click here to see our Getting Started Tags tutorial.

Step 4. Define Kafka Topic and Tags to Publish

Select Configure-Drivers.

Configure Drivers

Enter a meaningful Driver Interface Name that you will refer to this connection and define the properties.

Set the driver to Kafka. Set the Bootstrap Servers of the brokers you wish to publish to.

Kafka localhost

If sending data to the cloud set the appropriate Security Protocol.

When connecting with Confluent Cloud set the SASL Username with the provided API Key and the SASL Password with the Secret Key.

Kakfa Sasl Ssl

Enable Publish Selected Tags at the bottom of the Driver configuration.

Select to publish data continuously at a specified interval, based on event, or at a specific time of day.

IoT Publish Type

If Event Driven browse for a local or remote OAS tag that will trigger the publish.  Select a Boolean tag that will change state from false to true, true to false, or both.  Or choose an Integer tag that trigger a publish anytime the value changes other than 0.

IoT Publish by Event

Specify the Publish Topic Id if choosing to Publish All Tags As One Topic.

When publishing as one topic to optimize communications with Kafka multiple messages are sent for each publish when the number of tag value changes exceeds the Max Tags Per Publish limit.

Max Tags per Publish

Specify the Tag Id, Value Id, an optional Quality Id, and Timestamp Id for each tag value that is sent.

IoT Publish Ids

When including the Timestamp Id also specify the timestamp format, use Custom to specify your own date and time format.

Specify local and remote OAS tag variables to include in each publish and specify the Id.  Value is the current value from the data source or you can select any of the over 600 tag variables of each tag to publish.

IoT Publish Tags

Optionally use CSV Export and CSV Import buttons to set up additional tags to publish using Microsoft Excel.

When selecting remote tags use Basic Networking syntax or Live Data Cloud syntax in the tag path.

Select the Add Driver button in the lower part of the form to add the Driver Interface.

Add Driver

Data is now publishing to the Kafka broker.

Select the Save button from the toolbar to save your configuration.

Save Button

You can view the Getting Started with Kafka Video to familiarize yourself with the following steps to setup OAS as a producer and consumer of Kafka.

  • 00:00 – Introduction
  • 01:20 – Kafka Producer
  • 09:07 – Kafka Consumer
  • 10:20 – Save Configuration
  • 11:00 – Transaction Logging
  • 11:45 – Publish 100,000 Values per Second
  • 15:50 – Contact Us