Kafka vendor logo

Vendor

Kafka

Product

Kafka v2

Method

Library

Category

Notifications

Project Type

Adapter

View Repository
Adapter

Adapter for Integration to Kafka using kafkajs

Overview

This adapter is used to integrate the Itential Automation Platform (IAP) with the Kafka System. The adapter utilizes the KafkaJS client to integrate with Apache Kafka. The ReadMe file is intended to provide information on this adapter it is generated from various other Markdown files.

Details

The Kafka v2 adapter from Itential is used to integrate the Itential Automation Platform (IAP) with Kafka. With this adapter you have the ability to perform operations with Kafka on items such as:

  • Produce/Publish messages onto message topics for others to consume
  • Listen, receive and process messages placed on topics by other systems

For further technical details on how to install and use this adapter, please click the Technical Documentation tab.

Kafka v2

Table of Contents

Specific Adapter Information

Authentication

This document will go through the steps for authenticating the Kafka v2 adapter with. Properly configuring the properties for an adapter in IAP is critical for getting the adapter online. You can read more about adapter authentication HERE.

Library Authentication

The Kafka v2 adapter supports SASL Authentication for Kafka v2 server. If you change authentication methods, you should change this section accordingly and merge it back into the adapter repository.

STEPS

  1. Ensure you have access to a Kafka v2 server and that it is running
  2. Follow the steps in the README.md to import the adapter into IAP if you have not already done so
  3. Use the properties below for the properties.client.sasl field
  4. Kafka v2 adapter supports both PLAIN and SCRAM-SHA authentication mechanism
PLAIN
"sasl": {
  "mechanism": "PLAIN",
  "username": "username",
  "password": "password"
}
SCRAM-SHA
"sasl": {
  "mechanism": "scram-sha-512",
  "username": "username",
  "password": "password"
}

For more details on sasl authentication follow the README.md 4. Restart the adapter. If your properties were set correctly, the adapter should go online.

Troubleshooting

  • Make sure you copied over the correct username and password.
  • Turn on debug level logs for the adapter in IAP Admin Essentials.
  • Investigate the logs
  • Credentials should be masked by the adapter so make sure you verify the username and password - including that there are erroneous spaces at the front or end.

    Sample Properties

Sample Properties can be used to help you configure the adapter in the Itential Automation Platform. You will need to update connectivity information such as the host, port, protocol and credentials.

  "properties": {
    "registry_url": "",
    "interval_time": 5000,
    "stub": false,
    "parseMessage": true,
    "wrapMessage": "myKey",
    "check_iap_apps": true,
    "check_wfe_status": true,
    "check_ops_manager_status": false,
    "fromBeginning": false,
    "iap_apps_check_interval": 15000,
    "client": {
      "brokers": [
        "broker:9093"
      ],
      "clientId": "my-app",
      "logLevel": "INFO",
      "ssl": {
        "enableTrace": true,
        "ca": "/path/to/crt.pem",
        "rejectUnauthorized": true
      },
      "sasl": {
        "username": "my-user",
        "password": "my-password",
        "mechanism": "scram-sha-512"
      }
    },
    "producer": {
      "requireAcks": 1,
      "ackTimeoutMs": 100,
      "partitionerType": 0
    },
    "consumer": {
      "groupId": "kafka-node-group"
    },
    "topics": [
      {
        "name": "t1",
        "always": true,
        "subscriberInfo": [
          {
            "subname": "default",
            "filters": [
              "PIXI",
              "DRED",
              "[abc]",
              "F: (\\w+), L: (\\w+)",
              "\\d{3,4}"
            ],
            "rabbit": "kafka",
            "throttle": {}
          }
        ]
      }
    ]
  }

Swagger

Note: The content for this section may be missing as its corresponding .json file is unavailable. This sections will be updated once adapter-openapi.json file is added.

Generic Adapter Information