Resources

OpenShift Credentials

  • username: username
  • password: user-password

Dev Spaces

Developer Console

Matrix

Rocket.Chat

BGCreated with Sketch.

Tasks to complete

Get Started
BGCreated with Sketch.

Congratulations!

Return to the dashboard to continue your journey.

Go to home

Kafka to Rocket.Chat

Bridging Matrix and Rocket.Chat bidirectionally requires four flows (as covered in the introduction). The path that carries customer messages to the support agent alone needs two:

  • Flow 1 — Matrix to Kafka (pre-configured)

  • Flow 2 — Kafka to Rocket.Chat ✏️ (you’ll build this one)

All four flows follow the same pattern — once you’ve built one, the others are nearly identical. To keep the lab short, you’ll only build flow 2 by hand; the remaining three will be made available and enabled at the end.
flowchart LR
    MX -->|"flow 1"| K --> F2 --> RC

    MX@{ img: "../../_images/module-03/common/element-icon.svg", label: "Matrix", pos: "b", h: 100, w: 80, constraint: "on" }
    K@{ img: "../../_images/module-03/common/kafka-icon.svg", label: "Kafka", pos: "b", h: 100, w: 80, constraint: "on" }
    RC@{ img: "../../_images/module-03/m4/rocketchat-icon.svg", label: "Rocket.Chat", pos: "b", h: 100, w: 100, constraint: "on" }

    F2["✏️ flow 2"]

    style MX fill:none,stroke:none
    style K fill:none,stroke:none
    style RC fill:none,stroke:none
    style F2 fill:#fff3e0,stroke:#fb8c00,stroke-width:2px,color:#e65100,font-size:14px

    linkStyle 0 stroke:#ccc,stroke-width:1px
    linkStyle 1 stroke:#d32f2f,stroke-width:2px
    linkStyle 2 stroke:#d32f2f,stroke-width:2px

You’ll build the listener side — the flow that picks up events from Kafka and delivers them to Rocket.Chat. Once it’s in place, any message that lands in the Kafka topic will reach the support agent automatically.


Open the Camel route

Find in your directory tree the following resource:

  • k2r.camel.yaml

    Click to open it in Kaoto.

    New to Kaoto?

    Most of the processing flows you’ll edit during this workshop use Kaoto, the visual integration designer for Apache Camel. Kaoto is a VS Code extension that renders Camel definitions graphically so you can design and configure routes without editing YAML by hand.

    vs code kaoto

    Among other tasks, Kaoto helps you to:

    • Create Camel routes (the primary way to define processing flows)

    • Create Kamelets (reusable building blocks for your routes)

    • Create data transformations

    • Apply EIPs (Enterprise Integration Patterns)


    Here’s what the Kaoto canvas looks like — each step in the route is rendered as a visual node that you can click to configure:

    kaoto overview
    The Kaoto extension is already pre-installed in your Dev Spaces environment.


The flow is already prepared — you just need to fill in three areas:

  1. Define Kafka as the source system

  2. Define a filter to prevent loopback events

  3. Define the data mapping to transform Kafka events for Rocket.Chat

The picture below highlights the focus areas:


flow k2r todo new

flow-k2r-todo-new

{x: 10, y: 72, rot: 0, size: 1, color: "red"} {x: 33, y: 26, rot: 220, size: 1, color: "red"} {x: 61.5, y: 68, rot: 0, size: 1, color: "red"}

{x: 10, y: 117, size: 1.2, color: "red"}
{x: 40.5, y: -8, size: 1.2, color: "red"}
{x: 61.5, y: 117, size: 1.2, color: "red"}



What is a loopback event?

All systems publish to and listen from a single Kafka topic. This means a message sent from Rocket.Chat ends up in the same topic this flow is consuming — and could circle right back:

flowchart LR
    RC -->|"message"| K -->|"❌ same event"| RC

    RC@{ img: "../../_images/module-03/m4/rocketchat-icon.svg", label: "Rocket.Chat", pos: "b", h: 100, w: 100, constraint: "on" }
    K@{ img: "../../_images/module-03/common/kafka-icon.svg", label: "Kafka topic", pos: "b", h: 100, w: 80, constraint: "on" }

    style RC fill:none,stroke:none
    style K fill:none,stroke:none

    linkStyle 0 stroke:#66bb6a,stroke-width:2px
    linkStyle 1 stroke:#d32f2f,stroke-width:2px,stroke-dasharray:5

The filter you’ll define prevents this by checking the source field in each event and blocking those that originate from Rocket.Chat.


Define Kafka as the source system

The first step of the flow is where you define Kafka as the source from where events arrive.

Your OpenShift namespace already contains an Streams for Apache Kafka cluster (Kafka) ready for you to use.
  1. Hover the ❗ To do: Kafka step and click the floating     button (replace step).

  2. Type Kafka in the 🔍 Filter to find it, then select the Kafka component.

  3. Click the Kafka step to open the configuration panel.

  4. Select the All configuration tab.

  5. Configure the following 2 properties:

    (Use the 🔍 Filter at the top to quickly find each one)

    kaoto kafka steps

    Topic:

    myroom

    Brokers:

    my-cluster-kafka-bootstrap:9092


    Click anywhere outside the configuration panel to close it.


Define the filter

Events arriving from Kafka can originate from any source system. Without a filter, a message originally sent from Rocket.Chat would flow through Kafka and be pushed back to Rocket.Chat — creating a loop:

  • Rocket.Chat → Kafka → Rocket.Chat ⟵ loopback!

The filter prevents this by checking where the event came from.

kaoto filter config

kaoto-datamapper-configure

{x: 43, y: 41, size: 4} {x: 80, y: 29, size: 4} {x: 80, y: 86, size: 4}

  1. Click the bar  To do: Filter 

  2. Ensure the Required tab is selected. You should see the JQ expression language selected by default.

  3. Copy and paste the following expression:

    .source | test("rocketchat")

    The expression picks the source field from the Kafka event and checks if it originates from Rocket.Chat. Events from Rocket.Chat are filtered out to prevent loopbacks.


Define the data transformation

The data arriving from Kafka uses a common schema. You need to map it to the format Rocket.Chat expects.


  1. Include in your Camel route a Kaoto data transformation step

    From Kaoto:

    1. Click the + button inside the  To do: Map...  step

    2. Type mapper in the 🔍 Filter to find the Kaoto DataMapper

    3. Click the Kaoto DataMapper tile

    4. Click the mapper to open the configuration panel

    5. Click the Configure button

      kaoto datamapper steps
    Overview of the data mapper UI.

    This is what the data mapper looks like — Source on the left (where data comes from) and Target on the right (where data is mapped to):

    kaoto datamapper overview


  2. Next, define the inputs and outputs

    From the DataMapper editor, define the source (left) and target (right):

    Source (left panel) Target (right panel)
    1. On Body click (Attach schema)

    2. Click Upload schema file(s)

    3. Enter common in the 🔍 Filter

    4. Tick support/schemas/common.json, click OK

    5. Click Attach

    1. Click (Attach schema)

    2. Click Upload schema file(s)

    3. Enter rocket in the 🔍 Filter

    4. Tick support/schemas/rocketchat-out.json, click OK

    5. Click Attach


  3. Define the data mapping rules

    Copy and Paste (in the Target) the values below to define the given field entries:

    1. string[@key = channel]:

      /roomid

      room11

      "REPLACE"

      To paste the value:

      Click ⠇ and select Add value selector (value-of), as shown below:

      mapper add expression

      mapper-add-expression

      {x: 96.5, y: 28, rot: -90, size: 1} {x: 94.0, y: 41, rot: -90, size: 1}

      {x: 120, y: 28, size: 2} {x: 120, y: 41, size: 2}

    2. string[@key = text]:

      concat("*",/fn:map/fn:string[@key='user'], "@", /fn:map/fn:string[@key='source'],":* ", /fn:map/fn:string[@key='text'])

      The mapping rendering should be similar to:

      mapping kafka rocketchat


To return to the canvas, click the Design tab:

kaoto design tab


Your completed flow should look like this:

flow k2r complete


Test the flow locally

Run the process in the background using Camel JBang:

labrun k2r --background

What does this command do?

labrun is a lab helper that runs Camel JBang behind the scenes. The command above is equivalent to:

  • camel run --name k2r — runs the integration named k2r

  • --source-dir k2r — loads route files from the k2r directory

  • --local-kamelet-dir ../support/kamelets — loads custom Kamelets (connectors), including the Rocket.Chat one

  • --background — runs the process in the background so you can continue working in the terminal


Verify the process is running:

camel get

You should see the process in Running status:

  PID   NAME  CAMEL   PLATFORM  PROFILE  READY  STATUS   RELOAD  AGE  ROUTE  MSG/S  TOTAL  FAIL  INFLIGHT  LAST  SINCE-LAST
 16763  k2r   4.18.0  Camel     dev       1/1   Running     0/0  16s    1/1   0.00      0     0         0             -/-/-


Before sending a test message, make sure Rocket.Chat is open and you’re in your designated room. If you haven’t logged in yet, follow the instructions in Rocket.Chat platform onboarding.

Once ready, send a test message to Kafka:

camel cmd send k2r --body \ '{ "user": "test", "source": "test", "text": "test message" }'

What does this command do?
  • camel cmd send — sends a message to a running Camel process

  • k2r — the name of the target process

  • --body '{…​}' — a JSON payload simulating a Kafka event. Only user, source, and text are needed — the target channel is hardcoded in the mapping


Open your Rocket.Chat room and check for the incoming message. You should see something like:


rocket.chat
U
username 3:42 PM
test@test: test message


Once you’ve confirmed the message, stop the background process:

camel stop

What does this command do?
  • camel stop — stops all running Camel processes started with Camel JBang

  • You can verify nothing is running with camel get


Did you see the test message in your Rocket.Chat room?

The flow works — your Kafka to Rocket.Chat route is receiving events and delivering them. Now that the flow has been validated locally, it’s time to deploy it on OpenShift.

Review the Camel route definition and check your terminal for errors.