Building Intelligent Apps with RAG on Kubernetes: From Raw Data to Real-Time Insights

Workshop Overview

Welcome! This lab will illustrate how to combine event-driven serverless applications, data science pipelines, and a vector database to solve a real-world enterprise challenge using Large Language Models (LLMs).

The scenario for this lab is that we all work for "Parasol Insurance", a large company looking to leverage AI to improve its IT support operations. Our goal is to build a prototype Retrieval-Augmented Generation (RAG) system that taps into the vast knowledge base of closed ServiceNow tickets, making it instantly searchable for support engineers through a conversational AI assistant.

AI Lab Assistant

AI Lab Assistant

Throughout this workshop, you have access to an AI-powered lab assistant that can help you navigate the material, answer questions, and troubleshoot issues. The assistant is integrated directly into the documentation and has knowledge of the workshop content, OpenShift operations, and the technologies covered in this lab.

The AI assistant can help you to:

  • Ask questions about the technology - Learn more about RAG, vector databases, data science pipelines, or any other technology covered in the workshop

  • Get troubleshooting advice - When things go wrong, describe what you’re experiencing. Be specific: "In exercise 3, I deployed the ServiceNow app but I’m getting a 404 error when trying to access it" or "What is wrong with my milvus deployment in my project?"

  • Clarify workshop instructions - If any step in the workshop is unclear, ask for clarification or additional context

  • Understand OpenShift resources - Get help with pods, deployments, services, routes, and other Kubernetes/OpenShift concepts

To get the best results, be explicit and specific in your questions. Include details about which exercise you’re working on, what you’ve tried, and what error messages or unexpected behavior you’re seeing.

Disclaimer

This lab is an example of what a developer could build using Red Hat OpenShift AI. Red Hat OpenShift AI itself does not include a ServiceNow integration or a vector database out-of-the-box.

This lab makes use of a Large Language Model (LLM), the docling document parsing library, and the Milvus vector database. These models and third-party tools are not included in the Red Hat OpenShift AI product. They are provided as a convenience for this lab and as a learning experience.

The quality of the models and the architecture are suitable for a prototype. Choosing the right components for a production environment is a complex task that requires significant experimentation and tuning. This lab does not cover production-level architecture design.

Timetable

This is a tentative timetable for the materials that will be presented.

Name Duration Type Description

The Parasol Company Scenario

15

Presentation

Building the API-to-RAG Pipeline

45

Hands-On

  • Lab Setup: Attendees log in and explore their pre-created OpenShift AI project.

  • Deploy & Verify Mock API: Attendees deploy a mock ServiceNow API using ArgoCD and test its endpoint.

  • Run KFP Pipeline: Attendees import and run a Kubeflow Pipeline that fetches data from the API, generates embeddings, and populates the Milvus vector database.

  • Query the RAG System: Attendees use a Jupyter Notebook to ask questions against the system and see the contextually relevant answers retrieved from Milvus.

Event-Driven PDF Pipeline

20

Hands-On

  • Explore the Pipeline: You will explore a real-time data ingestion pipeline.

  • Trigger an Event: An upload of a PDF to an S3 bucket will automatically trigger a Kubeflow Pipeline via Knative Eventing and Kafka.

  • See It in Action: This showcases the event-driven capabilities of the platform.

Resources

Contributing

If you are interested in contributing to this project, consult this GitHub Repo: https://github.com/rhpds/showroom-intelligent-apps-rag