AI Applications and Llama Stack: A practical workshop

Overview

Welcome to this hands-on lab where we’ll explore the exciting and evolving world of AI applications including inference, Retrieval Augmented Generation (RAG), and Agentic AI. We’ll specifically focus on how an open-source AI stack called Llama Stack simplifies the development process and provides a robust architecture for building powerful AI applications.

Think of this lab as your first, or early, step into understanding how AI can be practically applied. We’re not diving deep into complex AI theory or intricate Python programming. Instead, we’re focusing on how Llama Stack provides a structured framework and essential components to build AI-powered tools. These tools can, for example, help analyze issues, retrieve information from technical documentation, or even suggest troubleshooting steps, complementing your existing Linux and OpenShift operational skills.

But today our journey will be a fun trip built around a National Parks application taking basic but functional application elements built using Llama Stack and exploring their patterns and approach. This will allow us to highlight many but not all of the main features of Llama Stack, see how they contribute to the overall AI application architecture, and understand the value they bring without getting bogged down in starting from scratch.

What You Will Learn

By the end of this lab, you will have a foundational understanding of:

  • The Core Idea of an AI Stack like Llama Stack: You’ll see how Llama Stack brings together a collection of tools and libraries to streamline the creation of AI-driven applications, providing a clear architectural pattern.

  • Practical Application Development with Llama Stack: You’ll gain hands-on experience by working with a pre-existing code structure within the Llama Stack framework, focusing on understanding and modifying key parts rather than writing extensive new code.

  • Key Architectural Components in Modern AI Applications (and Llama Stack’s role):

    • LLM Providers: We’ll explore what "Large Language Model (LLM) Providers" are. Think of them as services that grant you access to powerful AI models capable of understanding and generating human-like text. A key architectural advantage of Llama Stack is its ability to seamlessly integrate with and manage multiple providers. This means your Llama Stack-built application isn’t locked into a single option, offering flexibility in terms of capabilities, cost, and specific model strengths.

    • RAG (Retrieval Augmented Generation): You’ll learn about "Retrieval Augmented Generation." This is an architectural pattern that makes LLMs even more powerful for specific tasks, and Llama Stack provides the components to build this into your applications. Imagine an AI that doesn’t just rely on its general knowledge, but can also look up information in a specific set of documents – like your company’s internal knowledge base, system logs, or technical manuals – before answering a question. Llama Stack facilitates this RAG architecture, allowing the LLM to access relevant and up-to-date information, which is crucial for providing accurate answers in IT operations scenarios.

    • Agents (Agentic AI): We’ll introduce "AI Agents" and how Llama Stack serves as the platform for building and orchestrating them. An Agent is more than just a passive model; it’s a system designed to understand a goal, make a plan, and use available tools (which can be other parts of your Llama Stack application or external APIs) to achieve that goal.

No deep AI or Python programming experience is required for this lab. If you have some general development or scripting experience (like with Bash or Ansible), you’ll find it helpful, but we’ll guide you through the necessary steps and explain the concepts clearly. Our goal is to make these cutting-edge AI techniques, as implemented within Llama Stack, understandable and show their potential value in your operational world.