MCP server management
|
Prerequisites: Complete Module 1: Lab Setup before starting this module. |
In the previous module, you experienced MCP in action — an autonomous agent diagnosing pipeline failures and creating issues, plus interactive infrastructure queries through LibreChat. The MCP servers in that demo were pre-configured for ease of use, with all tools enabled and minimal security constraints.
Production deployments require a different approach. When AI agents can interact with your infrastructure, you need the same rigor you apply to any other privileged access: principle of least privilege, comprehensive audit trails, and observability into what’s happening. This module covers the operational aspects of running MCP servers in enterprise environments.
You will learn how to:
-
Restrict tool access using allowlists to implement least-privilege access control
-
Enable observability by integrating MCP servers with Prometheus and your monitoring stack
-
Deploy additional MCP servers to extend AI capabilities
-
Understand the architecture including ToolHive, transports, authentication, and the emerging MCP registry ecosystem
Tool filtering: Implementing least privilege
Why tool filtering matters
The OpenShift MCP Server you used in Module 2 exposes 23 tools — including pods_delete, resources_delete, and pods_exec. While these are useful for troubleshooting, they represent significant risk if an AI agent (or a malicious prompt injection attack) decides to use them inappropriately.
Consider the business implications:
- Operational Risk
-
An AI agent with unrestricted access could accidentally delete production pods, scale deployments to zero, or execute arbitrary commands in containers. Even well-intentioned prompts can have unintended consequences.
- Compliance Requirements
-
Regulations like SOC 2, HIPAA, and PCI-DSS require access controls on automated systems. Auditors will ask: "What prevents your AI from modifying production resources?" You need a concrete answer.
- Attack Surface
-
Prompt injection attacks are a known vulnerability in AI systems. An attacker who can influence the AI’s prompts might be able to trick it into calling destructive tools. Limiting available tools reduces the blast radius.
- Principle of Least Privilege
-
Just like human users, AI agents should have only the permissions they need. The pipeline failure agent only needs to read logs and list resources — it doesn’t need delete capabilities.
Understanding the GitOps workflow
Before we make changes to the MCP server configuration, it’s important to understand how this lab environment is managed. This lab uses GitOps — a modern approach to infrastructure and application management where Git serves as the single source of truth for your desired system state.
What is GitOps?
GitOps is an operational framework that applies DevOps best practices — version control, collaboration, compliance, and CI/CD — to infrastructure automation. The core principle is simple: instead of manually running commands to configure systems, you declare your desired state in Git, and automation ensures the actual system matches that declaration.
The benefits are significant:
-
Auditability: Every change is a Git commit with a timestamp, author, and message. You always know who changed what and when.
-
Reproducibility: The entire system state is captured in version-controlled files. You can recreate environments exactly.
-
Rollback: Made a mistake?
git revertrestores the previous state, and the automation applies it. -
Collaboration: Use pull requests, code review, and approval workflows for infrastructure changes, just like application code.
OpenShift GitOps (ArgoCD)
This lab environment is managed by OpenShift GitOps, Red Hat’s supported distribution of ArgoCD. ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes. Here’s how it works:
-
ArgoCD monitors your Git repository — In this case, your personal
{user}/mcprepository in Gitea. -
ArgoCD compares desired state to actual state — It continuously compares the Kubernetes manifests in Git against what’s actually running in the cluster.
-
ArgoCD reconciles differences — When it detects a drift (the cluster doesn’t match Git), it automatically applies the changes to bring the cluster back in sync.
-
Sync happens automatically — ArgoCD polls your repository approximately every 30 seconds. When you commit a change, you’ll typically see it reflected in the cluster within a minute.
Your Gitea repository structure
Your {user}/mcp repository contains everything needed to run the lab environment. Let’s understand its structure:
{user}/mcp/
├── helm/
│ ├── mcp-openshift/ # OpenShift MCP Server
│ │ ├── Chart.yaml
│ │ ├── values.yaml
│ │ └── templates/
│ │ └── mcpserver.yaml # MCPServer custom resource
│ │
│ ├── mcp-gitea/ # Gitea MCP Server
│ │ ├── Chart.yaml
│ │ ├── values.yaml
│ │ └── templates/
│ │ └── mcpserver.yaml
│ │
│ └── agent/ # Pipeline failure agent
│ ├── Chart.yaml
│ ├── values.yaml
│ └── templates/
│ └── deployment.yaml
│
└── agent/ # Agent source code
├── main.py
├── requirements.txt
└── Containerfile
Each component is packaged as a Helm chart. Helm is a package manager for Kubernetes that uses templates to generate Kubernetes manifests. The values.yaml file contains configuration, and the templates/ directory contains the actual Kubernetes resource definitions.
The workflow you’ll use
Throughout this module, you’ll make changes using this workflow:
-
Navigate to the file in Gitea — Use the Gitea web interface to browse to the file or directory you want to modify.
-
Edit or create the file — Click the edit button (pencil icon) to modify the file directly in the browser. Click the Add file button when you have a directory selected to add a new file to that directory.
-
Commit the change — Save your changes with a meaningful commit message. This creates a new Git commit.
-
Wait for sync — ArgoCD detects the new commit within ~30 seconds and begins applying changes.
-
Verify in OpenShift — Check the OpenShift console to confirm your changes took effect.
This workflow means you never need to run kubectl apply or oc apply commands directly. Everything flows through Git, giving you a complete audit trail of all changes to your MCP infrastructure.
Why GitOps for MCP?
GitOps is particularly valuable for MCP server management:
- Configuration as Code
-
Your MCP server configurations — which tools are allowed, what credentials are used, what telemetry is enabled — are all captured in version-controlled files. This documentation-as-code approach means your infrastructure is self-documenting.
- Safe Experimentation
-
Want to try a new tool filter configuration? Make the change, test it, and if it doesn’t work, revert the commit. The previous configuration is automatically restored.
- Multi-Environment Consistency
-
The same Git repository structure can deploy to development, staging, and production clusters. Environment-specific values go in different
values.yamlfiles; the templates remain the same. - Compliance and Governance
-
For regulated environments, having a complete Git history of all MCP configuration changes helps demonstrate compliance. Auditors can see exactly what was running at any point in time.
Now that you understand how changes flow from Git to the cluster, let’s create a tool filter configuration.
Hands-on: Creating a tool filter
ToolHive provides the MCPToolConfig custom resource for declarative tool filtering. You define which tools are permitted, and the ToolHive proxy enforces these restrictions at the protocol level — the AI never even sees the filtered tools in its tools/list response.
-
In Gitea navigate to
{user}/mcp/helm/mcp-openshift/templatesand create a new filemcptoolconfig.yaml:--- apiVersion: toolhive.stacklok.dev/v1alpha1 kind: MCPToolConfig metadata: name: openshift-tool-filter namespace: {{ .Values.namespace }} spec: toolsFilter: - events_list - namespaces_list - pods_get - pods_list - pods_list_in_namespace - pods_log - resources_get - resources_list -
Commit and save the file.
-
Next edit the file
mcpserver.yamlto pick up the new tool filter. -
Add this to the
specwhich will allow only the listed tools. This should still be enough for our use agent.Add the
toolConfigRefblock in thespec:section of your MCPServer, at the same indentation level asimageandtransport.toolConfigRef: name: openshift-tool-filter -
Commit the file
-
Check that the new MCP Server has been rolled out. OpenShift GitOps is configured to check for repository updates every 30 seconds - it shouldn’t take much longer for the changes to appear on the cluster. You can check in the OpenShift Console in project
mcp-openshift-{user}that new versions of the pods have been created. -
Test that the change worked:
-
In Librechat try to list the pods in the
agent-{user}namespace again:List the running pods in namespace agent-{user}.You see that the two running pods are returned.
-
If you did the optional parts in the previous lab you saw that were able to delete any pod in the
agent-{user}namespace.Try to delete one of the pods in that namespace again (replace the pod name with the name of the agent pod that the previous question returned):
Delete pod agent-xxxxxxx in namespace agent-{user}You should see that the request got cancelled. If you click the twistie you can see this result:
Assistant sent this info to openshift { "name": "agent-6d8777b969-v2z9p", "namespace": "agent-{user}" } Result Error processing tool: [MCP][openshift][pods_delete] tool call failed: Error POSTing to endpoint (HTTP 400):
-
-
You effectively protected your OpenShift cluster from pod deletions and other potentially dangerous operations.
Telemetry: Observability for AI operations
Why MCP observability matters
When AI agents interact with your systems through MCP, you need visibility into what’s happening. Without observability, MCP servers are black boxes — you can’t answer basic operational questions:
-
How many AI requests are hitting your MCP servers?
-
Which tools are being called most frequently?
-
Are there error spikes that indicate problems?
-
Is usage growing in ways that require capacity planning?
- Incident Response
-
When something goes wrong, you need audit trails. If an AI agent makes an unexpected change, you need to trace back: which tool was called, with what parameters, at what time? Prometheus metrics provide the foundation for this investigation.
- Capacity Planning
-
MCP servers consume resources — CPU, memory, network connections. Understanding usage patterns helps you right-size deployments and plan for growth. If your AI adoption is successful, MCP traffic will increase.
- SLO/SLA Management
-
If you’re offering AI-assisted tools to your organization, you need to track availability and performance. Prometheus metrics enable you to define SLOs (Service Level Objectives) and alert when they’re at risk.
- Security Monitoring
-
Unusual patterns in MCP usage might indicate compromise or abuse. A sudden spike in
pods_deletecalls, even if they’re failing due to tool filtering, could indicate an attempted attack.
Available metrics
ToolHive exposes several Prometheus metrics for MCP servers:
-
toolhive_mcp_active_connections— Current number of active MCP client connections -
toolhive_mcp_requests_total— Total number of MCP requests processed (with labels for tool name, status) -
toolhive_mcp_request_duration_seconds— Histogram of request processing times
These metrics integrate with OpenShift’s built-in monitoring stack, allowing you to create dashboards, alerts, and use the same observability tools you use for other workloads.
Hands-on: Enabling Prometheus metrics
-
Edit the file
mcpserver.yamlto include this underspec::Add this
telemetry:block underspec:at the same indentation level as other spec fields.telemetry: prometheus: enabled: true -
Commit the file.
-
When the change has rolled out you can test by using the route:
curl -s https://mcp-openshift-mcp-openshift-{user}.{openshift_cluster_ingress_domain}/metricsThe
-sflags:-ssuppresses progress output.You should see Prometheus-formatted metrics output. This confirms the MCP server is exposing metrics, but Prometheus isn’t collecting them yet.
Understanding ServiceMonitor
At this point, your MCP server (proxy) is exposing metrics on its /metrics endpoint, but OpenShift’s Prometheus instance doesn’t yet know about it. This is where the ServiceMonitor comes in.
How Prometheus discovers targets
Prometheus uses a pull-based model for metrics collection. Instead of applications pushing metrics to Prometheus, Prometheus reaches out to applications and "scrapes" their metrics endpoints at regular intervals. But how does Prometheus know which endpoints to scrape?
In traditional Prometheus deployments, you would manually configure scrape targets in a configuration file. This doesn’t scale well in dynamic Kubernetes environments where pods come and go constantly.
The ServiceMonitor custom resource
The Prometheus Operator (which powers OpenShift’s monitoring stack) solves this with the ServiceMonitor custom resource. A ServiceMonitor tells Prometheus:
-
Which Services to scrape — Using label selectors to match Kubernetes Services
-
Which port to use — The named port on the Service that exposes metrics
-
The metrics path — Usually
/metrics, but configurable -
Scrape interval — How often to collect metrics (e.g., every 30 seconds)
-
Additional configuration — Authentication, TLS settings, relabeling rules
When you create a ServiceMonitor, the Prometheus Operator automatically updates Prometheus’s configuration to include the new scrape target. When you delete the ServiceMonitor, Prometheus stops scraping that target. This dynamic discovery is essential for Kubernetes-native observability.
How it works in this lab
Here’s the flow for MCP server metrics:
-
ToolHive creates a Service — When you deploy an
MCPServerwith telemetry enabled, ToolHive creates a Kubernetes service that exposes the metrics endpoint. -
You create a ServiceMonitor — The ServiceMonitor uses label selectors to find the ToolHive-created service.
-
Prometheus Operator detects the ServiceMonitor — The operator watches for ServiceMonitor resources in namespaces it’s configured to monitor.
-
Prometheus configuration is updated — The operator generates a new Prometheus configuration that includes your MCP server as a scrape target.
-
Prometheus begins scraping — Every 30 seconds (or whatever interval you specify), Prometheus fetches metrics from your MCP server.
-
Metrics become queryable — You can now query MCP metrics in the OpenShift Console’s Observe → Metrics interface, create dashboards, and set up alerts.
The ServiceMonitor manifest explained
-
Let’s create the ServiceMonitor. Create a new file
servicemonitor.yamlin the OpenShift MCP server’stemplatesdirectory:--- apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: mcp-openshift namespace: {{ .Values.namespace }} labels: team: mcp spec: namespaceSelector: matchNames: - {{ .Values.namespace }} selector: matchLabels: app: mcpserver toolhive-name: openshift endpoints: - port: http path: /metrics interval: 30s scheme: http -
Commit the file and wait for the service monitor to be synced to your cluster (approximately 30-60 seconds).
-
Switch to the
mcp-openshift-{user}project. -
In the OpenShift Console navigate to Observe → Metrics and query for
toolhive_mcp_active_connectionsortoolhive_mcp_requests_total. If you see "No datapoints found" the service monitor hasn’t synced yet. Try again a few seconds later.
Extending AI capabilities
One of MCP’s strengths is the growing ecosystem of MCP servers. Adding new capabilities to your AI agents is often as simple as deploying another MCP server.
|
We are taking a few small shortcuts in the section below. For one, LibreChat has been preconfigured for your user - actually for all users - using a GitOps ApplicationSet. Which means that you can’t really change the LibreChat configuration that’s stored in a Second you are creating the YAML definitions for the additional MCP servers in the Helm Chart for the Gitea MCP server. We do that because you don’t have a way to deploy an additional GitOps application using Helm - so we just "abuse" the Gitea MCP server helm chart. In the real world you would create your own Helm Chart for each additional MCP Server and then create a GitOps (ArgoCD) Application to deploy it. |
Adding a simple MCP server: Fetch service
In this section, you’ll deploy the Fetch MCP Server — a lightweight service that enables AI agents to retrieve content from web URLs.
The Fetch MCP server provides a single, focused capability:
-
Fetching web content — Retrieve HTML, JSON, or text content from any accessible URL
While simple, this demonstrates the pattern for adding any MCP server to your environment. The same process applies whether you’re adding a Slack notification server, a database query server, or a custom internal tool.
Business value of web fetching
The ability to fetch web content opens up powerful use cases for AI agents:
- Real-Time Information Access
-
AI agents can retrieve current documentation, API responses, or status pages rather than relying solely on their training data. When troubleshooting an issue, the agent can fetch the latest release notes or known issues from a vendor’s website.
- Integration with External APIs
-
Many services expose REST APIs that return JSON. The Fetch MCP server enables AI agents to query these APIs directly — checking deployment status, retrieving configuration data, or pulling metrics from external monitoring systems.
- Documentation Lookup
-
Rather than hallucinating answers, AI agents can fetch authoritative documentation. "What are the parameters for this Kubernetes resource?" becomes a lookup rather than a guess.
- Competitive Intelligence
-
AI agents can retrieve and summarize publicly available information — pricing pages, feature lists, or technical specifications — to support business decisions.
- Link Validation
-
AI agents can verify that URLs in documentation or configuration files are accessible, identifying broken links before they cause problems.
Deploying the Fetch MCP server
-
In Gitea, navigate to
{user}/mcp/helm/mcp-gitea. -
Create a file
templates/mcpserver-fetch.yaml:--- apiVersion: toolhive.stacklok.dev/v1alpha1 kind: MCPServer metadata: name: fetch namespace: {{ .Values.namespace }} spec: image: ghcr.io/stackloklabs/gofetch/server:1.0.2 transport: streamable-http port: 8080 targetPort: 8080 permissionProfile: type: builtin name: network resources: limits: cpu: "100m" memory: "128Mi" requests: cpu: "50m" memory: "64Mi"Notice the
permissionProfilesetting — this grants the MCP server network access, which is required for fetching external URLs. Without this, the container would be network-isolated. -
Commit the file.
-
Wait until GitOps has synced and the Fetch MCP server has been deployed — you can check in namespace
mcp-gitea-{user}that two new pods are running. The MCP server in thefetch-0pod and the ToolHive proxy whose name also starts withfetch-. -
Back in LibreChat, click the MCP Servers button and add the new fetch server to your session by clicking the orange initialize button. The MCP Server dropdown should now show 3 selected.
Using the Fetch MCP server
Now let’s test the Fetch MCP server with some practical examples (if any question fails simply try to run it again):
-
Example 1: Fetch documentation
-
Ask the AI to retrieve and summarize documentation:
Fetch the MCP specification overview from https://modelcontextprotocol.io/introduction and give me a brief summary.
-
-
Example 2: Check a JSON API
-
Retrieve data from a public JSON API:
Fetch the data from https://jsonplaceholder.typicode.com/users/1 and tell me about this user.
-
-
Example 3: Verify link accessibility
-
Check if a URL is accessible:
Can you verify that https://www.redhat.com is accessible? Just check if you can fetch it.
-
-
Example 4: Compare Information
-
Combine fetching with analysis:
Fetch the ToolHive documentation from https://docs.stacklok.com/toolhive and tell me what custom resources ToolHive provides.The Fetch MCP server can only access URLs that are reachable from within the OpenShift cluster. Internal corporate resources behind firewalls or VPNs won’t be accessible unless the cluster has network connectivity to them.
-
Adding a stdio MCP server: Yardstick
Why stdio bridging matters
The Fetch MCP server you just deployed uses the streamable-http transport — it was designed from the ground up for network communication. But the vast majority of MCP servers in the wild weren’t built this way.
Most MCP servers were originally created for desktop use cases — running locally alongside Claude Desktop, VS Code, or other AI-powered applications. These servers use stdio (standard input/output) as their transport mechanism: they read JSON-RPC messages from stdin and write responses to stdout. This is simple, efficient, and works perfectly when the MCP client and server run on the same machine.
But stdio presents a fundamental challenge for Kubernetes deployments:
-
No network interface: stdio is process-local communication — there’s no TCP port to connect to
-
No service discovery: Kubernetes services work with network endpoints, not process pipes
-
No load balancing: You can’t distribute stdin/stdout across multiple pods
-
No external access: Applications outside the pod (like LibreChat) have no way to reach the server
This is where ToolHive’s stdio bridging becomes invaluable.
How ToolHive bridges stdio to HTTP
When you deploy an MCP server with transport: stdio, ToolHive automatically injects a proxy sidecar that bridges the gap between network protocols and process I/O:
-
ToolHive spawns the MCP server process inside the container
-
The proxy listens on an HTTP port (exposed via a Kubernetes service)
-
Incoming HTTP requests are converted to JSON-RPC messages and written to the server’s stdin
-
Server responses from stdout are captured and returned as HTTP responses
-
The conversion is transparent — clients interact via HTTP without knowing the server uses stdio internally
This means you can take any existing stdio-based MCP server — even one designed purely for desktop use — and deploy it in Kubernetes without modifying a single line of code. The entire MCP server ecosystem becomes available for enterprise deployments.
Business value of stdio bridging
- Massive Ecosystem Access
-
The MCP ecosystem includes hundreds of community-built servers for databases, APIs, developer tools, and more. The vast majority use stdio transport. Without bridging, you’d be limited to the handful of servers that support HTTP natively.
- No Code Modifications Required
-
Deploy existing MCP servers as-is. You don’t need to fork repositories, add HTTP server code, or wait for maintainers to add network support. Just point ToolHive at the container image.
- Faster Time to Value
-
Found an MCP server that does exactly what you need? Deploy it today. No development work, no custom integration — just a simple
MCPServermanifest. - Consistent Operations
-
Whether the underlying server uses stdio, SSE, or streamable HTTP, ToolHive presents a uniform interface. Your monitoring, security policies, and operational procedures work the same way for all MCP servers.
- Future-Proof Investment
-
As new MCP servers are created (often using stdio for simplicity), you can adopt them immediately in your Kubernetes environment.
Deploying the Yardstick MCP server
Yardstick is a simple MCP server created specifically for testing and validation. It provides an echo tool that returns whatever you send it — perfect for verifying that your MCP infrastructure is working correctly.
While Yardstick itself is a testing tool, deploying it demonstrates the pattern for any stdio-based MCP server.
-
In Gitea, navigate to
{user}/mcp/helm/mcp-gitea/templates. -
Create a file
mcpserver-yardstick.yaml:--- apiVersion: toolhive.stacklok.dev/v1alpha1 kind: MCPServer metadata: name: yardstick namespace: {{ .Values.namespace }} spec: image: ghcr.io/stackloklabs/yardstick/yardstick-server:1.1.0 transport: stdio port: 8080 permissionProfile: type: builtin name: network resources: limits: cpu: "100m" memory: "128Mi" requests: cpu: "50m" memory: "64Mi"Notice
transport: stdio— this tells ToolHive to inject the bridging proxy. The server itself has no HTTP code; all network functionality comes from ToolHive’s proxy. -
Commit the file.
-
Wait until GitOps has synced and the Yardstick MCP server has been deployed — you can check in namespace
mcp-gitea-{user}that two new pods with names starting withyardstick-are running (you may need to scroll out in the Topology view to see all 6 pods). -
In LibreChat, click the MCP Servers button and initialize the new yardstick server to your session. You should now see 4 selected.
Using the Yardstick MCP server
Yardstick provides an echo tool for testing. Try these examples:
-
Example 1: Verify Tool Discovery
-
Confirm the stdio bridging is working by checking available tools:
What tools does the yardstick MCP server provide? List them with their descriptions.
-
-
Example 2: Basic Echo Test
-
Verify the MCP connection is working:
Use the yardstick echo tool to echo back the message "HelloFromOpenShift"
-
The bigger picture
While Yardstick itself is a simple testing tool, the deployment pattern you just used works for any stdio-based MCP server. Consider some examples from the MCP ecosystem:
-
SQLite MCP Server: Query and modify SQLite databases (stdio transport)
-
Filesystem MCP Server: Read and write files (stdio transport)
-
Git MCP Server: Interact with Git repositories (stdio transport)
-
Puppeteer MCP Server: Control headless browsers (stdio transport)
All of these — and hundreds more — can be deployed on OpenShift using exactly the same pattern: create an MCPServer resource with transport: stdio, and ToolHive handles the rest.
For more information on deploying MCP servers with ToolHive, see the Red Hat Developer article on ToolHive deployments.
Deep Dive: Understanding MCP architecture
|
In early January or 2026 it was decided to go a different direction for the Red Hat official MCP server / registry tooling. This means that Red Hat will not use ToolHive in the supported solution. However the new direction means that no real code is available to demonstrate where we will be going. Therefore we decided to keep the current lab - since most concepts will stay exactly the same. MCP Registry is a standard API - so whatever you learned in this lab will still be useful regardless of how the final solution will shape out to be. |
This section provides a deeper understanding of the MCP ecosystem, from the ToolHive operator that manages server lifecycles to emerging standards like MCP registries and OAuth authentication.
Understanding ToolHive
ToolHive is a Kubernetes operator developed by Stacklok that manages the lifecycle of MCP servers in containerized environments. It solves a fundamental problem: most MCP servers were designed for desktop use with stdio (standard input/output) communication, but Kubernetes workloads need network-accessible services.
What ToolHive Provides:
-
MCPServer Custom Resource: A Kubernetes-native way to declare MCP servers. You specify the container image and configuration; ToolHive handles the rest.
-
Automatic Proxy Injection: ToolHive wraps stdio-based MCP servers with an HTTP proxy, making them accessible over the network without modifying the original server code.
-
Tool Filtering: The
MCPToolConfigresource lets you restrict which tools are exposed, implementing least-privilege access control. -
Telemetry Integration: Built-in Prometheus metrics export for observability.
-
Transport Abstraction: Whether the underlying server uses stdio, SSE, or streamable HTTP, ToolHive presents a consistent interface to clients.
How MCP servers get deployed
When you create an MCPServer custom resource, ToolHive’s controller performs the following:
-
Validates the specification — Checks that the image exists, transport is valid, and configuration is complete.
-
Creates a Deployment — Runs the MCP server container with the ToolHive proxy sidecar.
-
Creates a Service — Exposes the MCP server within the cluster on a predictable hostname.
-
Applies tool filtering — If an
MCPToolConfigis referenced, configures the proxy to filter tools. -
Configures telemetry — Sets up Prometheus metrics endpoints if enabled.
In this lab, GitOps (OpenShift GitOps/ArgoCD) automates the deployment. When you commit changes to your Gitea repository, ArgoCD detects the change and applies the updated manifests to the cluster. This typically happens within 30 seconds.
MCP transport protocols
MCP supports several transport mechanisms, each with different trade-offs:
- stdio (Standard I/O)
-
The original MCP transport, designed for local process communication. The MCP client spawns the server as a subprocess and communicates via stdin/stdout. This is simple and efficient for desktop applications but doesn’t work in networked environments.
Most community MCP servers use stdio because it’s the easiest to implement. ToolHive’s proxy converts stdio servers to network-accessible services.
- SSE (Server-Sent Events)
-
A unidirectional streaming protocol built on HTTP. The client opens a long-lived HTTP connection, and the server pushes events as they occur. Used by the OpenShift MCP server in this lab.
SSE is widely supported and works through most proxies and firewalls. However, it’s unidirectional — the client can only send data through separate HTTP requests.
- Streamable HTTP
-
A bidirectional HTTP-based transport that supports true request/response patterns. Used by the Gitea MCP server in this lab.
This is the most flexible transport, supporting all MCP features including streaming responses and bidirectional communication.
- The ToolHive Proxy Approach
-
ToolHive runs a proxy sidecar alongside each MCP server container. For stdio servers, this proxy:
-
Spawns the MCP server process
-
Reads JSON-RPC messages from the server’s stdout
-
Exposes these over HTTP (SSE or streamable HTTP)
-
Translates incoming HTTP requests to stdin writes
This means you can use any existing MCP server — even one designed for Claude Desktop — in a Kubernetes environment without modification.
-
RBAC and authentication
MCP servers need credentials to access the systems they interact with. This lab demonstrates two common patterns:
- OpenShift MCP server: Kubernetes RBAC
-
The OpenShift MCP server runs with a Kubernetes service account. This service account has specific RBAC (Role-Based Access Control) bindings that determine what resources it can access.
In this lab, the service account has permissions scoped to specific namespaces. If you tried to use the MCP server to access resources in a namespace it’s not authorized for, the Kubernetes API would reject the request.
This is a powerful pattern because it uses Kubernetes' native security model. You manage AI agent permissions the same way you manage any other workload’s permissions.
- Gitea MCP server: Token authentication
-
The Gitea MCP server authenticates to the Gitea API using a personal access token. This token is stored as a Kubernetes Secret and mounted into the MCP server container.
The token’s scope determines what the MCP server can do. A token with read-only repository access would limit the server to read operations, even if the MCP server code supports writes.
Best practices for credential management:
-
Store credentials in Kubernetes secrets, never in ConfigMaps or environment variables in plain text. Even better store credentials outside of the cluster in HashiCorp Vault or a similar platform.
-
Use the most restrictive scope possible for tokens
-
Rotate credentials regularly
-
Consider using short-lived tokens where possible
-
Audit credential access through Kubernetes audit logging
MCP Registry: Centralized server discovery
As organizations deploy more MCP servers, they face discovery and governance challenges:
-
How do developers know what MCP servers are available?
-
How do administrators control which servers are approved for use?
-
How do you track versions and ensure consistency?
The MCP Registry specification addresses these needs. A registry provides:
- Server Catalog
-
A searchable index of available MCP servers with metadata including name, description, capabilities, and connection information.
- Version Management
-
Track different versions of MCP servers and their compatibility requirements.
- Governance Controls
-
Administrators can mark servers as approved, deprecated, or restricted.
- Discovery API
-
A standardized API that tools can use to find and connect to MCP servers programmatically.
Organizations can run private registries to catalog their internal MCP servers. The registry API follows a standard specification, enabling tool interoperability.
Future versions of ToolHive will integrate with registries to enable automatic server discovery — you’ll be able to browse available servers and deploy them directly from the registry.
Image signing and provenance
Supply chain security is critical for MCP servers. When you deploy an MCP server, you’re giving it access to your systems. How do you know the container image hasn’t been tampered with?
- Container Image Signing
-
Tools like Sigstore and
cosignenable cryptographic signing of container images. Publishers sign images with their private key; consumers verify signatures before deployment.For MCP servers, this means you can verify that:
-
The image was built by a trusted publisher
-
The image hasn’t been modified since signing
-
The image matches the expected content hash
-
- Provenance Tracking
-
The MCP Registry specification includes fields for provenance information:
-
Build timestamp
-
Source repository and commit
-
Builder identity
-
Signature verification status
This information helps organizations make informed decisions about which MCP servers to trust.
-
- Future Enforcement
-
While current tools support signing and provenance tracking, enforcement is still evolving. Future versions of ToolHive may support policies like "only deploy MCP servers with valid signatures from approved publishers."
OAuth/OIDC authentication: The future direction
Current MCP deployments typically use static tokens for authentication. This works but has limitations:
-
Tokens are long-lived, increasing exposure if compromised
-
No connection to enterprise identity providers
-
Difficult to implement per-user access control
The MCP community is developing OAuth 2.0 and OpenID Connect (OIDC) support to address these limitations:
- OAuth 2.0 Integration
-
MCP servers will be able to participate in OAuth flows, obtaining short-lived access tokens that can be refreshed. This enables:
-
Integration with enterprise identity providers (Okta, Azure AD, Keycloak)
-
Fine-grained, time-limited access
-
Token revocation when access should be terminated
-
- Identity Propagation
-
When an AI agent calls a tool on behalf of a user, the user’s identity can flow through the entire call chain. The backend system knows not just that "an AI called this tool" but "Alice’s AI assistant called this tool on her behalf."
This enables:
-
Per-user audit trails
-
User-specific access control (Alice can access her repositories, not Bob’s)
-
Compliance with data access regulations
-
- Enterprise SSO
-
With OIDC support, organizations can require that MCP access go through their SSO system. Users authenticate once, and their AI tools inherit that authentication.
These capabilities are still being standardized, but the November 2025 MCP specification update included significant progress on enterprise authorization patterns.