Overview #

LiteMaaS guardrails use LiteLLM's built-in litellm_content_filter to intercept requests and responses at the proxy layer. No external service or additional infrastructure is required — all pattern matching runs in-process using precompiled regex.

Two guardrail layers are active by default:

How It Works #

Guardrail configuration lives in a Kubernetes ConfigMap (litellm-guardrails-config) mounted into the LiteLLM deployment at /etc/litellm/config.yaml. LiteLLM is started with --config /etc/litellm/config.yaml, which loads the guardrails on startup.

Because models are stored in the database (STORE_MODEL_IN_DB=true), the config file only needs to define guardrails — no model_list is required. The two systems coexist cleanly.

Request flow

User request
    │
    ▼
[pii-and-credentials guardrail]  ← pre_call: regex scan
    │ match → 403 Blocked
    │ no match ↓
    ▼
LiteLLM routes to model
    │
    ▼
Model response
    │
    ▼
[output-pii-mask guardrail]  ← post_call: regex scan
    │ match → MASK redaction
    │ no match ↓
    ▼
Response returned to user

Deploying with the Playbook #

The configure_guardrails.yml playbook handles everything: creating the ConfigMap, patching the deployment, waiting for rollout, and verifying the guardrails are active.

Prerequisites

• Logged into the target OpenShift cluster (oc login ...)
• kubernetes.core Ansible collection installed
• Access to the target namespace

Apply to production

copyansible-playbook playbooks/configure_guardrails.yml \
  -e litemaas_namespace=litellm-rhpds

Apply to dev or staging

copyansible-playbook playbooks/configure_guardrails.yml \
  -e litemaas_namespace=litellm-dev

ansible-playbook playbooks/configure_guardrails.yml \
  -e litemaas_namespace=litellm-staging

Check current guardrails without applying

copyansible-playbook playbooks/configure_guardrails.yml \
  -e litemaas_namespace=litellm-rhpds \
  -e litemaas_guardrails_check_only=true

What the playbook does

1. Resolves the LiteLLM route and master key from OpenShift secrets
2. Creates or updates the litellm-guardrails-config ConfigMap
3. Patches the litellm deployment to mount the ConfigMap and set --config startup args
4. Waits for rolling restart to complete (180s timeout)
5. Verifies the /guardrails/list API endpoint returns both guardrails
6. Runs a live SSN blocking test and reports pass/fail

Prebuilt Patterns Reference #

These regex patterns ship with LiteLLM 1.81.0 and are available without any additional setup.

PII Patterns BLOCK

Payment Card Patterns BLOCK

Credential Patterns BLOCK

Dangerous Content Patterns BLOCK

Harmful Content Categories #

Categories use dictionary-based keyword matching (not LLM inference) loaded from YAML files shipped with LiteLLM. Three categories are enabled by default:

Additional available categories (not enabled by default):

Testing Guardrails #

Run these against the production LiteLLM endpoint to verify guardrails are active. Use the team test key.

Quick smoke test

copyLITELLM_URL="https://litellm-prod.apps.maas.redhatworkshops.io"
TEST_KEY="sk-wqUDDPcLy5hKHnlP-3AOCg"

# SSN — expect 403
curl -s -o /dev/null -w "%{http_code}" -X POST "$LITELLM_URL/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_KEY" -H "Content-Type: application/json" \
  -d '{"model":"granite-3-2-8b-instruct","messages":[{"role":"user","content":"My SSN is 123-45-6789"}]}'

# AWS key — expect 403
curl -s -o /dev/null -w "%{http_code}" -X POST "$LITELLM_URL/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_KEY" -H "Content-Type: application/json" \
  -d '{"model":"granite-3-2-8b-instruct","messages":[{"role":"user","content":"key AKIAIOSFODNN7EXAMPLE"}]}'

# Violence — expect 403
curl -s -o /dev/null -w "%{http_code}" -X POST "$LITELLM_URL/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_KEY" -H "Content-Type: application/json" \
  -d '{"model":"granite-3-2-8b-instruct","messages":[{"role":"user","content":"I will kill you"}]}'

# Normal message — expect 200
curl -s -o /dev/null -w "%{http_code}" -X POST "$LITELLM_URL/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_KEY" -H "Content-Type: application/json" \
  -d '{"model":"granite-3-2-8b-instruct","messages":[{"role":"user","content":"What is the capital of France?"}],"max_tokens":10}'

View active guardrails (admin)

copyMASTER_KEY=$(oc get secret litellm-secret -n litellm-rhpds \
  -o jsonpath='{.data.LITELLM_MASTER_KEY}' | base64 -d)

curl -s "$LITELLM_URL/guardrails/list" \
  -H "Authorization: Bearer $MASTER_KEY" | python3 -m json.tool

Expected blocked response

copy{
  "error": {
    "message": "{'error': 'Content blocked: us_ssn pattern detected', 'pattern': 'us_ssn'}",
    "type": "None",
    "param": "None",
    "code": "403"
  }
}

Customizing Patterns #

To change which patterns are active, edit the litemaas_guardrails_config variable in playbooks/configure_guardrails.yml and re-run the playbook. Changes take effect after the rolling restart completes.

Add a custom regex pattern

copy# In configure_guardrails.yml, under patterns:
- pattern_type: "regex"
  name: "redhat-internal-id"
  pattern: "RH-[0-9]{6}"
  action: "BLOCK"

Change action from BLOCK to MASK

Use MASK to redact the matched content instead of blocking the entire request. Useful for output filtering where you want the response but with sensitive data removed.

copy- pattern_type: "prebuilt"
  pattern_name: "email"
  action: "MASK"   # replaces with [email_REDACTED]

Enable additional categories

copy# Add under categories in the pii-and-credentials guardrail:
- category: "denied_medical_advice"
  enabled: true
  action: "BLOCK"
  severity_threshold: "high"   # high | medium | low

Update ConfigMap without full playbook run

copy# Edit the ConfigMap directly, then restart to pick up changes
oc edit configmap litellm-guardrails-config -n litellm-rhpds
oc rollout restart deployment/litellm -n litellm-rhpds

Removing Guardrails #

To disable guardrails entirely:

copyansible-playbook playbooks/configure_guardrails.yml \
  -e litemaas_namespace=litellm-rhpds \
  -e litemaas_guardrails_state=absent

This removes the ConfigMap and restores the LiteLLM deployment to its original startup command (no --config flag).