Module 3: Expose the Model as a Model-as-a-Service

Overview

A model that only one team can reach is not production. In this module you transform the single-team deployment from Module 1 into a shared Model-as-a-Service (MaaS): the model is published to a governed gateway where consumers authenticate with API keys and call a stable, OpenAI-compatible endpoint.

The endpoint URL and API key you create here are the explicit handoff to Module 4, where the sample application consumes them.

Prerequisites

Modules 1–2 complete — the model is deployed and serving inference requests.

Duration

20 minutes (3 min review / 7 min publish the model / 5 min access controls and API key / 5 min scale and test)

Learning objectives

By the end of this module you will be able to:

  • Publish a deployed model to the Model-as-a-Service gateway

  • Create API keys scoped to a subscription and consume the model through the shared endpoint

Exercise 1: Review the Model-as-a-Service approach

3 minutes

Models-as-a-Service (MaaS) provides subscription-based governance for large language model serving. It acts as a governance layer between users and the model serving infrastructure: consumers call a single managed API endpoint, and the platform enforces centralized policies — authentication, token rate limits — without modifying the underlying model serving components.

This is what changes from Module 1:

Module 1 — direct deployment Module 3 — Model-as-a-Service

One team reaches the model directly through its inference endpoint

Multiple teams consume the model through a shared, governed API endpoint

No per-team quota or usage tracking

Subscriptions enforce token rate limits and track consumption per group

Access controlled by OpenShift project membership

Access controlled by subscriptions, API keys, and authorization policies

Three custom resources do the governing:

  • MaaSModelRef — references the model and registers it with MaaS. It lives in the model’s project namespace, next to the LLMInferenceService it names.

  • MaaSSubscription — defines quota: which groups have access to which models, with token rate limits. Subscriptions are read only from the models-as-a-service namespace.

  • MaaSAuthPolicy — authorizes groups to reach model endpoints through the API gateway. A subscription without a matching policy returns HTTP 429; a policy without a matching subscription returns HTTP 403.

MaaS governance-layer diagram showing the API gateway between consumers and models

Consumers authenticate with an API key carrying the sk-oai- prefix and call the model through the standard OpenAI-compatible /v1/chat/completions endpoint with the model name in the request body — drop-in compatible with the Python openai library, LangChain, and LlamaIndex.

The subscription model replaced the tier-based model used in OpenShift AI 3.3, starting with OpenShift AI 3.4. MaaS APIs are evolving across releases, so verify API names against your installed version.

Exercise 2: Configure the model as a shared service

7 minutes

The model you deployed in Module 1 — redhataigpt-oss-20b-essential, serving on the NVIDIA L4 GPU node in your project — is attached to the direct-serving gateway for single-team access. Publishing it to MaaS takes two steps: attach the model’s route to the shared MaaS gateway, then register the model with a MaaSModelRef.

Review the serving configuration for the deployed model:

  1. In the OpenShift AI console, navigate to Projects and open your project.

  2. Click the Deployments tab and locate redhataigpt-oss-20b-essential.

  3. Confirm the model row shows Ready in the Status column.

Attach the model’s route to the shared MaaS gateway by patching the LLMInferenceService router:

oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
  -p '{"spec":{"router":{"gateway":{"refs":[{"name":"maas-default-gateway","namespace":"openshift-ingress"}],"route":{}}}}}'

The model keeps serving through its existing route; the patch adds the MaaS gateway as the route’s parent, which is what makes the deployment shared.

Register the model with MaaS through a MaaSModelRef in the model’s project namespace:

cat <<'EOF' | oc apply -f -
apiVersion: maas.opendatahub.io/v1alpha1
kind: MaaSModelRef
metadata:
  name: redhataigpt-oss-20b-essential
  namespace: lab-user-dev
spec:
  modelRef:
    kind: LLMInferenceService
    name: redhataigpt-oss-20b-essential
  tenantRef: models-as-a-service
EOF

Verify the registration:

oc get maasmodelref -n lab-user-dev

The PHASE column moves through three states as the platform catches up — Failed while the model’s route is still attached to the direct-serving gateway, then Pending once the route resolves to the MaaS gateway, then Ready when governance pairing from Exercise 3 activates:

The MaaSModelRef pending with the route attached to the maas-default-gateway

At the Pending stage, RuntimeReady=True and ModelIdentityUnique=True are already set; the GovernanceAttached=False / NoPairingFound condition clears in Exercise 3 when the subscription pairing activates.

The model also appears on the shared service surface consumers use:

  1. In the console, navigate to Gen AI studio → AI asset endpoints.

  2. On the Models tab, locate publishers/lab-user-dev/models/redhataigpt-oss-20b-essential. The row shows Ready with View in the Endpoints column.

  3. Click View to open the Endpoints dialog. The dialog shows a Model as a Service badge, your subscription, and the external API endpoint URL.

AI asset endpoints is a Technology Preview feature in Red Hat OpenShift AI 3.6.

AI asset endpoints page with the model showing Ready status and View in the Endpoints column

Exercise 3: Configure access controls and API endpoints

5 minutes

Now that the model is published to the shared gateway, you will review who can consume it and create the credentials.

Access is governed by subscriptions. This environment has one tier, provisioned by automation and backed by the group every authenticated user belongs to:

Subscription Owner group Priority Token rate limits

maas-free-subscription

system:authenticated:oauth

0

20,000 tokens/min and 1,000 tokens/sec

The owner group is the group your OpenShift identity presents to the gateway — a subscription backed by plain system:authenticated never matches, so every call is denied with HTTP 403. The subscription and its matching MaaSAuthPolicy live in the models-as-a-service namespace — the only namespace the MaaS API reads subscriptions from. Verify both exist:

oc get maassubscription,maasauthpolicy -n models-as-a-service

Both report PHASE Active:

The subscription and authorization policy Active in the models-as-a-service namespace

The same pairing is visible in the console. Navigate to Settings → MaaS governance: the Overview tab lists each published model with its subscription and authorization policy pairing — redhataigpt-oss-20b-essential shows Ready with 1 subscription and 1 authorization policy. Use Create subscription and Create authorization policy on this page to add tiers.

The MaaS governance page under Settings showing the model Ready with its subscription and policy pairing

Review your own subscription from the consumer view:

  1. Navigate to Gen AI studio → API keys. The page combines API keys and subscriptions as tabs.

  2. Open the Subscriptions tab, then click Details on the maas-free-subscription row. The expanded row lists the model it includes with its token rate limits — 20,000 / 1 minute and 1,000 / 1 second.

Subscriptions view showing the subscription with its token rate limits

Create an API key:

  1. On the API keys tab, click Create API key.

  2. In the Create API key dialog, enter Workshop in the Name field.

  3. In the Subscription field, select maas-free-subscription. The key will work only with models that belong to the selected subscription.

  4. Leave the Expiration at its 30 day default (the field accepts 1–365 days).

  5. Click Create. The dialog displays the generated key with a sk-oai- prefix.

The plaintext key is displayed only during creation and cannot be retrieved later. Click the copy icon next to the API key field and save the key in a secrets manager or a secure local file before closing the dialog. If you lose it, you must revoke the key and create a new one.

  1. Click Close. The new key appears in the table with Status = Active and the Expires date you selected.

Create API key dialog with the name
API key created dialog showing the one-time key display

Retrieve the external endpoint URL:

  1. Navigate to Gen AI studio → AI asset endpoints.

  2. Click View in the Endpoints column for your model.

  3. In the Endpoints dialog, copy the External API endpoint URL and store it with your API key. The dialog also offers Generate API key for short-lived keys minted directly from the dialog.

Record both values now — you will use them in Exercise 4 and in Module 4:

Value Source

External API endpoint URL

Endpoints dialog on the AI asset endpoints page (shape: https://maas.apps.<cluster-domain>/)

API key (sk-oai- prefix)

Copied from the one-time API key created dialog

Your group membership is captured at API key creation time. If your group membership changes after the key is created, the key retains the original group associations — revoke the existing key and create a new one to reflect updated groups.

Exercise 4: Configure scaling policies

5 minutes

Now that you hold a working API key, you will scale the service and confirm it is ready for concurrent consumers.

Module 1 deployed the model with a single replica — the deployment wizard default. A shared endpoint with concurrent consumers needs headroom: for production, deploy at least 2 replicas so the service survives pod restarts and absorbs concurrent requests.

Scale the model to two replicas:

oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
  -p '{"spec":{"replicas":2}}'

Watch the second model pod:

oc get pods -n lab-user-dev -w

Expected output:

NAME                                            READY   STATUS            RESTARTS   AGE
redhataigpt-oss-20b-essential-kserve-6ccdd4bddc-krgvf   2/2   Running           0          5h24m
redhataigpt-oss-20b-essential-kserve-6ccdd4bddc-mk8xd   0/2   SchedulingGated   0          10s

Pod names vary. In this lab the second pod stays SchedulingGated: the GPU node carries a single NVIDIA L4 and the Kueue cluster queue’s GPU quota is 1, so the second replica cannot be admitted. The service keeps serving with the original replica — nothing is lost. On a multi-GPU cluster the second pod comes up Running the same way the first one did.

Scale back to one replica and delete the gated pod’s request:

oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
  -p '{"spec":{"replicas":1}}'

Press Ctrl+C to stop watching. The SchedulingGated pod is removed and the original replica keeps serving.

The scaling configuration carries forward to Module 6, where you monitor serving behavior under load and connect metrics to scaling decisions. The token rate limits from Exercise 3 govern load generation there.

Before testing the endpoint, confirm the model finished MaaS governance pairing — the MaaSModelRef status conditions must all report True:

oc get maasmodelref redhataigpt-oss-20b-essential -n lab-user-dev -o json \
  | jq -r '.status.conditions[] | "\(.type)=\(.status)"'

Expected output — all four conditions True:

The MaaSModelRef status conditions all reporting True

GovernanceAttached confirms the subscription and authorization policy pairing from Exercise 3 is active, RuntimeReady confirms the serving runtime is up, and Ready=True reports the model is published and serving through the MaaS gateway.

Verify the endpoint is externally reachable. Set the values you recorded in Exercise 3 as environment variables (run these in the same terminal session so the variables remain available):

export MAAS_URL="https://maas.apps.cluster.example.com"
export MAAS_API_KEY="<your sk-oai- API key>"
curl -s "${MAAS_URL}/v1/models" -H "Authorization: Bearer ${MAAS_API_KEY}" | jq .

The response lists the models accessible through your subscription in OpenAI-compatible JSON format:

The /v1/models response listing the published model with its subscription

Export the model ID from the catalog — request bodies must use the full ID returned by /v1/models (publishers/lab-user-dev/models/redhataigpt-oss-20b-essential):

export MODEL_ID=$(curl -s "${MAAS_URL}/v1/models" \
  -H "Authorization: Bearer ${MAAS_API_KEY}" \
  | jq -r '[.data[] | select(.ready==true)][0].id')
echo "Model ID: ${MODEL_ID}"

Send a test inference request through the MaaS endpoint using body-based routing — the standard OpenAI-compatible surface:

curl -s "${MAAS_URL}/v1/chat/completions" \
  -H "Authorization: Bearer ${MAAS_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"${MODEL_ID}\",\"messages\":[{\"role\":\"user\",\"content\":\"What is Model-as-a-Service in one sentence?\"}],\"max_tokens\":1000}" \
  | jq .

A successful response returns HTTP 200 with a completion in OpenAI-compatible format. GPT-OSS handles reasoning natively through its harmony response format: the model thinks before it answers, so the response carries the model’s thinking in the reasoning field and the final answer in content:

The /v1/chat/completions response with the reasoning field and the final content answer

Note the model field echoes the short model name, while request bodies use the full publishers/<project>/<model> ID from the catalog.

Module 4’s sample application streams responses — the same endpoint accepts "stream": true and returns OpenAI-compatible server-sent events.

Troubleshooting: 401 Unauthorized means the API key is invalid, expired, or revoked — create a new key from the API keys page; 429 Too Many Requests means the subscription’s token limit was reached — wait for the window to reset and retry; 403 Forbidden means the authorization policy does not grant your group gateway access, or the subscription owner group does not match the group your identity presents.

Keep request payloads small: the MaaS gateway enforces a 4 MiB request body limit (RHOAIENG-99765), so large payloads routed through the gateway — long prompts, batched requests, or attached files — must stay below roughly 3.8 MB.

Module summary

What you accomplished:

  • Reviewed the Model-as-a-Service approach and how it differs from direct deployment.

  • Published the model to the shared gateway with a MaaSModelRef and the router patch.

  • Created an API key scoped to the subscription and verified governance pairing.

Key takeaways:

  • Model-as-a-Service transforms a single-team model deployment into a shared, governed resource available across the organization.

  • Subscriptions and authorization policies govern who can consume the model; API keys carry that access.

  • Scaling headroom for a shared endpoint is bounded by cluster GPU quota — Kueue gates replicas the cluster cannot admit.

  • The endpoint URL and API key you recorded here are the handoff to Module 4 — reuse them there without re-deriving.

Next steps:

The endpoint URL and API key you recorded are ready to be consumed: continue to Module 4 to connect the sample application to the endpoint.