Module 3: Expose the Model as a Model-as-a-Service
Overview
A model that only one team can reach is not production. In this module you transform the single-team deployment from Module 1 into a shared Model-as-a-Service (MaaS): the model is published to a governed gateway where consumers authenticate with API keys and call a stable, OpenAI-compatible endpoint.
The endpoint URL and API key you create here are the explicit handoff to Module 4, where the sample application consumes them.
Modules 1–2 complete — the model is deployed and serving inference requests.
20 minutes (3 min review / 7 min publish the model / 5 min access controls and API key / 5 min scale and test)
Learning objectives
By the end of this module you will be able to:
-
Publish a deployed model to the Model-as-a-Service gateway
-
Create API keys scoped to a subscription and consume the model through the shared endpoint
Exercise 1: Review the Model-as-a-Service approach
Models-as-a-Service (MaaS) provides subscription-based governance for large language model serving. It acts as a governance layer between users and the model serving infrastructure: consumers call a single managed API endpoint, and the platform enforces centralized policies — authentication, token rate limits — without modifying the underlying model serving components.
This is what changes from Module 1:
| Module 1 — direct deployment | Module 3 — Model-as-a-Service |
|---|---|
One team reaches the model directly through its inference endpoint |
Multiple teams consume the model through a shared, governed API endpoint |
No per-team quota or usage tracking |
Subscriptions enforce token rate limits and track consumption per group |
Access controlled by OpenShift project membership |
Access controlled by subscriptions, API keys, and authorization policies |
Three custom resources do the governing:
-
MaaSModelRef— references the model and registers it with MaaS. It lives in the model’s project namespace, next to theLLMInferenceServiceit names. -
MaaSSubscription— defines quota: which groups have access to which models, with token rate limits. Subscriptions are read only from themodels-as-a-servicenamespace. -
MaaSAuthPolicy— authorizes groups to reach model endpoints through the API gateway. A subscription without a matching policy returns HTTP 429; a policy without a matching subscription returns HTTP 403.
Consumers authenticate with an API key carrying the sk-oai- prefix and call
the model through the standard OpenAI-compatible /v1/chat/completions
endpoint with the model name in the request body — drop-in compatible with the
Python openai library, LangChain, and LlamaIndex.
|
The subscription model replaced the tier-based model used in OpenShift AI 3.3, starting with OpenShift AI 3.4. MaaS APIs are evolving across releases, so verify API names against your installed version. |
Exercise 2: Configure the model as a shared service
The model you deployed in Module 1 — redhataigpt-oss-20b-essential, serving
on the NVIDIA L4 GPU node in your project — is attached to the direct-serving
gateway for single-team access. Publishing it to MaaS takes two steps: attach
the model’s route to the shared MaaS gateway, then register the model with a
MaaSModelRef.
Review the serving configuration for the deployed model:
-
In the OpenShift AI console, navigate to Projects and open your project.
-
Click the Deployments tab and locate
redhataigpt-oss-20b-essential. -
Confirm the model row shows Ready in the Status column.
Attach the model’s route to the shared MaaS gateway by patching the
LLMInferenceService router:
oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
-p '{"spec":{"router":{"gateway":{"refs":[{"name":"maas-default-gateway","namespace":"openshift-ingress"}],"route":{}}}}}'
The model keeps serving through its existing route; the patch adds the MaaS gateway as the route’s parent, which is what makes the deployment shared.
Register the model with MaaS through a MaaSModelRef in the model’s project
namespace:
cat <<'EOF' | oc apply -f -
apiVersion: maas.opendatahub.io/v1alpha1
kind: MaaSModelRef
metadata:
name: redhataigpt-oss-20b-essential
namespace: lab-user-dev
spec:
modelRef:
kind: LLMInferenceService
name: redhataigpt-oss-20b-essential
tenantRef: models-as-a-service
EOF
Verify the registration:
oc get maasmodelref -n lab-user-dev
The PHASE column moves through three states as the platform catches up —
Failed while the model’s route is still attached to the direct-serving
gateway, then Pending once the route resolves to the MaaS gateway, then
Ready when governance pairing from Exercise 3 activates:
At the Pending stage, RuntimeReady=True and ModelIdentityUnique=True are
already set; the GovernanceAttached=False / NoPairingFound condition clears
in Exercise 3 when the subscription pairing activates.
The model also appears on the shared service surface consumers use:
-
In the console, navigate to Gen AI studio → AI asset endpoints.
-
On the Models tab, locate
publishers/lab-user-dev/models/redhataigpt-oss-20b-essential. The row shows Ready with View in the Endpoints column. -
Click View to open the Endpoints dialog. The dialog shows a Model as a Service badge, your subscription, and the external API endpoint URL.
|
AI asset endpoints is a Technology Preview feature in Red Hat OpenShift AI 3.6. |
Exercise 3: Configure access controls and API endpoints
Now that the model is published to the shared gateway, you will review who can consume it and create the credentials.
Access is governed by subscriptions. This environment has one tier, provisioned by automation and backed by the group every authenticated user belongs to:
| Subscription | Owner group | Priority | Token rate limits |
|---|---|---|---|
|
|
0 |
20,000 tokens/min and 1,000 tokens/sec |
The owner group is the group your OpenShift identity presents to the gateway —
a subscription backed by plain system:authenticated never matches, so every
call is denied with HTTP 403. The subscription and its matching
MaaSAuthPolicy live in the models-as-a-service namespace — the only
namespace the MaaS API reads subscriptions from. Verify both exist:
oc get maassubscription,maasauthpolicy -n models-as-a-service
Both report PHASE Active:
The same pairing is visible in the console. Navigate to Settings → MaaS
governance: the Overview tab lists each published model with its
subscription and authorization policy pairing — redhataigpt-oss-20b-essential
shows Ready with 1 subscription and 1 authorization policy. Use Create
subscription and Create authorization policy on this page to add tiers.
Review your own subscription from the consumer view:
-
Navigate to Gen AI studio → API keys. The page combines API keys and subscriptions as tabs.
-
Open the Subscriptions tab, then click Details on the
maas-free-subscriptionrow. The expanded row lists the model it includes with its token rate limits — 20,000 / 1 minute and 1,000 / 1 second.
Create an API key:
-
On the API keys tab, click Create API key.
-
In the Create API key dialog, enter
Workshopin the Name field. -
In the Subscription field, select
maas-free-subscription. The key will work only with models that belong to the selected subscription. -
Leave the Expiration at its 30 day default (the field accepts 1–365 days).
-
Click Create. The dialog displays the generated key with a
sk-oai-prefix.
|
The plaintext key is displayed only during creation and cannot be retrieved later. Click the copy icon next to the API key field and save the key in a secrets manager or a secure local file before closing the dialog. If you lose it, you must revoke the key and create a new one. |
-
Click Close. The new key appears in the table with Status = Active and the Expires date you selected.
Retrieve the external endpoint URL:
-
Navigate to Gen AI studio → AI asset endpoints.
-
Click View in the Endpoints column for your model.
-
In the Endpoints dialog, copy the External API endpoint URL and store it with your API key. The dialog also offers Generate API key for short-lived keys minted directly from the dialog.
Record both values now — you will use them in Exercise 4 and in Module 4:
| Value | Source |
|---|---|
External API endpoint URL |
Endpoints dialog on the AI asset endpoints page (shape:
|
API key ( |
Copied from the one-time API key created dialog |
|
Your group membership is captured at API key creation time. If your group membership changes after the key is created, the key retains the original group associations — revoke the existing key and create a new one to reflect updated groups. |
Exercise 4: Configure scaling policies
Now that you hold a working API key, you will scale the service and confirm it is ready for concurrent consumers.
Module 1 deployed the model with a single replica — the deployment wizard default. A shared endpoint with concurrent consumers needs headroom: for production, deploy at least 2 replicas so the service survives pod restarts and absorbs concurrent requests.
Scale the model to two replicas:
oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
-p '{"spec":{"replicas":2}}'
Watch the second model pod:
oc get pods -n lab-user-dev -w
Expected output:
NAME READY STATUS RESTARTS AGE redhataigpt-oss-20b-essential-kserve-6ccdd4bddc-krgvf 2/2 Running 0 5h24m redhataigpt-oss-20b-essential-kserve-6ccdd4bddc-mk8xd 0/2 SchedulingGated 0 10s
Pod names vary. In this lab the second pod stays SchedulingGated: the GPU
node carries a single NVIDIA L4 and the Kueue cluster queue’s GPU quota is 1,
so the second replica cannot be admitted. The service keeps serving with the
original replica — nothing is lost. On a multi-GPU cluster the second pod
comes up Running the same way the first one did.
Scale back to one replica and delete the gated pod’s request:
oc patch llminferenceservice redhataigpt-oss-20b-essential -n lab-user-dev --type merge \
-p '{"spec":{"replicas":1}}'
Press Ctrl+C to stop watching. The SchedulingGated pod is removed and the
original replica keeps serving.
|
The scaling configuration carries forward to Module 6, where you monitor serving behavior under load and connect metrics to scaling decisions. The token rate limits from Exercise 3 govern load generation there. |
Before testing the endpoint, confirm the model finished MaaS governance
pairing — the MaaSModelRef status conditions must all report True:
oc get maasmodelref redhataigpt-oss-20b-essential -n lab-user-dev -o json \
| jq -r '.status.conditions[] | "\(.type)=\(.status)"'
Expected output — all four conditions True:
GovernanceAttached confirms the subscription and authorization policy
pairing from Exercise 3 is active, RuntimeReady confirms the serving runtime
is up, and Ready=True reports the model is published and serving through the
MaaS gateway.
Verify the endpoint is externally reachable. Set the values you recorded in Exercise 3 as environment variables (run these in the same terminal session so the variables remain available):
export MAAS_URL="https://maas.apps.cluster.example.com"
export MAAS_API_KEY="<your sk-oai- API key>"
curl -s "${MAAS_URL}/v1/models" -H "Authorization: Bearer ${MAAS_API_KEY}" | jq .
The response lists the models accessible through your subscription in OpenAI-compatible JSON format:
Export the model ID from the catalog — request bodies must use the full ID
returned by /v1/models (publishers/lab-user-dev/models/redhataigpt-oss-20b-essential):
export MODEL_ID=$(curl -s "${MAAS_URL}/v1/models" \
-H "Authorization: Bearer ${MAAS_API_KEY}" \
| jq -r '[.data[] | select(.ready==true)][0].id')
echo "Model ID: ${MODEL_ID}"
Send a test inference request through the MaaS endpoint using body-based routing — the standard OpenAI-compatible surface:
curl -s "${MAAS_URL}/v1/chat/completions" \
-H "Authorization: Bearer ${MAAS_API_KEY}" \
-H "Content-Type: application/json" \
-d "{\"model\":\"${MODEL_ID}\",\"messages\":[{\"role\":\"user\",\"content\":\"What is Model-as-a-Service in one sentence?\"}],\"max_tokens\":1000}" \
| jq .
A successful response returns HTTP 200 with a completion in OpenAI-compatible
format. GPT-OSS handles reasoning natively through its harmony response
format: the model thinks before it answers, so the response carries the
model’s thinking in the reasoning field and the final answer in content:
Note the model field echoes the short model name, while request bodies use
the full publishers/<project>/<model> ID from the catalog.
Module 4’s sample application streams responses — the same endpoint accepts
"stream": true and returns OpenAI-compatible server-sent events.
|
Troubleshooting: 401 Unauthorized means the API key is invalid, expired, or revoked — create a new key from the API keys page; 429 Too Many Requests means the subscription’s token limit was reached — wait for the window to reset and retry; 403 Forbidden means the authorization policy does not grant your group gateway access, or the subscription owner group does not match the group your identity presents. Keep request payloads small: the MaaS gateway enforces a 4 MiB request body limit (RHOAIENG-99765), so large payloads routed through the gateway — long prompts, batched requests, or attached files — must stay below roughly 3.8 MB. |
Module summary
What you accomplished:
-
Reviewed the Model-as-a-Service approach and how it differs from direct deployment.
-
Published the model to the shared gateway with a
MaaSModelRefand the router patch. -
Created an API key scoped to the subscription and verified governance pairing.
Key takeaways:
-
Model-as-a-Service transforms a single-team model deployment into a shared, governed resource available across the organization.
-
Subscriptions and authorization policies govern who can consume the model; API keys carry that access.
-
Scaling headroom for a shared endpoint is bounded by cluster GPU quota — Kueue gates replicas the cluster cannot admit.
-
The endpoint URL and API key you recorded here are the handoff to Module 4 — reuse them there without re-deriving.
Next steps:
The endpoint URL and API key you recorded are ready to be consumed: continue to Module 4 to connect the sample application to the endpoint.










