6.1 GitOps + admission
On this page you prove that stage runs the signed digest, and that an unsigned tag on the same ImageStream fails at pull.
Track 6 is the production container. This is the first of its two pages. You tag the signed image into lw-poc-staging, copy the signature onto that repository, commit the digest to the stage GitOps remote, and fill admission/trust-policy.yaml. A CronJob renders a namespaced ImagePolicy on lw-poc-staging and on lw-poc-prod. 6.2 Promote to prod is a second Gitea remote. Leave that remote for the next page. NetworkPolicy app-operate on staging stays open. That policy is not the 4.3 Build NetworkPolicy Check.
Why this control exists
Runtime enforcement means only the signed digest runs. GitOps sets that digest on stage. Unsigned deny is a pull failure, ImagePullBackOff, not an error from oc apply. The Lightwell pins inside the image do not admit an unsigned tag.
You edit one file on the stage remote: admission/trust-policy.yaml, kind: TrustPolicy. You set enforce: true, the live digest, and the pipeline service-account subject from 5.1. CronJob trust-policy-apply in namespace tssc-admission polls about every two minutes and renders ImagePolicy into both stage and prod. Applying a ClusterImagePolicy yourself fails the Check. Red Hat Advanced Cluster Security trusted image signers are a different gate.
Native ImagePolicy signedEmail has to be an email. The 5.1 identity is a Kubernetes service-account URI, so the CronJob renders policyType: PublicKey from ConfigMap lab-cosign-pubkey, the cosign.pub you published on 5.3. The TrustPolicy file still records the 5.1 service-account URI. That remains the Fulcio identity. oc tag copies the image and leaves the .sig tags behind. You cosign copy those tags, then cosign sign --key the destination repository, after the tag.
Lightwell Network and Lightwell Clearinghouse Premier are already inside the image as GAVs. Admission does not inspect Maven. A Premier member version does not skip unsigned deny.
The figure is two remotes. This page is the top row: lw-student/gitops-spring-boot-lw-poc, Application lw-poc-staging, namespace lw-poc-staging. The bottom row is 6.2. Retargeting Application lw-poc-prod at the stage remote fails that later Check.
In a customer engagement you promote a signed digest with GitOps and admit only that identity. You commit to Gitea and fill TrustPolicy. The cluster renders ImagePolicy. A production virtual machine is the same digest on a different runtime, such as bootc or Satellite. This cluster has no virtual machine.
Learning objectives
By the end of this module, you will be able to:
-
Tag the signed image into
lw-poc-stagingand copy its signature onto that repository -
Commit the digest to the stage GitOps remote and leave
replicasat 0 until the nodes are enforcing ImagePolicy -
Fill TrustPolicy so
enforce: trueand noREPLACE_ME_*remains -
Set
replicas: 1only after both MachineConfigPools have finished applying that policy -
Show an unsigned tag on the same ImageStream stuck in
ImagePullBackOffwhile stage stays Healthy
The two remotes
ConfigMap demo-userinfo-gitea in namespace gitea holds the URLs. student_gitops_repo_url is stage. student_prod_gitops_repo_url is prod. student_promote_namespace is lw-poc-staging. student_build_namespace is where the signed image lives. student_argocd_app is the stage Application. The password is the value of key student_password. Leave that value out of Git and out of screenshots.
ConfigMap demo-userinfo-admission in namespace tssc-admission names the scored kind and the live kind. Success is the stage and prod URLs, the two namespaces, the Argo CD application name, and scored=TrustPolicy with live=ImagePolicy.
oc -n gitea get configmap demo-userinfo-gitea \
-o jsonpath='stage={.data.student_gitops_repo_url}{"\n"}prod={.data.student_prod_gitops_repo_url}{"\n"}promote_ns={.data.student_promote_namespace}{"\n"}build_ns={.data.student_build_namespace}{"\n"}argo={.data.student_argocd_app}{"\n"}'
oc -n tssc-admission get configmap demo-userinfo-admission \
-o jsonpath='scored={.data.scored_kind}{"\n"}live={.data.live_kind}{"\n"}hint={.data.unsigned_deny_hint}{"\n"}'
The block below is a TrustPolicy that is still an example: enforce is false, the scope is a foreign registry, and the digest is all zeros. Copying admission-docs example-trust-policy.yaml over the scored file fails the Check.
# Worked example ONLY — wrong name, enforce still false, fake identity.
# Do not copy admission-docs example-trust-policy.yaml as the scored file.
apiVersion: tssc.workshop/v1
kind: TrustPolicy
metadata:
name: example-not-scored
spec:
enforce: false
scopes:
- quay.io/example/not-the-lab-repo
identity:
issuer: https://example.invalid/oidc
subject: student@example.invalid
digest: sha256:0000000000000000000000000000000000000000000000000000000000000000
Your change
Tag the signed image into staging. The digest you print has to match the 5.1 digest, including the sha256: prefix. oc tag copies the image. cosign copy copies the signature tags from the build repository to the stage repository. cosign sign --key then signs that destination tag with the 5.3 key, which is what ImagePolicy will check. HOME is /tmp/lab-home for the registry login. The key path stays /home/lab-user/lab-trust/cosign.key.
BUILD_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_build_namespace}')"
PROMOTE_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_promote_namespace}')"
oc tag "${BUILD_NS}/spring-boot-lw-poc:latest" "${PROMOTE_NS}/spring-boot-lw-poc:latest"
oc -n "${PROMOTE_NS}" get istag spring-boot-lw-poc:latest \
-o jsonpath='{.image.metadata.name}{"\n"}'
DIGEST="$(oc -n "${PROMOTE_NS}" get istag spring-boot-lw-poc:latest \
-o jsonpath='{.image.metadata.name}')"
SRC="image-registry.openshift-image-registry.svc:5000/${BUILD_NS}/spring-boot-lw-poc@${DIGEST}"
DST="image-registry.openshift-image-registry.svc:5000/${PROMOTE_NS}/spring-boot-lw-poc@${DIGEST}"
# oc tag copies the image, not .sig tags. Copy, then sign the dest repo with the 5.3 key.
export HOME=/tmp/lab-home
mkdir -p "${HOME}"
cosign login -u unused -p "$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
image-registry.openshift-image-registry.svc:5000
cosign copy --allow-http-registry --allow-insecure-registry "${SRC}" "${DST}"
COSIGN_PASSWORD="" cosign sign --key /home/lab-user/lab-trust/cosign.key \
--tlog-upload=false --allow-http-registry --allow-insecure-registry \
"${DST}"
Clone the stage GitOps remote, student_gitops_repo_url. A fresh clone does not inherit git config from the application repository. Set the author locally, the same identity as 3.2. The password is in the clone URL for this push. It does not belong in a committed file. Success is a clone at /tmp/gitops-stage.
STAGE_GITOPS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_gitops_repo_url}')"
: "${STUDENT_USER:=$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_username}')}"
: "${STUDENT_PASS:=$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_password}')}"
rm -rf /tmp/gitops-stage
hostpath="${STAGE_GITOPS#https://}"; hostpath="${hostpath#http://}"
git clone "https://${STUDENT_USER}:${STUDENT_PASS}@${hostpath}" /tmp/gitops-stage
cd /tmp/gitops-stage
git config user.name student
git config user.email student@workshop.local
Set image.digest to the digest you just tagged, and leave replicas: 0. Commit and push. Leave the prod remote untouched. PROMOTE.md in the clone names the fields if you need them.
Automated sync often stays on the previous commit, and replicas: 0 can still look Healthy. After the push, hard-refresh the Application. Success of that refresh is sync status, health, and a Deployment whose replicas is 0. Leave the pod stopped. A pull that happens before the node is enforcing ImagePolicy stores the image with no signature. After the policy is on that node, a restart reports already present on machine and no signature exists, and those retries do not start the container.
ARGO="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_argocd_app}')"
PROMOTE_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_promote_namespace}')"
oc -n openshift-gitops annotate applications.argoproj.io "${ARGO}" \
argocd.argoproj.io/refresh=hard --overwrite
oc -n openshift-gitops get applications.argoproj.io "${ARGO}" \
-o jsonpath='sync={.status.sync.status} health={.status.health.status} rev={.status.sync.revision}{"\n"}'
oc -n "${PROMOTE_NS}" get deploy spring-boot-lw-poc \
-o jsonpath='replicas={.spec.replicas} ready={.status.readyReplicas}{"\n"}'
The capture is Application lw-poc-staging after that refresh: Synced and Healthy, on the signed digest.
Edit admission/trust-policy.yaml on the same stage remote. Leave apiVersion and kind as they are. Set enforce: true. Replace every REPLACE_ME_*. The issuer is https://kubernetes.default.svc, or the userinfo hint oidc_issuer_hint. The subject is the pipeline service-account URI from 5.1. The digest is the signed sha256:…. The scopes are the internal ImageStream repositories that staging and prod actually pull. Discover those names with oc get is. Confirm ConfigMap tssc-admission/lab-cosign-pubkey still holds the 5.3 PEM. The CronJob injects the Fulcio and Rekor material. Pasting those PEMs into TrustPolicy fails the render. The CronJob sets signedIdentity.matchPolicy: ExactRepository to the build ImageStream, so a pull in stage or prod accepts a signature whose identity is lw-poc-build.
The figure is three layers. You edit the top layer, the TrustPolicy file in Gitea. The middle layer is CronJob trust-policy-apply. The bottom layer is the live ImagePolicy in lw-poc-staging and lw-poc-prod. ClusterImagePolicy and the Advanced Cluster Security signer list sit outside that gate.
Success is an ImagePolicy in both namespaces. The object name comes from userinfo policy_name. The CronJob interval is about two minutes.
ImagePolicy reaches the node as a CRI-O policy file, written by the Machine Config Operator. oc get imagepolicy can succeed while the pools are still updating. Wait until both pools list the same updated count as the machine count, and UPDATING is False. Then set replicas: 1, push, and hard-refresh. That first pull is the one the node checks. A later restart of that same pod finds the image already on the machine and starts the container. The Check accepts the Ready pod created after ImagePolicy, once the pools are settled.
PROMOTE_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_promote_namespace}')"
PROD_NS="$(oc -n tssc-admission get configmap demo-userinfo-admission -o jsonpath='{.data.prod_namespace}')"
POLICY="$(oc -n tssc-admission get configmap demo-userinfo-admission -o jsonpath='{.data.policy_name}')"
ARGO="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_argocd_app}')"
oc -n "${PROMOTE_NS}" get imagepolicy "${POLICY}"
oc -n "${PROD_NS}" get imagepolicy "${POLICY}"
oc get mcp
# After UPDATING is False on master and worker, set replicas: 1 in the stage clone, commit, and push.
oc -n openshift-gitops annotate applications.argoproj.io "${ARGO}" \
argocd.argoproj.io/refresh=hard --overwrite
oc -n "${PROMOTE_NS}" rollout status deploy/spring-boot-lw-poc --timeout=180s
oc -n "${PROMOTE_NS}" get pods -l app=spring-boot-lw-poc \
-o 'custom-columns=NAME:.metadata.name,CREATED:.metadata.creationTimestamp,PHASE:.status.phase,READY:.status.containerStatuses[0].ready'
Check
Native ImagePolicy is CRI-O policy.json. oc apply of an unsigned Deployment can succeed. The pod fails when the node pulls the image. If this cluster is using the Kyverno fallback, the deny happens at admission instead. Either way, read the pod and the events.
The gate covers spec.scopes. Tag an unsigned digest onto the same ImageStream and deploy that tag. Leave GitOps image.digest on the signed digest. Pointing GitOps at the unsigned tag would flap the Healthy check.
Success is a throwaway pod in ImagePullBackOff or a signature error, the stage Application still Healthy, replicas=1 on the signed Deployment, and ImagePolicy present in both lw-poc-staging and lw-poc-prod. Both MachineConfigPools are done updating. The stage pod’s creation time is later than that ImagePolicy. TrustPolicy has no REPLACE_ME, and enforce is true.
BUILD_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_build_namespace}')"
PROMOTE_NS="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_promote_namespace}')"
ARGO="$(oc -n gitea get configmap demo-userinfo-gitea -o jsonpath='{.data.student_argocd_app}')"
# Unsigned digest, in-scope name:
oc tag "${BUILD_NS}/openjdk-21-builder:latest" "${PROMOTE_NS}/spring-boot-lw-poc:unsigned"
oc -n "${PROMOTE_NS}" create deployment unsigned-deny-test \
--image="image-registry.openshift-image-registry.svc:5000/${PROMOTE_NS}/spring-boot-lw-poc:unsigned"
oc -n "${PROMOTE_NS}" rollout status deploy/unsigned-deny-test --timeout=30s || true
oc -n "${PROMOTE_NS}" get pods -l app=unsigned-deny-test
oc -n "${PROMOTE_NS}" get events --field-selector reason=Failed --sort-by=.lastTimestamp | tail -10
oc -n "${PROMOTE_NS}" delete deployment unsigned-deny-test
oc -n openshift-gitops get applications.argoproj.io "${ARGO}" \
-o custom-columns=NAME:.metadata.name,SYNC:.status.sync.status,HEALTH:.status.health.status
oc -n "${PROMOTE_NS}" get deploy spring-boot-lw-poc \
-o jsonpath='replicas={.spec.replicas} ready={.status.readyReplicas}{"\n"}'
oc -n "${PROMOTE_NS}" get imagepolicy
oc -n lw-poc-prod get imagepolicy
POLICY="$(oc -n tssc-admission get configmap demo-userinfo-admission -o jsonpath='{.data.policy_name}')"
echo -n "policy created "
oc -n "${PROMOTE_NS}" get imagepolicy "${POLICY}" -o jsonpath='{.metadata.creationTimestamp}{"\n"}'
oc -n "${PROMOTE_NS}" get pods -l app=spring-boot-lw-poc \
-o 'custom-columns=NAME:.metadata.name,CREATED:.metadata.creationTimestamp,PHASE:.status.phase,READY:.status.containerStatuses[0].ready'
The capture is namespace lw-poc-staging during that test. unsigned-deny-test is ImagePullBackOff. The signed spring-boot-lw-poc pod stays Running. Delete the throwaway Deployment before you leave the page. The command above already does that.
The Job also grades ConfigMap report-15-gitops-admission key live_api. Set it to imagepolicy. Empty or REPLACE_ME fails even if cluster state is correct. Do not copy validate-docs.
oc -n lw-poc-validate edit configmap report-15-gitops-admission
The scored Check is Job validate-15-gitops-admission in lw-poc-validate. Direct nav URLs still work (honor system). Unlimited retries. There is no Solve.
oc -n lw-poc-validate delete job validate-15-gitops-admission --ignore-not-found
TPL='job-15.yaml'
oc -n lw-poc-validate get configmap validate-job-templates \
-o go-template="{{ index .data \"${TPL}\" }}" | oc create -f -
oc -n lw-poc-validate logs -f job/validate-15-gitops-admission
Expect CHECK PASSED: before you continue. CHECK FAILED: names the object to fix. Delete the Job and create it again as often as you need.
The Validate Job grades TrustPolicy, the rendered ImagePolicy, a stage pod created after that policy, and the Healthy stage Application. The next page still opens if you skip this one. Leave prod Argo CD pointed at the prod remote.
|
A production virtual machine would run this same signed digest under bootc or Satellite. Canary and blue-green are other rollout strategies for that digest. This lab scores the default rolling Deployment. This cluster has no virtual machine to boot. |
Official documentation
-
Trusted Artifact Signer (the signatures admission consumes)
Key takeaways
-
Admission is a namespaced
ImagePolicyrendered from one TrustPolicy file. -
Start the stage Deployment after MachineConfigPools have applied that policy. A pull made before the node enforces it does not become Ready on a later restart.
-
A service-account subject renders as
PublicKeyfrom the 5.3 ConfigMaplab-cosign-pubkey. -
oc tagneeds a followingcosign copy, and acosign sign --keyon the destination. -
Lightwell GAVs inside the image do not admit an unsigned tag.
-
Production is a commit to a second remote, on the next page.



