Creating the RHOSO Controlplane Overlay Configuration

Creating the initial overlay

Change directory to your forked labrepo repo, in your bastion:

cd /home/lab-user/labrepo/gitops

The base/controlplane directory already consumes the upstream RHOSO controlplane catalog component (openstack-k8s-operators/gitops). In this chapter you create an environment overlay that patches that catalog default into the lab control plane.

Create the environment directory to store the managed cluster overlay configuration:

mkdir -p environments/demo-env/controlplane
cat > environments/demo-env/controlplane/kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - controlplane-patches
EOF

Creating the Controlplane Environment Overlay

Create the controlplane patches directory to store the overlay patches:

mkdir -p environments/demo-env/controlplane/controlplane-patches

Configuring RHOSO Controlplane patches

Creating the patches configuration

Create the kustomization.yaml for the demo-env environment overlay configuration. It starts from base/controlplane (the catalog controlplane component). Patches are added after you create the thematic files below:

cat > environments/demo-env/controlplane/controlplane-patches/kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../../../base/controlplane
EOF

Validate the catalog base configuration can be generated with kustomize build:

kustomize build environments/demo-env/controlplane/controlplane-patches

Applying thematic control plane patches

The catalog OpenStackControlPlane is a production-oriented default. The following patches adapt it for this lab: storage class, DNS and MetalLB addresses, replica scale, Cinder NFS, Nova/OVN networking, and which optional services are enabled. Each patch uses $patch: replace on the service sections it owns so catalog fields that do not apply to the lab are removed cleanly.

The commands below use a quoted heredoc (<<'EOF'). That keeps $patch literal in the YAML. If you use unquoted <<EOF, the shell expands $patch to empty and kustomize fails with unable to parse SM or JSON patch.
Configuring storage

Set the lab ODF storage class, shrink Galera PVC requests, and configure Glance storage.

cat > environments/demo-env/controlplane/controlplane-patches/patch-storage.yaml <<'EOF'
# Lab storage settings: ODF storage class and smaller Galera PVCs.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  storageClass: ocs-external-storagecluster-ceph-rbd
  galera:
    $patch: replace
    templates:
      openstack:
        storageRequest: 500M
      openstack-cell1:
        storageRequest: 500M
  glance:
    $patch: replace
    enabled: true
    template:
      databaseInstance: openstack
      storage:
        storageRequest: 10G
        storageClass: "ocs-external-storagecluster-ceph-rbd"
      keystoneEndpoint: default
      glanceAPIs:
        default:
          type: single
          replicas: 1
          override:
            service:
              internal:
                metadata:
                  annotations:
                    metallb.universe.tf/address-pool: internalapi
                    metallb.universe.tf/allow-shared-ip: internalapi
                    metallb.universe.tf/loadBalancerIPs: 172.17.0.89
                spec:
                  type: LoadBalancer
          networkAttachments:
            - storage
EOF
Configuring DNS

Point the DNS service MetalLB VIP at the lab ctlplane address and use the cluster DNS resolver.

cat > environments/demo-env/controlplane/controlplane-patches/patch-dns.yaml <<'EOF'
# DNS for the lab: ctlplane MetalLB VIP and cluster DNS resolver.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  dns:
    $patch: replace
    template:
      override:
        service:
          metadata:
            annotations:
              metallb.universe.tf/address-pool: ctlplane
              metallb.universe.tf/allow-shared-ip: ctlplane
              metallb.universe.tf/loadBalancerIPs: 172.22.0.89
          spec:
            type: LoadBalancer
      options:
      - key: server
        values:
        - 172.30.0.10
      replicas: 1
EOF
Configuring MetalLB shared IPs

Move Keystone, Neutron, and Placement onto the shared internalapi VIP used in the lab (.89).

cat > environments/demo-env/controlplane/controlplane-patches/patch-metallb-ips.yaml <<'EOF'
# Shared internalapi MetalLB VIP (.80 -> .89) for identity/network/placement APIs.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  keystone:
    $patch: replace
    apiOverride:
      route: {}
    template:
      override:
        service:
          internal:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/allow-shared-ip: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.89
            spec:
              type: LoadBalancer
      databaseInstance: openstack
      secret: osp-secret
  neutron:
    $patch: replace
    apiOverride:
      route: {}
    template:
      override:
        service:
          internal:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/allow-shared-ip: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.89
            spec:
              type: LoadBalancer
      databaseInstance: openstack
      secret: osp-secret
      networkAttachments:
      - internalapi
  placement:
    $patch: replace
    apiOverride:
      route: {}
    template:
      override:
        service:
          internal:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/allow-shared-ip: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.89
            spec:
              type: LoadBalancer
      databaseInstance: openstack
      secret: osp-secret
EOF
Scaling down for the lab

Reduce Memcached and RabbitMQ from the catalog HA defaults to a lab-sized footprint.

cat > environments/demo-env/controlplane/controlplane-patches/patch-scale.yaml <<'EOF'
# Scale down HA replicas for the single-node lab footprint.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  memcached:
    $patch: replace
    templates:
      memcached:
        replicas: 1
  rabbitmq:
    $patch: replace
    templates:
      rabbitmq:
        override:
          service:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.85
            spec:
              type: LoadBalancer
      rabbitmq-cell1:
        override:
          service:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.86
            spec:
              type: LoadBalancer
EOF
Configuring the Cinder NFS backend

Replace the catalog volume1 placeholder with the lab NFS volume backend.

cat > environments/demo-env/controlplane/controlplane-patches/patch-cinder-nfs.yaml <<'EOF'
# Cinder NFS backend for the lab (replace catalog volume1 placeholder).
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  cinder:
    $patch: replace
    enabled: true
    apiOverride:
      route: {}
    template:
      databaseInstance: openstack
      secret: osp-secret
      cinderAPI:
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
      cinderScheduler:
        replicas: 1
      cinderBackup:
        networkAttachments:
        - storage
        replicas: 0 # backend needs to be configured
      cinderVolumes:
        nfs:
          networkAttachments:
          - storage
          customServiceConfig: |
            [nfs]
            volume_backend_name=nfs
            volume_driver=cinder.volume.drivers.nfs.NfsDriver
            nfs_snapshot_support=true
            nas_secure_file_operations=false
            nas_secure_file_permissions=false
          customServiceConfigSecrets:
          - cinder-nfs-config
EOF
Configuring Nova cells and OVN

Wire Nova cell message buses and noVNC attachments, and set the OVN controller NIC mapping.

cat > environments/demo-env/controlplane/controlplane-patches/patch-nova-ovn.yaml <<'EOF'
# Nova cells (message bus + noVNC) and OVN controller NIC mapping for the lab.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  nova:
    $patch: replace
    apiOverride:
      route: {}
    template:
      apiServiceTemplate:
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
      metadataServiceTemplate:
        override:
          service:
            metadata:
              annotations:
                metallb.universe.tf/address-pool: internalapi
                metallb.universe.tf/allow-shared-ip: internalapi
                metallb.universe.tf/loadBalancerIPs: 172.17.0.89
            spec:
              type: LoadBalancer
      schedulerServiceTemplate:
        customServiceConfig: ""
        replicas: 1
        resources: {}
      cellTemplates:
        cell0:
          cellDatabaseAccount: nova-cell0
          cellDatabaseInstance: openstack
          cellMessageBusInstance: rabbitmq
          hasAPIAccess: true
        cell1:
          cellDatabaseAccount: nova-cell1
          cellDatabaseInstance: openstack-cell1
          cellMessageBusInstance: rabbitmq-cell1
          noVNCProxyServiceTemplate:
            enabled: true
            networkAttachments:
            - internalapi
            - ctlplane
          hasAPIAccess: true
      secret: osp-secret
  ovn:
    $patch: replace
    template:
      ovnDBCluster:
        ovndbcluster-nb:
          dbType: NB
          networkAttachment: internalapi
          storageRequest: 10G
          replicas: 1
        ovndbcluster-sb:
          dbType: SB
          networkAttachment: internalapi
          storageRequest: 10G
          replicas: 1
      ovnController:
        networkAttachment: tenant
        nicMappings:
         datacentre: enp7s0
EOF
Enabling and disabling services

Enable Horizon for the lab UI and disable optional services that are not required.

cat > environments/demo-env/controlplane/controlplane-patches/patch-services.yaml <<'EOF'
# Enable Horizon; disable optional services not used in the lab.
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
  name: openstack-control-plane
spec:
  barbican:
    $patch: replace
    enabled: false
    apiOverride:
      route: {}
    template:
      databaseInstance: openstack
      secret: osp-secret
      barbicanAPI:
        replicas: 1
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
      barbicanWorker:
        replicas: 1
      barbicanKeystoneListener:
        replicas: 1
  horizon:
    $patch: replace
    enabled: true
    apiOverride:
      route: {}
    template:
      replicas: 1
      secret: osp-secret
  manila:
    $patch: replace
    enabled: false
    apiOverride:
      route: {}
    template:
      manilaAPI:
        replicas: 1
        networkAttachments:
        - internalapi
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
      manilaScheduler:
        replicas: 1
      manilaShares:
        share1:
          replicas: 1
          networkAttachments:
          - storage
  heat:
    $patch: replace
    apiOverride:
      route: {}
    cnfAPIOverride:
      route: {}
    enabled: false
    template:
      databaseInstance: openstack
      heatAPI:
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
        replicas: 1
      heatEngine:
        replicas: 1
        resources: {}
      secret: osp-secret
  ironic:
    $patch: replace
    enabled: false
    template:
      databaseInstance: openstack
      ironicAPI:
        replicas: 1
      ironicConductors:
      - conductorGroup: ""
        customServiceConfig: "# add your customization here"
        replicas: 1
        resources: {}
        storageClass: ""
        storageRequest: 10G
      ironicInspector:
        replicas: 1
      ironicNeutronAgent:
        replicas: 1
      secret: osp-secret
  telemetry:
    $patch: replace
    enabled: true
    template:
      metricStorage:
        enabled: false
        monitoringStack:
          alertingEnabled: true
          scrapeInterval: 30s
          storage:
            strategy: persistent
            retention: 24h
            persistent:
              pvcStorageRequest: 20G
      autoscaling:
        enabled: false
        aodh:
          passwordSelectors:
          databaseAccount: aodh
          databaseInstance: openstack
          secret: osp-secret
        heatInstance: heat
      ceilometer:
        enabled: true
        secret: osp-secret
      logging:
        enabled: false
        annotations:
          metallb.universe.tf/address-pool: internalapi
          metallb.universe.tf/allow-shared-ip: internalapi
          metallb.universe.tf/loadBalancerIPs: 172.17.0.89
        port: 10514
        targetPort: 10514
        cloNamespace: openshift-logging
        rsyslogQueueSize: 10000
        rsyslogQueueType: linkedList
        rsyslogRetries: 100
  swift:
    $patch: replace
    enabled: false
    proxyOverride:
      route: {}
    template:
      swiftRing:
        ringReplicas: 1
      swiftStorage:
        replicas: 1
        networkAttachments:
        - storage
      swiftProxy:
        replicas: 1
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
        networkAttachments:
        - storage
  octavia:
    $patch: replace
    enabled: false
    template:
      databaseInstance: openstack
      octaviaAPI:
        replicas: 1
      secret: osp-secret
  redis:
    $patch: replace
    enabled: false
  designate:
    $patch: replace
    enabled: false
    apiOverride:
      route: {}
    template:
      databaseInstance: openstack
      secret: osp-secret
      designateAPI:
        override:
          service:
            internal:
              metadata:
                annotations:
                  metallb.universe.tf/address-pool: internalapi
                  metallb.universe.tf/allow-shared-ip: internalapi
                  metallb.universe.tf/loadBalancerIPs: 172.17.0.89
              spec:
                type: LoadBalancer
      designateCentral:
        replicas: 1
      designateWorker:
        replicas: 0
        networkAttachments:
          - designate
      designateProducer:
        replicas: 0
      designateMdns:
        replicas: 0
        networkAttachments:
          - designate
      designateBackendbind9:
        replicas: 1
        storageClass: local-storage
        storageRequest: 10G
        networkAttachments:
          - designate
EOF

Wire the patches into kustomization.yaml

Update controlplane-patches/kustomization.yaml so it applies every thematic patch:

cat > environments/demo-env/controlplane/controlplane-patches/kustomization.yaml <<'EOF'
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../../../base/controlplane
patches:
  - path: patch-storage.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-dns.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-metallb-ips.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-scale.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-cinder-nfs.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-nova-ovn.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
  - path: patch-services.yaml
    target:
      kind: OpenStackControlPlane
      name: openstack-control-plane
EOF

Validate the patched overlay:

kustomize build environments/demo-env/controlplane/controlplane-patches

Validate the full environment overlay:

kustomize build environments/demo-env/controlplane

Commit the files created and push them to your GitHub repo:

git add environments/demo-env/controlplane/
git commit -m "Base and demo-env environment controlplane"
git push origin

Configure the controlplane validated patterns section in the values-rhoso-gitops.yaml file

On the bastion, in your forked pattern repository:

cd /home/lab-user/rhoso-gitops
Make sure to replace $your_github_id with your GitHub ID.
your_github_id=pnavarro
echo $your_github_id

Enable openstack-controlplane applications in the values-rhoso-gitops.yaml file:

cat >> overrides/values-rhoso-gitops.yaml <<EOF
  openstack-controlplane:
    repoURL: https://github.com/$your_github_id/osp-on-ocp-showroom
    path: gitops/environments/demo-env/controlplane
    targetRevision: main
    kustomize:
      patches:
        - target:
            kind: Job
            name: ctlplane
          patch: |-
            [
              {"op": "replace", "path": "/metadata/namespace", "value": "openshift-gitops"},
              {"op": "replace", "path": "/spec/template/spec/serviceAccountName", "value": "openshift-gitops-argocd-application-controller"}
            ]
EOF
After ./pattern.sh make install, openstack-controlplane can show OutOfSync even when Healthy and Setup complete. Argo compares your Git manifest to the live OpenStackControlPlane; the OpenStack operator adds hundreds of defaulted fields (apiTimeout, tls, passwordSelectors, and so on) that are not in these thematic patches. Without ignoreDifferences, you cannot eliminate that drift by editing only the lab patches above. Options: treat Healthy as success for the lab, export the reconciled CR from the cluster and align Git to that export, or use ignoreDifferences on the Argo Application.

Commit the files created and push them to your GitHub repo:

git add overrides/values-rhoso-gitops.yaml
git commit -m "Configure the controlplane validated patterns section in the values-rhoso-gitops.yaml file"
git push origin