Network Management

Introduction

By default, all virtual machines are attached to the OpenShift software-defined network (SDN), which enables access from other workloads on the OpenShift cluster, including other VMs and any OpenShift native applications.

The SDN provides additional features for abstracting, connecting, and exposing applications in a controlled manner, whether deployed as VMs or Pods in the cluster. These include the Service and Route features of OpenShift.
OpenShift’s network policy engine allows the VM user or administrator to create rules which allow or deny network traffic to and from individual VMs or entire projects/namespaces.

However, virtual machines may also connect directly to one or more external networks, such as VLANs, when needed. This is in addition or in alternative to the SDN, which means that, for example, the administrator can connect to the VM from an external IP address, but the application communicates across the SDN with other VMs hosted by OpenShift Virtualization.

At a high level, this is done by configuring the host networking, by creating a NodeNetworkConfigurationPolicy such as creating a mode 4 (LACP) bond and a OVS bridge on top. With this networking infrastructure set up, we can define NetworkAttachmentDefinitions to allow VMs to connect to that bridge, and therefore, directly to the external network. The next section in this lab will cover those steps.

The OpenShift environment has already been configured with an OVS Bridge on each compute node your virtual machines will connect to, thus allowing for easy connectivity with/from outside network resources.

Goals

Create a network attachment definition
Connect a VM to the external network

Create a Network Attachment Definition

In order to use the OVS Bridge with your VM you need to create a Network Attachment Definition. (NAD) This is what tells OpenShift about the network and allows the virtual machines to connect to it. Network Attachment Definitions are specific to the project/namespace they’re created in, unless they’re created in the default project. This gives you, the administrator, the ability to control which networks are and aren’t available to users who have access to manage their own VMs. Once the Network Attachment Definition has been created, it can then be used by virtual machines when configuring their network adapters.

A network attachment definition instructs OpenShift to utilise an existing network device. In our case that device was previously created and is named br1.

How is this done?

To manage an OpenShift node’s network configuration you use a tool, available as an operator, called NMState. With NMState you can create network interfaces on OpenShift compute nodes using Kubernetes constructs or the NodeNetworkConfigurationPolicy (NNCP) wizard available in the OpenShift console. You can create devices and customize those so that they follow the naming conventions that suit your needs, and your network requirements. For more info about NMState, and to learn more about the host networking and how to view and manage the configuration, see the documentation here.

Before creating the network attachment definition, review the NNCP that was used to create the bridge it will use.

Ensure that you are working in the vmexamples project.
Navigate to Networking → NodeNetworkConfigurationPolicy:
Click br1:
Select the YAML tab and review the spec.desiredState section. In particular, note the value of spec.desiredState.ovn.bridge-mappings[0].localnet (localnet2); this value will be used in the network attachment definition:

With the bridge mapping name known, the network attachment definition can be created.

Navigate to Networking → Network Attachment Definitions and click Create network attachment definition:

Enter the following settings and click Create.

Name: vlan0
Description: l2 connection for vms
Network Type: OVN Kubernetes secondary localnet network

Bridge mapping: localnet2 (the value from the NNCP)

In most cases a single OVS bridge can support many Network Attachment Definitions each with their own designated VLAN number. In this lab we use an untagged network, so no VLAN number is required here, as such our Network Attachment Definition is labeled as vlan0.

Examine the details of the network attachment definition. Because this was created in the vmexamples project, it will be available only to attach to VMs that are in that project.

Connect a Virtual Machine to an External Network

Navigate to Virtualization → VirtualMachines, select the fedora01 VM. Click Configuration tab and then click the Network subtab:

Notice that the VM is currently using a single interface default which is connected to the Pod networking network. We can choose to modify this existing connection or add a new interface to the VM. Either action we choose currently requires either restarting or live migrating the VM. This is needed, because we need to move the VM into a new Kubernetes pod which has the new network interface attached.

Click the Add network interface button.
Change the interface Name to vlan0 and set the Network to vmexamples/vlan0 (the NAD that you created in the previous step). Then click the Save button.
Use the Actions menu or icon in the upper right corner to restart the VM. After rebooting, navigate to the Overview tab:
Once the machine restarts, you can see in the Network Interfaces section of the Overview screen that the vlan0 interface obtains a DHCP IP address from the flat network (192.168.64.0/18).

Before the next section of this lab, please repeat the actions to attach the fedora02 VM to the same vlan0 network. Instead of restarting the VM, live migrate it to another node to complete attaching the new NIC.

Using a MultiNetwork Policy

A multinetwork policy allows you to configure network access to a namespace and to define granular rules allowing ingress and egress from the namespace to enhance security of the applications and VMs that are running in the namespace.

This section of the lab is primarily performed through the CLI. You will need to ssh to your Bastion host where the CLI tools are already installed.

$ ssh lab-user@{bastion_public_hostname} -p {bastion_ssh_port}

The password is {bastion_ssh_password}

Create a MultiNetwork Policy

For this section of the lab we are going to create a MultiNetwork Policy that prevents all network traffic from reaching the VM’s that are attached to our vmexamples/vlan0 Network Attachment Definition including our VMs fedora01, and fedora02. We will then explictly allow one-way connectivity from fedora02 to fedora01 to show how we can fine tune network connections, even within the same namespace.

The IP addresses of your virtual guests will differ from those in the attached images or examples.

To begin, we are going to change the network configuration of the Fedora VMs to make their IP addresses static. This is required, because the policy will block traffic from the DHCP server to the VMs.

Navigate to the fedora01 console. If necessary, log in.
Examine the current IP addresses assigned to the system. Make a note of the IP address within the 192.168.64.0/18 subnet (probably on the enp2s0 network interface), as you will need it later in this module. Also note that the address has been dynamically assigned.
```
$ ip a
```
Display the NetworkManager connection profiles. Note the name of the connection profile (Wired connection 1) that corresponds to the network interface that you noted above.
```
$ nmcli con show
```
Modify the connection profile. Be sure to use the IP address that you noted above, not the address in the example. Also be sure to use the correct prefix length (18). After modifying the connection profile, restart NetworkManager to make the changes effective.
```
$ sudo nmcli con mod 'Wired connection 1' ipv4.method manual ipv4.addresses 192.168.64.36/18
$ sudo systemctl restart NetworkManager
```
Note that the IP address is no longer dynamic. (It is valid forever.)
Repeat the above steps on fedora02.
On the bastion host, edit the cluster network operator configuration to enable multi-network policy.
```
$ oc edit networks.operator.openshift.io cluster
```
Search for useMultiNetworkPolicy and ensure that it is set to true. Save and exit from the editor.

On the bastion host paste the following content into a new file called deny-all.yaml:

If vi/vim is messing yaml indentation, type :set paste to turn off automatic indenting. To turn back on, use :unset paste.

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: deny-by-default
  annotations:
    k8s.v1.cni.cncf.io/policy-for: vmexamples/vlan0
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []

Ping each of your Fedora VMs to confirm that you can currently connect to them. (This will also confirm that you are using the correct IP addresses.)

$ ping -c3 192.168.64.28
PING 192.168.64.28 (192.168.64.28) 56(84) bytes of data.
64 bytes from 192.168.64.28: icmp_seq=1 ttl=64 time=3.33 ms
64 bytes from 192.168.64.28: icmp_seq=2 ttl=64 time=0.957 ms
64 bytes from 192.168.64.28: icmp_seq=3 ttl=64 time=0.930 ms

--- 192.168.64.28 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.930/1.737/3.325/1.122 ms

$ ping -c3 192.168.64.29
PING 192.168.64.29 (192.168.64.29) 56(84) bytes of data.
64 bytes from 192.168.64.29: icmp_seq=1 ttl=64 time=3.12 ms
64 bytes from 192.168.64.29: icmp_seq=2 ttl=64 time=1.03 ms
64 bytes from 192.168.64.29: icmp_seq=3 ttl=64 time=0.870 ms

--- 192.168.64.29 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 0.870/1.673/3.118/1.023 ms

Apply the multinetwork policy with the following syntax:

$ oc create -f deny-all.yaml -n vmexamples
multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created

Now try again to ping the IP addresses your Fedora virtual machines; your ping attempts should now fail. (It may take a few seconds for the policy to become effective.)

$ ping -c3 192.168.64.36
PING 192.168.64.36 (192.168.64.36) 56(84) bytes of data.

--- 192.168.64.36 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2040ms

$ ping -c3 192.168.64.37
PING 192.168.64.37 (192.168.64.37) 56(84) bytes of data.

--- 192.168.64.37 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2034ms

Return to your OpenShift console, and click on Virtualization → VirtualMachines and select your fedora02 machine.
Click on the button to open it’s web console, and login with the provided credentials.
Attempt to ping the ip address for the fedora01 virtual machine, notice that it is also blocked, even though we are on the same subnet, in the same namespace. Leave the ping running.
Return to the bastion host console, and create a new file called allow-host.yaml, and paste in the following content:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name:  ingress-ipblock
  annotations:
    k8s.v1.cni.cncf.io/policy-for: vmexamples/vlan0
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress:
  - from:
    - ipBlock:
        cidr: <IP_ADDR_FROM_FEDORA02>/32
```
Make sure that you substitute the correct IP from the fedora02 VM and leave the the prefix length as 32. (We only want to allow traffic from this single IP address, not the subnet.)

Apply the policy using the following syntax:

$ oc apply -f allow-host.yaml -n vmexamples
multinetworkpolicy.k8s.cni.cncf.io/ingress-ipblock created

Attempt to ping from the bastion host. This attempt should still fail as we have not explictly allowed it.
Return to your fedora02 VM console, you should find that the ping has now resumed successfully.
Let’s clean up the policies for the next section.

For the next lab, it’s important that we clear out the two network policies we created.

Please run the following command:
```
$ oc delete -f allow-host.yaml -f deny-all.yaml -n vmexamples
```

Summary

In this section we learned a little bit more about how networking works in OpenShift Virtualization. We created a network attachment definition so that our VMs have network access from outside of the cluster. We also implemented a MultiNetwork Policy to demonstrate how we can secure connections to our VMs by only allowing specified hosts access.