Vulnerability Management
Module goals
-
Understand reports in the Vulnerability Management Dashboard
-
Set and manage risk acceptance workflows
-
Create a simple report to email to stakeholders
The locations and size of your panels may vary depending on your screen size and zoom. |
Introduction to vulnerability management in RHACS
You start with the topic of vulnerability management because it is familiar to most security teams, even those without prior experience with containers or Kubernetes. The vulnerability management process helps protect the software supply chain and prevent known vulnerabilities from being used as an entry point into your applications.
In this lab, you explore the vulnerability management features of Red Hat® Advanced Cluster Security.
The overview provides several important reports, such as those detailing the most widespread or recent vulnerabilities, where my container images are coming from, and important vulnerabilities in OpenShift itself.
More important than fixing any one vulnerability is establishing a process to keep container images updated and to prevent the promotion through the pipeline of images with serious, fixable vulnerabilities.
Definition review
The following scores will be prevalent throughout the next sections. It’s important that you understand the pros and cons of how vulnerabilities are scored, ranked and triaged.
CVE
A Common Vulnerabilities and Exposures (CVE) is a publicly disclosed cybersecurity vulnerability or exposure. The concept of CVEs was introduced to provide a standardized method for identifying and cataloging security vulnerabilities in software and hardware. Each CVE is assigned a unique identifier, such as "CVE-2024-1234," which helps distinguish one vulnerability from another.
Here are the key components of a CVE:
-
CVE Identifier (CVE ID): A unique identifier assigned to a vulnerability, in the format "CVE-YYYY-NNNN", where "YYYY" represents the year the CVE was issued, and "NNNN" is a sequence number.
-
Description: A brief and precise description of the vulnerability, including details about the nature of the issue, the affected products, and the potential impact.
-
References: Links to additional information about the vulnerability, such as advisories, technical documentation, and other relevant resources. These references help users understand the context and implications of the vulnerability.
-
Date Entry Created: The date when the CVE entry was created, which helps track the timeline of when the vulnerability was identified and disclosed.
-
Status: Indicates whether the CVE entry is in a draft state, has been published, or has been modified. This helps users know the current state of the CVE information.
-
CVSS Score (optional but common): The Common Vulnerability Scoring System (CVSS) score provides a standardized way to measure the severity of the vulnerability. The score is typically represented as a number between 0 and 10, with higher scores indicating more severe vulnerabilities.
-
CWE ID (optional but common): The Common Weakness Enumeration (CWE) identifier categorizes the type of vulnerability. For instance, a CWE ID can indicate if the issue is a buffer overflow, SQL injection, etc.
-
Acknowledgments (optional): Recognition of individuals or organizations that reported or discovered the vulnerability.
Find out more at the MITRE website.
CVSS
The Common Vulnerability Scoring System (CVSS) is an open framework for communicating the characteristics and severity of software vulnerabilities.
They consist of the following:
Base Metrics:These metrics capture the characteristics of a vulnerability that are constant over time and across user environments.
Temporal Metrics: These metrics measure the characteristics of a vulnerability that change over time but not across user environments.
Environmental Metrics: These metrics measure the characteristics of a vulnerability that are unique to a user’s environment.
The CVSS score is computed using a formula that combines these metrics, giving more weight to the base metrics. Temporal and environmental metrics adjust the base score to reflect the current threat landscape and specific user environment considerations. The resulting scores are categorized as follows:
-
Low (0.1 - 3.9)
-
Medium (4.0 - 6.9)
-
High (7.0 - 8.9)
-
Critical (9.0 - 10.0)
The structured approach of CVSS allows organizations to assess the risk of vulnerabilities consistently and to make informed decisions regarding vulnerability management and remediation prioritization. However, it does have its drawbacks, such as a Lack of context sensitivity, a static nature, and a potential for misinterpretation, to name a few.
Find out more at the NIST website.
Red Hat Security Advisory (RHSA)
The RHSA stands for Red Hat Security Advisory. It is a notification issued by Red Hat, a leading provider of enterprise open-source solutions, to inform users about security vulnerabilities and the corresponding updates or patches available to mitigate these risks.
Key components of RHSA:
-
Advisory ID: A unique identifier for the advisory, typically including the year and a sequential number (e.g., RHSA-2024:1234).
-
Description: A summary of the vulnerability, including affected components and potential impacts.
-
Severity: The criticality of the vulnerability, often classified as Low, Moderate, Important, or Critical.
-
Affected Products: A list of Red Hat products and versions impacted by the vulnerability.
-
Resolution: Information about the patch or update that resolves the issue, including links to download the necessary updates.
-
References: Additional resources such as CVE (Common Vulnerabilities and Exposures) identifiers and links to detailed reports.
Find out more at the link: Red Hat website
Vulnerability Management 0n RHACS
The Vulnerability Management - Dashboard Tab (Deprecated)
Let’s continue by looking at our customers' primary use case for RHACS, which is the Vulnerability Management features and dashboard—a familiar topic for most security teams. We are going to start with the deprecated dashboard so that you can evaluate the vulnerability management workflows for yourself.
For the following section, please note that the order in which the images appear or the number of components affected may vary depending on versions and other applications running in the cluster. |
-
Click the Vulnerability Management tab, and then select Dashboard (Deprecated)
Buttons along the top of the interface will list details by;
-
CVEs
-
Node vulnerabilities
-
Image vulnerabilities and risk
The Application & Infrastructure button displays a list that takes you to reports by;
-
Clusters
-
Namespace
-
Deployment
-
Node Component
-
Image Component
Also, note the Filter CVES buttons that limit the reports to only Fixable CVEs or ALL CVEs.
The dashboard options provide several critical vulnerability breakdowns, such as:
-
Top risky deployments/images
-
Recently detected image vulnerabilities
-
Most common image vulnerabilities
Above the panel information, there are buttons to link you to all policies, CVEs, and images, and a menu to bring you to reports by cluster, namespace, deployment, and component. The vulnerability dashboard can be filtered by clicking the Fixable CVSS score button.
-
Locate the Top riskiest images panel.
Here, you can see the CVEs associated with containers currently running in the cluster.
-
In the Top riskiest images panel, click on the VIEW ALL button.
The images in this dashboard are listed here in order of risk,
Risk is based on a multitude of security issues, such as
-
the severity of the vulnerabilities present
-
in the components in the images
-
vulnerability impact
-
the image is active
Notice which images are more exposed. Not only can we see the number of CVEs affecting the images, but which of them are fixable? We can also see:
-
Creation date
-
Scan time
-
Image OS
-
Image status
-
How many deployments are using the vulnerable image
-
The total components in the image
-
Next, find and click on the image ctf-web-to-system:latest-v2. You will review the images' components and violations.
If you cannot find the ctf-web-to-system:latest-v2 image, use the search bar to filter for the specific image you want. Try searching by deployment and then entering ctf-web-to-system |
You can move on to the next section only when the dashboard displays the image below.
-
Above the Image Findings section, click on the Dockerfile tab:
The Dockerfile tab view shows the layer-by-layer view, and, as you can see, the most recent layers are also several years old. Time is not kind to images and components - as vulnerabilities are discovered, RHACS will display newly discovered CVEs. The layers that are listed as Source=OS are not showing CVE data since the CVE feeds are stale or do not have any information. However, the Python libraries that are added to the container are showing vulnerabilities. For example, the 'mercurial' package in the four layers.
These core vulnerability management features were essential to the success of our customers during the early RHACS days. Since clusters have grown larger and there are more and more vulnerabilities, the Vulnerability Management workflows in RHACS required an update.
Vulnerability Management - Workload CVE
The Vulnerability Management tab has recieved a lot of love over the past year. Vulnerability management in RHACS has become more focused on the categorizing vulnerabilities by workload so that it can scan RHEL CoreOS and node-level scanning and correlate it with platform and application vulnerabilities. This is because security teams want to understand at what software layer of vulnerability resides so they know what team it can reach out to to resolve a fix.
Let’s start off this section by reviewing a similar use case in the Vulnerability Management Workload CVE tab.
More important than fixing any vulnerability is establishing a process to keep container images updated and to prevent the promotion through the pipeline for images with serious, fixable vulnerabilities. RHACS displays this through the Top Risky Deployments by CVE and CVSS Score and takes the container’s configuration and vulnerability details to show you the most at risk deployments in your cluster.
The Workload CVE dashboard aims to show the same information as the deprecated vulnerability management tab but in a more scalable and systematic approach. In the UI, you will see thousands of vulnerabilities, over 200 images and over 300 deployments. This is because multiple images are being used across different deployments.
The numbers may be different in your environments. |
Now it’s time to find the same Java application and do some dissecting.
-
Click the dropdown and select deployment.
-
Then, filter for the ctf-web-to-system image.
You will get the same information from the previous section.
However, if you click the deployments tab, you will see the specific deployments with all these vulnerabilities. This ability to see the individual deployments as well as their images is crucial. When you’re talking about multiple clusters and thousands of vulnerabilities, you’re going to have the same workloads across different clusters, and you will need to drill down into the individual deployments.
-
Click on the CVE severity tab on the left and filter by critical and important vulnerabilities.
You should see that all of the critical and important vulnerabilities are fixable. This is mostly due to the age of the container image and its contents.
Container OS age and the age of its components are a massive correlating factor to the number of vulnerabilites present. Speed is security when it comes to containers. |
Now, if you care about a specific vulnerability, it is extremely useful to be able to see all of the components affected by that vulnerability.
-
In the Workload CVE tab, search for the log4shell CVE (CVE-2021-44228)
Make sure to select CVE in the dropdown. |
-
Click on the CVE
-
Scroll down and look at the impact of the CVE.
How many deployments are impacted? How many images? Why would there be different numbers?
This is a glaringly obvious example of a critical vulnerability. Take a moment to think about how your team would triage this.
We will move onto vulnerability reporting workflow but take some time to think about how you and your teams would handle a situation such as log4shell.
Platform CVEs
The platform CVEs page provides information about vulnerabilities in clusters within your system. This refers to OpenShift, AKS, GKE, and other Kubernetes distribution components. The goal is to help you understand responsibility and determine the impact.
Let’s go through a simple use case to demonstrate.
Procedure
-
Click Vulnerability Management → Platform CVEs.
You can filter CVEs by entity by selecting the appropriate filters and attributes. You can select multiple entities and attributes by clicking the right arrow icon to add another criteria. Depending on your choices, enter the appropriate information such as text, or select a date or object. The filter entities and attributes are listed in the following table. |
Entity | Attributes |
---|---|
Cluster |
Name: The name of the cluster. Label: The label for the cluster. Type: The cluster type, for example, OCP. Platform type: The platform type, for example, OpenShift 4 cluster. |
CVE |
Name: The name of the CVE. Discovered time: The date when RHACS discovered the CVE. CVSS: The severity level for the CVE. You can select from the following options for the severity level: - is greater than - is greater than or equal to - is equal to - is less than or equal to - is less than Type: The type of CVE: - Kubernetes CVE - Istio CVE - OpenShift CVE |
-
Search by Cluster → Platform Type → OPENSHIFT4_CLUSTER
-
Click on RHSA-2024:2672
Here, you can see all of the RHSA and the link to the Security Advisory
Being able to quickly understand the difference between what is your responsibility to fix and what is Red Hat’s responsibility is one way RHACS is making vulnerability management easier for our users. |
Node CVEs
You can identify vulnerabilities in your nodes by using RHACS. The same logic that applied to the Platform CVEs applies here.
Procedure
-
In the RHACS portal, go to Platform Configuration → Integrations.
-
Find and review RHSA-2024:1780
What is the CVE associated with this RHSA? What would you do to fix it?
Vulnerability reporting
Internal vulnerability reporting significantly enhances software security and quality by allowing development teams to address issues early, reducing the risk of breaches and failures. This proactive approach fosters a security-aware culture and encourages best practices. Efficient reporting channels also enable teams to prioritize and promptly fix critical vulnerabilities, leading to a more robust and reliable product, which boosts user trust and satisfaction.
In this next session, we will draft up a vulnerability report around the log for Shell vulnerability, making sure that it never gets deployed into our cluster in the future.
-
Create a collection that targets the log4shell CVE (CVE-2021-44228)
-
Ensure that any detection of this vulnerability will trigger a report to the designated user.
-
Let’s start by clicking on the Vulnerability Reporting tab.
-
Click the Create report button.
You will see that creating a report is a three step process. It requires you to configure the report parameters and the delivery destination, and then you have to review and create your report.
The configurable parameters are the following:
-
Report Name
-
Report Description
-
CVE severity
-
CVE status
-
Image type
-
CVEs discovered since (with a date)
-
And a Report scope.
-
Go ahead and fill out the information.
The collection scope is where you are going to target the two images with the vulnerability. |
-
When you are done, select the Select a collection dropdown
-
Click Create Collection
You can create collection rules by deployment, namespace and cluster. The collections are setup this way so that you can easily attach policies, vulnerability reports and notifications by the logical groupings of your organization.
Since we want to target only two deployments, let’s add the two to the Collection rules.
-
Add the two deployments to the rules (frontend & sonarcube). You should also see the impacted deployments in the collection results on the right side of the UI.
-
Review the collection
-
Hit Save
-
Click Next once you are back in the Configure report parameters tab
-
Next, create an email notifier that will send YOU an email every Monday to remind you about the vulnerabilities in these two deployments.
Don’t worry if you don’t want to enable the notification. The exercise is about going through the workflow. |
-
Once you are happy with the destination, select the Email template option. Using this option, you can customize the report to say whatever you desire. Here is your chance to be cheeky :)
-
Select a frequency. For example, weekly on Monday.
-
Hit Next
-
Review your masterpiece and click Create
However, you don’t have to wait until Monday to view the report.
-
Click the vertical ellipses on the right side of the UI and click Generate Download
You will not be able to download the report unless you’ve set up the email notifier and integration correctly. |
What would you do?
It should be fairly clear that our notification selection and collection were not the most efficient way to target a single vulnerability.
Before the next module, it would be great if you could think about how you would format your notifications and collections. Would they be based on labels or groups? Would you ensure that emails are in the Kubernetes and OpenShift deployment labels so that groups are easy to contact?
Remember, for sending these communications, you must consider the following questions:
-
What schedule would have the most impact when communicating with stakeholders?
-
Who is the audience?
-
Should you include only specific severity vulnerabilities in your report?
-
Should you include only fixable vulnerabilities in your report?