Effective Modern Patch Management Processes and Best Practices for Patch Operations
Running a risk-based vulnerability management program is essential to maintaining a secure business computing environment. In a previous blog, “How Implementing Risk-Based Patch Management Prioritizes Active Exploits,” I provided perspective on how to prioritize vulnerabilities. Honing the operational aspect of securing your systems is essential to that process.
Conducting patch operations in your organization can be a complicated process. Even with some perspective on vulnerability priority, you still need to consider:
- The release cadence of patches.
- Supporting policies to make this process effective.
- Campaigns to deploy the updates.
- Service-level agreements (SLA) and compliance measurement.
This may seem like a lot to balance – but a flexible system that manages planned events and can account for the unplanned will put you in total control.
Do you have control?
There are some facets of patch releases you can plan for and others you can’t.
Take for example the cadence of security patch releases. On Patch Tuesday, the second Tuesday of each month, Microsoft attempts to release all its latest updates. These include updates for operating systems, Office and other user applications, development tools like Visual Studio and cloud components in Azure to name a few.
Patch Tuesday, which celebrated its 20th anniversary in October 2023, is the foundation for many patch programs – providing a point in time to distribute all the latest Microsoft and available third-party updates. No other vendor has had such a profound impact on driving organizational patch programs – and to this day the monthly patch cycle, anchored on Patch Tuesday, remains a standard in the industry.
Other vendors have attempted to follow suit and release their updates on a schedule:
- Oracle releases its Critical Patch Update once a quarter.
- Adobe typically releases its updates once a month, often in sync with Patch Tuesday.
- Google recently began releasing a single update each week.
However, most vendors release security updates as soon and as often they can to make sure they are addressing vulnerabilities as quickly as possible. This results in a random and never-ending stream of updates that need to be constantly prioritized and distributed across an organization.
Patch management process for policies and campaigns
Bringing the chaos of patch releases under control requires a clearly defined set of rules and an infrastructure to enforce them. In the patch realm these translate to policies and campaigns.
Patch policy requires a wide range of considerations beyond prioritizing vulnerabilities, including the impact of updates on business operations, applicability to different types of systems, degree of control over the updates and other factors.
Patch policies offer a study in contrast, depending on your business. For a server hosting critical business applications, its policy involves a clearly defined set of updates under strict configuration control, occurs only during a defined maintenance window, and always enforces a reboot upon completion to ensure the system is fully updated. The patch policy for a marketing user’s laptop identifies a series of applications with approved updates that may or may not be present and lets the user delay updates and reboot when convenient.
Campaigns take these policies into account but also bring control to the myriad patches constantly being released.
Modern patch management best practices require more frequent patching than just once a month. Google Chrome patches are being released weekly, and zero-day patches can be released at any time. A monthly campaign will leave many systems vulnerable for extended periods of time.
Establish three types of campaigns
A best practice is to establish three types of campaigns: regular maintenance, priority updates and critical deployments.
Regular maintenance campaign
Regular maintenance campaigns enforce the standard monthly ringed rollout of patches most organizations use today. This campaign includes:
- Initial testing in a controlled environment to ensure patches install as planned.
- Rollout to a larger pilot group of early adopters who are on the lookout to report issues.
- Rollout to the pre-defined groups of production systems to complete the overall distribution.
The patches in a maintenance campaign include security updates that are released less frequently, like Microsoft Patch Tuesday releases, as well as performance or application upgrades. The targeted systems in this campaign may also be those that have limited maintenance windows and cannot be interrupted without major business impact. Most patches will most likely fall into the priority updates campaign.
Priority updates campaign
The priority updates campaign is designed to quickly address those systems that have ongoing exposure to new vulnerabilities but can be updated more frequently. User systems running productivity applications and browsers fall under this campaign and are often at the highest risk with exposure to phishing, malware and ransomware.
The patches associated with this campaign are often the highest priority due to vulnerabilities with known exploitation but may also be of relatively low business impact requiring a browser or application restart. As a result, the policies may have a shorter test cycle before release and may be more quickly distributed to larger groups of systems which are not business critical, e.g. servers may not have a browser installed, but sales laptops do.
Zero-day response campaign
The zero-day response campaign is reserved for the “hair on fire” company- or industry-mandated emergency patch deployment that must be in place within a short period of time. This campaign takes precedence over all others.
The policy for this campaign could shorten the time or lower the standards between gated rollout – or it could ignore them altogether depending upon the SLA which must be met. Most importantly with zero-day response campaigns: This is still a controlled distribution of patches, and all activity is still reported to accurately track campaign events to completion.
Exposure time determines compliance
If compliance is measured in terms of fully patched machines, then under this metric most systems are only compliant for a limited number of hours each month. While technically accurate, this is a poor indicator to show the security of the system over time under a risk-based program. Showing the “exposure time” to a given vulnerability or group of vulnerabilities provides a better indicator of risk.
Here’s an example: CVE-2024-4761 was reported fixed in a Google Chrome update released May 14, which happened to be May Patch Tuesday. The next day, this Chrome patch was added to a priority update campaign that included a two-week distribution period across 500 systems in Group 1 and 1,000 systems in Group 2. Assuming most systems were successfully updated within that two-week window, a report would show when each system was updated, but more importantly how long each of these 1,500 systems remained unpatched – exposed to the vulnerability and at risk.
This is a simple one-vulnerability, one-patch example. But if there were multiple patches in the campaign with several vulnerabilities, the information could be aggregated to provide a more comprehensive campaign view. If multiple campaigns were run over the same period, the result could be overlaid or combined to provide an even more accurate risk assessment.
Armed with this data, you can show an auditor the true security state of your systems. Perhaps more importantly, you can start to assess the results and improve the effectiveness of your modern patch management operation. At that point, you are running the operation – it’s not running you.