Patching is one of the most critical system admin activities, but it is also one of the most frequently neglected. The stated reasons may vary, but usually come down to a simple lack of patch management strategy, and an application to make patching easy. To get from bad/non-existent patching strategy to sound and successful patch management strategy, like so many others, starts with a single step.
Decide patch management is important
IT needs to patch, but they also have to want to patch. It’s far too easy to push patching off, especially when most patches require reboots, and no one wants to stay up until 3AM on a Saturday. Security needs to patch, since many exploits take advantage of flaws that have been patched. Management needs to patch since patched systems are more stable and reliable, and have better performance against SLAs. Everybody knows patching is important, so you all need is to agree to it, and senior management needs to support that. With senior management support’s go ahead, the rest of the steps are easy.
Implement a patch management solution
That senior management support must include funding for a patch management solution. One of the biggest reasons why patching is so painful to many is because they try to do it manually, or with a combination of Windows Server Update Services (WSUS) and scripts, or other home-grown solutions. A good patch management solution can automate all the work, letting you approve and schedule patching, and then just check on status when it’s done.
Include third-party applications
Patching operating systems, but not third-party applications, is like locking all the windows and leaving the front door open. It’s the applications that are what the users interact with, and that process data submitted from the web, and these must be patched just as diligently as your operating systems. Good patch management solutions can patch third party apps just as easily as operating systems.
Commit to testing
The vendors do a lot to test their patches, but ultimately it is your responsibility to test patches before deploying them. Testing requires users to run patches on their workstations, and on test versions of your application servers, and to run things through their paces to ensure there are no issues. Senior management needs to allocate resources to perform this testing each month. Your patch management app should be able to deploy patches to a set of test machines to make it easier to evaluate patches before pushing them to all of production.
Have a way to rollback
Even with testing it’s possible to encounter an issue with a patch, so make sure your patch management solution can automate the rollback of a patch.
Assess, log, report and audit
The biggest risk with manually patching is that something will be missed. Patch management applications should be able to assess all systems, log all patching, generate scheduled and on-demand reports, and you need to audit these to ensure all machines are patched and compliant.
Respect the window
Establish a patching window and make sure everyone knows what that is. Make that window one that takes priority over other actions, and set the expectation that the business will have to work around patching, and not vice-versa. Again, you will need senior management support to get this through, but you don’t want to delay critical security patches just because the marketing team wants to update the content of the website.
Patch with confidence
With a good patch management application, the support of senior management, a sound testing plan, and windows where you are able to patch, proceed with confidence. Patching is a good thing and shouldn’t be a cause of pain or suffering. Leave that for when patches are missed, because it’s a safe bet that if you miss a critical patch, the pain and suffering will come.
If your IT organization and senior management see that patching is important, advocate patching within the organization, allocate a modest amount of resources to patching, and then set the expectation that patching will be done, you will soon find that patching is a normal and easy part of systems administration activities. Take that first step with your patch management process and you will be well on your way.
For more on patch management, see Security Patch Management.