Should all incidents be treated the same? Seems like a simple question, but the answer can have big implications.

Think about an employee who contacts the service desk, complaining they can’t log onto their email. If the issue is due to a ‘stale’ password, dropped connection or configuration issue after an update for the email server, then the impact on the organization can be quantified to the lost productivity for the impacted employee or employees. But if the outage is due to some malicious activity and the email outage is the first indicator of a larger security breach potentially affecting more mission critical applications, data or infrastructure, then the impact to the organization can be very far reaching.

The Service Desk as IT’s Front Line for Security Incident Responses

For most service management teams, incident management is focused on resolving incidents quickly and getting employees back up and running again. That practice works for most incidents, but as with the above security breech example, security-related incidents should be handled differently because of the higher potential risks and impacts. Even with dedicated security teams, the service desk will be IT’s face to the organization’s employees and the front line when a security incident occurs, so it needs to be an integral part of a coordinated response. Throw in the fact that service teams often act as the hub for communication and coordination during major incidents and the case becomes even stronger.

Since the worst time to plan for how to deal with a major security incident is in the middle of one, service teams need to proactively plan and prepare for how to handle security incidents. Otherwise, as one IT director said, “You’re trying to build the airplane while on final approach.” Given the increasing frequency and threat of security-related attacks, for most organizations it’s a question of “when” the next major security incident will occur, versus the question of “if.”

Your Security Incident Management Plan

One suggestion for service teams developing their Security Incident Management (SIM) plan is to do so in coordination with not just other IT teams, but ideally also with other departments that may potentially need to be involved. Why? Because a major security incident may have business impacts well beyond the scope of the immediate IT issues, such as legal responsibilities, privacy risks, and governance questions. That’s not to say everyone should be involved with each security incident, but a response plan should be comprehensive in dealing with and mitigating risks from a wide range of potential impacts.

When you start developing your SIM plan with your extended team, define the roles and responsibilities for involved team members. Think about leveraging models like RACI (Responsible, Accountable, Consulted, Informed) to help map out these roles and responsibilities based on type and scope of security incidents. Find and agree on the touch points for each team, not just for the Security team. Don’t wait until a breach occurs to determine who needs to approve specific actions; make it part of your SIM plan, along with response times and alternative approvers so requests don’t “hang” during critical moments and are instead automatically routed for timely approvals.

Also think about what data and information you need to capture during an incident. This can help in the moment when trying to figure out the incident scope and response, but also afterwards when things settle down and you’d like to evaluate and improve your response.

Similar to pilots preparing for a flight, one tactic IT teams use are checklists for what needs to be done, including for operational tasks like isolation, shutdown, recovery, and testing for different types of services, applications, devices, assets, and CIs. They also leverage automation tools as much as possible to remove as many manual steps, checks, notices, and approvals as they can, reducing the risk of things “falling through the cracks” when in the middle of a security response, as well as ensure additional levels of governance.

Once you complete your SIM plan, train and regularly practice the plan with your staff. Train them to quickly identify and confirm possible security incidents. Use practice runs to check the thoroughness and effectiveness of your procedures, including mitigation and recovery, looking for areas to improve.

SIMilar Position as Your Disaster Recovery Plan

One IT directory thinks of their SIM plan similar to their Disaster Recovery (DR) plan—"it’s good to have it ready but you hope you don’t need to use it.” But should you encounter a major security incident and need to activate your SIM plan, be sure to invest time soon after the incident to determine how you would improve your response. Plan to review the incident before memories fade, and gather the data and information collected during the incident.

During a review with the response team, investigate and determine the background for the incident. Answer the “news reporter’s” questions of “Who, What, When, Where, How and Why” for the incident. Keep in mind some of the answers and information may be needed for future legal proceedings.

Also evaluate your organization’s overall response. Analyze and grade how quickly threat identification, mitigation, and recovery happened. Gauge the effectiveness of current defenses and training, look for areas to improve, and apply lessons learned to be better prepared for the next threat.

For major incidents, prepare a report for the executive team along the lines of an “After Action” report used in the military. Summarize some of the key findings from your review, including an analysis of the speed and effectiveness of the response. Don’t forget to include possible financial and legal implications your extended team can provide.

Sample Questions to Begin Your Post-Incident Review

Here are some sample questions you may want to consider asking in your review. There are more questions you may have, but these are meant to help you get started as you work to improve your response to security incidents:

  • What type of incident was it?
  • How was the incident first detected?
  • Was the severity initially gauged correctly?
  • How well did the response plan work? Any steps not followed? What steps helped? What steps didn’t help?
  • Was response leadership clear? Was it effective and timely? Does anything need to change?
  • Any data or insights that could have helped?
  • How well did the security infrastructure work? Are there improvement opportunities in vulnerability management?
  • Was communication among teams effective and timely? What worked well? What didn’t work well?
  • Any other teams who should have been included? At what stage?
  • What could be improved to handle the next incident? For all types of possible security threats?

Forewarned is Forearmed

Security breaches and incidents, or at least attempted ones, are bound to occur given today’s changing threat landscape. But being forewarned is forearmed. IT service teams—along with the rest of IT and the larger organization—can be better protected and prepared, with well-documented plans for a coordinated team that’s ready to respond to and mitigate the risks from future security incidents.