Every Information Security manager I speak with sooner or later comments to me on how finding good security people is so difficult and even mediocre security staff are so expensive. How do you address the challenge of keeping your organization secure in face of tight budgets and rising workloads while also trying to bring some resiliency into your organization?
A colleague of mine asked for my help to tackle this issue with his Computer Forensics team. He was overwhelmed because his company was paying an external provider a fortune for doing Forensics analysis. He was having trouble retaining his one CCFE because of burnout and, because of the external spend, he was also having trouble making his budget numbers. I’ll try to reference some of that discussion in explaining the ideas in more detail.
I had discussed applying the ideas from Software Development that address this kind of a challenge. The SDLC offers an avenue for standardizing and streamlining the development process. In fact, my teams have very successfully executed the Best Practices over the past 6 years to reduce the cost of development and improve quality. I wanted to apply the same ideas here and I feel that the same benefits await the CISOs that are willing to see it through.
The Approach: Our goal is to build processes that can be repeatable and tackle the basics. In all processes, there are some common and well understood steps and activities that can be executed as a matter of routine. The goal is to lay these out in some detail and farm them out to be executed by more readily available personnel. You’re building a Information Security version of the Software Factory – leaving only the more complex tasks, analysis and strategic decision making for your core “senior staff”.
I want to build a system where these senior staff members are intellectually challenged and have the time and opportunity to stay abreast of the developments in the Information Security space so they have avenues for personal growth. Even so, churn is inevitable and if a couple of these team members leave and it is 6 months before I find replacements, my CISO will have confidence that things won’t fall apart.
The junior staff that execute the more routine events will be honed in Security basics and good practices. They will be required to pay attention to detail and those that are talented will form the bench and can be groomed for more senior roles. Over time, this allows you to develop a more resilient organization.
Start with a Checklist. Regardless of what the portfolio is for your Security organization, you should be developing Process descriptions of the activities that you own. It always starts (and ends) with a Checklist that lays out the steps in the process and who owns each one. As with any Business Process, clear ownership of activities and decisions is very important. The checklist tells you the actions. A RASCI chart tells you ownership. The RASCI (Responsible, Accountable, Support, Consult, Inform) Chart is a document that lays out the roles that members in the organization play during the process.
Make sure that the checklist and RASCI are:
- Complete: they shouldn’t have gaps and should leave little to the interpretation by the person executing the process
- Unambiguous and Prescriptive: if not, you will have variations in how staff members will execute the process. I’m anal, I go as far as to check and double check the grammar of the instructions.
- Don’t assume too much about the knowledge of the person executing the process … expertise level can vary significantly depending upon the background of the personnel.
Aligned with the checklist is generally some training for staff members that walks them through the process. A recorded “demo” that takes them through a live session executing the checklist is ideal – with commentary added to provide context. This would be required training for all staff members prior to them working on a project. And periodically they will be required to have a refresher as well.
Executing the Checklist: I’d like reference back to my colleague’s request and consider how this would work for Forensics. The checklist would lay out SOPs for preserving the Data, documenting each step meticulously to ensure that the copy of the data that you’re analyzing is true to the original (complete with MD5 checksums), making sure that you consider each potential issue in turn (hidden data, recoverable files, cookie analysis, etc.) and not make assumptions along the way that may be unwarranted. If you use software like Paraben or FTK Suite, the setup of the tools and preparing the host computer on which you’ll execute the analysis should be documented. This will ensure, for example, that your analysis is not contaminated by previous analyses done on the system.
Another example would be when asked to do Pen-Testing for a site, the checklist documents steps to prepare for the testing, the types of tests we’d run, what to do with results (especially vulnerabilities found), mitigation steps, etc.. The goal is to have junior staffers complete the basic PenTesting exercises to address known issues and items. We establish standardized tools/techniques and processes for communications (so the Pen Testing is not mistaken for an attack). This can be extended to other areas within Information Security as well.
The senior Forensic Examiners’ role is to provide oversight and to come in to finish off the job – to see what the process may have missed, and perhaps to apply new approaches that they have learnt. Ditto for Senior PenTesters. Periodically, they compile the new forensic developments or site attack vectors/vulnerabilities and add it to the checklist so that the process covers those as well.
In both these cases, the junior staff members are less expensive and, if you’ve reached a level of confidence in the maturity of your organization, they can even be located overseas. I’d not want to source them from an outside vendor, because you’re effectively training the vendor’s resources. Their work should be periodically audited by the Senior staffers to ensure consistency. As part of the process, they should be logging metrics to measure effectiveness of their work as well as data about the sites they’ve audited.
Maturing the Process using Metrics: At this point, you will have slowly built an approach that leverages lower cost and more abundantly available (less skilled) staff to reduce your overall cost per Forensic or PenTesting request, and you’ve also lightened the load of your highly skilled resources in the process so they can focus on more value-add activities – like keeping up with new developments in the area.
One goal that we’re trying to achieve is repeat-ability. The process should result in the same outcome if repeated – by another staff member, or at a different time. For Forensics, given that the legal department is also often involved in these, having a robust and repeatable process ensures that the results you obtain will hold up under scrutiny.
Metrics need to be developed to measure your progress in achieving goals. The broader metric is to not only increase capacity of the organization, but also to reduce cost. These are easy to measure – you can tally up how many of your Forensic requests were handled internally, and how many went out to an external provider because you were short of internal bandwidth. Another important metric is the speed with which a Forensic Analysis is completed – because the findings in an initial analysis will allow you to decide if more detailed analysis is appropriate.
There are other metrics that can be borrowed – re-use would be a good metric, as would any of the audit metrics (having two staffers perform the analysis on one case to measure variance in results) … but you should look at the what you want to measure based upon the value that the activity is bringing to the organization. Adopting metrics without understanding its meaning and the value to the organization would be counter-productive.
Challenges: Aside from organizational resistance (because people are threatened by change), there are considerations that must be taken into account in setting expectations. By far the hardest to measure is effectiveness. This is where Information Security differs so much from Software Development – there is no real equivalent of a Traceability Matrix that will tell you that you’ve delivered the goods.
Be aware that, as you go further down this road, this will also inevitably lead to adopting other aspects of and disciplines within the SDLC – following on the heels of the use of Metrics for measuring Quality, there will be Iterative Process Optimization (Deming’s Cycle) and Project Management methodologies. But the CISOs that want to change the nature of their organization will have to lead their organizations through this type of change. Those are the visionary CISO’s that will Get It Right.