Risktical Ramblings

This is no doubt one of many blog posts regarding the Verizon Business RISK Team “2009 Data Breach Investigations Supplemental Report” (DBISR). Below are a few of my thoughts.

1.    Quality of the Data. While it is neither the intent or spirit of the report to compare the usefulness of the information or the quality of the data to public data sources, I think it is important to recognize that the facts being collected by the Verizon team are generally more credible then the third-party sources that other public sources rely upon. In scenarios where I am trying to gather information about a breach or compiling a dataset for analysis – I am going to have a higher level of confidence in data / information from sources closer to the incident – then third parties just reporting on it. This does not mean that 3rd-party data is not legit – I am just suggesting the quality – from an accuracy and reliability perspective – is different and should be recognized.

2.    Data Overlap. On page 23 – is a table comparing the Verizon IR breaches and records lost to the equivalent DataLossDB values (keep in mind these are point in time values). The question I have is, how many of the 592 breaches in Verizon’s dataset are accounted for in the DataLossDB dataset? The reality is that in some US states (assuming all the breaches were in the US), data breach notification is not required, so an event can occur that does not result in breach notification to the consumer or the applicable State Attorneys General. If there were a difference between Verizon and DataLossDB – it only strengthens my confidence in their data because it contains credible data points not represented elsewhere (private consortium data aside).

3.    Threat Action “Profiles”. If you have not printed pages 5-21 and posted them on your cubicle / office wall – or recommended to your peers or other information security professionals – why not? Seriously. Threat actor / threat community profiles are such a valuable resource for security / risk practitioners to quickly reference, especially when we are dealing with dozens of threats and hundreds of controls. I can assure you that I will probably incorporate some of the DBSIR “threat action” profiles for some work I am doing in this same space with my employer – good job Verizon!

4.    Industry. My final observation is related to the industry and size of companies where breaches have occurred. I have blogged about this recently and I only mention this to remind folks that not every data point whether it is from Verizon, DataLossDB, PrivacyRights.Org, or other public / private data sources may be relevant to your industry or your company. The reality is that there are different expectations and regulatory requirements between industries and you have to keep that in mind while in the process of drawing conclusions from these types of reports.

Overall, two thumbs up to the Verizon Business RISK team. I commend them on their willingness to share this information and their desire to influence our industry as a whole.

In early October I began reviewing three external data repositories containing “loss event” data. I think it is important to state that what you are about to read is the result of me being guided by a real risk modeler at the company I work for. Modelers are very methodical, consistent, and have high expectations of quality – sort of like engineers. I understand information security, he understands modeling. I get to do the mundane work – he gets to build the mathematical relationships and distributions. No matter what though – I have to be able to explain everything in the model as well as maintain it moving forward. Thus, in this series, I want to share some observations and lessons I learned from the “gathering external” data exercise.

Really understand what you are looking to get from the data. It is too easy to jump into these data sets, perform some simple statistical calculations and then communicate outrageous findings to an audience. For me and my employer’s purpose we wanted to use “some” of this external data for use in a loss model. Specifically, to help establish a distribution of possible number of records that could be lost and potential loss magnitude per event in various types of security incidents. (Notice I said possible, not probable). The reality is that most companies do not have dozens let alone hundreds of loss events to develop loss models without needing to use external data. So, one of the benefits of using external data in a loss model is that it can really help understand worst-case loss magnitude also know as “tail risk”. Internal data may more influence the mean value of a loss model. For two of the data sources – dataloss.org and privacyrights.org – the number of records lost was the key data point. For the third and non-public data consortium source, the cost of security related events (not necessarily data loss events) was the most useful. Below are some considerations for narrowing down the number of data points in data set from all to “some”.

a.    Time. Technology and the regulatory landscape changes quickly. Thus, it is preferable to time limit data points to a period where a minimum level of technology was assumed as well as a consistent expectation of regulatory / industry standard requirements. For our purposes, we only chose data points dating back to 2005. Again, this time range will vary from model to model, person to person, company to company and industry to industry.

Note 1: One record in the dataloss.org set goes back to 1903. Seriously.

Note 2: In the dataloss.org data set dated 9/30/2010. There were 2013 data points. Using only records from 2005 to 9/30/2009; reduced the set down to 1945.

b.    Good Fit. Not all data points are a good fit to be included in your analysis. Security control expectations vary from industry to industry. Thus you need to have a way of methodically reviewing data points to determine which are a good fit. Below are just a few considerations:

i.    Industry. Most data sets are not industry specific – so they contain data points spanning all kinds of industries. The transportation industry has a different value proposition then the financial services industry. So, depending on your model – points outside your industry may not be relevant.

ii.    Service or Value Proposition. Somewhat related to industry but some services and value propositions are shared between industries. I think of health care insurance and property and casualty insurance. Both industries have to protect confidential information. This does not mean that if I am in the financial services industry that I would include ALL healthcare industry data points – it just means that I am acknowledging there is a shared value proposition and that some data points – depending on the loss form – can be used for my purposes.

iii.    Loss Form Categories. When I talk about loss form categories, I am referring mostly to BASEL II Operational Risk Categories (Level 1); “Internal fraud”, “External fraud”, “Employment Practices and Workplace Safety”, “Clients, Products & Business Practices”, “Damage to Physical Assets”, “Business Disruption and System failures” and “Execution, Delivery & Process Management”. Most data loss events will only map to a few of these categories and in some instances these categories may not even be applicable to your needs, your company or your industry – but classifying each data point to one of these categories or another category framework more relevant for your company / industry can allow you to refine your data set in a methodical and unbiased manner.

Note 3: After applying my good fit criteria, the total number of dataloss.org data points I am using for my model has been reduced from 1945 (note 2 above) down to 84.

Note 4: Of those 84 data points: 9 data points were categorized as “Internal Fraud”, 37 were categorized as “External Fraud” and 38 were categorized as “Execution, Delivery, and Process Management”.

c.    Duplicate Records. When you are using multiple data sets, you have to assume there is duplication of data points between data sets. This was definitely the case for the dataloss.org and privacyrights.org data sets. To compound matters, just expect that for a certain percentage of duplicate data points – the details might differ. This is not a super big deal – just understand that you will have duplicate data points and will have to choose one of the data points.

Note 5: Ok, there could be some duplicates where the variance in details is so wide and there is neither time to determine which one is more correct or there is not a valid source to determine which one is more accurate; you could throw them both out.

d.    Consistency. You have to be consistent in your approach to reviewing data points. Distributing the work between numerous people could be problematic if they are not all properly aligned on the goals of what you are doing and properly calibrated on determining if a data point meets the criteria for inclusion.

In the next post, I will focus more on “right-sizing” data points. In other words, adjusting data points to be commensurate with your particular company.

Note 6: Please do not take any of my remarks about dataloss.org or privacyrights.org having errors to be an attack against the fine folks that are maintaining those data sets. My intent for raising these points is related to taking personal responsibility for knowing the data points you are using to derive information from. It is too easy for our business partners and even others in the security industry to raise the “garbage in garbage out” argument when trying to understand risk or loss models.

***
Up front props:
1.    In the “risk universe” square, I used the “evolving change categorizations” from a Joshua Corman blog post found here.
2.    I heard the term “risk ecosystem” from Microsoft’s Mark Curphey in a video related to a risk repository web app they recently released called “Risk Tracker” (either here or here). I found the term to be valuable in the context of this blog post.
3.    The approach to the image above was not solely mine, I just embellished and sanitized on someone’s idea here at my employer.
***
Some terminology declarations:

I am using the word risk in a variety of capacities in this post.

In some cases, it is being used in the context of a threat (storm heading in my direction).

In other cases is being used in the context of a derived value; the probable magnitude and frequency of loss; $.

I am using the term “risk issue” or “risk finding” to mean a documented risk that requires a decision from management to either assume or mitigate.

Finally, in the database symbol titled “Risk Rep.” – that is short for “risk repository”.
***

I have recently been in a few conversations related to when a “risk” (or threat) becomes a “risk issue”. Most of these conversations have been with information security risk management executives; which implies “philosophying”, evangelizing, white boarding, and of course – excessive use of non-risk management analogies to reflect risk management concepts. In the end, these conversations turned out to be valuable because if forced the group to really understand when a risk (threat) becomes a risk issue in our environment. In other words, what are the various lenses we analyze threats or risk through to determine that we need to document a risk finding?

I will let you noodle the image and underlying concept on your own. However, below are a few parting points I would like to make.

1.    There is a difference between grandstanding on risks (threats) that pose no threat to your company versus managing risk issues within your own risk ecosystem. Think “solar storm heading directly to Mars” versus “a storm cell that is 10 miles away with 65 mile per hour winds headed directly towards us”.

2.    If a risk (threat) is important enough to grandstand on AND to begin mitigating – then it is no longer a risk, but a risk issue – and should be managed as such.

3.    Emerging risks – or threats – somewhat fall in between the two above. You may want to let management know about some potential exposure – but there is nothing that needs to be addressed today.

Feel free to share any thoughts you have!

Happy Birthday Mom!

My previous post was in early August (2009); a two post series on reputation risk. Since then, my professional and personal life has been pretty busy. Here is a quick update that will hopefully set some context for some upcoming (and hopefully more meaningful) posts between now and the end of the year.

No More PCI. OK, not 100% true – but let me explain. From about June 2008 until September 2009 – I helped lead a large information technology program (enterprise level program; containing numerous projects) to enhance some payment transaction applications as well as better manage compliance with the PCI DSS standard. Helping lead this program was truly one of the highlights of my information security / risk management career. It is not often in a big company that you get to be dedicated to a program for so long – as well as get to dive so deep to ensure that the solution being developed is not only compliant –but also secure. I transitioned away from the PCI program in early September to help lead some information risk management capability projects. I am still doing some ad-hoc / historical knowledge PCI consulting here and there – but for the most, I am not focused on PCI – and I am enjoying it.

So what am I doing now?

There are three efforts I am primarily working on.

Risk Quantification Methodology. Around April / May of 2008, I wrote a small proposal to our security leadership about transitioning from qualitative risk assessments to quantitative risk assessments. In late Q3 of 2008 – I was given the green light to lead a proof of concept of what I proposed earlier in 2008 – in my “spare time” when not dealing with PCI stuff. The proof of concept extended into early 2009. In late Q1 2009, I presented the POC findings to security leadership and shortly thereafter, a decision was made to transition to quantitative risk assessments. Since I was still primarily working on the PCI-related program – the risk quantification strategy was put on hold. Fast forward to September and now I have time to implement the risk quantification methodology and all the goodness that come with it (training, process changes, reporting, awareness, oversight, etc…). The goal is to have the methodology implemented in 2009 and focus on the related deliverables of reporting and oversight in 2010.

Risk Optimization Decision Model. This is really exciting and also dates back to Q4 of 2008. Very high level – I am working with a wicked smart data modeler to help build what I will refer to as a risk optimization model. The main purpose of the model is to aid decision making for information security (risk-related) funding decisions. An example of its use could be: A company has a lot of risk associated with “external fraud” and “internal fraud”; for example access control / authorization. The company has a loss model serving as a baseline. The company wants to invest $x dollars in a mitigation control that it expects to reduce loss frequency for “internal fraud” by 2% and “external fraud” by 10%. Based off the expected loss frequency reduction – what is the difference between the baseline loss model and the new loss model? Is there a risk reduction? If so, is the cost of the mitigation control a sound investment based of the risk reduction? I think there will be some interesting posts coming up related to this effort.

Risk Alignment. Around April of 2009, I was asked to represent the information risk management group (job family at my employer) in a working group with other risk assessment groups in our enterprise (Internal Audits, Financial Reporting Controls, SEC / FINRA, Privacy and Legal). I consider it a huge privilege and an even bigger growth opportunity. We have all heard of integrated operational risk management – and this working group is the epitome of that. Since my involvement with this working group, I have learned so much more about the company I work for as well as how other risk assessment programs assess and manage risk. The goal is alignment across risk assessment programs. Does that mean that every program assesses and manages exactly the same way – of course not. But there are opportunities to align on vernacular, risk concepts, risk categories, and in some cases risk repositories. I anticipate publishing a few blog posts that have been heavily influenced by my involvement with this alignment working group.

Finally, below are some books I have read since I took my vacation in late July. These books have nothing to do with IT or Information Security Risk Management whatsoever.

Crossfire by Andy McNab – Body guarding a TV crew on the streets of war-torn Basra, ex-deniable operator Nick Stone’s life is saved by a reporter’s swift action as a roadside bomb explodes. When the man later vanishes, Stone is asked to find him. The trail leads from Iraq to Bermuda, London and Kabul, the dark and brutal city where governments, terrorism and big business inexorably collide. Caught in the crossfire, his nightmare is only just beginning, for the hunter has suddenly become the hunted. . .

Brute Force by Andy McNab – Days after his car erupts in a ball of flame, Nick Stone narrowly cheats death a second time when a gunman opens fire on him from the back of a motorcycle. Who knows his movements? Who wants him dead, and why? Stone’s only chance of survival is to carry the fight to his attackers – but first he must uncover a trail of clues that leads from his own dark and complex past into the heart of a chilling conspiracy that threatens us all…Nick Stone’s eleventh adventure is McNab at his explosive best.

The Last Templar by Raymond Khoury – The war between the Catholic Church and the Gnostic insurgency drags on in this ponderous Da Vinci Code knockoff. The latest skirmish erupts when horsemen dressed as knights raid New York’s Metropolitan Museum of Art, lopping off heads and firing Uzis as they go. Their trail leads FBI agent Sean Ryan and fetching archeologist Tess Chaykin to the medieval crusading order of the Knights Templars. Anachronistic Gnostic champions of feminism and tolerance against Roman hierarchy and obscurantism, the Templars, they learn, discovered proof that Catholic dogma is a “hoax” and were planning to use it to unite all religions under a rationalist creed that would usher in world peace.

Moscow Rules by Daniel Silva – The death of a journalist leads Israeli spy Gabriel Allon to Russia, where he finds that, in terms of spycraft, even he has something to learn if he wants to prevent a former KGB colonel from delivering Russia’s most sophisticated weapons to al-Qaeda.

The Defector by Daniel Silva – Six months after the dramatic conclusion of Moscow Rules, Gabriel has returned to the tan hills of Umbria to resume his honeymoon with his new wife, Chiara, and restore a seventeenth-century altarpiece for the Vatican. But his idyllic world is once again thrown into turmoil with shocking news from London. The defector and former Russian intelligence officer Grigori Bulganov, who saved Gabriel’s life in Moscow, has vanished without a trace. British intelligence is sure he was a double agent all along, but Gabriel knows better. He also knows he made a promise. “If an injury has to be done to a man it should be so severe that his vengeance need not be feared.”