What We Learn from Cyber Exercises, or Not Jim Duncan CSIRT Coordinator, BB&T 2007 June 20 2007 FIRST Annual Technical Conference – Sevilla, España
Overview • Background • Purpose of exercises • Examples of what we can learn… • And what we fail to learn (repeatedly) • Purpose of exercises, redux • Future improvements • What else? Duncan - What We Learn from Cyber Exercises, or Not 2 2007 FIRST Annual Technical Conference – Sevilla, España
Background • General cyber-security expertise with a special focus on incident response • Product security work as well as critical infrastructure protection issues (ISACs) • Varying amounts of involvement with many different cyber exercises including Cyber Storm, Livewire, various ISACs… • And many real disasters, too • Exercise details not included Duncan - What We Learn from Cyber Exercises, or Not 3 2007 FIRST Annual Technical Conference – Sevilla, España
Why Conduct Exercises? • So we know what to expect & what to do! ▪ People in disasters fall into 3 categories: • 10% -- 15% remain calm & act quickly • 15% or less COMPLETELY FREAK OUT! • Remainder are “stunned and bewildered”. [John Leach in Aviation, Space, and Environmental Medicine , 2004] ▪ Survivors anticipate & plan accordingly ▪ Do you review the safety card every time you fly? Duncan - What We Learn from Cyber Exercises, or Not 4 2007 FIRST Annual Technical Conference – Sevilla, España
What Can We Learn? • How will we react? • Who will be the real stakeholders? • What capabilities will succeed or fail? • What are the unforeseen obstacles? • What serendipity awaits us? • What better estimates can we calculate for cost-benefit analyses? Duncan - What We Learn from Cyber Exercises, or Not 5 2007 FIRST Annual Technical Conference – Sevilla, España
How Will We React? • Perhaps the most obvious goal is to test an organization’s response to a crisis • When handling new information, the brain slows down (e.g.,1977 Tenerife accident) • Under stress, it slows down even more! 45% of people “shut down” in a crisis • Minimize “milling”; time is very valuable • Mitigate “disbelief”; Act now! Duncan - What We Learn from Cyber Exercises, or Not 6 2007 FIRST Annual Technical Conference – Sevilla, España
Who Will Be the Real Stakeholders? • A critical point in the development of an incident response plan is to identify who has authority over an asset and who pays for it, too; they might not be the same unit • Exercises have the potential to expose that information, at times with great relief • Results should be included in plan review • Good justification for exercise Duncan - What We Learn from Cyber Exercises, or Not 7 2007 FIRST Annual Technical Conference – Sevilla, España
What Capabilities Succeed or Fail? • Text paging has failed, but not noticed because the monitoring system pages the operators to report problems • How many of you provision your support teams with toll-free numbers? • How many of you know that toll-free dialing won’t be available in a disaster? • Or that it can’t be dialed from outside the region (overseas)? Duncan - What We Learn from Cyber Exercises, or Not 8 2007 FIRST Annual Technical Conference – Sevilla, España
What are the Unforeseen Obstacles? • Another obvious reason for an exercise; many hope to find the “gotchas” before a real crisis occurs • Unfortunately, it’s based totally on luck • TIP: review your toll-free number uses • TIP: make sure your teams really know how to use PGP and have had their keys signed & published Duncan - What We Learn from Cyber Exercises, or Not 9 2007 FIRST Annual Technical Conference – Sevilla, España
What Serendipity Awaits Us? • Exercises are a good thing, and every one in which I have participated has produced valuable results with practical application • It’s easy to forget about positive stuff when we worry so much about negative things • One example: other teams rewrote my faux advisory and discovered aspects that hadn’t occurred to me earlier Duncan - What We Learn from Cyber Exercises, or Not 10 2007 FIRST Annual Technical Conference – Sevilla, España
What Estimates Can We Calculate? • Cyber security is catching up with metrics • Still horribly lacking with incident response • Exercises can expose unforeseen costs as well as unanticipated rewards • Both help to reinforce the value of CSIRTs to management up to the board room level • Also helps to reveal intangibles like sharing opportunities and potential future relationships Duncan - What We Learn from Cyber Exercises, or Not 11 2007 FIRST Annual Technical Conference – Sevilla, España
What We Fail To Learn • We fail to bring in the existing experts • We fail to discover existing stakeholders, groups, capabilities, relationships • We fail to assess authority & responsibility • We fail to appreciate the resources and time involved in anticipated responses • We fail to imagine the threats • We fail to keep it secure Duncan - What We Learn from Cyber Exercises, or Not 12 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Bring in the Experts, 1 • FS-ISAC tabletop considered a power failure at a telephone switching facility due to sabotaged diesel backup systems • Organizers unaware of battery systems and alternative fueling systems • Credibility was suspended and the participants were unmotivated • Value of exercise questionable Duncan - What We Learn from Cyber Exercises, or Not 13 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Bring in the Experts, 2 • Exercise planners spent considerable time on scenario involving railroad cars and the lack of real-time tracking ability; expected major fumbling by participants to resolve • In reality, locomotives are needed to move train cars & their locations are well known! • As before, credibility was suspended, etc. • Exercise value plummeted! Duncan - What We Learn from Cyber Exercises, or Not 14 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Discover Current Players • “FIRST” means many things to many folks ▪ The “Federal Incident Response Support Team” might not be who you think it is; insist on clarification • Misunderstandings about FIRST influence incorrect conclusions favoring involvement ▪ Information sharing ▪ Web of Trust Duncan - What We Learn from Cyber Exercises, or Not 15 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Assess Authority, 1 • ISACs are defined per CIP sector for information-sharing and analysis ▪ IT-ISAC handles information technology ▪ Telecom-ISAC handles telephony ▪ Who handles the ISPs? Each ISAC says the other has superior authority • And the ISPs “just want to be left alone, thank you…” Duncan - What We Learn from Cyber Exercises, or Not 16 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Assess Authority, 2 • The U.S. National Response Plan divides activities by defined functional areas • Emergency Support Function #2 handles telecom and information technology, while ESF#7 supports office equipment • When a server in a disaster agency’s remote field office starts attacking other systems, who will handle it? • Answer: “No one, immediately” Duncan - What We Learn from Cyber Exercises, or Not 17 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Anticipate Response Cost • Most plans (and thus most exercises) are oriented toward physical events • In cyber-space, most planning ignores the international angle (Cyber Storm is trying hard to get this right, and will succeed) • For example, for an international attack I was instructed to notify the Department of State’s 24-hour Watch Desk... • Guess how long that takes! Duncan - What We Learn from Cyber Exercises, or Not 18 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail to Imagine Threats • Following Hurricane Katrina, IT/Telecom restoration initially followed rules oriented toward public safety, not toward critical infrastructure protection issues • A major bank couldn’t get essential parts for back-office transaction processing • “Instant cash” was unusable because bank was completely unreachable Duncan - What We Learn from Cyber Exercises, or Not 19 2007 FIRST Annual Technical Conference – Sevilla, España
We Fail To Keep It Secure • Multi-site exercise connected to the Internet reduces cost but poses risks • Collected diverse set of security experts connect to web pages for net simulation • Traffic is not SSL-enabled nor tunneled • Links to “bad sites” were genuine and HTTP referrers had not been disabled! • To their credit, Cyber Storm staff fixed that within hours Duncan - What We Learn from Cyber Exercises, or Not 20 2007 FIRST Annual Technical Conference – Sevilla, España
Recommend
More recommend
