vast
play

VAST A Unified Platform for Interactive Network Forensics Matthias - PowerPoint PPT Presentation

Apr 05, 2024 •236 likes •1.01k views

VAST A Unified Platform for Interactive Network Forensics Matthias Vallentin 1 , 2 Vern Paxson 1 , 2 Robin Sommer 2 , 3 1 UC Berkeley 2 International Computer Science Institute (ICSI) 3 Lawrence Berkeley National Laboratory (LBNL) March 17, 2016

  1. VAST A Unified Platform for Interactive Network Forensics Matthias Vallentin 1 , 2 Vern Paxson 1 , 2 Robin Sommer 2 , 3 1 UC Berkeley 2 International Computer Science Institute (ICSI) 3 Lawrence Berkeley National Laboratory (LBNL) March 17, 2016 USENIX NSDI 1 / 28

  2. Omnipresent Data Breaches 2 / 28

  3. Breach Timeline Detection Compromise Forensics Time 3 / 28

  4. Breach Timeline Detection Compromise Time 3 / 28

  5. Breach Timeline Detection ? Compromise Time 3 / 28

  6. Network Forensics — Characteristics 4 / 28

  7. Network Forensics — Characteristics 4 / 28

  8. Network Forensics — Characteristics Organization 4 / 28

  9. Network Forensics — Characteristics 4 / 28

  10. Network Forensics — Characteristics 4 / 28

  11. Network Forensics — Characteristics 4 / 28

  12. Network Forensics — Characteristics ? 4 / 28

  13. Network Forensics — Characteristics Interactive data exploration ◮ Iterative query refinement ◮ High-dimensional search ? 4 / 28

  14. Network Forensics — Characteristics Interactive data exploration ◮ Iterative query refinement ◮ High-dimensional search Disparate data access ◮ Temporal ◮ Spatial ? 4 / 28

  15. Network Forensics — Characteristics Interactive data exploration ◮ Iterative query refinement ◮ High-dimensional search Disparate data access ◮ Temporal ◮ Spatial Massive data volumes ? ◮ 50–100K events/sec ◮ 10s TBs/day 4 / 28

  16. Log Example — Bro Connection Log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path conn #open 2016-01-06-15-28-58 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_.. #types time string addr port addr port enum string interval count count string bool bool count string 1258531.. Cz7SRx3.. 192.168.1.102 68 192.168.1.1 67 udp dhcp 0.163820 301 300 SF - - 0 Dd 1 329 1 328 (empty) 1258531.. CTeURV1.. 192.168.1.103 137 192.168.1.255 137 udp dns 3.780125 350 0 S0 - - 0 D 7 546 0 0 (empty) 1258531.. CUAVTq1.. 192.168.1.102 137 192.168.1.255 137 udp dns 3.748647 350 0 S0 - - 0 D 7 546 0 0 (empty) 1258531.. CYoxAZ2.. 192.168.1.103 138 192.168.1.255 138 udp - 46.725380 560 0 S0 - - 0 D 3 644 0 0 (empty) 1258531.. CvabDq2.. 192.168.1.102 138 192.168.1.255 138 udp - 2.248589 348 0 S0 - - 0 D 2 404 0 0 (empty) 1258531.. CViJEOm.. 192.168.1.104 137 192.168.1.255 137 udp dns 3.748893 350 0 S0 - - 0 D 7 546 0 0 (empty) 1258531.. CSC2Hd4.. 192.168.1.104 138 192.168.1.255 138 udp - 59.052898 549 0 S0 - - 0 D 3 633 0 0 (empty) 1258531.. Cd3RNm1.. 192.168.1.103 68 192.168.1.1 67 udp dhcp 0.044779 303 300 SF - - 0 Dd 1 331 1 328 (empty) 1258531.. CEwuIl2.. 192.168.1.102 138 192.168.1.255 138 udp - - - - S0 - - 0 D 1 229 0 0 (empty) 1258532.. CXxLc94.. 192.168.1.104 68 192.168.1.1 67 udp dhcp 0.002103 311 300 SF - - 0 Dd 1 339 1 328 (empty) 1258532.. CIFDQJV.. 192.168.1.102 1170 192.168.1.1 53 udp dns 0.068511 36 215 SF - - 0 Dd 1 64 1 243 (empty) 1258532.. CXFISh5.. 192.168.1.104 1174 192.168.1.1 53 udp dns 0.170962 36 215 SF - - 0 Dd 1 64 1 243 (empty) 1258532.. CQJw4C3.. 192.168.1.1 5353 224.0.0.251 5353 udp dns 0.100381 273 0 S0 - - 0 D 2 329 0 0 (empty) 1258532.. ClfEd43.. fe80::219:e3ff:fee7:5d23 5353 ff02::fb 5353 udp dns 0.100371 273 0 S0 - - 0 D 2 369 0 0 1258532.. C67zf02.. 192.168.1.103 137 192.168.1.255 137 udp dns 3.873818 350 0 S0 - - 0 D 7 546 0 0 (empty) 1258532.. CG1FKF1.. 192.168.1.102 137 192.168.1.255 137 udp dns 3.748891 350 0 S0 - - 0 D 7 546 0 0 (empty) 1258532.. CNFkeF2.. 192.168.1.103 138 192.168.1.255 138 udp - 2.257840 348 0 S0 - - 0 D 2 404 0 0 (empty) 1258532.. Cq4eis4.. 192.168.1.102 1173 192.168.1.1 53 udp dns 0.000267 33 497 SF - - 0 Dd 1 61 1 525 (empty) 1258532.. CHpqv31.. 192.168.1.102 138 192.168.1.255 138 udp - 2.248843 348 0 S0 - - 0 D 2 404 0 0 (empty) 1258532.. CFoJjT3.. 192.168.1.1 5353 224.0.0.251 5353 udp dns 0.099824 273 0 S0 - - 0 D 2 329 0 0 (empty) 1258532.. Cc3Ayyz.. fe80::219:e3ff:fee7:5d23 5353 ff02::fb 5353 udp dns 0.099813 273 0 S0 - - 0 D 2 369 0 0 5 / 28

  17. Existing Solutions MapReduce (Hadoop) ✓ Scalability ✗ Batch-oriented: no iterative, exploratory analysis 6 / 28

  18. Existing Solutions MapReduce (Hadoop) ✓ Scalability ✗ Batch-oriented: no iterative, exploratory analysis In-Memory Cluster Computing (Spark) ✓ Efficient & complex analysis ✗ Thrashing when working set does not fit in aggregate memory 6 / 28

  19. Contribution VAST V isibility A cross S pace and T ime 7 / 28

  20. Contribution VAST V isibility A cross S pace and T ime Architecture ◮ Performance : concurrent & modular design ◮ Scaling : intra-machine & inter-machine ◮ Typing : strong & rich 7 / 28

  21. Contribution VAST V isibility A cross S pace and T ime Architecture ◮ Performance : concurrent & modular design ◮ Scaling : intra-machine & inter-machine ◮ Typing : strong & rich Implementation ◮ Composition : high-level bitmap indexing framework ◮ Adaptation : fine-grained component flow-control ◮ Asynchrony : finite state machines for query execution 7 / 28

  22. Outline 1. Architecture 2. Implementation 3. Evaluation

  23. VAST Architecture — Single Machine 8 / 28

  24. VAST Architecture — Single Machine node archive 10.0.0.1 10.0.0.254 53/udp 10.0.0.2 10.0.0.254 80/tcp source importer exporter sink index 8 / 28

  25. VAST Architecture — Ingestion 10.0.0.1 53/udp 10.0.0.2 80/tcp … generate event batch source meta type 10.0.0.1 53/udp meta type 10.0.0.2 80/tcp 9 / 28

  26. VAST Architecture — Ingestion 10.0.0.1 53/udp assign IDs 10.0.0.2 80/tcp … generate event batch source importer meta type 10.0.0.1 53/udp meta type 10.0.0.2 80/tcp 9 / 28

  27. VAST Architecture — Ingestion archive compress batch 10.0.0.1 53/udp assign IDs 10.0.0.2 80/tcp … generate event batch source importer meta type 10.0.0.1 53/udp meta type 10.0.0.2 80/tcp 9 / 28

  28. VAST Architecture — Ingestion archive compress batch 10.0.0.1 53/udp assign IDs 10.0.0.2 80/tcp … generate event batch source importer meta type 10.0.0.1 53/udp meta type 10.0.0.2 80/tcp index 9 / 28

  29. VAST Architecture — Ingestion archive compress batch 10.0.0.1 53/udp assign IDs 10.0.0.2 80/tcp … append data to bitmap index generate event batch type source importer 10.0.0.1 53/udp 10.0.0.2 80/tcp meta type 10.0.0.1 53/udp meta type 10.0.0.2 80/tcp index 9 / 28

  30. VAST Architecture — Index index meta index partition partition partition 10 / 28

  31. VAST Architecture — Index index meta index partition partition partition conn 10.0.0.2 53/udp 8.8.4.4 53/udp “dns” indexer 10 / 28

  32. VAST Architecture — Querying exporter X in 10.0.0.0/8 || X == 80/tcp 11 / 28

  33. VAST Architecture — Querying exporter X in 10.0.0.0/8 || X == 80/tcp index 11 / 28

  34. VAST Architecture — Querying lookup bit vectors from partitions exporter X in 10.0.0.0/8 X in 10.0.0.0/8 || X == 80/tcp _ index X == 80/tcp 11 / 28

  35. VAST Architecture — Querying lookup bit vectors from partitions exporter X in 10.0.0.0/8 X in 10.0.0.0/8 || X == 80/tcp _ index X == 80/tcp 11 / 28

  36. VAST Architecture — Querying archive locate & ship event batch for ID lookup bit vectors from partitions exporter X in 10.0.0.0/8 X in 10.0.0.0/8 || X == 80/tcp _ index X == 80/tcp 11 / 28

  37. VAST Architecture — Querying archive decompress locate & ship batch event batch for ID candidate check lookup bit vectors from partitions exporter X in 10.0.0.0/8 X in 10.0.0.0/8 || X == 80/tcp _ index X == 80/tcp 11 / 28

  38. VAST Architecture — Querying archive decompress locate & ship batch event batch for ID candidate check lookup bit vectors from partitions exporter sink X in 10.0.0.0/8 X in 10.0.0.0/8 || X == 80/tcp _ index X == 80/tcp 11 / 28

  39. VAST Architecture — Querying archive decompress locate & ship batch event batch for ID candidate check lookup bit vectors from partitions exporter sink X in 10.0.0.0/8 meta type 10.0.0.1 53/udp X in 10.0.0.0/8 meta type 10.0.0.2 80/tcp || X == 80/tcp render results _ 10.0.0.1 53/udp index 10.0.0.2 80/tcp … X == 80/tcp 11 / 28

  40. VAST Architecture — Distributed 12 / 28

  41. VAST Architecture — Distributed 12 / 28

  42. VAST Architecture — Distributed 12 / 28

  43. VAST Architecture — Distributed 12 / 28

  44. VAST Architecture — Distributed 12 / 28

  45. VAST Architecture — Distributed 12 / 28

  46. VAST Architecture — Distributed 12 / 28

  47. VAST Architecture — Distributed 12 / 28

  48. Outline 1. Architecture 2. Implementation 3. Evaluation

  49. Indexing Basics — Tree Indexes 13 / 28

  50. Indexing Basics — Composition ( ) _ _ 14 / 28

  51. Indexing Basics — Composition ( ) _ _ 14 / 28

  52. Indexing Basics — Inverted Index A B C D 1 0 2 2 3 5 4 4 5 8 6 9 0 1 2 3 4 5 6 7 8 9 15 / 28

  53. Indexing Basics — Bitmap Index A B C D 0 0 1 0 0 1 1 0 0 0 2 0 0 1 1 3 1 0 0 0 1 0 1 0 4 5 0 1 1 0 0 1 2 3 4 5 6 7 8 9 16 / 28

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

IAMI / VAST, Vietnam Grid enabled Medical Informatics In Vietnam (VAST/IAMI, formerly VAST/

IAMI / VAST, Vietnam Grid enabled Medical Informatics In Vietnam (VAST/IAMI, formerly VAST/

IAMI / VAST, Vietnam Grid enabled Medical Informatics In Vietnam (VAST/IAMI, formerly VAST/ IOIT-HCM) Tran Van Lang, Dao Van Tuyet, et all tuyetdv@vast-hcm.ac.vn ISGC 2010, ASGC March 10, 2010 1 Outline 1 Overview 2 Activities of IAMI

337 views • 18 slides
RBINS VAST GTI collaboration A 10 years+ success story! RBINS and 2 VAST institutions:

RBINS VAST GTI collaboration A 10 years+ success story! RBINS and 2 VAST institutions:

RBINS VAST GTI collaboration A 10 years+ success story! RBINS and 2 VAST institutions: IEBR and VNMN Collaboration between taxonomists working on Insects First step: 2007 Visit of Mr Pham Hong Thai GTI project involving H.T. Pham

131 views • 12 slides
VAST CHALLENGE 2017 Bianca Barnucz & Stephanie Wegscheidl OVERVIEW VAST Challenge

VAST CHALLENGE 2017 Bianca Barnucz & Stephanie Wegscheidl OVERVIEW VAST Challenge

VAST CHALLENGE 2017 Bianca Barnucz & Stephanie Wegscheidl OVERVIEW VAST Challenge User Challenge 1 Challenge 2 Challenge 3 Grand Challenge Implementation Project Management & Outlook VAST CHALLENGE 2017

270 views • 16 slides
Nov. 5-16, 2007, IoIT (VAST) ACGRID Denis Perret-Gallix IoIT (VAST) Hanoi IN2P3/CNRS Nov. 5

Nov. 5-16, 2007, IoIT (VAST) ACGRID Denis Perret-Gallix IoIT (VAST) Hanoi IN2P3/CNRS Nov. 5

Nov. 5-16, 2007, IoIT (VAST) ACGRID Denis Perret-Gallix IoIT (VAST) Hanoi IN2P3/CNRS Nov. 5 2007 Sponsors IN2P3: Institute of Nuclear and Particule Physics, a CNRS institute ICT-ASIA network: French sponsored IT programmme: Foreign Affairs

507 views • 28 slides
VAST s 52nd Annual Meeting 2019 Meeting Description & Overview Health of the

VAST s 52nd Annual Meeting 2019 Meeting Description & Overview Health of the

VAST s 52nd Annual Meeting 2019 Meeting Description & Overview Health of the Association Leadership Summit Liability Insurance Signing Trails VAST Safety Ambassador Program Funding and Clubs TMAs Registrations Miles Groomed

579 views • 15 slides
VAST: Visibility Across Space and Time Architecture & Usage Matthias Vallentin

VAST: Visibility Across Space and Time Architecture & Usage Matthias Vallentin

VAST: Visibility Across Space and Time Architecture & Usage Matthias Vallentin matthias@bro.org BroCon August 19, 2014 2 / 27 3 / 27 4 / 27 Outline 1. Introduction: VAST 2. Architecture Overview Example Workflow: Query Data Model

485 views • 46 slides
WORK IN THE GIG ECONOMY Huma Humans a ns as a s a Se Service rvice @JeremiasPrassl VAST

WORK IN THE GIG ECONOMY Huma Humans a ns as a s a Se Service rvice @JeremiasPrassl VAST

WORK IN THE GIG ECONOMY Huma Humans a ns as a s a Se Service rvice @JeremiasPrassl VAST HETEROGENEITY @JeremiasPrassl VAST HETEROGENEITY @JeremiasPrassl VAST HETEROGENEITY @JeremiasPrassl VAST HETEROGENEITY @JeremiasPrassl One common

412 views • 39 slides
North-East Visualization and Analytics Center (NEVAC) VAST Grand Challenge Award: Data

North-East Visualization and Analytics Center (NEVAC) VAST Grand Challenge Award: Data

North-East Visualization and Analytics Center (NEVAC) VAST Grand Challenge Award: Data Integration Visualization and Collaboration in the VAST 2008 Challenge Don Pellegrino, Drexel University, don@drexel.edu October 23, 2008 | NEVAC VAST

173 views • 15 slides
U N I V E R S A L S T O R A G E I N T R O D U C T I O N M A Y 2 0 1 9 VAST COMPANY TI MELI NE

U N I V E R S A L S T O R A G E I N T R O D U C T I O N M A Y 2 0 1 9 VAST COMPANY TI MELI NE

U N I V E R S A L S T O R A G E I N T R O D U C T I O N M A Y 2 0 1 9 VAST COMPANY TI MELI NE Disruptive Concept Driving Historic Success FOUNDED ALPHA V1 GA COMPANY LAUNCH FASTEST GROWING EARLY STAGE IT COMPANY IN HISTORY DOZENS

196 views • 17 slides
A Presentation by About Avalon AVALON GROUP Avalon has a vast experience of almost three

A Presentation by About Avalon AVALON GROUP Avalon has a vast experience of almost three

A Presentation by About Avalon AVALON GROUP Avalon has a vast experience of almost three decades in the real estate industry and very ably utilizes this storehouse of knowledge to provide an ultra modern lifestyle at affordable prices

636 views • 34 slides
1 Examples of protein functionality: Enzymatic catalysis vast majority of reactions

1 Examples of protein functionality: Enzymatic catalysis vast majority of reactions

A primer on the structure and function of proteins Protein is derived from the Greek proteios , for of first rank (Jns J. Berzelius, 1838) 1 Examples of protein functionality: Enzymatic catalysis vast majority of

861 views • 13 slides
D I S C O V E R A vast choice of activities, excursions and adventures awaits Almyra guests, all

D I S C O V E R A vast choice of activities, excursions and adventures awaits Almyra guests, all

D I S C O V E R A vast choice of activities, excursions and adventures awaits Almyra guests, all designed to transport you to the unspoiled heart of Cyprus. From a jeep safari deep into the dazzling Akamas Peninsula and lush Troodos Mountains to

762 views • 29 slides
FOUR|PARK MANAGEMENT, LLC An Introduction and Guided Discussion EXECUTIVE SUMMARY VAST

FOUR|PARK MANAGEMENT, LLC An Introduction and Guided Discussion EXECUTIVE SUMMARY VAST

FOUR|PARK MANAGEMENT, LLC An Introduction and Guided Discussion EXECUTIVE SUMMARY VAST EXPERIENCE STEWARDS AND SHEPHARDS DYNAMIC STRUCTURE We We co collab aborat rate wi with our ur clie lients, s, We are We are steward ewards,

432 views • 16 slides
How Will We Populate the Semantic Web on a Vast Scale? Tom M. Mitchell Weam AbuZaki, Justin

How Will We Populate the Semantic Web on a Vast Scale? Tom M. Mitchell Weam AbuZaki, Justin

How Will We Populate the Semantic Web on a Vast Scale? Tom M. Mitchell Weam AbuZaki, Justin Betteridge, Andrew Carlson, Estevam R. Hruschka Jr., Bryan Kisiel, Burr Settles, Richard Wang Machine Learning Department Carnegie Mellon University

665 views • 34 slides
the private and multidisciplinary Services public sectors backgrounds Vast experience and

the private and multidisciplinary Services public sectors backgrounds Vast experience and

Experience in A Team with Customized the private and multidisciplinary Services public sectors backgrounds Vast experience and connections Partners directly with governments worldwide, involved with academic institutions, and think tanks

530 views • 33 slides
1 Ocean Green Energy MICROALGAE : A VAST SOURCE OF PRODUCTIVE ORGANISMS A MIXOTHOPHIC

1 Ocean Green Energy MICROALGAE : A VAST SOURCE OF PRODUCTIVE ORGANISMS A MIXOTHOPHIC

Fermentalg Atlantic Forum Presentation 21 Sept 2012 1 Ocean Green Energy MICROALGAE : A VAST SOURCE OF PRODUCTIVE ORGANISMS A MIXOTHOPHIC MICROORGANISM VACUOLES (EPA, ARA, DHA) EXOPOLYSACCHARIDES MEMBRANE LIPIDS ADN Internal protein

3.16k views • 11 slides
Unpacking tips and tricks Protector Techniques Conclusion Samuel Chevet w4kfu@lse.epita.fr

Unpacking tips and tricks Protector Techniques Conclusion Samuel Chevet w4kfu@lse.epita.fr

Unpacking tips and tricks Samuel Chevet Presentation Process Unpacking tips and tricks Protector Techniques Conclusion Samuel Chevet w4kfu@lse.epita.fr http://www.lse.epita.fr 12 February 2013 Why this talk ? Unpacking tips and tricks

439 views • 26 slides
Heads and Tails A Variable-Length Instruction Format Supporting Parallel Fetch and Decode Heidi

Heads and Tails A Variable-Length Instruction Format Supporting Parallel Fetch and Decode Heidi

Heads and Tails A Variable-Length Instruction Format Supporting Parallel Fetch and Decode Heidi Pan and Krste Asanovi MIT Laboratory for Computer Science CASES Conference, Nov. 2001 Motivation Tight space constraints Cost, power

679 views • 46 slides
Recorder 2.0: Efficient Parallel I/O Tracing and Analysis Chen Wang, Jinghan Sun and Marc Snir

Recorder 2.0: Efficient Parallel I/O Tracing and Analysis Chen Wang, Jinghan Sun and Marc Snir

Recorder 2.0: Efficient Parallel I/O Tracing and Analysis Chen Wang, Jinghan Sun and Marc Snir Kathryn Mohror and Elsa Gonsiorowski Department of Computer Science Center for Applied Scientific Computing University of Illinois at

307 views • 11 slides
OpenBox: A Software-Defined Framework for Developing, Deploying, and Managing Network Functions

OpenBox: A Software-Defined Framework for Developing, Deploying, and Managing Network Functions

OpenBox: A Software-Defined Framework for Developing, Deploying, and Managing Network Functions Yotam Harchol The Hebrew University of Jerusalem Joint work with Anat Bremler-Barr and David Hay THE HEBREW This research was supported by the

467 views • 24 slides
UNIX Commands CIS 218 Advanced UNIX Commands (UNIX) File/Directory information ls

UNIX Commands CIS 218 Advanced UNIX Commands (UNIX) File/Directory information ls

UNIX Commands CIS 218 Advanced UNIX Commands (UNIX) File/Directory information ls LiSt. Shows contents of current directory. cd Change Directory mv

264 views • 7 slides
HICAMP Bitmap A Space-Efficient Updatable Bitmap Index for In-Memory Databases Bo Wang,

HICAMP Bitmap A Space-Efficient Updatable Bitmap Index for In-Memory Databases Bo Wang,

HICAMP Bitmap A Space-Efficient Updatable Bitmap Index for In-Memory Databases Bo Wang, Heiner Litz, David R. Cheriton Stanford University DAMON14 Database Indexing Databases use precomputed indexes to speed up processing avoid

518 views • 18 slides
Kernel Address Space Layout Randomization http://outflux.net/slides/2013/lss/kaslr.pdf gholzer

Kernel Address Space Layout Randomization http://outflux.net/slides/2013/lss/kaslr.pdf gholzer

Kernel Address Space Layout Randomization http://outflux.net/slides/2013/lss/kaslr.pdf gholzer Linux Security Summit, New Orleans 2013 Kees Cook <keescook@google.com> (pronounced Case) Overview Classic Attack Structure

431 views • 15 slides
BGP Scanner Isolario BGP-MRT Data Reader: C library & tool Lorenzo Cogotti lorenzo.cogotti

BGP Scanner Isolario BGP-MRT Data Reader: C library & tool Lorenzo Cogotti lorenzo.cogotti

BGP Scanner Isolario BGP-MRT Data Reader: C library & tool Lorenzo Cogotti lorenzo.cogotti < at > alphacogs.com Luca Sani luca.sani < at > isolario.it Isolario Project What is a BGP route collector? Route collectors (RCs) are

1.01k views • 20 slides

More recommend