broker bro s new communication library
play

Broker Bros New Communication Library Robin Sommer ICSI / LBNL / - PowerPoint PPT Presentation

Aug 20, 2022 •188 likes •880 views

Broker Bros New Communication Library Robin Sommer ICSI / LBNL / Broala robin@icsi.berkeley.edu robin@broala.com http://www.icir.org/robin Bro Communication 2 Broker Bro Communication Exchanging Events Separates event generation from

  1. Broker Bro’s New Communication Library Robin Sommer ICSI / LBNL / Broala robin@icsi.berkeley.edu robin@broala.com http://www.icir.org/robin

  2. Bro Communication 2 Broker

  3. Bro Communication Exchanging Events Separates event generation from handling. 2 Broker

  4. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. 2 Broker

  5. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. Distributing State Provides shared view across instances. 2 Broker

  6. Bro Communication Exchanging Events Separates event generation from handling. Logging Remotely Aggregates logs on remote side. Distributing State Provides shared view across instances. 2 Broker

  7. Use Case: Bro Cluster Bro Manager Logs & Events Bro Bro Bro Worker 1 Worker 2 Worker 3 Load-balanced Traffic Stream 3 Broker

  8. Use Case: Bro Cluster Bro Bro Manager Proxy Logs & Events &synchronized State Bro Bro Bro Worker 1 Worker 2 Worker 3 Load-balanced Traffic Stream 3 Broker

  9. Use Case: External Integration Bro Events Application Application Application A B C 4 Broker

  10. Traditional Implementation Events Logs State Updates Bro A Bro B 5 Broker

  11. Traditional Implementation Events Logs State Updates Child Child Bro A Bro B Process Process 5 Broker

  12. Traditional Implementation Events Logs State Updates Child Child Bro A Bro B Process Process Events libbroccoli C Python Ruby Perl 5 Broker

  13. Traditional Implementation Events Logs State Updates It’s showing its age … Race conditions with &synchronized. Child Child Bro A Bro B No good persistence story. Process Process Not much control over data flow. Events Complex & inefficient protocol. Implementation deficiencies. libbroccoli Two independent implementations. C Python Ruby Perl 5 Broker

  14. Introducing the Broker Library Events Logs State Updates Bro A Bro B 6 Broker

  15. Introducing the Broker Library Events Logs State Updates Bro A Broker Broker Bro B 6 Broker

  16. Introducing the Broker Library Events Logs State Updates Bro A Broker Broker Bro B Broker C C++ Application Python 6 Broker

  17. Introducing the Broker Library Events Logs State Updates Broker Bro A Broker Broker Bro B Unified Library New protocol Broker Publish/subscribe C Limiting type system’s flexibility. C++ Application Explicit state operations. Python 6 Broker

  18. Exchanging Events with Broker my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  19. Exchanging Events with Broker Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  20. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver 7 Broker

  21. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver my_event(“Bro”, 42) 7 Broker

  22. Exchanging Events with Broker subscribe_to_events(“bro/event”) Topic: “bro/event” my_event(“Bro”, 42) my_event(“Bro”, 42) Bro Broker Broker Bro Sender Receiver my_event(“Bro”, 42) 7 Broker

  23. Exchanging Events with Broker Demo 8 Broker

  24. Forwarding Logs with Broker Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  25. Forwarding Logs with Broker Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  26. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver 9 Broker

  27. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Bro Broker Broker Bro Sender Receiver Log::write(…) 9 Broker

  28. Forwarding Logs with Broker subscribe_to_logs(“bro/log”) Topic: “bro/log” Log::write(…) Log::write(…) Bro Broker Broker Bro Sender Receiver Log::write(…) 9 Broker

  29. Data Stores with Broker 10 Broker

  30. Data Stores with Broker Broker maintains global, persistent key/value stores. 10 Broker

  31. Data Stores with Broker Broker maintains global, persistent key/value stores. Broker Bro Bro Broker Client Master 10 Broker

  32. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) Broker Bro Bro Broker Key Value Client Master Authoritative Version 10 Broker

  33. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) insert(s, 192.150.187.12, 21) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 Authoritative Version 10 Broker

  34. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Authoritative Version 10 Broker

  35. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Authoritative Version 10 Broker

  36. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) clone() Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 131.159.14.1 5 Cached Version Authoritative Version 10 Broker

  37. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) clone() Broker Bro Bro Broker { content } Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Cached Version Authoritative Version 10 Broker

  38. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  39. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  40. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  41. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) insert(…) Broker Bro Bro Broker Key Value Key Value Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker 192.150.187.43 1947 Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 10 Broker Cached Version

  42. Data Stores with Broker Broker maintains global, persistent key/value stores. s = create_master(“my_store”) c = create_clone(“my_store”) insert(s, 192.150.187.12, 21) n = lookup(c, 192.150.187.12) insert(s, 131.159.14.1, 5) insert(c, 192.150.187.43, 1947) insert(…) Broker Bro Bro Broker insert(…) Key Value Key Value insert(…) Client Master 192.150.187.12 21 192.150.187.12 21 131.159.14.1 5 131.159.14.1 5 Bro Broker 192.150.187.43 1947 192.150.187.43 1947 Cached Version Authoritative Version Key Value 192.150.187.12 21 131.159.14.1 5 192.150.187.43 1947 10 Broker Cached Version

  43. Data Stores with Broker Demo 11 Broker

  44. Data Store Features 12 Broker

  45. Data Store Features Data Types Supports Bro data types for keys & values. 12 Broker

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third

Who is Broker Direct? Broker Direct is a blend of Underwriting Agency, Wholesale Broker and Third Party Administrator focussed on the UK Personal and Commercial Lines market. The brand was established in 1997 with a unique proposition: a general

234 views • 8 slides
Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment

Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment

Broker Seminar Oct 17 Agenda Broker Solutions Manufacturer & Wholesaler of Investment Product 100% Broker focused Broker FX A new FX and International Payments Platform MiFID II & the New Consumer Protection

1.07k views • 80 slides
Training, Development and Communication at the University Library Introduction by Ilaria Moroni

Training, Development and Communication at the University Library Introduction by Ilaria Moroni

Training, Development and Communication at the University Library Introduction by Ilaria Moroni September, 2012 TDC - University Library Summary TDC Office: History, Staff and Activities Communication and Marketing Organizational

721 views • 27 slides
DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2

DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2

Broker & TPA JAD May 23, 2013 DRAFT DRAFT A service of Maryland Health Benefit Exchange Broker Information DRAFT DRAFT 2 Broker Topics Transaction files Broker XSD Group file 834 Individual initial enrollment with broker

730 views • 25 slides
Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and

Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and

Library - Stefano Giani august 2016 Effective searching with the UvA Library What does the UvA Library offer you? 1. Books and journals 2. Databases for subject-specific search: Communication and mass media complete , Web of science ,

845 views • 49 slides
SK Telecom 1 U U U U U U U- U - - communication - - - - - communication

SK Telecom 1 U U U U U U U- U - - communication - - - - - communication

U U U U U U U- U - - communication - - - - - communication communication communication communication communication communication communication Korea Mobile Market Trend for Convergence & Ubiquitous Communication 2004.

400 views • 16 slides
SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do?

SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do?

PCS Term Project: SMS Communication Library Lee, Chia-Peng 2015/12/09 Outline What can this lib do? Architecture Prerequisite Data structure How to use this lib? Example : FindMe! Error reporting Reference

460 views • 31 slides
XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous

XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous

XMLBlaster Rokey MOM (Message oriented Middleware) Mediator for messages (Broker) Asynchronous communication architecture Accessing to remote data through an internet or intranet Building applications with multiple servers and clients

116 views • 7 slides
UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE

UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE

UCX-PYTHON: A FLEXIBLE COMMUNICATION LIBRARY FOR PYTHON APPLICATIONS March 21, 2018 OUTLINE Motivation and goals Implementation choices Features/API Performance Next steps 2 WHY PYTHON-BASED GPU COMMUNICATION? Python use growing

315 views • 29 slides
K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs

K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs

K.SKOURAS CUSTOMS BROKER (AEOF CERTIFIED AGENT) Brief Company Presentation K.Skouras Customs Broker K.SKOURAS Customs Broker Kyriakos Skouras Customs Broker, is one of the leading agencies in the customs brokers industry in Greece, operating

534 views • 12 slides
ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company

ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company

ASN Broker Public Company Limited ASN Broker Public Company Limited Opportunity Day 26 November 2018 ASN Broker Public Company Limited Agenda Company Overview I Q3 2018 Financial Performance II Q3 2018 Key Developments III 2 ASN Broker Public

542 views • 16 slides
Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon

Broker Matthias Vallentin UC Berkeley International Computer Science Institute (ICSI) BroCon '16 Communication in Bro Exploiting Independent State For Network Intrusion Detection Tap Internal Internet Firewall Tap Network Robin Sommer

802 views • 26 slides
eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive

eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive

eBooks In Your Library: Discoverability, Communication and The Future of eBooks. An interactive event . Simply drop your business card into the bowl and listen out at the end of this session it could be you! Interactive = have your say

465 views • 42 slides
Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65

Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65

Washington Health Benefit Exchange Broker/Navigator Changes 4.1 Washington Healthplanfinder System Re l e as e April 11, 2017 65 Broker/Navigator Changes Broker/Navigator Partnership Enhancements Navigators can Terminate Partnership with

512 views • 12 slides
Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a

Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a

Software Engineering and Architecture Broker IPC over HTTP History In the 1990, there was a lot of hype about building distributed systems in the OO paradigm / Broker based CORBA Version 1 - 1991 Microsoft DCOM About the same

554 views • 22 slides
Library support for your teaching LAW LIBRARY All commencing students are enrolled in online

Library support for your teaching LAW LIBRARY All commencing students are enrolled in online

Library support for your teaching LAW LIBRARY All commencing students are enrolled in online units: ACE / CARS / ISE Academic Conduct Essentials (ACE) In addition, undergrads are enrolled in: Communication and Research Skills (CARS)

405 views • 16 slides
Welcome Child Concern / CAF Training Aims of the Day Introduce partner agencies to: Data

Welcome Child Concern / CAF Training Aims of the Day Introduce partner agencies to: Data

Welcome Child Concern / CAF Training Aims of the Day Introduce partner agencies to: Data protection & Information sharing Issues of Consent Child Concern Common Assessment Framework (CAF) Good assessments CAF outcomes

465 views • 22 slides
Implementation of flux fit in Cafana - To a large extent, it is done. Cafana oscillation - It

Implementation of flux fit in Cafana - To a large extent, it is done. Cafana oscillation - It

Implementation of flux fit in Cafana - To a large extent, it is done. Cafana oscillation - It builds a two dimensional histogram when loading trees, true energy vs. desired variable. - Each time when oscillates, the weights in above two

229 views • 6 slides
Evaluation of Productivity and Performance Characteristics of CCE CAF and UPC Compilers Sadaf

Evaluation of Productivity and Performance Characteristics of CCE CAF and UPC Compilers Sadaf

Evaluation of Productivity and Performance Characteristics of CCE CAF and UPC Compilers Sadaf Alam, William Sawyer, Tim Stitt, Neil Stringfellow, and Adrian Tineo , Swiss National Supercomputing Center (CSCS) Motivation Upcoming CSCS

549 views • 18 slides
CAF Benchmarking CAF Benchmarking Marco MEONI CERN - Offline Week C N O e Wee Alice Offline

CAF Benchmarking CAF Benchmarking Marco MEONI CERN - Offline Week C N O e Wee Alice Offline

CAF Benchmarking CAF Benchmarking Marco MEONI CERN - Offline Week C N O e Wee Alice Offline Thu, 11 Oct 2007 # / 25 Outline SpeedUp test: scalability SpeedUp test: scalability. Cocktail test: usability.

777 views • 25 slides
Study abroad programs = Academic programs Take classes seriously

Study abroad programs = Academic programs Take classes seriously

Academics Educa&on Abroad Pre-Departure Orienta&on Video Study abroad programs = Academic programs Take classes seriously Contribute to

419 views • 14 slides
METRONET Railcars Procurement and Maintenance Industry Briefing 12/12/2018 Momentum West: A

METRONET Railcars Procurement and Maintenance Industry Briefing 12/12/2018 Momentum West: A

METRONET Railcars Procurement and Maintenance Industry Briefing 12/12/2018 Momentum West: A Joint Venture between CAF and UGL METRONET Railcars - Industry Briefing 12/12/2018 2 CAF Global leader in rolling stock design and manufacturing

295 views • 11 slides
CAF 2.0: A Next-generation Coarray Fortran Laksono Adhianto, John Mellor-Crummey, and Bill

CAF 2.0: A Next-generation Coarray Fortran Laksono Adhianto, John Mellor-Crummey, and Bill

CAF 2.0: A Next-generation Coarray Fortran Laksono Adhianto, John Mellor-Crummey, and Bill Scherer Department of Computer Science, Rice University WPSE Workshop, Tsukuba, Japan 25 March 2009 QuickTime and a decompressor are needed to see

1.75k views • 14 slides
Disclaimer This presentation is based on CAOTs reference document, Working for the

Disclaimer This presentation is based on CAOTs reference document, Working for the

2017-12-05 Bridge, K. (2017, December 14). Working for the Canadian Armed Forces and Veterans Affairs Canada: Guidance for Occupational Therapists. In CAOT News & Resources Webinar series. Retrieved from

440 views • 14 slides

More recommend