Runaway complexity in Big Data And a plan to stop it Nathan Marz - PowerPoint PPT Presentation

Apr 25, 2023 •109 likes •1.33k views

Runaway complexity in Big Data And a plan to stop it Nathan Marz @nathanmarz 1 Agenda Common sources of complexity in data systems Design for a fundamentally better data system What is a data system? A system that manages the storage

Batch views are optimized for the queries they serve
Batch views • Batch-writable from MapReduce • Fast random reads • Examples: ElephantDB, Voldemort
Batch view database No random writes required!
Properties All Batch data view Function ElephantDB is only a few thousand lines of code Simple
Properties All Batch data view Function Scalable
Properties All Batch data view Function Highly available
Properties All Batch data view Function Can be heavily optimized (b/c no random writes)
Properties All Batch data view Function Normalized

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal

Thermal Runaway Warning signs and preventive actions Presented by Peter DeMar 1 If thermal Thermal runaway runaway does not scare you, then you are 2 Thermal runaway Brand new to the stationary battery industry Ignorant of thermal

360 views • 20 slides

Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality

Complexity, Big Data Science, and Happiness Complexity, Big Data Science, and Complexity Introduction Happiness Emergence Universality Symmetry Breaking Discrete Days, St. Michaels College, 2011 The Big Theory Revolution: Big Data

910 views • 74 slides

Data Streams & Communication Complexity Lecture 3: Communication Complexity and Lower Bounds

Data Streams & Communication Complexity Lecture 3: Communication Complexity and Lower Bounds Andrew McGregor, UMass Amherst 1/23 Basic Communication Complexity Three friends Alice, Bob, and Charlie each have some information x , y , z

1.11k views • 98 slides

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1,

Partnering with Foster Care and Runaway and Homeless Youth Programs October 1, 2019 October 1, 2019 2 Runaway and Homeless Youth Programs October 1, 2019 3 Runaway and Homeless Youth (RHY) Definitions Runaway youth: a person under 18

525 views • 34 slides

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The

Philosophical Ethics Philosophical Ethics an introduction George Matthews 2020 1 / 17 The case of the runaway trolley The case of the runaway trolley You are standing near a switch next a railroad track when you notice a runaway trolley

565 views • 34 slides

An Explanation for the High-Beta Runaway: the Non-Zonal Transition M.J. Pueschel many thanks to :

An Explanation for the High-Beta Runaway: the Non-Zonal Transition M.J. Pueschel many thanks to : P .W. Terry, F. Jenko, D.R. Hatch, W.M. Nevins, T. G orler, D. Told, A.E. White 55th Annual Meeting of the APS Division of Plasma Physics

304 views • 19 slides

Measurement and Simulation of Thermal Runaway Conditions in the LHC Interconnects L. Bottura and

Measurement and Simulation of Thermal Runaway Conditions in the LHC Interconnects L. Bottura and A. Verweij Based on work and many contributions from: L. Gaborit, P. Fessia, L. Fiscarelli, V. Inglese, G. Montenero, G. Peiro, H. Prin, C.

570 views • 45 slides

threshold conditions for runaway electron generation and suppression R. Granetz, A. DuBois, B.

An ITPA joint experiment to study threshold conditions for runaway electron generation and suppression R. Granetz, A. DuBois, B. Esposito, J. Kim, R. Koslowski, M. Lehnen, J. Martin-Solis ,C. Paz-Soldan, T.-N. Rhee, P. de Vries, J. Wesley, and L.

507 views • 28 slides

Implementation of a fluid model for the non-linear interaction between runaway electrons and

Summary Implementation of a fluid model for the non-linear interaction between runaway electrons and background plasma V. Bandaru 1 , M. Hoelzl 1 , G. Papp 1 , P. Aleynikov 2 , G. Huijsmans 3 1 Max-Planck-Institute for Plasma Physics, Garching,

555 views • 16 slides

Complexity of DLs RWTH Aachen 1 Germany Complexity of DLs: Overview of the Complexity of

Complexity of DLs RWTH Aachen 1 Germany Complexity of DLs: Overview of the Complexity of Concept Consistency P (co-)NP PSpace ExpTime NExpTime ALC reg ALUN (NP) ALCN add regular roles without , (wrt acyc. TBoxes) ALC u only A ALN

745 views • 16 slides

Lecture 7Limits on Climate Stability, Part 1 The runaway greenhouse/ Future evolution of the

41st Saas-Fee Course From Planets to Life 3-9 April 2011 Lecture 7Limits on Climate Stability, Part 1 The runaway greenhouse/ Future evolution of the Earth J. F. Kasting Venus 93-bar, CO 2 -rich atmosphere Practically no water

373 views • 26 slides

Novel Approaches for Mitigating Plasma Disruptions and Runaway Electrons in Tokamak ADITYA by R.

Novel Approaches for Mitigating Plasma Disruptions and Runaway Electrons in Tokamak ADITYA by R. L. Tanna Institute for Plasma Research, India (Contribution from ADITYA Team) 23-01-2015 25th IAEA/Fusion Engineering Conference (FEC)-2014,

554 views • 20 slides

Whose technology policy? Hegemony of Runaway Technology Policy??!! FATALISM +ve Grid

Power, Policy and Samkhya Philosophy Dipak Gyawali What technology policy? Whose technology policy? Hegemony of Runaway Technology Policy??!! FATALISM +ve Grid HIERARCHISM Asymmetrical Transactions: Risk Absorbing: Strong Ascribed

282 views • 4 slides

The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database

CS 5614: Database Management Systems 5 The DB Industry RDBMSs are a runaway success of simple theoretical ideas Big 3 Database companies are among the largest in the s/w industry Oracle, Informix, Sybase Other Market Forces: Microsoft,

688 views • 31 slides

A Comprehensive Scenario Agnostic Data LifeCycle Model for an Efficient Data Complexity

2016 IEEE 12th International Conference on eScience, Baltimore A Comprehensive Scenario Agnostic Data LifeCycle Model for an Efficient Data Complexity Management Amir Sinaeepourfard, Jordi Garcia , Xavier Masip-Bruin, Eva Marn-Tordera

335 views • 15 slides

Generating Networks with Sur- plus Complexity p. 1/17 Complexity, or amount of infor-

Generating Networks with Sur- plus Complexity p. 1/17 Complexity, or amount of information Note: Information = data + meaning Meaning B Meaning A Semantic Space B is more complex than A

497 views • 17 slides

Lecture 10 Seasonal Arima 10/05/2018 1 Australian Wine Sales Example (Lecture 6) Australian

Lecture 10 Seasonal Arima 10/05/2018 1 Australian Wine Sales Example (Lecture 6) Australian total wine sales by wine makers in bottles <= 1 litre. Jan 1980 2 Aug 1994. wineind 40000 30000 20000 1980 1985 1990 1995 0.6 0.6

631 views • 38 slides

present with Google Trends - Hyunyoung Choi - Hal Varian Outline Problem Statement Goal

Predicting the present with Google Trends - Hyunyoung Choi - Hal Varian Outline Problem Statement Goal Methodology Analysis and Forecasting Evaluation Applications and Examples Summary and Future work Problem

646 views • 28 slides

Section 4.1: Time Series I Jared S. Murray The University of Texas at Austin McCombs School of

Section 4.1: Time Series I Jared S. Murray The University of Texas at Austin McCombs School of Business 1 Time Series Data and Dependence Time-series data are simply a collection of observations gathered over time. For example, suppose y 1 . .

558 views • 44 slides

On Deploying Property- Preserving Encryp4on Paul Grubbs Cornell

On Deploying Property- Preserving Encryp4on Paul Grubbs Cornell University/Skyhigh Networks Outline Encryp4on that trades leakage for func4onality Look at

672 views • 23 slides

Fundamentals of Programming Session 24 Instructor: Reza Entezari-Maleki Email:

Fundamentals of Programming Session 24 Instructor: Reza Entezari-Maleki Email: entezari@ce.sharif.edu 1 Fall 2013 These slides have been created using Deitels slides Sharif University of Technology Outlines Data Members, set Functions

829 views • 35 slides

Aggregations Wednesday, February 22, 2017 Agenda Announcements Reading Quiz

Aggregations Wednesday, February 22, 2017 Agenda Announcements Reading Quiz Aggregations Discussion 2 Practice Problems Group Bys Discussion 2 Practice Problems Announcements Heads-up on Lab 3

453 views • 24 slides

Stored Procedures and Functions Rose-Hulman Institute of Technology Curt Clifton Outline

Stored Procedures and Functions Rose-Hulman Institute of Technology Curt Clifton Outline Stored Procedures or Sprocs Functions Statements Reference Defining Stored Procedures Named Collections of Transact-SQL Statements

511 views • 33 slides

61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y)

61A Extra Lecture 13 Announcements Prediction Regression 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values 4 Regression Given a set of (x, y) pairs, find a function f(x) that returns good y values

764 views • 41 slides