bigdata
play

BigData R EBECCA P ARSONS M ARTIN F OWLER CTO - PowerPoint PPT Presentation

Mar 15, 2023 •553 likes •1.68k views

BigData R EBECCA P ARSONS M ARTIN F OWLER CTO http://martinfowler.com rparsons@thoughtworks.com http://thoughtworks.com http://thoughtworks.com @martinfowler Thursday, March 8, 2012 Big Data R EBECCA P ARSONS M ARTIN F OWLER CTO

  1. BigData R EBECCA P ARSONS M ARTIN F OWLER CTO http://martinfowler.com rparsons@thoughtworks.com http://thoughtworks.com http://thoughtworks.com @martinfowler Thursday, March 8, 2012

  2. Big Data R EBECCA P ARSONS M ARTIN F OWLER CTO http://martinfowler.com rparsons@thoughtworks.com http://thoughtworks.com http://thoughtworks.com @martinfowler Thursday, March 8, 2012

  3. The Evolving Panorama of Data R EBECCA P ARSONS M ARTIN F OWLER CTO http://martinfowler.com rparsons@thoughtworks.com http://thoughtworks.com http://thoughtworks.com @martinfowler Thursday, March 8, 2012

  4. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  5. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  6. Data is: Thursday, March 8, 2012

  7. Data is: Growing Thursday, March 8, 2012

  8. Data is: Growing Thursday, March 8, 2012

  9. Data is: Growing Walmart : 1 million transactions per hour Facebook : 40 billion photos The Economist: Feb 25th 2010 Thursday, March 8, 2012

  10. 640K ought to be enough for anybody Thursday, March 8, 2012

  11. Lots of T raffic Thursday, March 8, 2012

  12. Thursday, March 8, 2012

  13. Thursday, March 8, 2012

  14. Data is: Thursday, March 8, 2012

  15. Data is: Distributed Thursday, March 8, 2012

  16. 1,482,824 Data is: Distributed 1,287,537 1,080,872 853,698 616,308 356,191 127,942 40,223 8,640 442 1,990 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Monthly Contributors to Wikipedia souce: wikipedia Thursday, March 8, 2012

  17. Data is: Distributed Thursday, March 8, 2012

  18. Data is: Distributed ��� � � � Thursday, March 8, 2012

  19. Data is: Distributed 98% of internet access points in Africa are mobile 30 million networked sensor nodes growing 30% per year McKinsey Global Institute: Big data: The next frontier for innovation, competition, and productivity Thursday, March 8, 2012

  20. Data is: Thursday, March 8, 2012

  21. Data is: Valuable Thursday, March 8, 2012

  22. Data is: Valuable $300 billion / year for US health care 60% increase in retail margins McKinsey Global Institute: Big data: The next frontier for innovation, competition, and productivity Thursday, March 8, 2012

  23. Data is: Thursday, March 8, 2012

  24. Data is: Urgent Thursday, March 8, 2012

  25. Data is: Thursday, March 8, 2012

  26. Data is: Connected Thursday, March 8, 2012

  27. Data Silos Thursday, March 8, 2012

  28. App App Enterprise Data Model App App Thursday, March 8, 2012

  29. App App Enterprise Data Model App App Thursday, March 8, 2012

  30. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  31. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  32. Thursday, March 8, 2012

  33. SQL SQL Thursday, March 8, 2012

  34. Thursday, March 8, 2012

  35. Bigtable Dynamo Thursday, March 8, 2012

  36. Thursday, March 8, 2012

  37. "NoSQL" Thursday, March 8, 2012

  38. Definition of NoSQL Thursday, March 8, 2012

  39. Characteristics of NoSQL Thursday, March 8, 2012

  40. Characteristics of NoSQL non-relational open-source cluster-friendly 21st Century Web schema-less Thursday, March 8, 2012

  41. Document Column- family Graph Key-value aph Thursday, March 8, 2012

  42. Aggregate Thursday, March 8, 2012

  43. Aggregate Eric Evans Thursday, March 8, 2012

  44. orders ID: 1001 customer: Ann line items: customers 0321293533 2 $48 $96 0321601912 1 $39 $39 order lines 0131495054 1 $51 $51 payment details: Card: Amex CC Number: 12345 expiry: 04/2001 credit cards Thursday, March 8, 2012

  45. ID: 1001 customer: Ann line items: 0321293533 2 $48 $96 Order 0321601912 1 $39 $39 1001 0131495054 1 $51 $51 aggregate payment details: Card: Amex CC Number: 12345 expiry: 04/2001 Thursday, March 8, 2012

  46. ID: 1001 customer: Ann line items: 0321293533 2 $48 $96 Order 0321601912 1 $39 $39 1001 0131495054 1 $51 $51 aggregate payment details: Card: Amex CC Number: 12345 document expiry: 04/2001 key-value column family Thursday, March 8, 2012

  47. Order 1001 Order 1004 Order 1026 Order 2843 Order 1001 Order 1004 Order 1001 Order 1004 Order 1026 Order 3754 Thursday, March 8, 2012

  48. Document Column- family Graph Key-value aph Thursday, March 8, 2012

  49. Document Column- family Graph Key-value aph Thursday, March 8, 2012

  50. aph Thursday, March 8, 2012

  51. aph Thursday, March 8, 2012

  52. aph Thursday, March 8, 2012

  53. P olyglot P ersistence aph Thursday, March 8, 2012

  54. Billing Inventory Thursday, March 8, 2012

  55. Billing Integration Database Inventory Thursday, March 8, 2012

  56. Billing Billing Integration Database Inventory Inventory Thursday, March 8, 2012

  57. Billing Billing Integration Database Application Database Inventory Inventory Thursday, March 8, 2012

  58. Event Sourcing Thursday, March 8, 2012

  59. John Smith : Person :Address city = Portland Thursday, March 8, 2012

  60. John Smith : change my Person address :Address city = Portland Thursday, March 8, 2012

  61. John Smith : change my Person address :Address city = Boston Thursday, March 8, 2012

  62. John Smith : change my Person address :Address city = Portland Thursday, March 8, 2012

  63. John Smith : change my Person address :Address city = Portland :Event change address Thursday, March 8, 2012

  64. John Smith : change my Person address :Address city = Boston :Event change address Thursday, March 8, 2012

  65. Application State Log Thursday, March 8, 2012

  66. Application State Log Thursday, March 8, 2012

  67. Application State Log Thursday, March 8, 2012

  68. on-demand self-service rapid elasticity broad network access resource pooling measured service special publication 800-145 Thursday, March 8, 2012

  69. Data Sources Thursday, March 8, 2012

  70. Data Sources were Thursday, March 8, 2012

  71. Data Sources were will be text image video connections Thursday, March 8, 2012

  72. Analytics Thursday, March 8, 2012

  73. Analytics were roll-ups trends variance Thursday, March 8, 2012

  74. Analytics were will be pattern recognition roll-ups data mining trends chasing connections variance Thursday, March 8, 2012

  75. Thursday, March 8, 2012

  76. Thursday, March 8, 2012

  77. Thursday, March 8, 2012

  78. map map map map per order map map map map map map map map Thursday, March 8, 2012

  79. map map map reduce map per order per month map map map reduce map map map map reduce map Thursday, March 8, 2012

  80. map map map reduce map per order per month map map map reduce map reduce map map map reduce map Thursday, March 8, 2012

  81. Thursday, March 8, 2012

  82. 10,000 ft view (literally) CodeCity by Richard Wettel http://www.inf.unisi.ch/phd/wettel/codecity.html Thursday, March 8, 2012

  83. Thursday, March 8, 2012

  84. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  85. Changing Nature of Data Response How we use data now Thursday, March 8, 2012

  86. Data Scientist Thursday, March 8, 2012

  87. Data Thursday, March 8, 2012

  88. Data Journalist Thursday, March 8, 2012

  89. Thursday, March 8, 2012

  90. Thursday, March 8, 2012

  91. Thursday, March 8, 2012

  92. Data Warehousing Thursday, March 8, 2012

  93. Data Warehousing Thursday, March 8, 2012

  94. Thursday, March 8, 2012

  95. Thursday, March 8, 2012

  96. http://ureport.ug/ Thursday, March 8, 2012

  97. http://ushahidi.com/ Thursday, March 8, 2012

  98. http://libyacrisismap.net/ Thursday, March 8, 2012

  99. http://opendata.go.ke/ Thursday, March 8, 2012

  100. http://datawithoutborders.cc/ Thursday, March 8, 2012

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

3548 Hypothetical Solution using Lambda Architecture Where BigData Industry is Going? SQL

3548 Hypothetical Solution using Lambda Architecture Where BigData Industry is Going? SQL

Enterprise Data Problems in Investment Banks BigData History and Trend Driven by Google CAP Theorem for Distributed Computer System Open Source Building Blocks: Hadoop, Solr, Storm.. 3548 Hypothetical Solution using Lambda

571 views • 25 slides
WP2 - Scalability and distributed Bigdata Marc X. Makkes Email: m.x.makkes@vu.nl WP2 Status

WP2 - Scalability and distributed Bigdata Marc X. Makkes Email: m.x.makkes@vu.nl WP2 Status

WP2 - Scalability and distributed Bigdata Marc X. Makkes Email: m.x.makkes@vu.nl WP2 Status Scalability and distributed Bigdata Marc X. Makkes (AP@ Vrije Universiteit, but currently @ CWI) A1: Reduced resource requirements for

524 views • 11 slides
The convergence of HPC and BigData What does it mean for HPC sysadmins? damienfrancois FOSDEM

The convergence of HPC and BigData What does it mean for HPC sysadmins? damienfrancois FOSDEM

The convergence of HPC and BigData What does it mean for HPC sysadmins? damienfrancois FOSDEM 2019 Feb 03, 2019 Brussels | damien.francois@uclouvain.be Scientists are never happy Some have models but they want data Please do not ask me

555 views • 22 slides
Welcome It used to be easy they all looked pretty much alike NoSQL BigData MapReduce Graph

Welcome It used to be easy they all looked pretty much alike NoSQL BigData MapReduce Graph

Welcome It used to be easy they all looked pretty much alike NoSQL BigData MapReduce Graph Document Shared Column Eventual BigTable CAP Nothing Oriented Consistency ACID BASE Mongo Coudera Hadoop Voldemort Cassandra Dynamo

1.04k views • 75 slides
#IoT #BigData Seema Jethani @seemaj @basho 10/31/14 1 Why should we care? 11/2/14 2 Source:

#IoT #BigData Seema Jethani @seemaj @basho 10/31/14 1 Why should we care? 11/2/14 2 Source:

#IoT #BigData Seema Jethani @seemaj @basho 10/31/14 1 Why should we care? 11/2/14 2 Source: http://en.wikipedia.org/wiki/Internet_of_Things Motivation for Specialized Big Data Systems Rate of data capture started soaring Traditional data

510 views • 21 slides
BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer Wenji Wu

BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer Wenji Wu

FERMILAB-SLIDES-18-060-CD BigData Express: Toward Predictable, Schedulable, and High-performance Data Transfer Wenji Wu wenj@fnal.gov Internet2 Global Summit May 8, 2018 This manuscript has been authored by Fermi Research Alliance, LLC under

494 views • 20 slides
Attacking a Big Data Developer Dr. Olaf Flebbe of t oflebbe.de ApacheCon Bigdata Europe

Attacking a Big Data Developer Dr. Olaf Flebbe of t oflebbe.de ApacheCon Bigdata Europe

Attacking a Big Data Developer Dr. Olaf Flebbe of t oflebbe.de ApacheCon Bigdata Europe 16.Nov.2016 Seville About me PhD in computational physics Former projects: Minix68k (68k FP Emulation), Linux libm.so.5 (High Precision FP), perl and

1.12k views • 56 slides
Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing

Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing

Canonical Text Service Jochen Tiepmar BigData Competence Center ScaDS Naural Language Processing Leipzig University Canonical Text Service - Jochen Tiepmar 2015 www.urncts.de/survey Survey From 20.06.2015 to 30.08.2015 Anonym, no tracking,

372 views • 24 slides
NDS Turner Fairbank Highway Research Center RID Federal Highway Administration US Department of

NDS Turner Fairbank Highway Research Center RID Federal Highway Administration US Department of

SHRP2 Naturalistic Driving Study and some other DOT data activities 2016 NSF BIGDATA PI MEETING APRIL 20TH-21ST WASHINGTON, DC Charles Fay SHRP 2 Research Manager Safety Training &Analysis Center(STAC) Office of Safety Research &

386 views • 4 slides
Amyotrophic lateral sclerosis Dr Natalie Twine | Transformational Bioinformatics |

Amyotrophic lateral sclerosis Dr Natalie Twine | Transformational Bioinformatics |

Using Big Data technologies to uncover genetic causes of Amyotrophic lateral sclerosis Dr Natalie Twine | Transformational Bioinformatics | @nat_twine 11 October 2017 HEATH & BIOSECURITY Genomics will outpace other BigData

360 views • 17 slides
Apache Incubator: where it is coming from and where it is going. Roman Shaposhnik

Apache Incubator: where it is coming from and where it is going. Roman Shaposhnik

Apache Incubator: where it is coming from and where it is going. Roman Shaposhnik <rvs@apache.org> Whos this guy? Director of Open Source strategy @Pivotal Former VP of Apache Incubator Committer and PMC members on a few Bigdata

541 views • 34 slides
Alexander Dejanovski @alexanderdeja Consultant www.thelastpickle.com Datastax MVP for Apache

Alexander Dejanovski @alexanderdeja Consultant www.thelastpickle.com Datastax MVP for Apache

Real world tales of repair APACHE BIGDATA - MAY 2017 Alexander Dejanovski @alexanderdeja Consultant www.thelastpickle.com Datastax MVP for Apache Cassandra Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

1.24k views • 98 slides
Cyrus Cousins with Eli Upfal Brown University BigData Group Spring 2019 Web:

Cyrus Cousins with Eli Upfal Brown University BigData Group Spring 2019 Web:

Distance Based Learning k -Nearest Representatives Uniform Convergence The Curse of Dimensionality Generalization Bounds for Distance-Based Learning with High-Dimensional Domains and Codomains Cyrus Cousins with Eli Upfal Brown University

1.25k views • 86 slides
Data-Intensive Distributed Computing CS 431/461 451/651 (Fall 2019) Part 2: From MapReduce to

Data-Intensive Distributed Computing CS 431/461 451/651 (Fall 2019) Part 2: From MapReduce to

Data-Intensive Distributed Computing CS 431/461 451/651 (Fall 2019) Part 2: From MapReduce to Spark (2/2) Ali Abedi These slides are available at http://roegiest.com/bigdata-2019w/ This work is licensed under a Creative Commons

606 views • 27 slides
MIT Relational Database GraphiQL Expensive! Graph Intuitive Query Language for Relational

MIT Relational Database GraphiQL Expensive! Graph Intuitive Query Language for Relational

GraphiQL Graph Intuitive Query Language for Relational Databases at Talking on IEEE BigData 2014 Alekh Jindal collaborate Supervisors Amol Deshpande Sam Madden work sabbatical Mike Stonebraker work University work of Maryland

514 views • 40 slides
Security and Performance Analysis of Encrypted NoSQL Databases M.W. Grim BSc., Abe Wiersma BSc.

Security and Performance Analysis of Encrypted NoSQL Databases M.W. Grim BSc., Abe Wiersma BSc.

Security and Performance Analysis of Encrypted NoSQL Databases M.W. Grim BSc., Abe Wiersma BSc. Supervisor: F. Turkmen PhD February 6, 2017 University of Amsterdam Introduction Problem Securely storing BigData on NoSQL database systems.

669 views • 48 slides
Alternative Titles: What are the practical implications of the HIPAA privacy rules for

Alternative Titles: What are the practical implications of the HIPAA privacy rules for

Protecting the Privacy of Healthcare Data While Preserving the Utility of Geographic Location Information for Epidemiologic Research Daniel C. Barth-Jones, M.P.H., Ph.D. Center for Healthcare Effectiveness Research Wayne State University (

448 views • 31 slides
GARCH models Magnus Wiktorsson SW-[?]ARCH An advanced extension is the switching ARCH model.

GARCH models Magnus Wiktorsson SW-[?]ARCH An advanced extension is the switching ARCH model.

GARCH models Magnus Wiktorsson SW-[?]ARCH An advanced extension is the switching ARCH model. ARCH, GARCH or EGARCH (the later two are non-trivial, due to their non-Markovian structure) parameters. The conditional variance is given by a

1.52k views • 37 slides
DUNE Project Status Jolie Macier DUNE PMG Meeting 17 April 2018 Outline ES&H Update

DUNE Project Status Jolie Macier DUNE PMG Meeting 17 April 2018 Outline ES&H Update

DUNE Project Status Jolie Macier DUNE PMG Meeting 17 April 2018 Outline ES&H Update QA Update Schedule & Budget Status PM Update DUNE Update FD Engineering FD Installation FD TPC Electronics FD

441 views • 26 slides
ACRS MEETING WITH THE U.S. NUCLEAR REGULATORY COMMISSION December 6, 2012 Overview J. Sam

ACRS MEETING WITH THE U.S. NUCLEAR REGULATORY COMMISSION December 6, 2012 Overview J. Sam

ACRS MEETING WITH THE U.S. NUCLEAR REGULATORY COMMISSION December 6, 2012 Overview J. Sam Armijo 2 Accomplishments Since our last meeting with the Commission on June 7, 2012, we issued 19 Reports. Topics SECY-12-0064,

634 views • 60 slides
FUTURE WEB DESIGN MOBILES, TABLETS, TVS & GLASSES About 51Degrees.mobi Founded 2009

FUTURE WEB DESIGN MOBILES, TABLETS, TVS & GLASSES About 51Degrees.mobi Founded 2009

Making the mobile web work James Rosewell - @jwrosewell / @51degreesmobi 51Degrees.mobi FUTURE WEB DESIGN MOBILES, TABLETS, TVS & GLASSES About 51Degrees.mobi Founded 2009 Open Source Project Device Detection

1.1k views • 96 slides
CPSC 310 Requirements Elicitation, Analysis, Specification and Validation 2014-09-09 1

CPSC 310 Requirements Elicitation, Analysis, Specification and Validation 2014-09-09 1

CPSC 310 Requirements Elicitation, Analysis, Specification and Validation 2014-09-09 1 Learning goals By the end of this unit, you will be able to: Explain why its important to elicit and specify requirements well Specify or

921 views • 66 slides
MIKEYv2 ldondeti@qualcomm.com Why MIKEYv2? MIKEY is an efficient key management protocol

MIKEYv2 ldondeti@qualcomm.com Why MIKEYv2? MIKEY is an efficient key management protocol

MIKEYv2 ldondeti@qualcomm.com Why MIKEYv2? MIKEY is an efficient key management protocol designed for SRTP keying Used for broadcast keying in 3GPP and OMA Easy to add crypto algorithm and mode negotiation MIKEYv2 finishes in 1 RT

586 views • 9 slides
Programming Heterogeneous Systems F. Bodin June 2013 Uppsala Introduction HPC and embedded

Programming Heterogeneous Systems F. Bodin June 2013 Uppsala Introduction HPC and embedded

Programming Heterogeneous Systems F. Bodin June 2013 Uppsala Introduction HPC and embedded software going for dramatic changes to adapt to massive parallelism o Huge market issue o Many codes and users not ready directives based

1.37k views • 133 slides

More recommend