architecture at scale
play

Architecture at Scale Thursday, March 8, 12 About the speaker - PowerPoint PPT Presentation

Oct 24, 2023 •463 likes •838 views

Architecture at Scale Thursday, March 8, 12 About the speaker Manny Pelarinos- Sr. Director of Engineering, ESPN manole.pelarinos@espn.com Been with ESPN for 5 years First time at QCon! 2 Thursday, March 8, 12 Agenda

  1. Architecture at Scale Thursday, March 8, 12

  2. About the speaker • Manny Pelarinos- Sr. Director of Engineering, ESPN • manole.pelarinos@espn.com • Been with ESPN for 5 years • First time at QCon! 2 Thursday, March 8, 12

  3. Agenda • High Level Architecture –Overview –Data Ingest –Core Service Architecture • Advanced Systems –Content Management System –Template Language –Live Scores –Personalization • Q&A 3 Thursday, March 8, 12

  4. ESPN.com Facts & Figures 4 Thursday, March 8, 12

  5. ESPN.com Facts & Figures • ESPN.com has the 5th largest audience of men 18 to 54 across the entire Internet, behind only Facebook, Google, Yahoo and Microsoft! • Serves over 100K RPS with few servers • Almost entirely Java based • ESPN digital properties include –ESPN.com –Fantasy games –Mobile –WatchESPN –ESPN the Ocho • No, not yet but others (Deports, W, HS, etc) 4 Thursday, March 8, 12

  6. ESPN.com Mission 5 Thursday, March 8, 12

  7. ESPN.com Mission • Serve sports fans anytime, anywhere on any device • Availability & accuracy of the utmost important • Bring fan all stats and scores plus deeper content 5 Thursday, March 8, 12

  8. 6 Thursday, March 8, 12

  9. 7 Thursday, March 8, 12

  10. Data Ingest 7 Thursday, March 8, 12

  11. Data Ingest 7 Thursday, March 8, 12

  12. 8 Thursday, March 8, 12

  13. Application Services 8 Thursday, March 8, 12

  14. Application Services 8 Thursday, March 8, 12

  15. 9 Thursday, March 8, 12

  16. 10 Thursday, March 8, 12

  17. Cache Push • Moving to cache push model • Data ingest process has converts incoming XML message into JPA POJO –Inefficient to have DB send expire –MDBs and webapp both talk POJO so just push it –Remove biggest bottle neck (our DB) from the equation • All live event data gets delivered to the web application by ingest (MDB) process –Eliminates millions of DB calls per hour during peak times –Once an application has primed its caches with historical & meta data, DB could theoretically be turned off 10 Thursday, March 8, 12

  18. 11 Thursday, March 8, 12

  19. Hibernate Caching • Using Ehcache as 2 nd level cache provider with cache replication enabled • Works great for entity updates • BIG PROBLEM: almost everything on .com looked up via query… –Hibernate support for query caching inefficient –Can’t have getPlaysForGame(1234) query banging on DB all day b/c query caceh only supports TTL or dopey last update timestamp –What to do…. � 11 Thursday, March 8, 12

  20. 12 Thursday, March 8, 12

  21. JPA Query Replicator 12 Thursday, March 8, 12

  22. JPA Query Replicator 12 Thursday, March 8, 12

  23. 13 Thursday, March 8, 12

  24. Content Management System 13 Thursday, March 8, 12

  25. Content Management System 13 Thursday, March 8, 12

  26. 14 Thursday, March 8, 12

  27. ESPN.com Web Template Language • Proprietary, high performance template framework –http://teatrove.org/ • Stout –High performance page caching –Peer fetch enabled –Per URI, TTL based expiration –Blocking and non-blocking source fetch –Automatically demotes unresponsive source servers –Runs on cheap / low end hardware 14 Thursday, March 8, 12

  28. 15 Thursday, March 8, 12

  29. ESPN.com Live Scores 15 Thursday, March 8, 12

  30. ESPN.com Live Scores 15 Thursday, March 8, 12

  31. Personalization • Fan desire to have personalized content and presentation –And have it follow you everywhere (.com, mobile, TV, etc) • Need to build scalable, high performance, distribute cache –>500 GB of data –Constrained tree map - Lookup primarily by ID • But site is heavily page cached… not conducive to personalized user experience 16 Thursday, March 8, 12

  32. 17 Thursday, March 8, 12

  33. http://developer.espn.com/ 18 Thursday, March 8, 12

  34. Challenges • Designing APIs is hard –Fine grained vs. coarse grained debate –More generic APIs are more reusable, but often require the client to make many calls and caching is tough –More specific APIs are more efficient but the variations get out of hand really fast • What did we settle on –Generic APIs for now and build use case driven ones as needed –‘Coherent TTL’ cache on the way to help with mobile 19 Thursday, March 8, 12

  35. Q&A 20 Thursday, March 8, 12

  36. Q&A • We are hiring!!! • Go apply at http://espncareers.com … or better yet talk to me afterward • Questions? –Feedback on anything you heard? –Suggestions for ESPN.com features? –Click that little happy face for my rating and I’ll fix your fantasy football scores! 20 Thursday, March 8, 12

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Advanced features and capabilities Platform architecture Ayman Hamed Solutions Architect How We

Advanced features and capabilities Platform architecture Ayman Hamed Solutions Architect How We

| 1 Advanced features and capabilities Platform architecture Ayman Hamed Solutions Architect How We Built It: Web-Scale Engineering X86 Off-the-shelf x86 servers X86 X86 All intelligence in software Scale-out architecture Self-healing

751 views • 62 slides
kohn kohn architecture 466 AMSTERDAM AVE. STOREFRONT PROPOSAL P-01 EXISTING PHOTOS 4 4 6 6 6

kohn kohn architecture 466 AMSTERDAM AVE. STOREFRONT PROPOSAL P-01 EXISTING PHOTOS 4 4 6 6 6

BROWNSTONE BELTCOURSE TAX PHOTO OF 466 AMSTERDAM AVE. EXISTING BUILDING (GROUND LEVEL) EXISTING STOREFRONT 1 2 3 SCALE : NTS SCALE : NTS SCALE : NTS CORNICE BRACKETS FIRE- BELT-COURSE ESCAPE 9 SCALE : NTS DESIGNATION PHOTO CAST

366 views • 5 slides
Array-Based Architecture for Molecular Electronics Andr DeHon <andre@cs.caltech.edu>

Array-Based Architecture for Molecular Electronics Andr DeHon <andre@cs.caltech.edu>

Array-Based Architecture for Molecular Electronics Andr DeHon <andre@cs.caltech.edu> DeHon February 2002 Molecular Scale Building Blocks Chemists starting to demonstrate Molecular-scale switching devices Molecular-scale

343 views • 12 slides
emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network scale Regional scale

emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network scale Regional scale

Project Team: Gabriel Cira, AIA - Very Flat, Architecture Nick Lutsko - MIT EAPS, Atmospheric Dynamics Julia Hopkins - TU Delft & NEU, Coastal Engineering emerald.tutu@gmail.com vimeo.com/266553400 Unit scale Network

401 views • 5 slides
APIs, Architecture and Modeling for Extreme Scale Resilience Dagstuhl Seminar: Resilience in

APIs, Architecture and Modeling for Extreme Scale Resilience Dagstuhl Seminar: Resilience in

APIs, Architecture and Modeling for Extreme Scale Resilience Dagstuhl Seminar: Resilience in Exascale Computing 9/30/2014 Kento Sato LLNL-PRES-661421 This work was performed under the auspices of the U.S. Department of Energy by Lawrence

515 views • 24 slides
Practical OVN: Architecture, Deployment, and Scale of OpenStack Networking Justin Pettit

Practical OVN: Architecture, Deployment, and Scale of OpenStack Networking Justin Pettit

Practical OVN: Architecture, Deployment, and Scale of OpenStack Networking Justin Pettit (@Justin_D_Pettit) Ben Pfaff (@Ben_Pfaff) Han Zhou (@zhouhanok) Ryan Moats (@regxboi) The Case for Network Virtualization Network provisioning needs

787 views • 33 slides
A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services 1. Overview 2.

A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services 1. Overview 2.

A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services 1. Overview 2. Challenges and Solution. 3. Introduction to FPGA 4. Requirement and Architecture. 5.Infrastructure and Platform architecture. 5.1 Debugging support. 5.2

786 views • 36 slides
Contents Introduction Scalability and Accuracy in a Architecture Large-Scale Network

Contents Introduction Scalability and Accuracy in a Architecture Large-Scale Network

Contents Introduction Scalability and Accuracy in a Architecture Large-Scale Network Emulator Implementation and Evaluation Accuracy vs. Scalability Tradeoff Nov. 12, 2003 Case Studies Byung-Gon Chun 1 2 Motivation

299 views • 5 slides
Internet-Scale Event Notification: Architecture Alternatives Jie Ren Information and Computer

Internet-Scale Event Notification: Architecture Alternatives Jie Ren Information and Computer

Internet-Scale Event Notification: Architecture Alternatives Jie Ren Information and Computer Science University of California, Irvine 05/08/2000 Before Routing Event Types: protocol or business? Routing Paradigm: on subject, or

198 views • 3 slides
Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead Ang Chen, Yang

Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead Ang Chen, Yang

Data Provenance at Internet Scale: Architecture, Experiences, and the Road Ahead Ang Chen, Yang Wu, Andreas Haeberlen, Boon Thau Loo, Wenchao Zhou Motivation D E A foo.com Alice C B An example scenario: network routing System

960 views • 93 slides
Extreme Scale Computer Architecture: Energy Efficiency from the Ground Up Josep Torrellas

Extreme Scale Computer Architecture: Energy Efficiency from the Ground Up Josep Torrellas

Extreme Scale Computer Architecture: Energy Efficiency from the Ground Up Josep Torrellas Department of Computer Science University of Illinois at Urbana-Champaign http://iacoma.cs.uiuc.edu ASBD June 2014 Wanted: Energy-Efficient Computing

707 views • 24 slides
Wix Architecture at Scale Aviran Mordo Head of Back-End Engineering @ Wix @aviranm

Wix Architecture at Scale Aviran Mordo Head of Back-End Engineering @ Wix @aviranm

Wix Architecture at Scale Aviran Mordo Head of Back-End Engineering @ Wix @aviranm linkedin.com/in/aviran aviransplace.com Wix in Numbers Over 45,000,000 users 1M new users/month Static storage is >800TB of data 1.5TB new files/day 3

964 views • 39 slides
Effect of Network Architecture on Sparsely Synchronized Brain Rhythms in A Scale-Free Neural

Effect of Network Architecture on Sparsely Synchronized Brain Rhythms in A Scale-Free Neural

Effect of Network Architecture on Sparsely Synchronized Brain Rhythms in A Scale-Free Neural Network Sang-Yoon Kim and Woochang Lim Institute for Computational Neuroscience and Department of Science Education, Daegu National University of

252 views • 3 slides
Monarch Googles planet-scale streaming monitoring infrastructure. Background Architecture and

Monarch Googles planet-scale streaming monitoring infrastructure. Background Architecture and

Monarch Googles planet-scale streaming monitoring infrastructure. Background Architecture and Data Model Queries Using Monarch Monarch Platform Lessons Learned re: Scaling Monitoring at Google Ref:

854 views • 51 slides
VoltDB Things you learn as you massively scale David Rolfe Director of Solution Architecture,

VoltDB Things you learn as you massively scale David Rolfe Director of Solution Architecture,

VoltDB Things you learn as you massively scale David Rolfe Director of Solution Architecture, EMEA Tom Howcroft Director of Sales, EMEA 1 17-Jul-18 Scaling at the Architectural Level 2 How many servers will you need to start? HA

379 views • 21 slides
Using AWS to Build a Large Scale Dockerized Microservices Architecture Dr. Oliver Wahlen moovel

Using AWS to Build a Large Scale Dockerized Microservices Architecture Dr. Oliver Wahlen moovel

Using AWS to Build a Large Scale Dockerized Microservices Architecture Dr. Oliver Wahlen moovel Group GmbH Frankfurt, 30. Juni 2016 The moovel Group GmbH Our vision is an ecosystem that simplifies mobility now solves the mobility

384 views • 15 slides
Large Area Picosecond Photodetector (LAPPD) Pilot Production and Development Status Photo

Large Area Picosecond Photodetector (LAPPD) Pilot Production and Development Status Photo

14 th Pisa Meeting on Advanced Detectors La Biodola, Isola d'Elba (Italy) May 27 - Jun 2, 2018 Large Area Picosecond Photodetector (LAPPD) Pilot Production and Development Status Photo Detectors and PID, Monday May 28, starting at 18:50 Michael

362 views • 17 slides
Introducing Campo Verde high school AP Hall of FAME AP Scholar AP Scholar with Honor AP

Introducing Campo Verde high school AP Hall of FAME AP Scholar AP Scholar with Honor AP

Introducing Campo Verde high school AP Hall of FAME AP Scholar AP Scholar with Honor AP Scholar with Distinction National AP Scholar Class 2016 AP Scholar Ali Ahmed Joy Gregory Alex Moore Mallorie Alton Devin Haltom

1.72k views • 21 slides
King County Local Services Community Service Area Town Hall Meeting West-North Highline

King County Local Services Community Service Area Town Hall Meeting West-North Highline

King County Local Services Community Service Area Town Hall Meeting West-North Highline Thursday, Oct. 15, 2020 Interpretation This meeting is being simultaneously interpreted in Spanish, Somali, and Vietnamese Our interpreters today

1.08k views • 82 slides
Combined License Application Mandatory Hearing Florida Power & Light Company Turkey Point

Combined License Application Mandatory Hearing Florida Power & Light Company Turkey Point

Combined License Application Mandatory Hearing Florida Power & Light Company Turkey Point Units 6 and 7 Safety Panel December 12, 2017 Panelists Manny Comar Senior Project Manager, NRC Joseph Giacinto Lead

338 views • 17 slides
SWEN 262 Engineering of Software Subsystems The Laws of Software Evolution Beginning in 1974,

SWEN 262 Engineering of Software Subsystems The Laws of Software Evolution Beginning in 1974,

SWEN 262 Engineering of Software Subsystems The Laws of Software Evolution Beginning in 1974, Manny Lehman and Laszlo In other words, over time any software system must change to add new Belady began documenting the laws of software

576 views • 28 slides
Progress with the Prime Ideal Principle Manuel L. Reyes Bowdoin College Conference on Rings and

Progress with the Prime Ideal Principle Manuel L. Reyes Bowdoin College Conference on Rings and

Progress with the Prime Ideal Principle Manuel L. Reyes Bowdoin College Conference on Rings and Factorizations University of Graz February 22, 2018 Manny Reyes Progress with the PIP February 22, 2018 1 / 40 Todays themes Major

1.56k views • 90 slides
Pain Management CLeAR Webinar December 9, 2014 To reduce noise, participants phone lines are

Pain Management CLeAR Webinar December 9, 2014 To reduce noise, participants phone lines are

Pain Management CLeAR Webinar December 9, 2014 To reduce noise, participants phone lines are muted on entry Please type comments or questions in CHAT. Your CLeAR Team Ben Ridout, Improvement Advisor Manny Sahota, Project Coordinator Katie

583 views • 32 slides
The Role of MPPs at NASA and the Aerospace Industry Manny Salas May 22, 1997 Talk Outline

The Role of MPPs at NASA and the Aerospace Industry Manny Salas May 22, 1997 Talk Outline

The Role of MPPs at NASA and the Aerospace Industry Manny Salas May 22, 1997 Talk Outline NASA Aeronautics Computational Resources NASA Utilization & Availability Metrics NASA Aeronautics Parallel Software Aerospace

626 views • 47 slides

More recommend