atrapos adaptive transaction
play

ATraPos: Adaptive Transaction Processing on Hardware Islands Danica - PowerPoint PPT Presentation

Jan 31, 2024 •630 likes •932 views

ATraPos: Adaptive Transaction Processing on Hardware Islands Danica Porobic , Erietta Liarou, Pnar Tzn, Anastasia Ailamaki Data-Intensive Application and Systems Lab, EPFL Scaling up OLTP on multisockets Throughput 1 2 3 4 5 6 7 8

  1. ATraPos: Adaptive Transaction Processing on Hardware Islands Danica Porobic , Erietta Liarou, Pınar Tözün, Anastasia Ailamaki Data-Intensive Application and Systems Lab, EPFL

  2. Scaling up OLTP on multisockets Throughput 1 2 3 4 5 6 7 8 Number of sockets Multisocket servers are severely under-utilized 2

  3. Multisocket multicores <10 cycles 50 cycles 500 cycles threads Core Core Core Core Core Core Core Core L1 L1 L1 L1 L1 L1 L1 L1 L1 L2 L2 L2 L2 L2 L2 L2 L2 Island L3 L3 L3 Memory controller Memory controller Inter-socket links Inter-socket links Inter-socket links Inter-socket links Socket 1 Socket 0 Communication latencies vary by an order-of-magnitude 3

  4. OLTP on Hardware Islands Shared-everything Island shared-nothing Shared-nothing 4

  5. Scaling up on a 8-socket machine 8 socket x 10 core 800K row dataset 7 Probing one row Shared-nothing 6 Throughput (MTPS) Island shared-nothing 5 Shared-everything 4 3 2 1 0 1 2 3 4 5 6 7 8 Number of sockets Islands significantly challenge scalability 5

  6. Physical partitioning for Islands 4 socket x 6 core 240K row dataset 120 Shared-nothing Updating 10 rows Island shared-nothing Throughput (KTps) 100 Shared-everything 80 60 40 20 0 0 20 40 60 80 100 % multisite transactions No configuration is optimal for all environments 6

  7. OLTP on Hardware Islands Shared-everything Island shared-nothing Shared-nothing  Stable  Robust middle ground  Fast  Not optimal  Sensitive to workload • Challenges – Optimal configuration depends on workload and hardware – Expensive repartitioning due to physical data movement ATraPos: hardware and workload-aware shared-everything adaptive system 7

  8. ATraPos: Adaptive Transaction Processing • No unnecessary inter-socket synchronization • Workload & hardware-aware partitioning • Lightweight monitoring and repartitioning ATraPos: hardware and workload-aware shared-everything adaptive system 8

  9. Outline • Impact of Hardware Islands on OLTP • ATraPos – Avoiding unnecessary synchronization – Workload & hardware-aware partitioning & placement – Lightweight monitoring & repartitioning • Summary 9

  10. Critical path of transaction execution System state threads Core Core Core Core Core Core Core Core Data Many accesses to shared data structures 10

  11. Perfectly partitionable workload 8 socket x 10 core 800K row dataset 7 Probing one row Shared-nothing 6 Throughput (MTPS) Centralized shared-everything 5 4 3 2 1 0 1 2 3 4 5 6 7 8 Number of sockets Accessing centralized data structures limits scalability 11

  12. PLP: Physiologically partitioned SE* System state threads Core Core Core Core Core Core Core Core System state is still shared 12 *I. Pandis, et al: PLP: Page Latch-free Shared-everything OLTP, VLDB 2011

  13. Perfectly partitionable workload 8 socket x 10 core 800K row dataset 7 Probing one row Shared-nothing 6 Throughput (MTPS) PLP 5 Centralized shared-everything 4 3 2 1 0 1 2 3 4 5 6 7 8 Number of sockets Inter-socket accesses to system state are a bottleneck 13

  14. ATraPos: Island-aware SE System state System state threads Core Core Core Core Core Core Core Core 14

  15. Perfectly partitionable workload 8 socket x 10 core 800K row dataset 7 Probing one row Shared-nothing 6 ATraPos Throughput (MTPS) PLP 5 Centralized shared-everything 4 3 2 1 0 1 2 3 4 5 6 7 8 Number of sockets Island awareness brings scalability 15

  16. Outline • Impact of Hardware Islands on OLTP • ATraPos – Avoiding unnecessary synchronization – Workload & hardware-aware partitioning & placement – Lightweight monitoring & repartitioning • Summary 16

  17. Naive partitioning and placement 8 socket x 10 core 800K rows per table Probing 1 row each from A and B 2100 1800 Probe A Probe B Throughput (KTPS) 1500 1200 900 1.9x 600 300 0 PLP ATraPos ATraPos ATraPos HW-aware Load balanced Cores are overloaded with contending threads 17

  18. ATraPos partitioning and placement 8 socket x 10 core 800K rows per table Probing 1 row each from A and B 2100 1800 Probe A Probe B Throughput (KTPS) 1500 1200 4.4x 900 600 300 0 PLP ATraPos ATraPos ATraPos HW-aware Load balanced Ignoring Islands -> synchronization overhead 18

  19. ATraPos partitioning and placement 8 socket x 10 core 800K rows per table Probing 1 row each from A and B 2100 1800 Probe A Probe B Throughput (KTPS) 1500 4.8x 1200 900 600 300 0 PLP ATraPos ATraPos ATraPos HW-aware Load balanced ATraPos: balanced load + reduced synchronization 19

  20. Outline • Impact of Hardware Islands on OLTP • ATraPos – Avoiding unnecessary synchronization – Workload & hardware-aware partitioning & placement – Lightweight monitoring & repartitioning • Summary 20

  21. ATraPos monitoring Initialize with naive scheme stats Monitor the workload Evaluate cost model: Balance the load Probe A Minimize synchronization 21

  22. ATraPos monitoring Initialize with naive scheme stats Monitor the workload Evaluate cost model Probe A Probe B Repartition 22

  23. Repartitioning Multi-Rooted B-trees threads Splitting and merging B-trees accesses few pages 23

  24. ATraPos repartitioning 8 socket x 10 core 800K row table 180 merge Repartitioning cost (ms) split 150 rearrange (split+merge) 120 90 60 30 0 10 20 30 40 50 60 70 80 Number of repartitioning actions Repartitioning of a table takes < 200ms 24

  25. TATP - speedup over PLP 8 socket x 10 core 800K subscribers 7 ATraPos 6 Normalized throughput PLP 5 4 3 2 1 0 GetSubData GetNewDest UpdSubData Mix ATraPos improves performance of TATP by 3.1-6.7x 25

  26. Adapting to workload skew 8 socket x 10 core Monitoring Repartitioning 4.5 800K subscribers 50% requests to TATP GetSubData 4 Static 20% data Throughput (MTPS) 3.5 ATraPos 3 2.5 2 1.5 1 0.5 0 0 10 20 30 40 50 Time (s) ATraPos detects skew and quickly adapts 26

  27. Adapting to changing workload type Repartitioning Monitoring 8 socket x 10 core 360 800K subscribers Static TATP Throughput (KTPS) 300 ATraPos GetNewDest 240 180 120 Mix 60 UpdSubData 0 0 15 30 45 60 75 90 Time (s) ATraPos gracefully adapts to any change 27

  28. ATraPos: Adaptive OLTP for Islands • Challenges – Optimal configuration depends on workload and hardware – Expensive repartitioning due to physical data movement • ATraPos – Minimal inter-socket accesses in the critical path – Workload & hardware-aware partitioning & placement – Lightweight monitoring and repartitioning Thank you! 28

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

1 Transaction State Transaction State (Cont.) Active, the initial state; the transaction stays in

1 Transaction State Transaction State (Cont.) Active, the initial state; the transaction stays in

Transactions Transaction Concept Transaction State Implementation of Atomicity and Durability Lecture 8 Concurrent Executions Transactions Serializability Recoverability Implementation of Isolation Transaction

376 views • 6 slides
Introduction 2 Introduction - transaction All cash transaction Planned transaction $115 per

Introduction 2 Introduction - transaction All cash transaction Planned transaction $115 per

1 Introduction 2 Introduction - transaction All cash transaction Planned transaction $115 per share Delivering a 50% approximately cash premium $5 billion 3 Introduction - rationale Key strengths of StanCorp Consistent long-term

940 views • 43 slides
Transaction Processing Transaction Concept A transaction is a unit of program execution that

Transaction Processing Transaction Concept A transaction is a unit of program execution that

Transaction Processing Transaction Concept A transaction is a unit of program execution that accesses and possibly updates various data items. E.g. transaction to transfer $50 from account A to account B: 1. read ( A ) 2. A := A 50

861 views • 74 slides
Chapter 20: Advanced Transaction Processing Remote Backup Systems Transaction-Processing

Chapter 20: Advanced Transaction Processing Remote Backup Systems Transaction-Processing

' $ Chapter 20: Advanced Transaction Processing Remote Backup Systems Transaction-Processing Monitors High-Performance Transaction Systems Long-Duration Transactions Real-Time Transaction Systems Weak Levels of Consistency

521 views • 20 slides
Transaction Announcement Presentation July 2008 Summary of Transaction ! Acquisition by

Transaction Announcement Presentation July 2008 Summary of Transaction ! Acquisition by

Transaction Announcement Presentation July 2008 Summary of Transaction ! Acquisition by Kohlberg Kravis Roberts &amp; Co. (KKR) of all the assets and liabilities Transaction of KKR Private Equity Investors (KPE) ! KPE unitholders

820 views • 41 slides
Lecture 11: Transaction processing and concurrency in ADO.NET Lisa (Ling) Liu Transaction

Lecture 11: Transaction processing and concurrency in ADO.NET Lisa (Ling) Liu Transaction

Chair of Softw are Engineering C# Programming in Depth Prof. Dr. Bertrand Meyer March 2007 May 2007 Lecture 11: Transaction processing and concurrency in ADO.NET Lisa (Ling) Liu Transaction processing Transaction The execution of

417 views • 29 slides
Avnel Acquisition June 2017 AVNEL TRANSACTION TRANSACTION HIGHLIGHTS Endeavour to acquire

Avnel Acquisition June 2017 AVNEL TRANSACTION TRANSACTION HIGHLIGHTS Endeavour to acquire

Avnel Acquisition June 2017 AVNEL TRANSACTION TRANSACTION HIGHLIGHTS Endeavour to acquire Avnel, which owns the Kalana Gold project in Mali An all-share transaction values Avnel at 1 2 US$122m (C$159m), representing a 48% FITS OUR

480 views • 10 slides
1 Transaction management Example of a transaction Transaction management aims at handling

1 Transaction management Example of a transaction Transaction management aims at handling

DATABASE DESIGN I - 1DL300 Introduction to Transactions &amp; Concurrency Control Fall 2011 Elmasri/Navathe ch 20 and 21 Padron-McCarthy/Risch ch 23 and 24 An introductory course on database systems

619 views • 9 slides
OUR OBJECTIVE : OPTIMIZING YOUR TRANSACTION 1 OPTIMIZING YOUR TRANSACTION, is bringing added

OUR OBJECTIVE : OPTIMIZING YOUR TRANSACTION 1 OPTIMIZING YOUR TRANSACTION, is bringing added

OUR OBJECTIVE : OPTIMIZING YOUR TRANSACTION 1 OPTIMIZING YOUR TRANSACTION, is bringing added value to you appropriate for each stage of your project. 2 p.4 Presentation of Trianon Corporate Finance Our team p.12 References p.15 p.24

288 views • 24 slides
Opinion of the Independent Financial Advisor Regarding the Asset Acquisition Transaction and

Opinion of the Independent Financial Advisor Regarding the Asset Acquisition Transaction and

Opinion of the Independent Financial Advisor Regarding the Asset Acquisition Transaction and Connected Transaction April 2015 Strictly Private &amp; Confidential Table of Content P. Summary of the Transaction of Investing in

166 views • 16 slides
From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov

From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov

From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov convinced me that direct adaptive control can be robust Dimitri Peaucelle FRontiers of ADaptive and nonlinear CONtrol (FRADCON 2018) ECC Workshop,

628 views • 19 slides
Cross Border Update Dermot Corry Dealing/Transaction Accounts Dealing/transaction accounts

Cross Border Update Dermot Corry Dealing/Transaction Accounts Dealing/transaction accounts

Cross Border Update Dermot Corry Dealing/Transaction Accounts Dealing/transaction accounts for Portfolio Bonds Policies usually include a range of unit holdings and deposit accounts There is often a transaction or dealing account

78 views • 6 slides
ML&amp;P Sale Transaction Proceeds, the Sequel 11/12/19 Projected Transaction Proceeds Up

ML&amp;P Sale Transaction Proceeds, the Sequel 11/12/19 Projected Transaction Proceeds Up

ML&amp;P Sale Transaction Proceeds, the Sequel 11/12/19 Projected Transaction Proceeds Up Front Payment ($ 755,273,577.00 ) + Payments in Lieu of Tax (50 years) ($166,800,000.00) + Eklutna Power Purchase (35 years) ($75,000,000.00)

269 views • 9 slides
High Volume Low Latency Transaction Processing Transaction Processing Presenters : Tony Harrop,

High Volume Low Latency Transaction Processing Transaction Processing Presenters : Tony Harrop,

High Volume Low Latency Transaction Processing Transaction Processing Presenters : Tony Harrop, CTO, LiquidityHub Jeremy Vickers, Technical Architect, Detica 12 th March Date : Reference : QCon 2008 Client Confidential Contents Overview of

352 views • 19 slides
SPAC 101 Transaction Basics and Current Trends Transaction Basics What is a SPAC? Blank

SPAC 101 Transaction Basics and Current Trends Transaction Basics What is a SPAC? Blank

SPAC 101 Transaction Basics and Current Trends Transaction Basics What is a SPAC? Blank check company formed for the purpose of effecting a merger, share exchange, asset acquisition, share purchase, reorganization or similar business

516 views • 20 slides
3-509

3-509

3-509 liuzhen@sjtu.edu.cn 1 Introduction to Bitcoin SSL and IPSec 2 2. Transaction in Bitcoin A Transaction 2. Transaction in Bitcoin A

727 views • 25 slides
THE ENLIGHTENMENT: Still Burning Bright FERNANDO N ZIALCITA, PhD DEPARTMENT OF SOCIOLOGY AND

THE ENLIGHTENMENT: Still Burning Bright FERNANDO N ZIALCITA, PhD DEPARTMENT OF SOCIOLOGY AND

THE ENLIGHTENMENT: Still Burning Bright FERNANDO N ZIALCITA, PhD DEPARTMENT OF SOCIOLOGY AND ANTHROPOLOGY ATENEO DE MANILA UNIVERSITY We assume that we have a right to be Free THE ENLIGHTENMENT: STILL BURNING BRIGHT OUTLINE How the flame

515 views • 32 slides
Heterogeneous Networks Jie Tang*, Tiancheng Lou*, and Jon Kleinberg + *Tsinghua University +

Heterogeneous Networks Jie Tang*, Tiancheng Lou*, and Jon Kleinberg + *Tsinghua University +

Inferring Social Ties across Heterogeneous Networks Jie Tang*, Tiancheng Lou*, and Jon Kleinberg + *Tsinghua University + Cornell University 1 Real social networks are complex... Different social ties have different influence on people

717 views • 37 slides
Tandem modeling investigations Dan Ellis International Computer Science Institute, Berkeley CA

Tandem modeling investigations Dan Ellis International Computer Science Institute, Berkeley CA

Tandem modeling investigations Dan Ellis International Computer Science Institute, Berkeley CA &lt;dpwe@icsi.berkeley.edu&gt; Outline 1 What makes Tandem successful? 2 Can we make Tandem better? 3 Does Tandem work with LVCSR tricks?

716 views • 7 slides
Lecture 2 Signal Processing and Dynamic Time Warping Michael Picheny, Bhuvana Ramabhadran,

Lecture 2 Signal Processing and Dynamic Time Warping Michael Picheny, Bhuvana Ramabhadran,

Lecture 2 Signal Processing and Dynamic Time Warping Michael Picheny, Bhuvana Ramabhadran, Stanley F . Chen, Markus Nussbaum-Thom Watson Group IBM T.J. Watson Research Center Yorktown Heights, New York, USA

1.46k views • 134 slides
AGILE Speech to Text (STT) Contributors: BBN: Long Nguyen, Tim Ng, Kham Nguyen, Rabih Zbib, John

AGILE Speech to Text (STT) Contributors: BBN: Long Nguyen, Tim Ng, Kham Nguyen, Rabih Zbib, John

AGILE Speech to Text (STT) Contributors: BBN: Long Nguyen, Tim Ng, Kham Nguyen, Rabih Zbib, John Makhoul CU: Andrew Liu, Frank Diehl, Marcus Tomalin, Mark Gales, Phil Woodland LIMSI: Lori Lamel, Abdel Messaoudi, Jean-Luc Gauvain, Petr Fousek, Jun

1.09k views • 44 slides
Inexact Tensor Methods with Dynamic Accuracies Nikita Doikov Yurii Nesterov UCLouvain, Belgium

Inexact Tensor Methods with Dynamic Accuracies Nikita Doikov Yurii Nesterov UCLouvain, Belgium

Inexact Tensor Methods with Dynamic Accuracies Nikita Doikov Yurii Nesterov UCLouvain, Belgium ICML 2020 Plan of the talk 1. Introduction: Tensor Methods in Convex Optimization 2. Inexact Tensor Methods 3. Acceleration 4. Numerical Example 2

784 views • 22 slides
Multimodal Biometrics Josef Kittler Centre for Vision, Speech and Signal Processing University

Multimodal Biometrics Josef Kittler Centre for Vision, Speech and Signal Processing University

Centre for Vision, Speech and Signal Processing Multimodal Biometrics Josef Kittler Centre for Vision, Speech and Signal Processing University of Surrey, Guildford GU2 7XH J.Kittler@surrey.ac.uk Acknowledgements: Dr Norman Poh 1 Biometric

965 views • 59 slides
Probabilistic Logic Programming for Natural Language Processing Fabrizio Riguzzi, Evelina Lamma,

Probabilistic Logic Programming for Natural Language Processing Fabrizio Riguzzi, Evelina Lamma,

Probabilistic Logic Programming for Natural Language Processing Fabrizio Riguzzi, Evelina Lamma, Marco Alberti, Elena Bellodi , Riccardo Zese, Giuseppe Cota Dipartimento di Matematica e Informatica Dipartimento di Ingegneria Universit` a di

384 views • 20 slides

More recommend