an index advisor using deep reinforcement learning
play

An Index Advisor Using Deep Reinforcement Learning Hai Lan 1 , - PowerPoint PPT Presentation

Feb 09, 2024 •190 likes •581 views

An Index Advisor Using Deep Reinforcement Learning Hai Lan 1 , Zhifeng Bao 1 , Yuwei Peng 2 1 RMIT University 2 Wuhan University 1 Index Selection Problem (ISP) 2 Index Selection Problem (ISP) Choosing the right indexes to build is one of

  1. An Index Advisor Using Deep Reinforcement Learning Hai Lan 1 , Zhifeng Bao 1 , Yuwei Peng 2 1 RMIT University 2 Wuhan University 1

  2. Index Selection Problem (ISP) 2

  3. Index Selection Problem (ISP) • Choosing the right indexes to build is one of the central issues in database tuning. • Problem Definition: • Select a set of indexes (index configuration) to be built to maximize the performance of the given workload with some constraints. • Constraints: storage usage, index number, and so on. 2

  4. Index Selection Problem (ISP) • Choosing the right indexes to build is one of the central issues in database tuning. • Problem Definition: • Select a set of indexes (index configuration) to be built to maximize the performance of the given workload with some constraints. • Constraints: storage usage, index number, and so on. • Index interaction : an interaction exists between an index a and an index b if the benefit of a is affected by the existence of b and vice - versa . SELECT * FROM t WHERE a < 10 OR b < 10; (1) An index on a ✗ (2) An index on b ✗ (3) An index on a and an index on b ✓ 2

  5. Prior Work 3

  6. Prior Work Category Work Cost Index type Alog Cons IIA ✗ Non - Learning AutoAdmin [ VLD ’97] Estimated cost S / M Greedy index number method ✗ ILP [ ICDE ’07] Estimated cost S / M ILP storage ✓ ISRM [ ICDE 19] Estimated cost S / M Greedy storage Learning - based AI Meet AI [ SIGMOD ’19] Learning - model S / M Not sure Greedy index number method Estimated cost ✓ Welborn et al [ arxiv ’19] Not mention S / M DQN no ✓ DRL - Index [ ICDEW ’20] Estimated cost S DQN Not mention IIA means index interaction . Cons means constraints . Alog means search algorithm . S means single column index . M means multi - column index . Welborn ’ s work only focuses on single table . 3 DRL - index is not implemented yet .

  7. Prior Work Category Work Cost Index type Alog Cons IIA ✗ Non - Learning AutoAdmin [ VLD ’97] Estimated cost S / M Greedy index number method ✗ ILP [ ICDE ’07] Estimated cost S / M ILP storage ✓ ISRM [ ICDE 19] Estimated cost S / M Greedy storage Learning - based AI Meet AI [ SIGMOD ’19] Learning - model S / M Not sure Greedy index number method Estimated cost ✓ Welborn et al [ arxiv ’19] Not mention S / M DQN no ✓ DRL - Index [ ICDEW ’20] Estimated cost S DQN Not mention IIA means index interaction . Cons means constraints . Alog means search algorithm . S means single column index . M means multi - column index . Welborn ’ s work only focuses on single table . 3 DRL - index is not implemented yet .

  8. Prior Work Category Work Cost Index type Alog Cons IIA ✗ Non - Learning AutoAdmin [ VLD ’97] Estimated cost S / M Greedy index number method ✗ ILP [ ICDE ’07] Estimated cost S / M ILP storage ✓ ISRM [ ICDE 19] Estimated cost S / M Greedy storage Learning - based AI Meet AI [ SIGMOD ’19] Learning - model S / M Not sure Greedy index number method Estimated cost ✓ Welborn et al [ arxiv ’19] Not mention S / M DQN no ✓ DRL - Index [ ICDEW ’20] Estimated cost S DQN Not mention Our Goal : (1) Handle complex queries on multiple tables (2) Recommend multi - column indexes (3) Capture the index interaction IIA means index interaction . Cons means constraints . Alog means search algorithm . S means single column index . M means multi - column index . Welborn ’ s work only focuses on single table . 3 DRL - index is not implemented yet .

  9. Our Method - Overview 4

  10. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance 4

  11. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance 4

  12. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance workload 4

  13. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload 4

  14. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload The algorithm to select an index from candidates according to current workload and index configuration 4

  15. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration 4

  16. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration • T is determined by the constraints . 4

  17. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration • T is determined by the constraints . • Framework 4

  18. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration • T is determined by the constraints . • Framework Index Candidates Rules Workload Sample 4

  19. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration • T is determined by the constraints . • Framework Index Candidates Rules Workload Sample action DQN 4 Agent

  20. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps The algorithm to select an index from candidates according to current workload and index configuration • T is determined by the constraints . Create • Framework transform Index Candidates Rules Workload Sample action DQN 4 Agent

  21. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps Environment DB The algorithm to select an index from candidates according to What - If Caller current workload and index configuration • T is determined by the constraints . Create • Framework transform Index Candidates Rules Workload Sample action DQN 4 Agent

  22. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps Environment DB The algorithm to select an index from candidates according to What - If Caller current workload and index configuration • T is determined by the constraints . Create • Framework Reward Next state transform Index Candidates Rules Workload Sample action DQN 4 Agent

  23. Our Method - Overview • Formulate Index Selection as a reinforcement learning problem • Maximize the Performance Index configuration before starting the step t workload steps Environment DB The algorithm to select an index from candidates according to What - If Caller current workload and index configuration • T is determined by the constraints . Create • Framework Reward Next state transform Index Candidates Rules Workload Sample action DQN 4 Agent

  24. Our Method - Rules 5

  25. Our Method - Rules 5

  26. Our Method - Rules 5

  27. Our Method - Rules SELECT t 1. a 7 FROM t 1, t 2 WHERE t 1. a 1 = t 2. b 1 AND t 1. a 2 = t 2. b 2 AND t 1. a 3 = 4 AND t 2. b 3 < 10 ORDER BY t 1. a 5, t 1. a 6 5

  28. Our Method - Rules SELECT t 1. a 7 FROM t 1, t 2 WHERE t 1. a 1 = t 2. b 1 AND t 1. a 2 = t 2. b 2 AND t 1. a 3 = 4 AND t 2. b 3 < 10 ORDER BY t 1. a 5, t 1. a 6 J : t 1. a 2, t 1. a 3, t 2. b 2, t 2. b 3 EQ : t 1. a 3 RANGE : t 2. b 3 O : t 1. a 5, t 1. a 6 USED : t 1.( a 1- a 7), t 2.( b 1- b 3) 5

  29. Our Method - Rules SELECT t 1. a 7 FROM t 1, t 2 WHERE t 1. a 1 = t 2. b 1 AND t 1. a 2 = t 2. b 2 AND t 1. a 3 = 4 AND t 2. b 3 < 10 ORDER BY t 1. a 5, t 1. a 6 t 1. a 1, t 1. a 2, t 1. a 3, t 2. b 1, t 2. b 2, t 2. b 3 Rule 1 J : t 1. a 2, t 1. a 3, t 2. b 2, t 2. b 3 EQ : t 1. a 3 RANGE : t 2. b 3 O : t 1. a 5, t 1. a 6 USED : t 1.( a 1- a 7), t 2.( b 1- b 3) 5

  30. Our Method - Rules SELECT t 1. a 7 FROM t 1, t 2 WHERE t 1. a 1 = t 2. b 1 AND t 1. a 2 = t 2. b 2 AND t 1. a 3 = 4 AND t 2. b 3 < 10 ORDER BY t 1. a 5, t 1. a 6 t 1. a 1, t 1. a 2, t 1. a 3, t 2. b 1, t 2. b 2, t 2. b 3 Rule 1 J : t 1. a 2, t 1. a 3, t 2. b 2, t 2. b 3 Rule 2 EQ : t 1. a 3 ( t 1. a 5, t 1. a 6) RANGE : t 2. b 3 O : t 1. a 5, t 1. a 6 USED : t 1.( a 1- a 7), t 2.( b 1- b 3) 5

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

540 views • 15 slides
1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement learning? Agent/Actor + Action + Environment + State + Reward How does reinforcement learning work?

792 views • 31 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature 2015] CS 486/686 University of Waterloo Lecture 20: July 10, 2017 Outline Value Function Approximation Linear approximation Neural

706 views • 19 slides
Cooperation and Communication in Multiagent Deep Reinforcement Learning Matthew Hausknecht Nov

Cooperation and Communication in Multiagent Deep Reinforcement Learning Matthew Hausknecht Nov

Cooperation and Communication in Multiagent Deep Reinforcement Learning Matthew Hausknecht Nov 28, 2016 Advisor: Peter Stone 1 Motivation Intelligent decision making is at the heart of AI. 2 Thesis Question How can the power of Deep

1.81k views • 121 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides
Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1.

Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1.

Introduction to Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 1 is due next Wednesday! Remember that Monday is a holiday, so no office hours 2. Remember to start forming final project

877 views • 48 slides
Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 is extended by one week, to Wednesday after next Todays Lecture 1. Managing overfitting in model-based RL

686 views • 40 slides
Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning for robots? slide from V. Vanhoucke End-to-end Deep Learning for robots? 2010 : Speech Recognition Audio Acoustic Model Phonetic Model

1.05k views • 65 slides
Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index Basics of Reinforcement Learning Model Based vs Model Free Reinforcement Learning Autonomous Car collision avoidance What is Reinforcement

714 views • 24 slides
Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays Lecture 1. So far: manually design reward function to define a task 2. What if we want to learn the reward function from observing an expert, and

650 views • 33 slides
CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe

CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe

DeepLoco by X. B. Peng, G. Berseth &amp; M. van de Panne CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe University // Spring 2018 Neural Face by Taehoon Kim Previously on CMP784 Generative

895 views • 71 slides
Deep Reinforcement Learning for Robotics:

Deep Reinforcement Learning for Robotics:

Deep Reinforcement Learning for Robotics: Frontiers and Beyond Shixiang (Shane) Gu ( ) 2018.5.27 01 Deep RL: successes and limitations Computation-Constrained Data-Constrained

634 views • 21 slides
Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina

Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina Fragkiadaki Course Logistics Course website : all you need to

1.13k views • 88 slides
Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement

Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement

Reinforcement Learning Q-Learning Deep Q-Learning on Atari Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement Learning Q-Learning Deep Q-Learning on Atari Table of Contents Reinforcement Learning

938 views • 63 slides
MATH 676 Finite element methods in scientific computing Wolfgang Bangerth, Texas A&amp;M

MATH 676 Finite element methods in scientific computing Wolfgang Bangerth, Texas A&amp;M

MATH 676 Finite element methods in scientific computing Wolfgang Bangerth, Texas A&amp;M University http://www.dealii.org/ Wolfgang Bangerth Lecture 41: Parallelization on a cluster of distributed memory machines Part 1: Introduction

685 views • 32 slides
FEBRUARY 2016 WEBINAR Quality Assurance Visit Information Critical Elements 1-3 Re-Accreditation

FEBRUARY 2016 WEBINAR Quality Assurance Visit Information Critical Elements 1-3 Re-Accreditation

FEBRUARY 2016 WEBINAR Quality Assurance Visit Information Critical Elements 1-3 Re-Accreditation Planning Danielle Morago HFO QA/TA Specialist 2016 Quality Assurance Visits Self-Study Reminders Critical Element 1 AGENDA Critical

535 views • 16 slides
Why should the THRIVE Framework for system change and the National i-THRIVE Programme be

Why should the THRIVE Framework for system change and the National i-THRIVE Programme be

Why should the THRIVE Framework for system change and the National i-THRIVE Programme be considered as a basis for transforming the offer for children and young peoples mental health and wellbeing ? What is the THRIVE Framework for system

399 views • 3 slides
How to build and maintain an effective Mental Health First Aid programme Vanessa Sallows Claims

How to build and maintain an effective Mental Health First Aid programme Vanessa Sallows Claims

How to build and maintain an effective Mental Health First Aid programme Vanessa Sallows Claims &amp; Governance Director 81% 1 in 3 of the UK workforce have been of LGBT+ people have experienced a formally diagnosed with a mental mental

242 views • 12 slides
Enabling IPv4 access over IPv6 mobility NEMO WG, 64th IETF Vancouber Keiichi SHIMA

Enabling IPv4 access over IPv6 mobility NEMO WG, 64th IETF Vancouber Keiichi SHIMA

Enabling IPv4 access over IPv6 mobility NEMO WG, 64th IETF Vancouber Keiichi SHIMA &lt;keiichi@iijlab.net&gt; Internet Initiative Japan Nautilus6 Purpose of this presentation To fill missing pieces of draft-ietf-mip6-nemo-

370 views • 10 slides
Did I happen to mention? Final exam Final Exam Review The date for the Final has been

Did I happen to mention? Final exam Final Exam Review The date for the Final has been

Did I happen to mention? Final exam Final Exam Review The date for the Final has been decided: Saturday, November 16 th 8am 10am 01-2000 No makeup finals! Project Notes Today Grand tour through CS4

307 views • 4 slides
Micah 6:8 NIV He has shown you, O mortal, what is good. And what does the LORD require of you? To

Micah 6:8 NIV He has shown you, O mortal, what is good. And what does the LORD require of you? To

Micah 6:8 NIV He has shown you, O mortal, what is good. And what does the LORD require of you? To act justly and to love mercy and to walk humbly with your God. Micah 6:1-2 NIV 1 Listen to what the LORD says: Stand up, plead my case before

300 views • 18 slides
Logo slide Peter, an apostle of Jesus Christ, To Gods elect, strangers in the world, scattered

Logo slide Peter, an apostle of Jesus Christ, To Gods elect, strangers in the world, scattered

Logo slide Peter, an apostle of Jesus Christ, To Gods elect, strangers in the world, scattered throughout Pontus, Galatia, Cappadocia, Asia and Bithynia, 2 who have been chosen according to the foreknowledge of God the Father, through the

399 views • 21 slides

More recommend