A Reinforcement Learning Based System for Minimizing Cloud Storage - PowerPoint PPT Presentation

Sep 24, 2022 •176 likes •360 views

A Reinforcement Learning Based System for Minimizing Cloud Storage Service Cost Haoyu Wang 1 , Haiying Shen 1 , Qi Liu 1 , Kevin Zheng 1 , and Jie Xu 2 1 University of Virginia and 2 George Mason University ICPP2020 Online presentation Web

A Reinforcement Learning Based System for Minimizing Cloud Storage Service Cost Haoyu Wang 1 , Haiying Shen 1 , Qi Liu 1 , Kevin Zheng 1 , and Jie Xu 2 1 University of Virginia and 2 George Mason University ICPP2020 Online presentation
Web application:
Type of storage User Hot Cloud Storage Cold Web Application Archive CSP: Cloud Service Provider
US West Hot $0.0055 US East Cold $0.01 Hot $0.005 Cold $0.01
Outline • How to minimize storage monetary cost • Related work • Wikipedia trace analysis • Markov decision process problem formulation • Main design • Performance evaluation • Conclusion
Minimize storage monetary cost Different price is determined by: • Storage type • Read/write operation frequencies • Storage period
Related work • Cloud storage payment minimization • Cloud resource pricing • Combining cloud providers Unlike the above methods, the goal of our method is to minimize the total payment a cloud storage service customer made to a CSP by leveraging the different types of storage provided by the CSP.
Trace analysis
Trace analysis
Trace analysis
Problem formulation Markov Decision Process M=(S,A,P,R) State space: Action space: Reward:
Main design 1. A3C algorithm used in MiniCost
Main design 2. Concurrent requested data files aggregation
Performance evaluation
Performance evaluation
Conclusion Analysis on the Wikipedia trace demonstrates that the substantial • request frequency variabilities may make it cost-inefficient for cloud storage service customer. An RL based data storage types assignment algorithm that generates • data storage types assignment plans periodically can save monetary cost significantly. Trace-driven experiment shows that our online RL based method can • achieve significant cost savings.
Thank you! Questions?

Download Presentation

Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Reinforcement Learning-Based End-to-End Parking for Automatic Parking System CS885

Reinforcement Learning-Based End-to-End Parking for Automatic Parking System CS885 Reinforcement Learning Paper by: P. Zhang, L. Xiong, Z. Yu, P. Fang, S. Yan, J. Yao, and Y. Zhou (Sensors 2019) Presented by: Neel Bhatt Context and

286 views • 14 slides

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey

Advanced Model-Based Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 is extended by one week, to Wednesday after next Todays Lecture 1. Managing overfitting in model-based RL

686 views • 40 slides

Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers (the class labels) Reinforcement learning: In reinforcement learning, the agent learns Markov Decision Processes based on a punishment/reward

89 views • 6 slides

A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong

A Robotic Auto-Focus System based on Deep Reinforcement Learning Peking University A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong Yang, Xiaohui Duan* School of EECS, Peking University Beijing,

340 views • 21 slides

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning Haarnoja, Tang et al. (2017) Reinforcement Learning with Deep Energy Based Policies, ICML . Haarnoja, Zhou et al. (2018) Soft Actor-Critic: Off-Policy

682 views • 24 slides

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Introduction to Reinforcement Learning RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem Inside an RL agent Temporal difference learning Many faces of Reinforcement Learning What is

551 views • 35 slides

Model Based Reinforcement Learning Oriol Vinyals (DeepMind) @OriolVinyalsML May 2018 Stanford

Model Based Reinforcement Learning Oriol Vinyals (DeepMind) @OriolVinyalsML May 2018 Stanford University The Reinforcement Learning Paradigm OBSERVATIONS GOAL Agent Environment ACTIONS The Reinforcement Learning Paradigm Maximize Return -

1.26k views • 72 slides

Paths to Learning Reinforcement Learning Ever wonder how the orcas and dolphins at Sea World are

Paths to Learning Reinforcement Learning Ever wonder how the orcas and dolphins at Sea World are trained? They learn through a simple system of reinforcement learning, where the desired behaviors are met with rewards such as raw fish and back

248 views • 5 slides

Model Based Reinforcement Learning Katerina Fragkiadaki Model Anything the agent can use to

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Model Based Reinforcement Learning Katerina Fragkiadaki Model Anything the agent can use to predict how the environment will respond to its actions,

826 views • 65 slides

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine playing a new game whose rules you dont know; after a hundred or so moves your don t know; after a hundred or so moves, your opponent announces, You

511 views • 30 slides

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 4: Q-Value based RL Animesh

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 4: Q-Value based RL Animesh Garg Deep Reinforcement Learning with Double Q-learning Hado van Hasselt, Arthur Guez, David Silver Dueling Network Architectures for Deep

535 views • 24 slides

Reinforcement Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 13

0. Reinforcement Learning Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 13 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. Reinforcement Learning Overview Task: Control

396 views • 24 slides

Reinforcement Learning and Simulation-Based Search David Silver Reinforcement Learning and

Reinforcement Learning and Simulation-Based Search Reinforcement Learning and Simulation-Based Search David Silver Reinforcement Learning and Simulation-Based Search Outline 1 Reinforcement Learning 2 Simulation-Based Search 3 Planning Under

425 views • 20 slides

Symbolic Regression for Reinforcement Learning and Dynamic System Modeling Robert Babuka 1

Symbolic Regression for Reinforcement Learning and Dynamic System Modeling Robert Babuka 1 Research interests Clustering for building locally linear models Reinforcement learning for continuous dynamic systems Neural

604 views • 31 slides

SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning Marvin Zhang*,

SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning Marvin Zhang*, Sharad Vikram*, Laura Smith, Pieter Abbeel, Matthew J Johnson, Sergey Levine UC Berkeley, UC San Diego, Google Efficient reinforcement learning from

212 views • 20 slides

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index Basics of Reinforcement Learning Model Based vs Model Free Reinforcement Learning Autonomous Car collision avoidance What is Reinforcement

714 views • 24 slides

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow) & David Robinson (DataCamp) https://www.tidytextmining.com/ R for Data Science: Garrett Grolemund & Hadley Wickham http://r4ds.had.co.nz/

866 views • 15 slides

Programming Style Crista Lopes impressionism abstract expressionism modernism realism

Exercises in Programming Style Crista Lopes impressionism abstract expressionism modernism realism photorealism surrealism cubism Rules and constraints in software construction PROGRAMMING STYLES Programming Styles Ways of expressing

900 views • 62 slides

Advanced Analytics in Business [D0S07a] Big Data Platforms & Technologies [D0S06a] Text

Advanced Analytics in Business [D0S07a] Big Data Platforms & Technologies [D0S06a] Text Mining Overview Featurization Traditional approaches Word embeddings and representational learning A look ahead Tooling 2 Text is everywhere

1.61k views • 61 slides

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu February 17, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Decision Tree;

337 views • 17 slides

TERENA Trusted Cloud Drive pilot STATUS UPDATE Motivations Since 2 0 1 0 , TERENA has been

TF-Storage meeting 26-27 September 2012 Dubrovnik, Croatia Peter Szegedi Project Development Officer szegedi@terena.org www.terena.org TERENA Trusted Cloud Drive pilot STATUS UPDATE Motivations Since 2 0 1 0 , TERENA has been actively

545 views • 21 slides

An Additional Survey of In-Network Storage Systems

278 views • 10 slides

OpenStack Summit Primer: The Who, What, Why, and How of OpenStack Presented by Ben Silverman,

OpenStack Summit Primer: The Who, What, Why, and How of OpenStack Presented by Ben Silverman, Cincinnati Bell Technology Services Monday, May 13, 2019 BS DETAILS 30+ years of IT experience 6+ years experience with OpenStack Former

376 views • 36 slides

INFRASTRUCTURE IN THE NETHERLANDS WITH IRODS IRODS USER GROUP MEETING JUNE 11, 2020 Agenda

BUILDING A NATIONAL RDM INFRASTRUCTURE IN THE NETHERLANDS WITH IRODS IRODS USER GROUP MEETING JUNE 11, 2020 Agenda Short introduction RDM infrastructure, why? How did we start? of SURF What do we build? How do we do this? WHO ARE WE?

609 views • 31 slides