the landscape of empirical risk for non convex losses
play

The landscape of empirical risk for non-convex losses Song Mei - PowerPoint PPT Presentation

Apr 10, 2024 •11 likes •341 views

The landscape of empirical risk for non-convex losses Song Mei ICME, Stanford December 3, 2016 Joint work with Yu Bai and Andrea Montanari Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 1 / 17 Binary linear

  1. The landscape of empirical risk for non-convex losses Song Mei ICME, Stanford December 3, 2016 Joint work with Yu Bai and Andrea Montanari Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 1 / 17

  2. Binary linear classification The model Z ✐ ❂ ✭ X ✐ ❀ ❨ ✐ ✮ . X ✐ ✷ R ❞ , ❨ ✐ ✷ ❢ ✵ ❀ ✶ ❣ , i = 1,. . . , n. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 2 / 17

  3. Non-convex formulation of binary classification The model Z ✐ ❂ ✭ X ✐ ❀ ❨ ✐ ✮ . X ✐ ✷ R ❞ , ❨ ✐ ✷ ❢ ✵ ❀ ✶ ❣ , i = 1,. . . , n. ◮ Convex logit loss ( ❵ ❝ is cvx in θ ) ✏ ✑ ❵ ❝ ✭ θ ❀ Z ✮ ❂ ❨ ❤ X ❀ θ ✐ � ❧♦❣ ✶ ✰ ❡①♣✭ ❤ X ❀ θ ✐ ✮ ✿ ◮ Non-convex loss ( ❵ is not cvx in θ ) ✏ ✑ ✷ ❵ ✭ θ ❀ Z ✮ ❂ ❨ � ✛ ✭ ❤ X ❀ θ ✐ ✮ ❀ where ✛ ✭ t ✮ ❂ ✶ ❂ ✭✶ ✰ ❡①♣✭ t ✮✮ ✿ ◮ Empirical Risk ♥ ❳ ❘ ♥ ✭ θ ✮ ❂ ✶ ❜ ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ ◮ Empirical risk minimizer ❫ ❜ θ ♥ ❂ ❛r❣ ♠✐♥ ❘ ♥ ✭ θ ✮ ✿ θ ✷ B ❞ ✭ ❘ ✮ Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 3 / 17

  4. Non-convex formulation of binary classification The model Z ✐ ❂ ✭ X ✐ ❀ ❨ ✐ ✮ . X ✐ ✷ R ❞ , ❨ ✐ ✷ ❢ ✵ ❀ ✶ ❣ , i = 1,. . . , n. ◮ Convex logit loss ( ❵ ❝ is cvx in θ ) ✏ ✑ ❵ ❝ ✭ θ ❀ Z ✮ ❂ ❨ ❤ X ❀ θ ✐ � ❧♦❣ ✶ ✰ ❡①♣✭ ❤ X ❀ θ ✐ ✮ ✿ ◮ Non-convex loss ( ❵ is not cvx in θ ) ✏ ✑ ✷ ❵ ✭ θ ❀ Z ✮ ❂ ❨ � ✛ ✭ ❤ X ❀ θ ✐ ✮ ❀ where ✛ ✭ t ✮ ❂ ✶ ❂ ✭✶ ✰ ❡①♣✭ t ✮✮ ✿ ◮ Empirical Risk ♥ ❳ ❘ ♥ ✭ θ ✮ ❂ ✶ ❜ ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ ◮ Empirical risk minimizer ❫ ❜ θ ♥ ❂ ❛r❣ ♠✐♥ ❘ ♥ ✭ θ ✮ ✿ θ ✷ B ❞ ✭ ❘ ✮ Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 3 / 17

  5. Non-convex formulation of binary classification The model Z ✐ ❂ ✭ X ✐ ❀ ❨ ✐ ✮ . X ✐ ✷ R ❞ , ❨ ✐ ✷ ❢ ✵ ❀ ✶ ❣ , i = 1,. . . , n. ◮ Convex logit loss ( ❵ ❝ is cvx in θ ) ✏ ✑ ❵ ❝ ✭ θ ❀ Z ✮ ❂ ❨ ❤ X ❀ θ ✐ � ❧♦❣ ✶ ✰ ❡①♣✭ ❤ X ❀ θ ✐ ✮ ✿ ◮ Non-convex loss ( ❵ is not cvx in θ ) ✏ ✑ ✷ ❵ ✭ θ ❀ Z ✮ ❂ ❨ � ✛ ✭ ❤ X ❀ θ ✐ ✮ ❀ where ✛ ✭ t ✮ ❂ ✶ ❂ ✭✶ ✰ ❡①♣✭ t ✮✮ ✿ ◮ Empirical Risk ♥ ❳ ❘ ♥ ✭ θ ✮ ❂ ✶ ❜ ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ ◮ Empirical risk minimizer ❫ ❜ θ ♥ ❂ ❛r❣ ♠✐♥ ❘ ♥ ✭ θ ✮ ✿ θ ✷ B ❞ ✭ ❘ ✮ Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 3 / 17

  6. Non-convex formulation of binary classification The model Z ✐ ❂ ✭ X ✐ ❀ ❨ ✐ ✮ . X ✐ ✷ R ❞ , ❨ ✐ ✷ ❢ ✵ ❀ ✶ ❣ , i = 1,. . . , n. ◮ Convex logit loss ( ❵ ❝ is cvx in θ ) ✏ ✑ ❵ ❝ ✭ θ ❀ Z ✮ ❂ ❨ ❤ X ❀ θ ✐ � ❧♦❣ ✶ ✰ ❡①♣✭ ❤ X ❀ θ ✐ ✮ ✿ ◮ Non-convex loss ( ❵ is not cvx in θ ) ✏ ✑ ✷ ❵ ✭ θ ❀ Z ✮ ❂ ❨ � ✛ ✭ ❤ X ❀ θ ✐ ✮ ❀ where ✛ ✭ t ✮ ❂ ✶ ❂ ✭✶ ✰ ❡①♣✭ t ✮✮ ✿ ◮ Empirical Risk ♥ ❳ ❘ ♥ ✭ θ ✮ ❂ ✶ ❜ ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ ◮ Empirical risk minimizer ❫ ❜ θ ♥ ❂ ❛r❣ ♠✐♥ ❘ ♥ ✭ θ ✮ ✿ θ ✷ B ❞ ✭ ❘ ✮ Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 3 / 17

  7. Why use non-convex loss? ◮ Comparing to logistic regression, non-convex formulation is robust to ourliers. ◮ This model is the same as neural network with a single layer and a single node. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 4 / 17

  8. Why use non-convex loss? ◮ Comparing to logistic regression, non-convex formulation is robust to ourliers. ◮ This model is the same as neural network with a single layer and a single node. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 4 / 17

  9. ❜ ❘ ♥ ✭ ✮ A negative theoretical result 1996 [AHW ✰ 96]) Theorem (Auer et. al. For the non-convex binary classification problem, for any ♥ and ❞ , ✐ ❂✶ such that the empirical risk ❜ there exists a dataset ✭ x ✐ ❀ ② ✐ ✮ ♥ ❘ ♥ ✭ θ ✮ ❞ ❝ ❞ distinct local minima. has ❜ ♥ Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 5 / 17

  10. A negative theoretical result 1996 [AHW ✰ 96]) Theorem (Auer et. al. For the non-convex binary classification problem, for any ♥ and ❞ , ✐ ❂✶ such that the empirical risk ❜ there exists a dataset ✭ x ✐ ❀ ② ✐ ✮ ♥ ❘ ♥ ✭ θ ✮ ❞ ❝ ❞ distinct local minima. has ❜ ♥ Seems to imply the landscape of the non-convex empirical risk ❜ ❘ ♥ ✭ θ ✮ is very rough. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 5 / 17

  11. A negative theoretical result 1996 [AHW ✰ 96]) Theorem (Auer et. al. For the non-convex binary classification problem, for any ♥ and ❞ , ✐ ❂✶ such that the empirical risk ❜ there exists a dataset ✭ x ✐ ❀ ② ✐ ✮ ♥ ❘ ♥ ✭ θ ✮ ❞ ❝ ❞ distinct local minima. has ❜ ♥ Seems to imply the landscape of the non-convex empirical risk ❜ ❘ ♥ ✭ θ ✮ is very rough. Is this the end of the world of non-convex binary classification? Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 5 / 17

  12. Non-convex formulation of binary classification On real data, we "always" observe a unique minimum! Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 6 / 17

  13. Non-convex formulation of binary classification On real data, we "always" observe a unique minimum! Why? Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 6 / 17

  14. Non-convex formulation of binary classification On real data, we "always" observe a unique minimum! Why? Data generated by nature is not against us! Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 6 / 17

  15. A negative theoretical result Theorem (Auer et. al. . 1996 [AHW ✰ 96]) For the non-convex binary classification problem, for all ♥ ❃ ✵ ✐ ❂✶ such that the empirical risk ❜ there exists a dataset ✭ x ✐ ❀ ② ✐ ✮ ♥ ❘ ♥ ✭ θ ✮ ❞ ❝ ❞ distinct local minima. has ❜ ♥ Seems to imply the landscape of the non-convex empirical risk ❜ ❘ ♥ ✭ θ ✮ is very rough. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 7 / 17

  16. Our main positive result Theorem (Mei, Bai, Montanari. 2016 [MBM16]) Assume X ✐ are i.i.d. sub-Gaussian random vectors, and ❨ ✐ are generated via P ✭ ❨ ✐ ❂ ✶ ❥ X ✐ ✮ ❂ ✛ ✭ ❤ X ✐ ❀ θ ✵ ✐ ✮ . Then there exists a constant ❈ depending on ✍ , such that as long as ♥ ✕ ❈❞ ❧♦❣ ❞ , the following happens with probability at least ✶ � ✍ : ❘ ♥ ✭ θ ✮ has a unique local minimizer ❫ ✭ ❛ ✮ ❜ θ ♥ in B ❞ ✭ 0 ❀ ❘ ✮ . ♣ ✭ ❜ ✮ ❫ θ ♥ satisfies ❦ ❫ θ ♥ � θ ✵ ❦ ✷ ✔ ❈ ✭ ❞ ❧♦❣ ♥ ✮ ❂♥ . ✭ ❝ ✮ Gradient descent converges exponentially fast to ❫ θ ♥ . The landscape of the non-convex empirical risk ❜ ❘ ♥ ✭ θ ✮ is actually smooth! Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 8 / 17

  17. Why assuming a statistical model make the landscape of emprical risk smooth? ✐✿✐✿❞✿ 1 Assuming a statistical model Z ✐ ✘ P Z , ✐ ❂ ✶ ❀ ✿ ✿ ✿ ❀ ♥ , we can define the population risk ✧ ★ ❤ ✐ ♥ ❳ ✶ ❜ ❘ ✭ θ ✮ ❂ E Z ❘ ♥ ✭ θ ✮ ❂ E Z ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ The population risk is usually very smooth. 2 We can transfer the good properties of the population risk to the empirical risk using uniform convergence argument. So empirical risk will be also smooth. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 9 / 17

  18. Why assuming a statistical model make the landscape of emprical risk smooth? ✐✿✐✿❞✿ 1 Assuming a statistical model Z ✐ ✘ P Z , ✐ ❂ ✶ ❀ ✿ ✿ ✿ ❀ ♥ , we can define the population risk ✧ ★ ❤ ✐ ♥ ❳ ✶ ❜ ❘ ✭ θ ✮ ❂ E Z ❘ ♥ ✭ θ ✮ ❂ E Z ❵ ✭ θ ❀ Z ✐ ✮ ✿ ♥ ✐ ❂✶ The population risk is usually very smooth. 2 We can transfer the good properties of the population risk to the empirical risk using uniform convergence argument. So empirical risk will be also smooth. Song Mei (ICME, Stanford) The landscape of empirical risk December 3, 2016 9 / 17

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Di ff erentially Private Empirical Risk Minimization with Non-convex Loss Functions Di Wang ,

Di ff erentially Private Empirical Risk Minimization with Non-convex Loss Functions Di Wang ,

Di ff erentially Private Empirical Risk Minimization with Non-convex Loss Functions Di Wang , Changyou Chen and Jinhui Xu State University of New York at Bu ff alo International Conference on Machine Learning 2019 Di Wang Non-convex DP-ERM ICML

544 views • 27 slides
The landscape of non-convex losses for statistical learning problems Song Mei Stanford

The landscape of non-convex losses for statistical learning problems Song Mei Stanford

The landscape of non-convex losses for statistical learning problems Song Mei Stanford University October 19, 2017 Song Mei (Stanford University) The landscape of non-convex optimization October 19, 2017 1 / 32 Deep learning Song Mei

834 views • 66 slides
Introduction to Machine Learning Vapnik Chervonenkis Theory Barnabs Pczos Empirical Risk

Introduction to Machine Learning Vapnik Chervonenkis Theory Barnabs Pczos Empirical Risk

Introduction to Machine Learning Vapnik Chervonenkis Theory Barnabs Pczos Empirical Risk and True Risk 2 Empirical Risk Shorthand: True risk of f (deterministic) : Bayes risk : Let us use the empirical counter part: Empirical risk: 3

1.46k views • 69 slides
Empirical Methods Empirical Methods t= a +b Research Landscape Quantitative =

Empirical Methods Empirical Methods t= a +b Research Landscape Quantitative =

Empirical Methods Empirical Methods t= a +b Research Landscape Quantitative = Positivist/post-positivist approach Evaluate hypotheses via experimentation Qualitative = Constructivist approach Build theory from data Overview:

435 views • 25 slides
Overparametrization for Landscape Design in Non-convex Optimization Jason D. Lee University of

Overparametrization for Landscape Design in Non-convex Optimization Jason D. Lee University of

Overparametrization for Landscape Design in Non-convex Optimization Jason D. Lee University of Southern California October 8, 2018 Jason Lee The State of Non-Convex Optimization Practical observation: Empirically, non-convexity is not an

889 views • 30 slides
Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization Zhenxun

Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization Zhenxun

Surrogate Losses for Online Learning of Stepsizes in Stochastic Non-Convex Optimization Zhenxun Zhuang 1 , Ashok Cutkosky 2 , Francesco Orabona 1 , 3 1 Department of Computer Science, Boston University 2 Google 3 Department of Electrical &

121 views • 10 slides
Calibration of Convex Surrogate Losses via Property Elicitation Jessie Finocchiaro October 10,

Calibration of Convex Surrogate Losses via Property Elicitation Jessie Finocchiaro October 10,

Calibration of Convex Surrogate Losses via Property Elicitation Jessie Finocchiaro October 10, 2019 Introduction Machine Learning Predictions about Some property of data future events Outline Background Surrogates Calibration

904 views • 44 slides
Empirical Risk Minimization October 29, 2015 Outline Empirical risk minimization view

Empirical Risk Minimization October 29, 2015 Outline Empirical risk minimization view

Empirical Risk Minimization October 29, 2015 Outline Empirical risk minimization view Perceptron CRF Notation for Linear Models Training data: {(x 1 , y 1 ), (x 2 , y 2 ), , (x N , y N )} Testing data: {(x N+1 , y N+1 ),

418 views • 21 slides
Contents of Presentation Types of losses Causes of losses Prevention of losses

Contents of Presentation Types of losses Causes of losses Prevention of losses

Contents of Presentation Types of losses Causes of losses Prevention of losses Documenting and reporting of losses Types of Losses Marine Internal Inland Internal Loss location: At Port At Warehouse At Food

407 views • 23 slides
CSC2412: Private Gradient Descent & Empirical Risk Minimization Sasho Nikolov 1 Empirical

CSC2412: Private Gradient Descent & Empirical Risk Minimization Sasho Nikolov 1 Empirical

CSC2412: Private Gradient Descent & Empirical Risk Minimization Sasho Nikolov 1 Empirical Risk Minimization Learning: Reminder Known data universe X and an unknown probability distribution D on X Known concept class C and an unknown

341 views • 17 slides
An Empirical Look at the Loss Landscape HEP AI - September 4, 2018 Components of training an

An Empirical Look at the Loss Landscape HEP AI - September 4, 2018 Components of training an

An Empirical Look at the Loss Landscape HEP AI - September 4, 2018 Components of training an image classifier For fixed architecture of ResNet 56 we have: 1. Preprocessing: normalize, shift and flip (show examples) 2. Momentum 3. Weight decay

949 views • 35 slides
Convex hull 1 - 1 Convex hull 1 - 2 Convex hull 1 - 3 Convex hull Definition, extremal

Convex hull 1 - 1 Convex hull 1 - 2 Convex hull 1 - 3 Convex hull Definition, extremal

Convex hull 1 - 1 Convex hull 1 - 2 Convex hull 1 - 3 Convex hull Definition, extremal point Jarvis algorithm Orientation predicate Buggy degenerate example Real RAM model and general position hypothesis Graham

1.41k views • 114 slides
Output of a convex-hull algorithm: a, b, c, g, j A&DS Lecture 16 2 Mary Cryan Polar Angles

Output of a convex-hull algorithm: a, b, c, g, j A&DS Lecture 16 2 Mary Cryan Polar Angles

The Convex Hull Definition 16.1 (1) A set C of points is convex if for all p, q C the whole line segment pq is contained in C . (2) The convex hull of a set S of points is the smallest convex set C that contains S . Observation 16.2 The convex

290 views • 14 slides
ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration

ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration

ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration with D.Freeman (UC Berkeley), Luca Venturi & Afonso Bandeira (NYU) MOTIVATION We consider the standard Empirical Risk Minimization setup: `

683 views • 46 slides
On the Local Minima of the Empirical Risk Chi Jin * 1 , Lydia T. Liu* 1 , Rong Ge 2 , Michael I.

On the Local Minima of the Empirical Risk Chi Jin * 1 , Lydia T. Liu* 1 , Rong Ge 2 , Michael I.

On the Local Minima of the Empirical Risk Chi Jin * 1 , Lydia T. Liu* 1 , Rong Ge 2 , Michael I. Jordan 1 1EECS, University of California, Berkeley. 2Duke University. 1 / 6 Chi Jin On the Local Minima of the Empirical Risk Overview Nonconvex

425 views • 14 slides
Reducing Losses from Flood Risks: Role of the Behavioral Risk Audit Howard Kunreuther

Reducing Losses from Flood Risks: Role of the Behavioral Risk Audit Howard Kunreuther

Reducing Losses from Flood Risks: Role of the Behavioral Risk Audit Howard Kunreuther kunreuth@wharton.upenn.edu James G. Dinan Professor Emeritus of Decision Sciences & Public Policy Co-Director, Risk Management and Decision Processes

345 views • 19 slides
CS485/685 Lecture 15: Feb 28, 2012 Probably Approximately Correct Learning [BDSS] Chapter 1

CS485/685 Lecture 15: Feb 28, 2012 Probably Approximately Correct Learning [BDSS] Chapter 1

28/02/2012 CS485/685 Lecture 15: Feb 28, 2012 Probably Approximately Correct Learning [BDSS] Chapter 1 CS485/685 (c) 2012 P. Poupart 1 Quick Recap Tom Mitchell (1998): A computer program is said to learn from experience E with respect to some

288 views • 9 slides
Introduction to Machine Learning CMU-10701 10. Risk Minimization Barnabs Pczos 10. Risk

Introduction to Machine Learning CMU-10701 10. Risk Minimization Barnabs Pczos 10. Risk

Introduction to Machine Learning CMU-10701 10. Risk Minimization Barnabs Pczos 10. Risk Minimization 2 What have we seen so far? Several algorithms that seem to work fine on training datasets: Linear regression Nave Bayes

1.13k views • 51 slides
9.54 class 8 Supervised learning Optimization, regularization, kernels Shimon Ullman + Tomaso

9.54 class 8 Supervised learning Optimization, regularization, kernels Shimon Ullman + Tomaso

9.54 class 8 Supervised learning Optimization, regularization, kernels Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert 9.54, fall semester 2014 The Regularization Kingdom Loss functions and empirical risk

786 views • 67 slides
Introduction to Machine Learning CART: Splitting Criteria compstat-lmu.github.io/lecture_i2ml

Introduction to Machine Learning CART: Splitting Criteria compstat-lmu.github.io/lecture_i2ml

Introduction to Machine Learning CART: Splitting Criteria compstat-lmu.github.io/lecture_i2ml TREES Classification Tree: Regression Tree: Iris Data 2.5 -1.20 -0.42 0.98 -0.20 -0.01 2.0 Species Petal.Width 1.5 setosa versicolor

401 views • 13 slides
Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks Maksym

Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks Maksym

Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks Maksym Andriushchenko (EPFL ) Matthias Hein (University of T ubingen) Work done at the University of T ubingen SMLD 2019, NeurIPS 2019 Stumps Trees

848 views • 70 slides
Risk bounds for cl classification and re regre ression rules that interpolate Daniel Hsu

Risk bounds for cl classification and re regre ression rules that interpolate Daniel Hsu

Risk bounds for cl classification and re regre ression rules that interpolate Daniel Hsu Computer Science Department & Data Science Institute Columbia University Google Research, 2019 Feb 20 Spoilers Springer Series in Statistics

619 views • 29 slides
Recent Results on Algorithmic Fairness and Meta-Learning Massimiliano Pontil Computational

Recent Results on Algorithmic Fairness and Meta-Learning Massimiliano Pontil Computational

Recent Results on Algorithmic Fairness and Meta-Learning Massimiliano Pontil Computational Statistics and Machine Learning Istituto Italiano di Tecnologia and Department of Computer Science University College London 4th Annual Machine

439 views • 26 slides
The Failure of a Clearinghouse: Empirical Evidence Vincent Bignon Guillaume Vuillemey Banque de

The Failure of a Clearinghouse: Empirical Evidence Vincent Bignon Guillaume Vuillemey Banque de

The Failure of a Clearinghouse: Empirical Evidence Vincent Bignon Guillaume Vuillemey Banque de France HEC Paris & CEPR Conseil scientifique de lAMF Paris April 2017 Vincent Bignon, Guillaume Vuillemey The Failure of a

809 views • 80 slides

More recommend