Ellipse and Gaussian Distribution Prof. Seungchul Lee Industrial AI - - PowerPoint PPT Presentation

β–Ά
ellipse and gaussian distribution
SMART_READER_LITE
LIVE PREVIEW

Ellipse and Gaussian Distribution Prof. Seungchul Lee Industrial AI - - PowerPoint PPT Presentation

Oct 05, 2023 671 likes β€’1.13k views

Ellipse and Gaussian Distribution Prof. Seungchul Lee Industrial AI Lab. Coordinates 2 Coordinates with Basis basis 1 2 basis 1 2 3 Coordinate Transformation 4 Equation of an Ellipse 5 Equation of an

slide-1
SLIDE 1

Ellipse and Gaussian Distribution

  • Prof. Seungchul Lee

Industrial AI Lab.

slide-2
SLIDE 2

Coordinates

2

slide-3
SLIDE 3

Coordinates with Basis

basis ො 𝑦1 ො 𝑦2 basis ො 𝑧1 ො 𝑧2

3

slide-4
SLIDE 4

Coordinate Transformation

4

slide-5
SLIDE 5

Equation of an Ellipse

5

slide-6
SLIDE 6

Equation of an Ellipse

  • Unit circle

6

slide-7
SLIDE 7

Equation of an Ellipse

  • Independent ellipse

7

slide-8
SLIDE 8
  • Dependent ellipse (Rotated ellipse)

Equation of an Ellipse

To find the equation of dependent ellipse

8

slide-9
SLIDE 9
  • Dependent ellipse (Rotated ellipse)
  • Coordinate changes

Equation of an Ellipse

To find the equation of dependent ellipse

9

slide-10
SLIDE 10
  • Dependent ellipse (Rotated ellipse)
  • Coordinate changes

Equation of an Ellipse

To find the equation of dependent ellipse

10

slide-11
SLIDE 11
  • Dependent ellipse (Rotated ellipse)
  • Coordinate changes

Equation of an Ellipse

To find the equation of dependent ellipse

11

slide-12
SLIDE 12

Equation of an Ellipse

  • Dependent ellipse (Rotated ellipse)

To find the equation of dependent ellipse

12

slide-13
SLIDE 13

Equation of an Ellipse

13

slide-14
SLIDE 14

Equation of an Ellipse

14

slide-15
SLIDE 15

Question (Reverse Problem)

  • Given Σ𝑧

βˆ’1 (or Σ𝑧),

– How to find 𝑏 (major axis) and 𝑐 (minor axis) or – How to find the Σ𝑦 or – How to find the proper matrix 𝑉

15

slide-16
SLIDE 16

Question (Reverse Problem)

  • Given Σ𝑧

βˆ’1 (or Σ𝑧),

– How to find 𝑏 (major axis) and 𝑐 (minor axis) or – How to find the Σ𝑦 or – How to find the proper matrix 𝑉

16

slide-17
SLIDE 17

Question (Reverse Problem)

  • Given Σ𝑧

βˆ’1 (or Σ𝑧),

– How to find 𝑏 (major axis) and 𝑐 (minor axis) or – How to find the Σ𝑦 or – How to find the proper matrix 𝑉

  • Eigenvectors of Ξ£

17

slide-18
SLIDE 18

Question (Reverse Problem)

18

slide-19
SLIDE 19

Question (Reverse Problem)

19

slide-20
SLIDE 20

Summary

  • Independent ellipse in ො

𝑦1, ො 𝑦2

  • Dependent ellipse in ො

𝑧1, ො 𝑧2

  • Decouple

– Diagonalize – Eigen-analysis

20

slide-21
SLIDE 21

Gaussian Distribution

21

slide-22
SLIDE 22

Standard Univariate Normal Distribution

  • It is a continuous pdf, but

– Parameterized by only two terms, 𝜈 = 0 and 𝜏 = 1 – This is a big advantage of using Gaussian

22

slide-23
SLIDE 23

Standard Univariate Normal Distribution

23

slide-24
SLIDE 24

Standard Univariate Normal Distribution

  • How to generate data from Gaussian distribution

24

slide-25
SLIDE 25

Univariate Normal Distribution

  • Gaussian or normal distribution, 1D (mean 𝜈, variance 𝜏2)
  • It is a continuous pdf, but parameterized by only two terms, 𝜈 and 𝜏

Affine transformation

25

slide-26
SLIDE 26

Univariate Normal Distribution

26

slide-27
SLIDE 27

Multivariate Gaussian Models

  • Similar to a univariate case, but in a matrix form
  • Multivariate Gaussian models and ellipse

– Ellipse shows constant Ξ”2 value… – The contours of equal probability is ellipse

  • Ellipsoidal probability contours
  • Bell shaped

27

slide-28
SLIDE 28

Two Independent Variables

  • In a matrix form

– Diagonal covariance

28

slide-29
SLIDE 29

Two Independent Variables

  • Geometry of Gaussian
  • Summary in a matrix form

29

slide-30
SLIDE 30

Two Independent Variables

30

slide-31
SLIDE 31

Two Dependent Variables in π’›πŸ, π’›πŸ‘

  • Compute 𝑄𝑍 𝑧 from 𝑄

π‘Œ 𝑦

  • Relationship between 𝑧 and 𝑦

31

slide-32
SLIDE 32

Two Dependent Variables in π’›πŸ, π’›πŸ‘

  • Σ𝑦 : covariance matrix of 𝑦
  • Σ𝑧 : covariance matrix of 𝑧
  • If 𝑣 is an eigenvector matrix of Σ𝑧, then Σ𝑦 is a diagonal matrix

32

slide-33
SLIDE 33

Two Dependent Variables in π’›πŸ, π’›πŸ‘

  • Remark

33

slide-34
SLIDE 34

Two Dependent Variables in π’›πŸ, π’›πŸ‘

34

slide-35
SLIDE 35

Decouple using Covariance Matrix

  • Given data, how to find Σ𝑧 and major (or minor) axis (assume πœˆπ‘§ = 0)
  • Statistics

35

slide-36
SLIDE 36

Decouple using Covariance Matrix

36

slide-37
SLIDE 37

Nice Properties of Gaussian Distribution

37

slide-38
SLIDE 38

Properties of Gaussian Distribution

  • Symmetric about the mean
  • Parameterized
  • Uncorrelated β‡’ independent
  • Gaussian distributions are closed to

– Linear transformation – Affine transformation – Reduced dimension of multivariate Gaussian

  • Marginalization (projection)
  • Conditioning (slice)

– Highly related to inference

38

slide-39
SLIDE 39

Affine Transformation of Gaussian

  • Suppose 𝑦~π’ͺ(πœˆπ‘¦, Σ𝑦)
  • Consider affine transformation of 𝑦
  • Then it is amazing that 𝑧 is Gaussian with

39

slide-40
SLIDE 40

Component of Gaussian Random Vector

  • Suppose 𝑦~π’ͺ(0, Ξ£), 𝑑 ∈ β„π‘œ be a unit vector
  • 𝑧 is the component of 𝑦 in the direction 𝑑
  • 𝑧 is Gaussian with 𝐹 𝑧 = 0, cov 𝑧 = π‘‘π‘ˆΞ£π‘‘
  • So E 𝑧2 = π‘‘π‘ˆΞ£π‘‘
  • The unit vector that minimizes π‘‘π‘ˆΞ£π‘‘ is the eigenvector of Ξ£ with the smallest eigenvalue
  • Notice that we have seen this in PCA

40

slide-41
SLIDE 41

Marginal Probability of Gaussian

  • Suppose 𝑦~π’ͺ(𝜈, Ξ£)
  • Let’s look at the component 𝑦1
  • In fact, the random vector 𝑦1 is also Gaussian.

– (this is not obvious)

41

slide-42
SLIDE 42

Marginalization (Projection)

42

slide-43
SLIDE 43

Conditional Probability of Gaussian

  • The conditional pdf of 𝑦 given 𝑧 is Gaussian
  • The conditional mean is
  • The conditional covariance is
  • Notice that conditional confidence intervals are narrower. i.e., measuring 𝑧 gives information about 𝑦

43

slide-44
SLIDE 44

Conditioning (Slice)

44