Inner Product Spaces and Orthogonality Mongi BLEL King Saud - PowerPoint PPT Presentation

Inner Product Spaces and Orthogonality Mongi BLEL King Saud University August 30, 2019 Mongi BLEL Inner Product Spaces and Orthogonality

Table of contents Mongi BLEL Inner Product Spaces and Orthogonality

Inner Product Definition Let V be a vector space on R . We say that a function � , � : V × V − → R is an inner product on V if it satisfies the following: For all u , v , w ∈ V , α ∈ R . 1 � u , v � = � v , u � 2 � u + v , w � = � u , w � + � v , w � 3 � α u , v � = α � u , v � 4 � u , u � ≥ 0 5 � u , u � = 0 ⇐ ⇒ u = 0 Mongi BLEL Inner Product Spaces and Orthogonality

Examples 1 The Euclidean inner product on R n defined by: n � � u , v � = x j y j = x 1 y 1 + . . . + x n y n , j =1 where u , v ∈ R n , u = ( x 1 , . . . , x n ) and v = ( y 1 , . . . , y n ). 2 If E = C ([0 , 1]) the vector space of continuous functions on [0 , 1]. For all f , g ∈ E , we define the inner product of f and g by: � 1 � f , g � = f ( t ) g ( t )t . . 0 Mongi BLEL Inner Product Spaces and Orthogonality

Remarks If ( E , � , � ) is an inner product space and u , v , w , x ∈ E , a , b , c , d ∈ R , we have: � u + v , w + x � = � u , w � + � u , x � + � v , w � + � v , x � . � au + bv , cw + dx � = ac � u , w � + ad � u , x � + bc � v , w � + bd � v , x � . Mongi BLEL Inner Product Spaces and Orthogonality

Example Let u = ( x , y ) and v = ( a , b ), we define � u , v � = 2 ax + by − bx − ay � , � is an inner product on R 2 . It is enough to prove that � u , u � ≥ 0 and � u , u � = 0 ⇐ ⇒ u = 0. � u , u � = 2 x 2 + y 2 − 2 xy = ( x − y ) 2 + x 2 ≥ 0 and � u , u � = 0 ⇐ ⇒ u = 0. Mongi BLEL Inner Product Spaces and Orthogonality

Example Let u = ( x , y , z ) and v = ( a , b , c ), we define � u , v � = 2 ax + by + 3 cz − bx − ay + cy + bz � , � is an inner product on R 3 . It is enough to prove that � u , u � ≥ 0 and � u , u � = 0 ⇐ ⇒ u = 0. ( y + z − x ) 2 − ( z − x ) 2 + 2 x 2 + 3 z 2 � u , u � = ( y + z − x ) 2 + ( x + z ) 2 + z 2 ≥ 0 = � u , u � = 0 ⇐ ⇒ z = x = y = 0 ⇐ ⇒ u = 0 . Mongi BLEL Inner Product Spaces and Orthogonality

Example Let u = ( x , y , z ) and v = ( a , b , c ), we define � u , v � = 2 ax + by + cz − bx − ay + cy + bz � , � is not an inner product on R 3 . ( y + z − x ) 2 − ( z − x ) 2 + 2 x 2 + z 2 � u , u � = ( y + z − x ) 2 + x 2 + 2 xz = ( y + z − x ) 2 + ( x + z ) 2 − z 2 . = Mongi BLEL Inner Product Spaces and Orthogonality

Example If A = ( a j , k ) ∈ M n ( R ), we define the trace of the matrix A by: n � tr ( A ) = a j , j j =1 and � A , B � = tr ( AB T ) for all A , B ∈ M n ( R ). � A , B � is an inner product on the vector space M n ( R ). Mongi BLEL Inner Product Spaces and Orthogonality

Exercise If u = ( x 1 , x 2 , x 3 ), v = ( y 1 , y 2 , y 3 ), we define the following functions: f , g , h , k : R 2 × R 3 − → R . 1 f ( u , v ) = x 1 y 1 + x 2 y 2 + 2 x 3 y 3 + x 2 y 1 + 2 x 1 y 2 + x 2 y 3 + y 2 x 3 . 2 g ( u , v ) = x 1 y 2 + x 2 y 1 + x 2 y 3 + x 3 y 2 + 3 x 1 y 3 + 3 x 3 y 1 . 3 h ( u , v ) = x 1 y 1 + x 2 y 2 + x 3 y 3 + x 2 y 1 + x 1 y 2 + x 2 y 3 + y 2 x 3 + x 3 y 1 + x 1 y 3 . 4 k ( u , v ) = x 1 y 1 + x 2 y 2 + x 3 y 3 x 2 y 3 x 3 y 2 + x 1 y 3 + y 1 x 3 . Select from which the functions f , g , h , k is an inner product on R 3 . Mongi BLEL Inner Product Spaces and Orthogonality

Solution 1 f ( u , v ) − f ( v , u ) = x 1 y 2 − x 2 y 1 . Then f is not an inner product on R 3 . 2 g ( u , u ) = 2 x 1 x 2 +2 x 2 x 3 +6 x 1 x 3 = 2( x 1 + x 3 )( x 2 +3 x 3 ) − 6 x 2 3 = ( x 1 + x 2 + 4 x 3 ) 2 − ( x 1 − x 2 − 2 x 3 ) 2 − 6 x 2 3 . . Then g is not an inner product on R 3 . 3 x 2 1 + x 2 2 + x 2 h ( u , u ) = 3 + 2 x 1 x 2 + 2 x 2 x 3 + 2 x 1 x 3 ( x 1 + x 2 + x 3 ) 2 = Then h is not an inner product on R 3 because h ( u , u ) = 0 �⇒ u = 0 . 4 Mongi BLEL Inner Product Spaces and Orthogonality

x 2 1 + x 2 2 + x 2 k ( u , u ) = 3 − 2 x 2 x 3 + 2 x 1 x 3 ( x 1 + x 3 ) 2 + x 2 = 2 − 2 x 2 x 3 ( x 1 + x 3 ) 2 + ( x 2 − x 3 ) 2 − x 2 = 3 Then k is not an inner product on R 3 because k ( u , u ) = 0 �⇒ u = 0 . Mongi BLEL Inner Product Spaces and Orthogonality

Example Find the values of a , b such that � ( x 1 , x 2 ) , ( y 1 , y 2 ) � = x 1 y 1 + x 2 y 2 + ax 1 y 2 + bx 2 y 1 is an inner product on R 2 . Mongi BLEL Inner Product Spaces and Orthogonality

Solution � ( x 1 , x 2 ) , ( y 1 , y 2 ) � = � ( y 1 , y 2 ) , ( x 1 , x 2 ) � if a = b . x 2 1 + x 2 � ( x 1 , x 2 ) , ( x 1 , x 2 ) � = 2 + 2 ax 1 x 2 ( x 1 + ax 2 ) 2 + x 2 2 (1 − a 2 ) . = Then � , � is an inner product on R 2 if and only if | a | < 1. Mongi BLEL Inner Product Spaces and Orthogonality

Definition Let ( E , � , � ) be an inner product space. 1 If u ∈ E , we define the norm of the vector u by: � � u � = � u , u � . 2 If u , v ∈ E , we define distance between u and v by: d ( u , v ) = � u − v � . 3 We define the angle 0 ≤ θ ≤ π between the vectors u , v ∈ E by: � u , v � cos θ = � u � . � v � Mongi BLEL Inner Product Spaces and Orthogonality

Let the inner product space M 2 ( R ) , � , � ) defined by: � A , B � = tr ( AB T ) . Find cos θ If θ is the angle between the matrices � 1 � � 2 � − 1 1 A = and B = . 2 3 1 1 � 1 � 0 AB T = , � A � 2 = 15, � B � 2 = 7. 7 5 Then √ cos θ = 2 3 √ . 35 Mongi BLEL Inner Product Spaces and Orthogonality

Theorem (Cauchy-Schwarz Inequality) If ( E , � , � ) is an inner product space and u , v ∈ E , then |� u , v �| ≤ � u �� v � . (1) We have the equality in ( ?? ) if the vectors u , v are linearly dependent. Mongi BLEL Inner Product Spaces and Orthogonality

Proof Let Q ( t ) be the polynomial Q ( t ) = � u + tv � 2 = � u � 2 + 2 t � u , v � + t 2 � v � 2 . Since Q ( t ) ≥ 0 for all t ∈ R , then the discriminant of Q ( t ) is non positive. Then � u , v � 2 ≤ � u � 2 � v � 2 . If |� u , v �| = � u �� v � , this mean that the discriminant of Q ( t ) is zero. Then the equation Q ( t ) = 0 has a solution. This means that the vectors u , v are linearly dependent. Mongi BLEL Inner Product Spaces and Orthogonality

Theorem If ( E , � , � ) is an inner product space and u , v ∈ E , then � u + v � ≤ � u � + � v � . Proof � u � 2 + � v � 2 + 2 � u , v � � u + v � 2 = � u � 2 + � v � 2 + 2 � u � � v � = ( � u � + � v � ) 2 . ≤ Mongi BLEL Inner Product Spaces and Orthogonality

Definition If ( E , � , � ) is an inner product space. We say that the vectors u , v ∈ E are orthogonal and we denote u ⊥ v if � u , v � = 0. Theorem (Pythagor’s Theorem) If u ⊥ v if and only if � u + v � 2 = � u � 2 + � v � 2 . Proof � u + v � 2 = � u � 2 + � v � 2 + 2 � u , v � = � u � 2 + � v � 2 . Mongi BLEL Inner Product Spaces and Orthogonality

Definition If ( E , � , � ) is an inner product space. We say that set S = { e 1 , . . . , e n } of non zeros vectors is orthogonal if � e j , e k � = 0 , ∀ 1 ≤ j � = k ≤ n . and we say that S is normal if � e j � = 1 , ∀ 1 ≤ j ≤ n . and we say that it is orthonormal if � e j , e k � = δ j , k , ∀ 1 ≤ j , k ≤ n . ( δ j , k = 0 If j � = k and δ j , j = 1.) Mongi BLEL Inner Product Spaces and Orthogonality

Theorem Any set of non zero orthogonal vectors is linearly independent . Mongi BLEL Inner Product Spaces and Orthogonality

Theorem If ( E , � , � ) is an inner product space and if S = { e 1 , . . . , e n } is an orthonormal basis of E , then for all u ∈ E u = � u , e 1 � e 1 + . . . + � u , e n � e n . Mongi BLEL Inner Product Spaces and Orthogonality

Proof n � a j e j , then � u , e k � = � n If u = j =1 a j � e j , e k � = a k . j =1 Mongi BLEL Inner Product Spaces and Orthogonality

Theorem (Gramm-Schmidt Algorithm) If ( E , � , � ) is an inner product space and ( v 1 , . . . , v n ) a set of linearly independent vectors in E , there is a unique orthonormal set ( e 1 , . . . , e n ) such that 1 for all k ∈ { 1 , . . . , n } , Vect ( e 1 , . . . , e k ) = Vect ( v 1 , . . . , v k ) , 2 for all k ∈ { 1 , . . . , n } , � e k , v k � > 0 . Mongi BLEL Inner Product Spaces and Orthogonality

Proof We construct in the first time an orthogonal set ( u 1 , . . . , u n ) such that:  = u 1 v 1   v 2 − � u 1 , v 2 �   u 2 = � u 1 � 2 u 1      . . .   n − 1  � u i , v n �  �  u n = v n − � u i � 2 u i .     i =1 We construct the set ( e 1 , . . . , e n ) from ( u 1 , . . . , u n ) as follows: u k e k = k ∈ { 1 , . . . , n } . � u k � , Mongi BLEL Inner Product Spaces and Orthogonality

Example Let F be the vector sub-space of R 4 spanned by the vectors S = { u = (1 , 1 , 0 , 0) , v = (1 , 0 , − 1 , 0) , w = (0 , 0 , 1 , 1) } . 1 Prove that S is a basis of the sub-space F . 2 In use of Gramm-Schmidt Algorithm, find an orthonormal basis of F . (with respect to the Euclidean inner product). Mongi BLEL Inner Product Spaces and Orthogonality