lgebra Linear e Aplicaes LINEAR EQUATIONS Start with a few examples - PowerPoint PPT Presentation

Álgebra Linear e Aplicações

LINEAR EQUATIONS

Start with a few examples • One way of designing a reconstruction kernel • Cubic B-Spline • Leads to linear equations • Two ways of computing the sums of squares • Both lead to linear equations • One simpler than the other

Desirable properties • A smooth, piecewise polynomial function Q 3 ( − x ) P 3 ( − x ) P 3 ( x ) Q 3 ( x ) -2 -1 0 1 2 P 3 ( x ) = ax 3 + bx 2 + cx + d Q 3 ( x ) = ex 3 + fx 2 + gx + h P 0 3 (0) = 0 Q 3 (2) = 0 P 3 (1) = Q 3 (1) Z 1 Z 2 Q 3 ( x ) d x = 1 P 0 3 (1) = Q 0 Q 0 3 (2) = 0 3 (1) P 3 ( x ) d x + Q 00 3 (2) = 0 P 00 3 (1) = Q 00 3 (1) 2 0 1

Resulting linear equations • Substituting, we get the linear system c = 0 8 e + 4 f + 2 g + h = 0 12 e + 4 f + g = 0 12 e + 2 f = 0 a + b + c + d − e − f − g − h = 0 3 a + 2 b + c − 3 e − 2 f − g = 0 6 a + 2 b − 6 e − 2 f = 0 1 4 a + 1 3 b + 1 2 c + d + 15 4 e + 7 3 f + 3 2 g + h = 1 2

Result: the Cubic B-Spline function • Solving, we get P 3 ( x ) = x 3 / 2 − x 2 + 2 / 3 Q 3 ( x ) = − x 3 / 6 + x 2 − 2 x + 4 / 3 Q 3 ( − x ) P 3 ( − x ) P 3 ( x ) Q 3 ( x ) -2 -1 0 1 2

Sum of squares • Find a formula for the sum of squares n X i 2 S 2 ( n ) = i =1 • Seems to be a cubic polynomial P 2 ( n ) = a n 3 + b n 2 + c n + d • Use first 4 results to solve for a , b , c , and d P 2 (0) = d = 0 = S 2 (0) P 2 (1) = a + b + c + d = 1 = S 2 (1) P 2 (2) = 8 a + 4 b + 2 c + d = 5 = S 2 (2) P 2 (3) = 27 a + 9 b + 3 c + d = 14 = S 2 (3)

Even simpler! • On one hand S 2 ( n + 1) − S 2 ( n ) = ( n + 1) 2 = n 2 + 2 n + 1 • On the other hand P 2 ( n + 1) − P 2 ( n ) = 3 a n 2 + (3 a + 2 b ) n + a + b + c • Equating coefficient by coefficient 3 a = 1 3 a + 2 b = 2 P 2 ( n ) = n ( n + 1)(2 n + 1) a + b + c = 1 6 d = 0

Generalizing • Formulate general problem • Propose a solution strategy • Elimination • Back-substitution • Propose a convenient notation • Investigate the cost of solution

Systems of linear equations • General problem formulation a 11 x 1 + a 12 x 2 + · · · a 1 n x n = b 1 a 21 x 1 + a 22 x 2 + · · · a 2 n x n = b 2 . . . a m 1 x 1 + a m 2 x 2 + · · · a mn x n = b m • a ij are the coefficients • b i is the right-hand side • x i are the variables or unknowns • m equations on n unknowns

Solutions to linear equations • There are only three possibilities • System has unique solution • System has no solution • System has infinitely many solutions • Easy to understand geometrically

The row-view of linear systems • Imagine m = 2 and n = 2 • Each equation represents a line in R 2 a 11 x + a 12 y = b 1 a 21 x + a 22 y = b 2 • Planes in 3D, hyper-planes in higher dimensions

How to solve a linear system? • Transform to an equivalent system • Same solution set • Easier to solve • Method known as Gaussian Elimination 2 x + y + z = 1 2 x + y + z = 1 6 x + 2 y + z = − 1 − y − 2 z = − 4 − 2 x + 2 y + z = 7 − 4 z = − 4

Elementary operations • Consider the following row operations: • Exchange two rows • Multiply a row by a non-zero constant • Add a multiple of a row into another • Can these operations change the solution set? • Are these operations reversible?

Elimination step-by-step • Let each row i be denoted by E i • Use E i to eliminate variable i from E i +1 to E n 2 x + y + z = 1 2 x + y + z = 1 6 x + 2 y + z = − 1 − 1 y − 2 z = − 4 − 2 x + 2 y + z = 7 3 y + 2 z = 8 2 x + y + z = 1 2 x + y + z = 1 y − 2 z = − 4 ( E 2 − 3 E 1 ) − 1 y − 2 z = − 4 − − 4 z = − 4 ( E 3 + 3 E 2 ) − 2 x + 2 y + z = 7 2 x + y + z = 1 2 x + y + z = 1 y − 2 z = − 4 − y − 2 z = − 4 − 3 y + 2 z = 8 ( E 3 + E 1 ) − 4 z = − 4

Back substitution • Start from triangularized system 2 x + y + z = 1 − y − 2 z = − 4 − 4 z = − 4 • Solve for z in E 3 z = 1 • Solve for y in E 2 y = 4 − 2 z = 4 − 2(1) = 2 • Solve for x in E 1 x = 1 2(1 − y − z ) = 1 2(1 − 2 − 1) = − 1

Simplifying notation • Variable names and operators are redundant   2 1 1 1 2 x + y + z = 1 6 2 1 − 1 6 x + 2 y + z = − 1   − 2 2 1 7 − 2 x + 2 y + z = 7 • Coefficient matrix A and right-hand side b • Augmented matrix [ A | b ] • A matrix is a rectangular array of scalars • A scalar is a real or complex number

Elimination on augmented matrix 2 x + y + z = 1   2 1 1 1 6 x + 2 y + z = − 1 6 2 1 − 1   − 2 x + 2 y + z = 7 − 2 2 1 7   2 1 1 1  R 2 − 3 R 1 0 − 1 − 2 − 4  R 3 + R 1 0 3 2 8   2 1 1 1 2 x + y + z = 1 0 − 1 − 2 − 4 − y − 2 z = − 4   0 0 − 4 − 4 R 3 + 3 R 2 − 4 z = − 4

Back-substitution on augmented matrix     R 1 − R 2 2 1 1 1 2 0 0 − 2 0 − 1 − 2 − 4 0 1 0 2     R 3 / ( − 4) 0 0 1 1 0 0 1 1     R 1 / 2 2 1 0 0 R 1 − R 3 1 0 0 − 1 0 − 1 0 − 2 R 2 + 2 R 3 0 1 0 2     0 0 1 1 0 0 1 1   = − 1 2 1 0 0 x R 2 / ( − 1) = 2 0 1 0 2 y   z = 1 0 0 1 1

Origins of method • Chinese book Chiu-chang Suan-shu • Nine chapters on arithmetic • From around 200 B.C. • Using a “counting board” • Row manipulations • Made its way to Japan then Europe • Gauss was a user of method, not inventor

How much computation? • Assume an n × n system of equations • Count number of additions/subtractions • Count number of multiplications/divisions • The elimination step is     a 11 a 12 a 1 n b 1 t 11 t 12 t 1 n c 1 · · · · · · a 21 a 22 a 2 n b 2 0 t 22 t 2 n c 2 · · · · · ·         . . . . ... . . . .  . . . .    ... . . . . . . . .     . . . . a n 1 a n 2 a nn b n 0 0 t nn c n · · · · · ·

Operation count of elimination step Elimination Pivot Multiplication & Addition & Division Subtraction (n-1) ( 1+(n-1)+1 ) (n-1) ( (n-1)+1 ) 1 (n-2) ( 1+(n-2)+1 ) (n-2) ( (n-2)+1 ) 2 … … … 1 ( 1+(1)+1 ) 1 ( (1)+1 ) n-1 n − 1 n − 1 X X i ( i + 2) i ( i + 1) i =1 i =1

How much computation? • The back-substitution step is     1 0 0 t 11 t 12 t 1 n c 1 s 1 · · · · · · 0 0 1 0 t 22 t 2 n c 2 s 2 · · · · · ·         . . . . . . . . ... ... . . . . . . . .     . . . . . . . .     0 0 0 0 1 t nn c n s n · · · · · · • And the solution is given by     x 1 s 1 x 2 s 2          = . .     . . . .    x n s n

Operation count of back-substitution Elimination Back-substitution Pivot Multiplication & Addition & Multiplication & Addition & Division Subtraction Division Subtraction (n-1) ( 1+(n-1)+1 ) (n-1) ( (n-1)+1 ) 1 1+(0) 0 (n-2) ( 1+(n-2)+1 ) (n-2) ( (n-2)+1 ) 2 1+(1) 1 … … … … … 1 ( 1+(1)+1 ) 1 ( (1)+1 ) n-1 1+(n-1) n-1 n n − 1 n − 1 n − 1 X X X X i ( i + 2) i ( i + 1) i i i =1 i =1 i =1 i =1

Total operation count of Gaussian Elimination • Number of multiplication and divisions n 3 3 + n 2 − n 3 • Number of additions and subtractions n 3 3 + n 2 2 − 5 n 6 n 3 • So about of each operation ( n is large!) 3

Gauss-Jordan method • Try to avoid back-substitution • Modify Gaussian Elimination • Scale E i so pivot is 1 • Annihilate all terms above pivot as well • Intermediate state would look like   c 13 c 1 n d 1 1 0 · · · c 23 c 2 n d 2 0 1 · · ·     0 0 c 33 c 3 n d 3 · · ·     . . . . . ... . . . . .   . . . . .   c n 3 c nn d n 0 0 · · ·

Total operation count of Gaussian-Jordan method • Number of multiplication and divisions n 3 2 + n 2 2 • Number of additions and subtractions n 3 2 − n 2 2 n 3 • So about of each operation 2 • Worse than Gaussian Elimination! Why?

Why do we care about cost? • Are there large linear systems? • Yes, very large systems • Look at two examples • Signal reconstruction • With our Cubic B-Spline! • Two-point boundary problem • Solve ODE numerically through discretization

Signal reconstruction • Assume you have sampled a function f at a range of integer positions � f (0) , f (1) , f (2) , . . . • Now you want to reconstruct the value of the function at all reals in a finite range • Use a compactly supported generating function and define ϕ ( x ) ˜ X f ( x ) = f ( i ) ϕ ( x − i ) i

Generating functions • Typical generating functions 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 - 2 - 1 0 1 2 - 2 - 1 0 1 2 - 2 - 1 0 1 2 nearest/box linear/tent cubic B-Spline • Typical reconstructions 5 5 5 4 4 4 3 3 3 2 2 2 1 1 1 0 0 0 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7