04/09/2018 Linear algebra A brush-up course Jeff Hindsborg - - PDF document

04/09/2018 1

Jeff Hindsborg

Linear algebra

A brush-up course

Agenda

• 1. Real numbers
• Operators
• Linear equations
• 2. Linear algebra
• 3. Systems of Linear equations

04/09/2018 2

Real numbers

04/09/2018 3

• The real number system consists of 4 parts:

A set of R of all real numbers

• A relation < on R. If a, b ∈ R, then a < b is either true or false.

We know it as the order-relation.

• A function +:

R + R ≠ R . The addition operation

• A function ∘

∘ ∘ ∘ : R ∘ R ≠ R . The multiplication operation

04/09/2018 2

Real numbers

• Overview
• R : Real numbers
• Q : Rational numbers
• Z : Integers
• N : natural numbers

Does that cover all real numbers? Is everything real numbers?

04/09/2018 4

Real numbers

• Axioms I
• Associative laws
• a + (b + c) = (a + b) + c
• a ∘ (b ∘ c) = (a ∘ b) ∘ c
• Commutative laws
• a + b = b + a
• a ∘ b = b ∘ a
• Distributive law
• a ∘ (b + c) = a ∘ b + a ∘ c

04/09/2018 5

Real numbers

• Axioms II
• Additive identity (”zero” element)
• There exist an element in R called 0 so that, for all a:
• a + 0 = a
• Additive inverse
• For all a there exists a b so that:
• a + b = 0, and b = − a
• Multiplicative identity (”one” element)
• There exists an element in R called 1 so that, for all a:
• 1 ∘ a = a
• Multiplicative inverse
• For all a ≠ 0 there exists a b so that:
• a ∘ b = 1, and b = a-1

04/09/2018 6

04/09/2018 3

Real numbers

• Solving equations

04/09/2018 7

• Let a ≠ 0 and b be known real numbers, and x be

an unknown real number.

• If, for some reason, we know that
• ∙

we say that we have an equation.

• We can solve the equation in a couple of stages

using the axioms: ∙ , ∙ ∙ ∙ 1 ∙ ∙

Real numbers

• Example

04/09/2018 8

Farmer Hansen has delivered milk to the dairy last week.

• He delivered 10.000kg
• He has received a total of 23.000 DKK as payment.

Using these 2 figures, we can find the milk-price per quantity.

(eq. 1) ∙

Where:

• 10.000
• 23.000
• !

We saw that solving eq. 1 w.r.t. yields

" #, such that

• $%%

&'

().*** $%%

*.*** &'

+. ,- $%%

&'

Agenda

• 1. Real numbers
• 2. Linear algebra

1. Background 2. What is a matrix? 3. Types of matrices 4. Operations 5. Where and why do we use it?

• 3. Systems of Linear equations

04/09/2018 9

04/09/2018 4

Linear algebra

• background
• We saw that a linear equation (eq. 1) could be expressed as
• We can generalize that to

. (( . ⋯ . 00 , for n=1,2,3,…,n Where and the ’s are (real numbers) coefficients that are often known in advance.

In this course we focus on linear equations.

04/09/2018 10

Linear algebra

• background

Class exercise 1 Reorder the following equations into the generalized form I. 3 . 5( . 2

• II. 4 . 5( (
• III. ( 2

6 5 . )

• IV. ( 2 5 6

When finished, we’ll discuss the results.

04/09/2018 11

. (( . ⋯ . 00

Agenda

• 1. Real numbers
• 2. Linear algebra

1. Background 2. What is a matrix? 3. Types of matrices 4. Operations 5. Where and why do we use it?

• 3. Systems of Linear equations

04/09/2018 13

04/09/2018 5

Linear algebra

• What is a matrix?
• Consider the system

3 . 6( . 2) 0 51 . 5( . 23) 0 52( . 4) 0

• We can align its coefficients in columns to form the matrix

3 6 2 51 5 23 52 4

Which is a 3x3 matrix (Notation: #? @ A #B@)

04/09/2018 14

Linear algebra

• What is a matrix?
• Another example

3 6 2 51 5 23 Which is a 2x3 matrix. Symbolically we can express this as

• (

) ( (( () Similar to the generalized equation, only that the first index denotes the equation.

04/09/2018 15

. (( . ⋯ . 00

Agenda

• 1. Real numbers
• 2. Linear algebra

1. Background 2. What is a matrix? 3. Types of matrices 4. Operations 5. Where and why do we use it?

• 3. Systems of Linear equations

04/09/2018 16

04/09/2018 6

Linear algebra

• Types of matrices

A matrix of dimension A is called a quadratic (or rectangular) matrix: A matrix of dimension 1 A is called a row vector: A matrix of dimension A 1 is called a column vector

04/09/2018 17

Agenda

• 1. Real numbers
• 2. Linear algebra

1. Background 2. What is a matrix? 3. Types of matrices 4. Operations 5. Where and why do we use it?

• 3. Systems of Linear equations

04/09/2018 18

Linear algebra

• Operations

Addition

• Two matrices may be added if they are of equal

dimensions (say A ) From the axioms of real numbers, it follows that the commutative law is valid for matrix operation: . .

04/09/2018 19

04/09/2018 7

Linear algebra

• Operations

Class exercise 2 Additive identity Does a set of A matrices have a ‘zero’ element 0 so that for any : . 0 If yes, what does it look like?

04/09/2018 20

Linear algebra

• Operations

04/09/2018 21

Even when ∙ exists, most often ∙ C ∙

Multiplication

• Two matrices ( and ) may be multiplied if
• is of dimension A
• is of dimension A

In other words, must have the same number of columns as has rows. Resulting in a A matrix. E.g. A ! A !

• Due to the dimension requirement, it is clear that the

commutative law is not valid for matrix multiplication:

Linear algebra

• Operations

Vector multiplication

• A row vector of dimension 1 A may be multiplied with

a column vector b of dimension A 1.

• The product ∙ is a 1 A 1 matrix (a number):
• The product ∙ is a A matrix:

04/09/2018 22

04/09/2018 8

Matrix multiplication – simple example

An element in the product is calculated as the product of a row and a column 5 4 3 6 1 2 2 3 2 1 2 4 3 2 1

21 30 15 24 22 26

A 3 x 3 matrix multiplied with a 3 x 2 matrix

a b Let’s visualize this: http://matrixmultiplication.xyz/

Linear algebra

• Operations

Multiplicative identity Does the set of matrices have a ‘one’ element F, so that if F is an A matrix, then for any A matrix , F ∙ If yes:

• What must the values of n necessarily be?
• What are the elements of F – what does the matrix look like?

Does a ‘one’ element F( exist such that for any matrix of given dimension, ∙ F( If yes: Same questions as before.

04/09/2018 24

Linear algebra

• Operations

It follows directly from the axioms for real numbers, that every matrix , has an additive inverse, , such that . 0, and, for the additive inverse, b 5a Resulting in the ‘zero’ or ‘null’ matrix.

04/09/2018 25

04/09/2018 9

Linear algebra

• Operations

Other operations

• A real number may be multiplied with a matrix
• The transpose G (often denoted as H or I) of a matrix is

formed by changing the columns to rows. Visualize it as a folding around the diagonal.

04/09/2018 26

Linear algebra

• Operations

Other operations – Examples If 2 and 1 2 3 4 5 6 then 2 1 2 3 4 5 6

• 2

4 6 8 10 12 The transpose of is, G

04/09/2018 27

Linear algebra

• Operations

Multiplicative inverse I Does every matrix that are non-zero ( C 0) have a multiplicative inverse, , such that ∙ F If yes,

• What does it look like?

04/09/2018 28

04/09/2018 10

Linear algebra

• Operations

Multiplicative Inverse II A matrix only has a multiplicative inverse under certain conditions:

• The matrix is quadratic (i.e. its dimension is A )
• The matrix is non-singular
• is singular if and only if KL 0 where KL is the

determinant of .

• Many quadratic matrices are singular!

04/09/2018 29 ns

Determinant

• The determinant of a quadratic matrix is a real

number.

• Calculation of the determinant is rather

complicated for large dimensions.

• The determinant of a 2 x 2 matrix:
• The determinant of a 3 x 3 matrix:

Linear algebra

• Operations

The inverse matrix If a quadratic matrix is non-singular (det C 0) it has an inverse , and:

• F
• F
• The inverse is complicated to find for matrices of high dimension.
• For large matrices (millions of rows and columns) inversion is a

challenge even to modern computers.

• Inversion of matrices is crucial to many applications in herd

management (and animal breeding)

04/09/2018 31

04/09/2018 11

Linear algebra

• Operations

The inverse matrix A 2x2 matrix a is inverted by ( ( ((

• 1
• (

( (( (( 5( 5(

• Example

1 3 2

• 1

1 3 2 53 52 1 1 56 53 52 1

• 5 1

2 5 1 3 5 1 6

04/09/2018 32

Linear algebra

• Operations

The inverse matrix Example of a 3x3 matrix

04/09/2018 33

Linear algebra

• Operations summary
• How to translate a complex and coupled system (or set of

equations) into matrix-form and identify nonlinear elements.

• Different types of matrices
• Quadratic, row vector, identity matrix etc.
• Operations
• Addition, multiplication (matrices and vectors), transposed, inverse (incl. determinant)

04/09/2018 34

04/09/2018 12

Agenda

• 1. Real numbers
• 2. Linear algebra

1. Background 2. What is a matrix? 3. Types of matrices 4. Operations 5. Where and why do we use it?

• 3. Systems of Linear equations

04/09/2018 35

Why?

04/09/2018 36

• Because they enable us to express very complex relations

in a very compact way.

• Because the algebra and notation are powerful tools in

mathematical proofs for correctness of methods and properties.

• Because they enable us to solve large systems of linear

equations.

Applications outside animal science

Engineers apply it too!

• Modelling complex physical

systems often results in a set of coupled differential equations

• Engineers reorder it according

to a (slightly different) standard form and translate it into matrices.

• They can then utilize specific

methods to construct a control system

In short

Linear algebra is your ticket to multidimensional space.

04/09/2018 37

04/09/2018 13

Complex relations

Modeling of drinking patterns of weaned piglets.

04/09/2018 38

Complex relations

04/09/2018 39

• Madsen et al. (2005) performed an on-line monitoring of

the water intake of piglets. The water intake Yt at time t was expressed as

• Where
• Simple, but …

Complex relations II

04/09/2018 40

F, θt and wt are of dimension 25 x 1, G and Wt are of dimension 25 x 25.

• The value of θ

θ θ θt is what we try to estimate.

04/09/2018 14

Agenda

• 1. Real numbers
• 2. Linear algebra
• 3. Systems of Linear equations

04/09/2018 41

Systems of linear equations

• Old McDonalds Farm

A naïve example: Old McDonalds has a farm…

• On his farm he has some sheep, but he has forgotten how
• many. Let us denote the number as x1 .
• On his farm he has some geese, but he has forgotten how
• many. Let us denote the number as x2 .
• He has no other animals, and the other day he counted the

number of heads of his animals. The number was 25. He knows that sheep and geese have one head each, so he set up the following equation:

• 1x1 + 1x2 = 25
• He also counted the number of legs, and it was 70. He

knows that a sheep has 4 legs and a goose has 2 legs, so he set up the following equation:

• 4x1 + 2x2 = 70

Recall that a linear system is expressed as

04/09/2018 42

Systems of linear equations

• Old McDonalds Farm
• We have two equations with two unknowns:
• 1x1 + 1x2 = 25
• 4x1 + 2x2 = 70
• Define the following matrix a and the (column-) vectors x

and b

• We may then express the two equations as one matrix

equation:

04/09/2018 43

04/09/2018 15

Systems of linear equations

• Old McDonalds Farm

Having brought the system of linear equation to its general form, solution for x is straight forward: This solution is valid whether it’s a system of 2 equations or a million (which is not unrealistic). Solution: Solving this system we get ( 15 10 , meaning 15 sheep and 10 geese.

04/09/2018 44

Linear regression and matrices I

• In a study of children born in Berkeley 1928-29

the height and weight of 10 18-year old girls were measured.

• It is reasonable to assume that the weight Yi

depends on the height xi according to the following linear regression model:

• Yi = β0 + β1xi + εi where,
• β0 and β1 are unknown parameters
• The εi are N(0, σ2)

04/09/2018 45

Linear regression and matrices II

Let us define the following matrices

• We may then write our model in matrix notation simply as:

Y = xβ + ε

04/09/2018 46

04/09/2018 16

Linear regression and matrices III

The least squares estimate of β is Define the vector of predictions as: Then an estimate,s2 , for the residual variance σ2 is:

• Where n = 10 is the number of observations and k = 2 is the number of parameters

estimated.

Yielding

04/09/2018 47

Linear regression (visual inspection)

04/09/2018 48

Weight versus heigth of 18-year old girls 45 50 55 60 65 70 75 80 150 160 170 180 190 Heigth, cm Weight, kg Observations Fitted regression

Linear regression and matrices IV

• A class variable: boys and girls
• If it had been 5 girls and 5 boys we had observed, the data

could have looked like this (where xi1 = 0 means girl and xi1 = 1 means boy):

• Y = xβ + ε

04/09/2018 49

04/09/2018 17

Linear regression and matrices V

• A class variable: boys and girls
• We obtain the following estimate for β
• The interpretation is that the weight of a boy is 4.49 kg

lower than the weight of a girl of exactly same height.

• (Since we have declared 5 arbitrarily selected girls for boys,

the result should not be interpreted at all)

04/09/2018 50 ns

R and R studio for statistical computing:

ns

R and R studio for statistical computing:

R R Studio Independant program GUI for R Command line based Command line AND ”Point and Click” Hard to keep an overview –

• ne thing at a time

Easy to keep an overvirew – multible tabs and windows Mac and Windows Mac and Windows www.r-project.org/ www.rstudio.com/products/rstudio/