Chapter 2 Linear Ill-Posed Problems Observations from previous - - PowerPoint PPT Presentation
Linear Ill-Posed Problems Michael Moeller Chapter 2 Linear Ill-Posed Problems Observations from previous chapter Ill-Posed Problems in Image and Signal Processing Finite dimensional WS 2014/2015 linear operators Some functional analysis
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Ill-Posed Problems in Image and Signal Processing WS 2014/2015 Michael Moeller Optimization and Data Analysis Department of Mathematics TU M¨ unchen
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
What we have seen so far...
x u(y)dy is ill-posed.
u(x, T) = π k(x, y, T)f(y) dy, k(x, y, T) = 2 π
∞
e−n2T sin(nx) sin(ny). is ill-posed.
k(x − y)u(y)dy with smoothing kernel k is ill-posed.
Question
Is the inversion of integral operators ill-posed in general?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Why are problems ill-posed? We want to study and understand our introductory examples in more detail.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Why are problems ill-posed? We want to study and understand our introductory examples in more detail. Observation: Our introductory problems can be written as f = Au for linear operators A : X → Y between Hilbert spaces X, Y.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Why are problems ill-posed? We want to study and understand our introductory examples in more detail. Observation: Our introductory problems can be written as f = Au for linear operators A : X → Y between Hilbert spaces X, Y. Strategy: Understand finite dimensional case first!
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Linear operators between two finite dimensional Hilbert spaces: Matrices.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Linear operators between two finite dimensional Hilbert spaces: Matrices. Making our life easier: Consider a linear operator from a finite dimensional Hilbert space into itself: A ∈ Rn×n.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Linear operators between two finite dimensional Hilbert spaces: Matrices. Making our life easier: Consider a linear operator from a finite dimensional Hilbert space into itself: A ∈ Rn×n. Corresponding finite dimensional linear inverse problem: Find u from given f = Au
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Linear operators between two finite dimensional Hilbert spaces: Matrices. Making our life easier: Consider a linear operator from a finite dimensional Hilbert space into itself: A ∈ Rn×n. Corresponding finite dimensional linear inverse problem: Find u from given f = Au Making our life even easier: A is symmetric and positive definite.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Symmetric positive definite A ∈ Rn×n: A = VSV T, with
Assume scaling: λ1 = 1. Condition number κ =
1 λn .
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Assume f = Au, f δ = Auδ, with f δ − f ≤ δ:
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Assume f = Au, f δ = Auδ, with f δ − f ≤ δ: uδ − u = VS−1V T(f δ − f) ⇒ uδ − u = VS−1V T(f δ − f) = S−1V T(f δ − f) ≤ 1 λn V T(f δ − f) = δ λn = κδ
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators Assume f = Au, f δ = Auδ, with f δ − f ≤ δ: uδ − u = VS−1V T(f δ − f) ⇒ uδ − u = VS−1V T(f δ − f) = S−1V T(f δ − f) ≤ 1 λn V T(f δ − f) = δ λn = κδ → Noise amplification: reciprocal of smallest eigenvalue! → Continuous dependence on the data! → Well-posed, but for small λn ill-conditioned! → In infinite dimensions: infinitely many λn → 0!
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators
Question
What can we do against the instability?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Finite dimensional linear operators
Question
What can we do against the instability? Idea:
by uα = A−1
α f δ.
Computation on the board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Some basics...
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Some basics...
Definition: Banach space
A normed vector space X which is complete is called a Banach
converges in X.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Some basics...
Definition: Banach space
A normed vector space X which is complete is called a Banach
converges in X.
Definition: Hilbert space
A vector space X equipped with a scalar product ·, · which is complete with respect to the induced norm x =
called a Hilbert space.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Some basics...
Definition: Banach space
A normed vector space X which is complete is called a Banach
converges in X.
Definition: Hilbert space
A vector space X equipped with a scalar product ·, · which is complete with respect to the induced norm x =
called a Hilbert space.
Convention
Unless stated otherwise, X and Y are real Hilbert spaces.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Banach and Hilbert spaces
Proposition: Closed subspaces
A nonempty subspace M ⊂ X is closed if and only if (xn) ⊂ M, limn→∞ xn = x implies x ∈ M.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Banach and Hilbert spaces
Proposition: Closed subspaces
A nonempty subspace M ⊂ X is closed if and only if (xn) ⊂ M, limn→∞ xn = x implies x ∈ M.
Definition: Closure
The closure M of M ⊂ X is defined as M = M ∪
lim
n→∞ xn = x
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Banach and Hilbert spaces
Definition: Orthogonal complemet
The orthogonal complement of the set M ⊂ X is M⊥ = {x ∈ X | x, m = 0 ∀m ∈ M}.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Banach and Hilbert spaces
Definition: Orthogonal complemet
The orthogonal complement of the set M ⊂ X is M⊥ = {x ∈ X | x, m = 0 ∀m ∈ M}.
Theorem: Direct sum
Let M ⊂ X be any closed subspace of X. Then X = M + M⊥.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Definition: Linear operators
A mapping A : D(A) ⊂ X → Y is called a linear operator, if the domain D(A) is a subspace of X and for all x1, x2 ∈ D(A), and all α ∈ R A(x1 + x2) = Ax1 + Ax2 A(αx1) = α Ax1
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Definition: Boundedness
We say that a linear operator A : D(A) ⊂ X → Y is bounded if there exists a c ∈ R such that for all x ∈ D(A) AxY ≤ cxX.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Definition: Boundedness
We say that a linear operator A : D(A) ⊂ X → Y is bounded if there exists a c ∈ R such that for all x ∈ D(A) AxY ≤ cxX.
Examples
is bounded.
∂x : C1([0, 1]) ⊂ L2([0, 1]) → L2([0, 1]) is unbounded. (Example with sin(2πkx))
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Theorem: Boundedness and Continuity
Let A : D(A) ⊂ X → Y be a linear operator. Then the following three statements are equivalent:
Proof: Exercises
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Theorem: Boundedness and Continuity
Let A : D(A) ⊂ X → Y be a linear operator. Then the following three statements are equivalent:
Proof: Exercises
Notation
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Definition: Open
An operator A : X → Y is called open, if for every open set M ⊂ X in X the set A(M) ⊂ Y is open in Y.
1See D. Werner, Funktionalanalysis. Springer 2005.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operators
Definition: Open
An operator A : X → Y is called open, if for every open set M ⊂ X in X the set A(M) ⊂ Y is open in Y.
Theorem: Open mapping theorem
If A ∈ L(X, Y) is surjective, then A is open. 1
1See D. Werner, Funktionalanalysis. Springer 2005.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
The previous analysis was done for symmetric positive definite matrices A ∈ Rn×n. What can we do for a general A ∈ L(X, Y)? What could happen?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
The previous analysis was done for symmetric positive definite matrices A ∈ Rn×n. What can we do for a general A ∈ L(X, Y)? What could happen?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
The previous analysis was done for symmetric positive definite matrices A ∈ Rn×n. What can we do for a general A ∈ L(X, Y)? What could happen?
Definition
We call u a least-squares solution of Au = f if Au − f = inf{Av − f | v ∈ X}
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
The previous analysis was done for symmetric positive definite matrices A ∈ Rn×n. What can we do for a general A ∈ L(X, Y)? What could happen?
Definition
We call u a least-squares solution of Au = f if Au − f = inf{Av − f | v ∈ X}
unique.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
The previous analysis was done for symmetric positive definite matrices A ∈ Rn×n. What can we do for a general A ∈ L(X, Y)? What could happen?
Definition
We call u a least-squares solution of Au = f if Au − f = inf{Av − f | v ∈ X}
unique.
Definition
We call u a minimal-norm solution of Au = f if u = inf{v | v is least-squares solution of Au = f}
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
Can we define a linear operator that computes minimal norm solutions?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Question
Can we define a linear operator that computes minimal norm solutions?
Definition (Moore-Penrose inverse)
Let A ∈ L(X, Y) and let ˜ A : N(A)⊥ → R(A) denote its
defined as the unique linear extension of ˜ A−1 to D(A†) := R(A) ⊕ R(A)⊥ with N(A†) = R(A)⊥.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations The Moore-Penrose inverse is well-defined.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations The Moore-Penrose inverse is well-defined.
Theorem: Moore-Penrose equations
The Moore-Penrose generalized inverse A† meets the following four Moore-Penrose equations
1 AA†A = A 2 A†AA† = A† 3 A†A = I − P 4 AA† = Q|D(A†)
where P : X → N(A) and Q : Y → R(A) are the orthogonal projectors onto the nullspace of A, N(A), and onto the closure
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations
Theorem: Minimal-norm solutions
For a given f ∈ D(A†), the equation Ax = f has a unique minimal-norm solution given by x† := A†f. The set of all least-squares solutions is given by {x†} + N(A). Proof: Board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equation A word about adjoint operators...
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equation A word about adjoint operators...
Theorem: Gaussian normal equation
For a given f ∈ D(A†), x ∈ X is a least-squares solution of Ax = f if and only if x satisfies the Gaussian normal equation A∗Ax = A∗f. Proof: Board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equations Observations:
least-squares solution with minimal norm.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equations Observations:
least-squares solution with minimal norm.
A∗Ax = A∗y (1)
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equations Observations:
least-squares solution with minimal norm.
A∗Ax = A∗y (1)
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Gaussian normal equations Observations:
least-squares solution with minimal norm.
A∗Ax = A∗y (1)
(cf. Landweber iteration!)
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations For any linear operator equation f = Au, A ∈ L(X, Y), we now have a (possibly naive) way of finding a solution via u = A†f. When is this approach naive?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear operator equations For any linear operator equation f = Au, A ∈ L(X, Y), we now have a (possibly naive) way of finding a solution via u = A†f. When is this approach naive?
Proposition: Discontinuity of A†
A† is continuous if and only if R(A) is closed. Proof: Board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators
Definition: Compact linear operator
A ∈ L(X, Y) is said to be compact if for every bounded sequence {xn} ⊂ X, {Axn} has a convergent subsequence.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators
Definition: Compact linear operator
A ∈ L(X, Y) is said to be compact if for every bounded sequence {xn} ⊂ X, {Axn} has a convergent subsequence. Remark: Be careful with the dimensions of your space! Example: Identity operator on X for finite and infinite dimensional X.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators
Theorem: Ill-posedness of compact linear operators
Let A ∈ L(X, Y) be compact, and let the dimension of R(A) be
Proof: Board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators What kind of operators are compact?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators What kind of operators are compact?
Theorem: Operators with Hilbert-Schmidt kernel are compact
Let Au(x) =
k(x, y)u(y) dy with kernel k ∈ L2(Ω × Ω). Then A ∈ L(L2(Ω), L2(Ω)) is compact. Proof: Board. A kernel k ∈ L2(Ω × Ω) is called a Hilbert-Schmidt kernel from Ω × Ω → R.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Examples for compact linear operators
x u(y)dy is ill-posed.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Examples for compact linear operators
x u(y)dy is ill-posed.
u(x, T) = π k(x, y, T)f(y) dy, k(x, y, T) = 2 π
∞
e−n2T sin(nx) sin(ny). is ill-posed.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Examples for compact linear operators
x u(y)dy is ill-posed.
u(x, T) = π k(x, y, T)f(y) dy, k(x, y, T) = 2 π
∞
e−n2T sin(nx) sin(ny). is ill-posed.
k(x − y)u(y)dy with smoothing kernel k is ill-posed.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Compact linear operators What kind of operators are compact?
Facts about compact operators
Then A is bounded, i.e. A ∈ L(X, Y).
Then AB is compact.
Then AB is compact.
Then A∗ is compact. X, Y, Z Hilbert spaces.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
A little summary What did we learn so far?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition
Theorem: Eigendecomposition2
Let A ∈ L(X, X) be self-adjoint and compact. Then there exist at most countably many nonzero eigenvalues {λn}, n ∈ I, of A. All eigenvalues are real and for a set of orthonormal eigenvectors {un} with un = 1 one has Ax =
λix, unun
2See D. Werner, Funktionalanalysis. Springer 2005. Thm VI.3.2 pp 265.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition Requirements for the eigendecomposition:
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition Requirements for the eigendecomposition:
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition Requirements for the eigendecomposition:
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition Requirements for the eigendecomposition:
Ideas: If A ∈ L(X, Y) is compact ...
Bx =
σ2
nunx, un
∀x ∈ X
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Eigendecomposition Requirements for the eigendecomposition:
Ideas: If A ∈ L(X, Y) is compact ...
Bx =
σ2
nunx, un
∀x ∈ X
Cy =
I
˜ σn
2vny, vn
∀y ∈ Y Further computations on the board.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Singular value decomposition (SVD)
Any compact linear operator A ∈ L(X, Y) has a representation Ax =
σnx, unvn A∗y =
σny, vnun with the orthonormal singular vectors un and vn, and singular values σn > 0.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Singular value decomposition (SVD)
Any compact linear operator A ∈ L(X, Y) has a representation Ax =
σnx, unvn A∗y =
σny, vnun with the orthonormal singular vectors un and vn, and singular values σn > 0. Convention: σ1 ≥ σ2 ≥ σ3 ≥ ...
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Singular value decomposition (SVD)
Any compact linear operator A ∈ L(X, Y) has a representation Ax =
σnx, unvn A∗y =
σny, vnun with the orthonormal singular vectors un and vn, and singular values σn > 0. Convention: σ1 ≥ σ2 ≥ σ3 ≥ ... Sanity check:
σnx, unvn2 =
N
σ2
nx, un2 ≤ σ1 N
x, un2 ≤ σ1x2
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition Can we use the SVD for the Moore-Penrose inverse?
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition Can we use the SVD for the Moore-Penrose inverse? Some considerations yield
Singular value representation of A†
For y ∈ D(A†) it holds that A†y =
1 σn y, vnun.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition Can we use the SVD for the Moore-Penrose inverse? Some considerations yield
Singular value representation of A†
For y ∈ D(A†) it holds that A†y =
1 σn y, vnun. Proof that compact linear operators yield ill-posed problems: A†vn =
1 σn .
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition Can we use the SVD for the Moore-Penrose inverse? Some considerations yield
Singular value representation of A†
For y ∈ D(A†) it holds that A†y =
1 σn y, vnun. Proof that compact linear operators yield ill-posed problems: A†vn =
1 σn .
Theorem: Singular values of compact operators
Let A ∈ L(X, Y) be compact. The zero is the only possible accumulation point for the singular values σn. Proof: Exercise.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition From A†y =
1 σn y, vnun we can see
1 σn ,
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition From A†y =
1 σn y, vnun we can see
1 σn ,
amplified much stronger,
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition From A†y =
1 σn y, vnun we can see
1 σn ,
amplified much stronger,
amplification.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Definition: Classification of ill-posedness
A problem Au = f with a compact linear operator A ∈ L(X, Y) with infinite dimensional range is called
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Definition: Classification of ill-posedness
A problem Au = f with a compact linear operator A ∈ L(X, Y) with infinite dimensional range is called
that σn ≥ Cn−γ for all n.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Definition: Classification of ill-posedness
A problem Au = f with a compact linear operator A ∈ L(X, Y) with infinite dimensional range is called
that σn ≥ Cn−γ for all n.
exist a γ > 1 and C > 0, such that σn ≥ Cn−γ for all n.
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Singular value decomposition
Definition: Classification of ill-posedness
A problem Au = f with a compact linear operator A ∈ L(X, Y) with infinite dimensional range is called
that σn ≥ Cn−γ for all n.
exist a γ > 1 and C > 0, such that σn ≥ Cn−γ for all n.
with polynomial speed. Example: Differentiation
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Summary What we have learned so far:
ill-posed!
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Summary What we have learned so far:
ill-posed!
A†y =
∞
1 σn y, vnun
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Summary What we have learned so far:
ill-posed!
A†y =
∞
1 σn y, vnun
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Summary What we have learned so far:
ill-posed!
A†y =
∞
1 σn y, vnun
Linear Ill-Posed Problems Michael Moeller Observations from previous chapter Finite dimensional linear operators Some functional analysis basics Linear operators in infinite dimensions Compact linear
The singular value decomposition
updated 30.10.2014
Summary What we have learned so far:
ill-posed!
A†y =
∞
1 σn y, vnun
Next chapter: Can we modify the σn to obtain stability?