Hybrid sparse stochastic processes and the resolution of linear - - PDF document

SLIDE 1

Hybrid sparse stochastic processes and
 the resolution of linear inverse problems

Michael Unser Biomedical Imaging Group EPFL, Lausanne, Switzerland

Statistical Models for Shape and Imaging, March 11-15, 2019, Institut Poincaré, Paris

Variational-MAP formulation of inverse problem

2

Linear forward model

y = Hs + n

Reconstruction as an optimization problem

srec = arg min ky Hsk2

2

| {z }

data consistency

+ λkLskp

p

| {z }

regularization

, p = 1, 2 − log Prob(s) : prior likelihood

linear
 model

noise

H n s

SLIDE 2

EDEE Course

3

An introduction to sparse stochastic processes

Random spline: archetype of sparse signal

4

Stochastic differential equation

Ds(t) = w(t)

with boundary condition s(0) = 0 Innovation:

w(t) = X

n

anδ(t − tn)

Random weights {an} i.i.d. and random knots {tn} (Poisson with rate λ)

Formal solution = Compound Poisson process

s(t) = D−1w(t) = X

n

anD−1{δ(· − tn)}(t) = b1 + X

n

an

+(t − tn)

SLIDE 3

Lévy processes: all admissible brands of innovations

5

(perfect decoupling!)

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Compound Poisson Brownian motion

Integrator

Gaussian Impulsive

Z t dτ

Lévy flight

s(t) w(t) White noise (innovation) Lévy process

SαS (Cauchy)

(Paul Lévy circa 1930) (Wiener 1923)

Generalized innovations : white Lévy noise with E{w(t)w(t0)} = σ2

wδ(t − t0)

Ds = w

∈ S0(Rd)

<latexit sha1_base64="sUc5qrFhIvsr51Rwv234bJX9R98=">A JaXiclVZtb9s2EFb vcTeW7IBw7B9IeYa 4vGkN1u2ToUKJCiSIF+6JKlLRJ5ASWdZdYUpZJUHI/Tn9nX7Q/tN+xP7CjZrl6SwRXghOA9fJ7j3elOfsqZ0q7 z7XrN957/4MPtzrdjz7+5NP tnc+f6GSTAZwHCQ8ka98qoAzAcea Q6vUgk09jm89Gf71v7yHKRi fhVL1IYxzQSbMICqnHrbPtLjwnixVRPA8rNUf7dLe/wt/D2 XbPHbjFQ9qL4XLRc5bP87OdrZ+9MAmyGIQO FXqdOimemyo1CzgkHe9TEFKgxmNwBRu56SPWyGZJBJ/QpNit4ajsVKL2EekdVA1bXbzMt p ic/jg0Ta ZB KXQJONEJ8TGgIRMQqD5gtAgQH8zqtGPGoPCaED4cPDTaGymwM9B4w0kCJgHSRxTERpvQmPGFyFMaMZ1bjw1Wa27/W6fFKFXBKHEz gHrR7gvocYzFRxVRNnUcLD3MjIz83wrjvY+/6um7cwEhZLjDtAQPlDWNUZn2dQyqAndT859bE6NMQMTSWpRefEkxkHY1XhIl/9zxvEVn1z4qW3G1Lj5d+J2gZrM2qfYx7fJR4I/x/qOoGegsp8hRWEb1BuiPGmCusG7phd /ADHqlHGcuUc3Kz9Em9yaiEm29PjKwEuUrjSpm9tzLrmGsmFlfq3FvpVFVK/vx0OF6trdYdc1mdDG1M7J8lpy3GgtNWPukNL2dvCGyqUREpk7HSaOaZRTaTVoAYm1qJnWET53fX3pe0pM6bUlnl3cxTm8olHylawFEKAaOc2B6W8PL9r6owcZibdf/1fXPYvB8TJ3XESROxj5WxMu43jYcVY4v7pGJs0R4w7oPUK4idDQdNDLzBRmVTo7HgJHDjnWO3l4yKCBu+eYj4+oFnSZpjNGVMnjW5Hq9Nj5ump2vT06YpZEIvbWFL7QmOSAayeoUnrUT7F/Y8Zicsk0QuWiU2ayBmLcS8gfCSGCJqHervFg85Ao1vZ0TwtSYpzhei2O9AQAcDfLqeApzbItLTsvJECPZebhC3+sJq1Cst8fQUa3AwarijqZ9XmgUWOvLgTKq aLjQcxZqS3AfzZ0+YROySDIiAEchzk avs6ULhwugCSZ4EKEybzOhP6qGUt uxzViOYURzoSY pkEmYB3hkdskHwQc8B AZC0kjSFIc3fm4Mmx8X7cWL0WB4b+D+cr/36MHyw2PL+cb51rnlDJ095 Fz4Dx3jp3A+cP50/nL+Xvr385O56vO1yX0+rXlmS+c2tPp/QfuE2vI</latexit>

Generalized innovation model

6

1 3

White noise
 Whitening operator

L−1 L

s = L−1w w

2

Solution of SDE (general operator) Proper definition of
 continuous-domain white noise X = hϕ, wi

Theoretical framework: Gelfand’s theory of generalized stochastic processes (Unser et al, IEEE-IT 2014)

innovation process sparse stochastic process

Regularization operator vs. wavelet analysis 4

Approximate decoupling

Main feature: inherent sparsity (few significant coefficients)

= hL−1∗ϕ, wi Y = hϕ, si = hϕ, L−1wi

Ls = w

Generic test function ϕ ∈ S(Rd) plays the role of index variable

<latexit sha1_base64="KWQGjUuYKXxaMcV8rvHV2 mFaC8=">A JqHiclVZdb9s2F W7r9j7aLo97oVY qALGkN2t2XrUKBAi 4F+pAlS9Ml8gJKupI5U5RKUrE9Qr9sv2SPe93+xC4l27WkZHAFOCF4D8+5vPeSl37GmdKu+9edu+ 9/8GH 21 uh9/8uln97bvf/5KpbkM4CxIeSpf+1QBZwLON McXmcSaOJzOPcnh9Z+fg1SsVT8oucZjBIaCxaxgGqcuto+ wkESBYQDUqTKBeBnSe73jWV2Zh5TBAvoXocUG5OiwfeyW/h17sk43SuiB4DkSkHk aEiRBmB cxitJX2ztu3y0/0h4MFoMdZ/EdX93f+tEL0yBPQOiAU6UuB26mR4ZKzQIORdfLFWQ0mNAYTLnrgvRwKiR KvEnNClnaziaKDVPfETaHaimzU7eZLvMdfT9yDCR5RpEUAlFOSc6JTaEJGQSAs3nhAYB+ptTjX7UGBSGC8In/R+GIzMGfg0adyAx0tMgTRIqQuNFNGF8HkJEc64L46loOe72uj1SZk4RhBI/5xy0eoz HmIw0eVWTZLHKQ8LI2O/MIOHbv/g24du0cJImC8wbh8B1Q9h6874PIdKBj2p+8mpj8WlIWFoqkgtuiCezDkYqwqzYvm/aB b9c2JF95uSI2bfydqG6zNqH2OeXyXeCD8f6jrBHhqVO4rKA9aY jx gr BvbMvtv/DpfUo4xlyjmex1JEvcmphN23K4ZWgtymcavMwVuZVcw1E/NbdR4td ZVKv7icjBajq3WnrmpTgY2JvbPgtMWY8lpK5/sDG5mbwhsqrEmUiVjqdHM 4t Jq0AMTa1Em+GTZzfX3lf0ZI6b4bX4BrvZp7aVC74SHkFnGYQM qJvcNSXp3/dRUmTgqzuqB935w098fER 1x0UQcYmUsjYdN48mascV9sWZs0R4x7oPUS4htHkdNDLzBi8qmRmPBSeDGthyN/UPEeOGbJ4ivL3iZ gVGUybkZ Pr2cr0rGl6sTK9aJpCJvTCFrbUnmOHZSDXt/C8lWh/ZtdjdsIqSWTWKrFJAzFpIaYNhJcmEFPrUG+/ MgpaDydcdlwM+wvRLE/gIAO+vh1PQXY9kWsx1XlYSe2+3KDpHUvLF8KSktcPcYa7A8b7mjqF2uXBRY68mBPWlfRMN TFmpL8A2aOz3CIjJPcyIAWyH2SRr+nuN7wjpcAu0TY qepdM6E/qrJiyz1+WwRjSl2NKRCFMk0zAPcM/okA2CD3oKIDAQksaSZti8 bkxaD4u2oNXw/7gUd/9ebjz9PHi4bHlfOl85TxwBs6B89Q5co6dMydw/nT+dv5x/u3sdY475 1fK+jdO4s1Xzi1r+P/ByVFhJY=</latexit>

SLIDE 4

Description of sparse stochastic process

7

1

Xid = hw, recti = h , i

1 n 1 n

X = hw, recti = h , i = h , i + · · · + h , i

1

i.i.d.

Definition: A random variable X with generic pdf pid(x) is infinitely divisible (id) iff., for any N ∈ Z+, there exist i.i.d. random variables X1, . . . , XN such that X

d

= X1 +· · ·+XN.

Specification of spatial dependencies

Whitening operator L

Canonical observation through a rectangular window

Specification of innovation (sparsity behavior) ⇒ s = L−1w w = white noise ) Xid = hw, recti is infinitely divisible

with canonical Lévy exponent f(ω) = log E{ejωXid}.

∈ S0(Rd)

<latexit sha1_base64="sUc5qrFhIvsr51Rwv234bJX9R98=">A JaXiclVZtb9s2EFb vcTeW7IBw7B9IeYa 4vGkN1u2ToUKJCiSIF+6JKlLRJ5ASWdZdYUpZJUHI/Tn9nX7Q/tN+xP7CjZrl6SwRXghOA9fJ7j3elOfsqZ0q7 z7XrN957/4MPtzrdjz7+5NP tnc+f6GSTAZwHCQ8ka98qoAzAcea Q6vUgk09jm89Gf71v7yHKRi fhVL1IYxzQSbMICqnHrbPtLjwnixVRPA8rNUf7dLe/wt/D2 XbPHbjFQ9qL4XLRc5bP87OdrZ+9MAmyGIQO FXqdOimemyo1CzgkHe9TEFKgxmNwBRu56SPWyGZJBJ/QpNit4ajsVKL2EekdVA1bXbzMt p ic/jg0Ta ZB KXQJONEJ8TGgIRMQqD5gtAgQH8zqtGPGoPCaED4cPDTaGymwM9B4w0kCJgHSRxTERpvQmPGFyFMaMZ1bjw1Wa27/W6fFKFXBKHEz gHrR7gvocYzFRxVRNnUcLD3MjIz83wrjvY+/6um7cwEhZLjDtAQPlDWNUZn2dQyqAndT859bE6NMQMTSWpRefEkxkHY1XhIl/9zxvEVn1z4qW3G1Lj5d+J2gZrM2qfYx7fJR4I/x/qOoGegsp8hRWEb1BuiPGmCusG7phd /ADHqlHGcuUc3Kz9Em9yaiEm29PjKwEuUrjSpm9tzLrmGsmFlfq3FvpVFVK/vx0OF6trdYdc1mdDG1M7J8lpy3GgtNWPukNL2dvCGyqUREpk7HSaOaZRTaTVoAYm1qJnWET53fX3pe0pM6bUlnl3cxTm8olHylawFEKAaOc2B6W8PL9r6owcZibdf/1fXPYvB8TJ3XESROxj5WxMu43jYcVY4v7pGJs0R4w7oPUK4idDQdNDLzBRmVTo7HgJHDjnWO3l4yKCBu+eYj4+oFnSZpjNGVMnjW5Hq9Nj5ump2vT06YpZEIvbWFL7QmOSAayeoUnrUT7F/Y8Zicsk0QuWiU2ayBmLcS8gfCSGCJqHervFg85Ao1vZ0TwtSYpzhei2O9AQAcDfLqeApzbItLTsvJECPZebhC3+sJq1Cst8fQUa3AwarijqZ9XmgUWOvLgTKq aLjQcxZqS3AfzZ0+YROySDIiAEchzk avs6ULhwugCSZ4EKEybzOhP6qGUt uxzViOYURzoSY pkEmYB3hkdskHwQc8B AZC0kjSFIc3fm4Mmx8X7cWL0WB4b+D+cr/36MHyw2PL+cb51rnlDJ095 Fz4Dx3jp3A+cP50/nL+Xvr385O56vO1yX0+rXlmS+c2tPp/QfuE2vI</latexit>

⇒ Probability laws of sparse processes are id

8

⇒ pY (y) = F−1{efφ(ω)}(y) = Z

R

efφ(ω)−jωy dω 2π

= explicit form of pdf

Unser and Tafti

An Introduction to Sparse Stochastic Processes

Analysis: go back to innovation process: w = Ls

Generic random observation: X = hϕ, wi with ϕ 2 S(Rd) or ϕ 2 Lp(Rd) (by extension) Linear functional: Y = hψ, si = hψ, L−1wi = h

z }| { L−1∗ψ, wi If φ = L−1∗ψ 2 Lp(Rd) then Y = hψ, si = hφ, wi is infinitely divisible with (modified) L´ evy exponent fφ(ω) =

R

Rd f

• ωφ(x)
• dx

<latexit sha1_base64="mFLHMsp3AJPrKENid/anCo4EUqg=">A Ke3iclVbdbts2FJa7v8T7a7bL3RCLgyZBYsju mwdAhRIUSRALrp0abuErkFJRzYX6qckFdsj9Ap7mt1u7 GHGbBDyU4tKRlcAbZpno/fd8hzdA69VHClXfef1r0P vzo40/W1tuf vb5F1/e3/jqpUoy6cO5n4hEv aYAsFjONdcC3idSmCRJ+CVd3Vk7a+uQSqexL/oWQqDiI1iHnKfaZwabrQenISkY2jBZDyRQU7TMc8P6WmSvjH7PcqUxinFCeUxOR2m2/TsTbDTIXoMcbvz6yEVLB4JIBazRxShsvh/M9/k3iOTBapDuEKEFxIKEeFxyGOuQcxIwK+54riLnFIy4XpMtqMkQMch2CkXnNIHcD0jME2TG Kdk04 vEVsmyYRjNjOIbqvh8Y6n5OQenw0t9y6xpvuWMgODXAVwb+d4f1Nt+sWD2kOevPBpjN/ng831n6iQeJnETrnC6bUZc9N9cAwqbmP+2rT EHK/Cs2AlN4kJMtnApImEj8oG4xW8GxSKlZ5CEyYnqs6jY7eZvtMtPhDwPD4zT EPulUJgJohNiswKPW4Kv8dyZ76O/GdPoR4VB+UxAcNj9sT8wYxDXoHEHEmKY+EkUsTgwNGQRF7MAQpYJnRuqwsW4vdXeIkUyKoJQ4mVCgFaPcZ4iBnO3DEGUjRIR5EaOvNz09tzuwaM9N29gJMzmGLeLgPKDsGVnbDRLGfSk6qdgHr4vGiKeV2KPSZkJMFYVpvniN68RW/XVieferkiNm38vantYq1F7AuP4PueB8P+hrhJgKVCZpzCDsKbkBt+oscK8gV2z73a/xyXVU8Y0FYJ0Sp/U24xJ6Lxb0bcS5C6NO2UO3sncnLnm8exOnYcLnW Vkj+/7A0WY6u1a27Lk549E/s157TJWHDazCebvdvZawKraiyJlMFYaNTjzEc2klaAGBtaiZVhFef3b7wvaUmVN2VymXc1T20o53ykKAEvUvA5E8TWsESU7/+yCo/PcvtNqC1hnmfO6v j8U VcVFH GFmLIxHdePZkrHBfbFkbNAec+GB1AsIVkNzXMfAWyxUNjQaE06CMPQaq73kRavLzSHiqwuwx+Z4mjIip3Wupzemp3XTyY3p G6y7WpuCxpqz/DSwE ub+FZI9De1K7H6ARlkMi0kWJXNcRVAzGpIcpOax3a2i8e8gI0vp0je4sgKfYXovjvQED7X zaVAHeZOKRHpeZFwe2wxvXjxp1YXH5UVri6jHmYLdfc0czL18qFpjoyIM9aVlFw1RPeKAtwXdoXt8iPCSzJCMxYCvEPsmC3zKlC4cLIElCHMRBMqkyob/qiqe2XPYrRBOGLR2JMEQyCTIf94wO2UPwQE8AYjwIyUaSpdi8 brRq18umoOX/W7vYdf9ub/5 PH84rHmfON862w7PefAe IcO8+dc8dv/dH6s/VX6+/1f9ub7d32Xgm915qv+dqpPO1H/wGgiNC4</latexit>

SLIDE 5

Examples of infinitely divisible laws

9

4 2 2 4 0.1 0.2 0.3 0.4 4 2 2 4 0.1 0.2 0.3 0.4 0.5 4 2 2 4 0.1 0.2 0.3 0.4 0.5 0.6 0.7 4 2 2 4 0.05 0.10 0.15 0.20 0.25 0.30

(a) Gaussian (b) Laplace (c) Compound Poisson (d) Cauchy (stable)

2000 5 5 2000 5 5 2000 5 5 2000 5 5

Sparser

pid(x) pCauchy(x) = 1 π (x2 + 1) pGauss(x) = 1 √ 2πσ2 e− x2

2σ2

pLaplace(x) = λ 2 e−λ|x| pPoisson(x) = F−1{eλ(ˆ

pA(ω)−1)}

Characteristic function:

b pid(ω) = Z

R

pid(x)ejωxdx = ef(ω)

Aesthetic sparse signal: the Mondrian process

10

λ = 30 L = DxDy

F

← → (jωx)(jωy)

SLIDE 6

High-level properties of SSP

11

Sparsifying transforms / ICA: SSP are (approximately) decoupled in a matched operator-like wavelet basis.

N-term approximation properties: SSP are truly “sparse” as described

by their inclusion in (weighted) Besov spaces.

Unser and Tafti

An Introduction to Sparse Stochastic Processes

(Pad-U., IEEE-SP 2015) (Fageot et al., ACHA 2015) Explicit calculations: Analytical determination of transform-domain statistics (including, joint pdfs). Infinite divisible probability laws: broadest class of distributions preserved through linear transformation. Unifying framework: includes all traditional families of stochastic processes (ARMA, fBm), as well as their non-Gaussian generalizations.

12

OUTLINE

■ Non-Gaussian statistical modeling ✔

■ Sparse stochastic processes

■ Hybrid sparse stochastic processes ■ Iterative image reconstruction (MAP formulation)

SLIDE 7

Hybrid stochastic processes

13

Hybrid sparse processes: shyb = s1 + · · · + sI

<latexit sha1_base64="LD9PB74uj WYRoWido0CRCzVHVE=">A JlHiclVZb 9s2F a7W+xdm 7AXvZCLAkwtI0huduyZQvQIV2RAMXQpUtbJDIMijqWOVOXklQcj9Dv2a/Z64b9mx1KdqpLMrgEZBM8H7/vkOfwkE muNKu+ t2+ 8+97 H2z0+h9+9PEndzbvfvpCpblkcMpSkcpXAVUgeAKnm sBrzIJNA4EvAxmh9b+8gKk4mnym15kMIp lPAJZ1Tj0HjzJ1/lgQJmjhaB5CFRGZUKSCZTBkqB2ifbPlFj48uYTBdBcaDG3n2fhalW9 X4eLvojze3 IFbNtLteMvOlrNsz8Z3N37w 5TlMS aCarUuedmemSo1JwJKPp+riCjbEYjMOUKC7KDQyGZpBK/RJNytIGjsVKLOEBkTPVUtW128Drbea4n340MT7JcQ8IqoUkuiE6J3S4ScglMiwWhjKG/OdXoR4NBMSogPBh8PxyZKYgL0LgC QnMWRrHNAmNP6ExF4sQJjQXujC+mqz6/Z3+DimjpAhCSZALAVrt47iPGAxquVQT51EqwsLIKCiM98Ad7H3zwC06GAmLJcYdIKD6EFZ3JhA5VDLoSdNPQ NMJA0xR1NFatEF8WUuwFhVuCxW/0WL2KqvT7z0dk1qXPxbUdvNWo86EBjHt9kPhP8PdZNAT6E6XvawFY f4oHjME9s+sOvsUpzV3GNBUCD1wpol7nVML2mxlDK0Fu0rhRZu+NzNWea54sbtR5uNKpq1T8xbk3WvWt1j1zXZ54dk/sz5LTJmPJaTOfbHnXs7cE1tWoiVTBWGm048wjG0krQIwNrcTKsI7zu1feV7SkyYv1s 67nqc2lEs+UpaA5xkwTgWxNSwV1fmvq/DkpLC/xLclLAjMSXt9PDlrIs7aiEPMjJXxsG08qRk73Gc1Y4f2iIsApF5BsBqaozYGXmOhsqHRmHAShPEvsNpLTpMIC745QHxzwtM0K0x56Txtcz2+Mj1um46vTMdtU8gTvbSFHbUneJtykPUlPOkEOri08zE6YRUkctlJsVkLMesg5i2En8YQUevQzm7ZyHPQeDojgseaZHi/EMX/A KaDbD1fQV4xSeRnlaZl4Rg1+WyuFMXVq8CpSXOnmIODoYtdzQNilqxwERH ryT6ioaLvWch9oSfI3m3g7hE7JIc5IAXoV4T9Lw91zp0uES NIJdpIwnTeZ7MtixjNbLocNojnFKx2JMEQyDXMG9hXC7CYEoOcACW6EpJGkGV7e+Nzw2o+LbufFcOA9HLi/Drce7S8fHhvOF86XzleO5+w5j5wj5 lz6jDnT+cv52/n 97nvR97h72fK+jtW8s5nzmN1v lP8TZfFM=</latexit>

c Psi(ϕ) = E{ejhsi,ϕi} = E{ejhL−1

i

wi,ϕi} = E{ejhwi,L−1∗

i

ϕi} = c

Pwi(L1⇤

i

ϕ)

<latexit sha1_base64="ninWQOfNgWqE/72GwEj09WyxjwI=">A K6XiclVbdbts2FJa7v9T7a7bL3RBLDaRFYshpt2wFAhRIW6RAL7J0aYNEqUFJxzJr6qckFdsj+BC7G3a7B9lz7AF2u73CDiU5lWV7cAjYIng+ft8hzyEP/YwzqVz3r9atDz786ONPNm63P/3s8y+ vLP51SuZ5iKA0yDlqTjzqQTOEjhVTHE4ywTQ2Ofw2h8dWv rKxCSpcnPaprBZUyjhA1YQBUO9TdbZ96YhTCkSnsxVUMZCH1sTF/LvmbGbHtXVGRDdu/AezrJPA1vtPZETN4a4nGaRByI7LMdUsGIJ4pB45mVE7RXeK19niPuRZr12Ru92zOGzCwCQjPuM3Mj2iWzVyh5VCpjl Ev34kmsdleh/he/86W23WLRhY7vaqz5VTtuL+5MfHCNMhjSFTAqZQXPTdTl5oKxQL0sO3lEjIajGgEunDAkA4OhWSQCvwlihSjczgaSzmNfUQWa2ra7OBSmx0RciBNbfAiV4MfLjVLslxBEpTqg5wTlRKbWyRkAgLFp4QGAS4ipwqdq9NeyIByCA+6P+5d6iHwK1C4LAEJjIM0jmkSam9AY8anIQxozpXRnhzM+u1Ou0OKlJYEocTPOQclH+G4hxg8AWVY4jxKeWi0iHyjeztud/+7HdcsYARMK4zbRUD5Q1jdGRvhUgY9mfeTUx9PnYKYmbl8wIzKMcutKkzM7GsaxFZ9feLK2zWpcfE3orabtR61z GON9kPhP8P9TyBGoLMfYkZhDeT0Xh8hxLzBu7rXbf7PU6Z32XMXc7J3dIn+S6nAu6+n7FnJcgqjZUy+ 9lrvdcsWS6UufBTKeuUvKbi97lrG+17utledKze2L/Kk6bjAWnzXy 1VvO3hBYV6MmUgZjptGM 4tsJK0A0Ta0IoK1nN+9 r6kJfO8GRV13vU8taGs+EhxBbzMIGCUE3uxpbw8/3UVlpwY+0+Ke9z39UlzfSw5n0ecNxGHmBkz42HTeFIzLnCf14wLtEeM+yDUDIK3oT5qYuAdXlQ2NAoT gDXtqIowco6pQ8QX9xlthKWpSqtvqj4FHcuSTmLmZIWVyfGQmXKsvmiqfnk2vSkaXp+bXreNIUsUZUtNE21Z/hEYSDqS32 kBD+xM7HKIZlM lkIRVHDcRoATFuILw0hohahzq7RSMvQeEpjg ef5JhHSKS/QIEVNDF1vYk4LspidSwzNAkBLsuN4gX7o/ZU0sqgbOHmKvdvY 7ivqmdqng UAerF1 FQUThY8NZQkeovl2h7ABmaY5SQBLJtZTGr7NpSocLoAkHWAnCdPxPBP6K0cs 9fq3hzRmOJ7AIkwRCIN8wDXjA7ZTfB jQES3AhBI0EzrPz4Vuk1XyaLnVd73d6Drv Tw63Hj6pXy4bzjfOts+30nH3nsXPkHDunTtD6s/V365/Wv+1R+9f2b+3fS+itVjXna2eutf/4D5JeB+I=</latexit>

c Pshyd(ϕ) =

I

Y

i=1

c Psi(ϕ) = exp Z

Rd I

X

i=1

fi

• L−1∗

i

ϕ(x)

• dx

!

<latexit sha1_base64="5zB5FOES2H0uyANFmFMrD4BtL9A=">A KQniclVZb 9s2FJa7W+xdm 6PeyGWBkiKxrDTbtkKBAiQokiAPmTp0hYJE4OSjmQu1KUkFdsj+O/2J7afsLdhr3vYoWSnkpwMKQHJNM/H7zvkOeKhnwu 9GDwR+feRx9/8ulnK93e5198+dX91Qdfv1Z IQM4CTKRybc+UyB4Ciea wFvcwks8QW8 S/3nf3NFUjFs/QXPcvhPGFxyiMeMI1Do9U/6YSHMGba0ITpsQqkObJ2ZNTIUJmQ8Sy0doNeMZmP+WZvl+YyC0eG7w7txSG5dS5vzIFpTgVEeoNQnmokPr4ILaGqSK6ZDC2XYiSENsLp1Ofx mLQFwVY+jL R/zCbA0pU9raOf8G9aebDr1JaIjsBP9TyeOx3hytrg36g7KR5c5w3lnz5u1o9GDliIZ UCSQ6kAwpc6Gg1yfGyY1DwTYHi0U5Cy4ZDGY0jVL1nEoJFEm8UH1crSBY4lSs8RHZLlHbZsbvNHmRqSKlK0NnhU6+vHc8DQvNKRBpR4VguiMuOCSkEsItJgRFgS4iIJpdK5Oe6YCJiDc7f+0fW7GIK5A47IkpDAJsiRhaWhoxBIuZiFErBDaGq iRb+3 lsnZU4pglDiF0KAVs9wnCIGU7AKWFLEmQitkbFvzfDxoL/z/eOBXcJImM0xgz4CqgdhdWdc7CsZ9KTp 2A+pr2GhNtGphAqCwHGqcLULn5ti9ip35147u0dqXHxH0TtNutu1L7AOH7IfiD8f6ibBHoMqvAVZhAeDdbgRzlWmDfwyGwN+j/glOYuY+4KQR5WPql3BZPw8P2MbSdBbtO4VWbnvcz1nmuezm7VebLQqatU/PZseL7oO61H5qY8Gbo9ca85p0vGktNlPlkb3szeErirRk2kCsZCox1nHrtIOgFiXGglHgl3cX7r2vuKljR5cybrvHfz1IVyzkfKI+BVDgFngriDLRPV919X4emxdW9S1gXfN8ft9fH0tIk4bSP2MTMWxv2 8bhmXOI+rRmXaA+48EHqBQRPQ3PQxsA7PKhcaDQmnARhXK3RkrM0xipgdhHfnIClyZqyXr5scz2/Nj1vmw6vTYdtk6tkc1u4pPYCaz8HWV/Ci6VA+1M3H6MTVkEi06U u2whLpcQkxaCZgnEzDm0vlU28go0fp0xwc+a5FhfiOK/AQEd9LH1qAK8kKSxHleZl4bg1jUIkqVzYXGHUVri7DHmYH+75Y5mvq0dFpjoyIM1qa6iYarxUqIdwVM0d9cJj8gsK0gKWAqxTrLw10Lp0uES LI O2mYTZpM6K+65Lk7LrcbRBOGdR6JMER4FyoCXDM65DbB z0BSHEjJIsly7Gi4x1k2L5xLHdeb/eHT/qDn5+u7T2b30ZWvG+97 wNb+jteHvegXfknXhBZ68TdbJO3v29+1f37+4/FfReZz7nG6/Ruv/+B9xYw7k=</latexit>

Description of independent components

Innovation model:

Lisi = wi ⇒ si = L1

i wi

Whitening operator: Li : S0(Rd) → S0(Rd) Lévy exponent of white noise:

fi : R → C

Characteristic functionals

c Pwi, c Psi : S(Rd) → C

<latexit sha1_base64="BvB3 yWVHoQpLy8Id3VWusdeE8 =">A Q3niclVdb xtF J4WCK25pfDIy4jEgqLEslOVQkSkSm rRgolJL2p2T M7o7tafbW3dnE7mhfeUO8 tf4BYhfwCvfzKydvSTFXcvrmXO+c5lz pwZu0kgMtnv/3Xl6nv f7D04bXrnY8+/uT z5ZvfP40i/PU40+8OIjT5y7LeCAi/kQKGfDnScpZ6Ab8mXuyrfnPTnmaiTh6LKcJPwrZKBJD4TEJUnzj6t/EIRnJiYs3Jx5R5B5+M4xSIkhCJN4xiQjFe4i3wNgHIsHXjiJgKPAxCUHVWEvLSAHdLmYjI6VIgN8MnAJjx0gE+KaYhfBghBEnU3ApPg5mOfgc3D7pkTvkNsYTI1ufF6RDuqUHVl9GNjF3MWOgnpA1zLRu34ysHe2jnoXAjEqPmaF48IGV3Dp2ilFgbJyVstprbdWvzO0qrE+zFa8jMgwcBgzHnBsvKfkG1oZYDSW7hpcY/2U5l2aF SNGOBzi9yEZh09CWRicpaQLXAs0jERG4IbGns28ltkA9TQ8IXRGl4gqfM9IuOWbB8+hibK59KU7IAXwYdT4+esRkL8+iZKm0C/RjwY5jqfq62c6/zkwOoq2TWrOY enf3MjChs05aUzWOBDFj0uRWH7M9XoP1PTZ6qXrxN78XevARnHTEqjF/ 58tqI0bP4IudRSaWI7OLEpN9ZjKYmkpdPDabJlOhkR5Dg GnyAE4X6OWdARewqOb5Q7SFhbDNz3fJf8gs1OMdHVUd/WsD5zN10ZNHQhTUfWst9c1i9iwtp79mrfbLW+2YcvuHq/cO3ZXCcwpdOXw JtXoV5jZjSdV0a9Eh14L0ydjk1kVCVGtvMpsmdyfozR27KuzCr0Z+0dtWYV6cuzWs1RO0K2A1f7aud4eaXf65uHtgeDcrBCymfv+Mb1fx0/9vKQR9ILWJYdDvqJPFIslcILeNFx8ownzDthI67MeVPQLkg+HcYpvpGkhlrDsTDLpqELZMjkOGvyNPFCnqak2TCrW3XdsOhUKYe5H 5/pESU5J Hn VomAdUxlQfctQXKfdkMKXM87CunEn4W9PweHCktPNavMbIPBZwf6v3w8aRGvPglEvwUx7xMy8OQxb5yhmyUARTnw9ZHshCOdlwNu50O1 qDt2MAkrdPAi4zDZBd4DBGW1ipcJ8FAd+odKRW6jBWr935/Zav2h Uj4tMf0eAPYLWNUZN8i5NQNP6n4GzMW9QPJQgGWVanRBnTQPuNJW+aTozAZFQ7M2v7jm0t1FdWP576Rbh2tB3W6AVL5LSAB/m+6 BjnmWe5mKC/cnwpFlTPOUDv8W7Xe730HkXqkUetBQFc71qvsdc5SvnousqFt0MuMXGrnzrmdedyliKaXG7o1M1Q1Yw0Uh9gLzrmxb9VF1TKA vMqdeoQmQhRvQHoyuBi9Q0LixqpWjGJmBtpJluMdDq1Bap0flM0jAXc1 Es3acdo5fWFScsrSpewFe4qtMJhSY phc JNwTLKC6G8aBbQRVKyLaL/SbOr 1ua7aby5QRC/qiBdNxDaqY8bcbjL3K8yW7hcVZkvtowrzUZP5UAQuT+UMgp6pHjYx/DXamU6cREGmPFDOKQ6V LBohHNFbRW6qVcFduOkQKjTkO42d 2bs+41WTtz1k6T5YtIljy/Ze0B/kIJnlaX8KBVBe4bLY/U+TaD9A30dOuQSQMyadVoE5G2EGcNhBOHfMRaPrsnDdxJu/k1E FLyQ xkQ0xeJg24+rj5BRyWtjY+vZ4cp6uDGoGlN08s6dLT7HBcf5Z9EHiGfwswiW4S82OKUE/WZXDc274I+pkYrk/J2AbBWloEJbM+ZzITdo08dWrOfHVnLgT+cJTZSmHajBn7KWxa+mZl6q9meYHMcSdM+HzMZPKoOYi9yeJlYiTmUZ1Hx0gigMRComLS6e7bh56wCVa8oi l9MENwuaiTec un18HScjO /ejS Y3R7PpEwhzuRGgw83HWqTN2HIp0KXAIMr35SzP7 ZzKFaq2ht9FIpGRuUTk+dEOFo LiqjhoXgzbg6cbvcGtXv+XjZW7m+Wl8Rr5knyFq+kAf8Lvkoe41D4h3tLtpcMlf4kv/7r82/Lvy39Y6NUrpcwXpPYs/ kfHkQh7w= </latexit>

c Pwi(ϕ)

M

= E{ejhw,ϕi} = exp ✓Z

Rd fi

• ϕ(x)
• dx

◆

<latexit sha1_base64="o4BpaWf2w+9Nx2v7YuWVxJqPwk4=">A KRniclVb hs3EF0lvVjqzWkf2weijgE7sAXJSeM2gIGgTgIH9YPr1Elg0xG4u7MrxtxLSK4l eBv9Rv6C/2JvhV97XBXclYru1AISKI4h+cMOUMO/VxwpXu9v1q3bn/08Sefr Q7n3 +xZdfrd75+qXKChnASZCJTL72mQLBUzjRXAt4nUtgiS/glX+x7+yvLkEqnqW/6UkO5wmLUx7xgGkcGqz+QUc8hCHThiZMD1UgzZG1A2NoyW0khHY04NZu0Esm8yHf7FB4F0JE6N xTg28QahMyFtLqGBpLICMtsgUS6gshy 1exQ LiDSG4TyVA8MPX4TWlLXiVCH+jyeSW1Qf7zpBjYJDXEOwf9U8nioNwera71ur2xksdOfdta8aTsa3FkZ0zALigRSHQim1Fm/l+tzw6TmAXrYoYWCnAUXLAZTumTJOg6FJMokflC9HJ3DsUSpSeIjsty7ps0NXmtzI1JFytYGzwod/XhueJoXGtKgUo8KQXRGXOBIyCUEWkwICwJcRME0OlenPVMBExDudX/aOTdDEJegcVkSUhgFWZKwNDQ0YgkXEw fK4S2hqpo1u+sd9ZJmS+KIJT4hRCg1SMcp4jB9KoClR xJkJrZOxb09/qdXd/2OrZBYyEyRT 6yKg+iCs7owvCqhk0JN5PwXzMaU1JNzOMsShMclkIcA4VRjb2a9tEDv15Ymn3i5JjYv/IGq3WctR+wLj+CH7gfD/oZ4n0ENQha8wg/DYW4Pnbqgwb+Ce2e51H+KU+V3G3BWC3K18Uu8KJuHu+xk7ToLcpHGjzO57mas91zyd3Khzf6ZTV6n47Vn/fNZ3WvfMdXnSd3vivqacLhlLTpf5ZK1/PXtDYFmNmkgVjJlGM848dpF0AsS40Eq8EpZxfv K+4qWzP mTNZ5l/PUhXLKR8or4EUOAWeCuIstE9X5r6vw9Ni6b1LWC983x8318fR0HnHaROxjZsyM+03jc 24wH1aMy7QHnDhg9QzCN6G5qCJKWuXC43GhJMgjKs1WvKqTpk9xJd3matuVUnMpr+o+BR3Ls0ET7hWDlcnPsxyW5XCw6bmkyvTk6bp+ZXpedPkKt7UFtqm2jOs/x kfanPFhLCH7v5GMWwCiYZL6TiRQNxsYAYNRA0SyBmzqH17bKRF6DxFMcEjz/JsQ4RxX8HAjroYutQBfgoSWM9rDI0DcGtqxckC/fH7B2jtMTZQ8zV7k7DHc18W7tU8EAgD9au oqGscZHjXYED9DcXic8IpOsIClgycR6ysK3hdKlwyWQZBF20jAbzTOhv+qC5+5a3ZkjGjF8DyARhkhmYRHgmtEhtwk+6BFAihshWSxZjpUf3yr95stksfNyp9u/3+39+mDt8aPpq2XF+9b73tvw+t6u9 g78I68Ey9ofdf6ufVL67D9Z/v 9j/tfyvordZ0zjfeXOt4/wGfncQl</latexit>

Hybrid processes: Audio signal modeling

14

Sparse, bandpass processes

L = dn dtn + an−1 dn−1 dtn−1 + · · · + a1 d dt + a0I

(a) Gaussian (b) Alpha stable 훂=1.2 Gaussian (Am) generalized Lévy (Am, S훂S)

Hybrid sparse processes: shyb = s1 + · · · + sI

<latexit sha1_base64="PE2RszhxdyfgInLzx0ZwS7n2Uo =">A JlHiclVZb 9s2F b XWLvlnbAXvZCLAkwtI0hudvSdQvQIV2RAMXQpUtbJDIMijqWOVOXklQcj9Dv2a/Z64b9mx1KdqpLMrgEZBM8H7/vkOfwkE muNKu+ Nm7fe /+D zd6/Y8+/uT z Zv3 mp0lwyOG pSOXrgCoQPIETzbWA15kEGgcCXgWzA2t/dQ5S8T 5TS8yGMU0SviEM6pxaLz5k6/yQAEzh4tA8pCojEoFJ MpA6VAPSLbPlFj48uYTBdBsa/G3j2fhalW9 T4aLvojze3 IFbNtLteMvOlrNsz8e3N37w 5TlMS aCarUmedmemSo1JwJKPp+riCjbEYjMOUKC7KDQyGZpBK/RJNytIGjsVKLOEBkTPVUtW128CrbWa4nD0eGJ1muIWGV0CQXRKfEbhcJuQSmxYJQxtDfnGr0o8GgGBUQ7g+ H47MFMQ5aFyBhATmLI1jmoTGn9CYi0UIE5oLXRhfTVb9/k5/h5R UgShJMiFAK0e4biPGAxquVQT51EqwsLIKCiMd98d7H173y06GAmLJcYdIKD6EFZ3JhA5VDLoSdNPQ NMJA0xR1NFatEF8WUuwFhVuChW/0WL2KqvT7z0dk1qXPw7UdvNWo86EBjHd9kPhP8PdZNAT6E6XvawFY f4oHjMFds+sOvsMpzV3GNBUCD1wpot7kVML2 xlDK0Gu07hWZu+tzOWea54srtV5sNKpq1T8xZk3WvWt1l1zVZ54dk/sz5LTJmPJaTOfbHlXs7cE1tWoiVTBWGm048wjG0krQIwNrcTKsI7zu5feV7SkyYv1s 67nqc2lEs+UpaAFxkwTgWxNSwV1fmvq/DkuLC/xLclLAjMcXt9PDltIk7biAPMjJXxoG08rhk73Kc1Y4f2kIsApF5BsBqawzYG3mChsqHRmHAShPHPsdpLTpMIC7 ZR3xzwrM0K0x56Txrcz25ND1pm4 uTUdtU8gTvbSFHbWneJtykPUlPO0EOriw8zE6YRUkctFJsVkLMesg5i2En8YQUevQzm7ZyAvQeDojgseaZHi/EMX/A KaDbD1fQV4xSeRnlaZl4Rg1+WyuFMXVq8CpSXOnmIODoYtdzQNilqxwERH ryT6ioaLvSch9oSfIPm3g7hE7JIc5IAXoV4T9Lw91zp0uES NIJdpIwnTeZ7MtixjNbLocNojnFKx2JMEQyDXMG9hXC7CYEoOcACW6EpJGkGV7e+Nzw2o+LbuflcOA9GLi/DrceP1w+PDacL52vnK8dz9lzHjuHznPnxGHOn85fzt/OP70vej/2Dno/V9CbN5ZzPncarf Lf8Q/fFE=</latexit>

c Pshyd(ϕ) =

I

Y

i=1

c Psi(ϕ) = exp Z

R I

X

i=1

fi

• L−1∗

i

ϕ(t)

• dt

!

<latexit sha1_base64="Oj/DNcqrpP50OxGSt5i2knskcng=">A KIXiclVbdbts2FJa7v9j7S7fLXYxYGiApGkNKs2UbEKBAuiIBi 1Ll7ZImBqUdCRzoX5KUnE8Qk+zp9gj7G7Y3bCX2aFkp5KcDC4B2QTPx+875DnkoZ8Lr Tr/tO78 67 3/wUp/8OFH 3/y6erdz56r JABnASZyORLnykQPIUTzbWAl7kElvgCXvgX+9b+4hKk4ln6i57mcJ6wO URD5jGodHqH3TCQxgzbWjC9FgF0hyV5ciokaEyIeNpWJYb9JLJfMw3B3s0l1k4MnzPK18dklvn8tYcuMqpgEhvEMpTjcTHJaGqSK5 ohGnPo836NMsH/FXZsujTOlyRrGhN61 k9AQpxN JY/HenO0u YO3aqRxY436 w5s3Y0urtyRM sKBJIdSCYUme m+tzw6TmgYByQAsFOQsuWAym2tmSrONQSKJM4ofa1WgLx KlpomPyGoLujY7eKPNjkgVqbIxeFbo6Ntzw9O80JAGtXpUCKIzYmNHQi4h0GJKWBDgIgqm0bkm7ZkKmIBwb/jd9rkZg7gEjcuSkMIkyJKEpaGhEUu4mIYQsULo0lAVzfuD9cE6qVJGEYQSvxACtPoexyliM Oq9ZukiDMRlkbGfm 8B+5w9+sHbrmAkTCdYdwhAuoPYU1nfF ALYOetP0UzMes1pBwN WkFo2pIwsBxqrCVTn/LzvEVn154pm3S1Lj4t+K2m7WctS+wDi+zX4g/H+o2wR6DKrwFWYQnvzSE PHCvMG7pstd/gNTmnvMuauEORe7ZN6XTAJ9 7M2LYS5DaNW2V238hc7 nm6fRWnYdznaZKzV+e fzvtW6b27KE8/uif2ZcdpkrDht5pM172b2jsCyGg2ROh zjW6ceWwjaQWIsaGVeCUs4/zWtfc1LWnz5kw2eZfz1IZyxkeqK+BZDgFngtiL RP1+W+q8PS4tL+kuvZ93x 318fT0zbitIvYx8yYG/e7xuOGcYH7tGFcoD3gwgep5xC8Dc1BFwOv8aKyodGYcBKEsXVGS87SGKuA2UN8ewJWpdJU5fBpl+vxtelx13R4bTrsm wdm9nCBbUnWNo5yOYSniwE2r+y8zE6YR0kcrWQYhcdxMUCYtJB0CyBmFmH1reqRp6BxtMZEz WJMf6QhT/DQjoYIhtQBXgeyON9bjOvDQEuy43SBbuhfkTRWmJs8eYg8Ptj ua+WXjs BERx6sSU0VDVca3xzaEuygub9OeESmWUFSwFKIdZKFvxZKVw5XQJ F2EnDbNJmQn/VBc/tdbndIpow+8bICIYInzpFgGtGh+wm+KAnACluhGSxZDlWdHyDeN0Xx2Ln+fbQezh0f95Ze7Qze42sOF84XzkbjufsOo+cA+fIOXGC3pe9H3o/9n7q/97/s/9X/+8aeqc3m/O502r9f/8DiCu0ag= </latexit>

SLIDE 8

Discretization of linear inverse problem

15

Discrete innovation model

(M ⇥ K) system matrix : [H]m,k = hηm, βki = Z

Rd ηm(r)βk(r)dr

s(r) = X

k∈Ω

s[k]βk(r) s(r) =

I

X

i=1

si(r): hybrid sparse process

Partial signals and innovation vectors: si =

• si[k]
• k∈Ω and ui =
• u[k]
• k∈Ω of dimension K

Li: (K × K) matrix representation of Li,d y = y0 + n = H(s1 + · · · + sI) + n

Measurement model (image formation)

ym = Z

Rd s(r)ηm(r)dr + n[m] = hs, ηmi + n[m],

(m = 1, . . . , M) ηm: sampling/imaging function (mth detector) n[·]: additive i.i.d. noise with pdf pN ui = Lisi

Posterior probability distribution

16

(Bayes’ rule)

(decoupling simplication) pS|Y (s1, . . . , sI|y) = pY |S(y|s1, . . . , sI) pS(s1, . . . , sI) pY (y) = pN

• y − H(s1 + · · · + sI)
• pS(s1, . . . , sI)

pY (y) = 1 Z pN

• y − H(s1 + · · · + sI)
• pS(s1, . . . , sI)

Independence of partial signals:

pS(s1, . . . , sI) =

I

Y

i=1

pS(si)

Innovation model:

ui = Lisi ⇒ pS(si) ∝ pUi(Lisi) pS|Y (s1, . . . , sI|y) ∝ pN

• y − H(s1 + · · · + sI)
• I

Y

i=1

pUi(ui) ≈ pN

• y − H(s1 + · · · + sI)

Y

k∈Ω I

Y

i=1

pUi

• [Lisi]k

SLIDE 9

Statement of MAP reconstruction problem

17

Maximum a posteriori (MAP) estimator

arg min

s1,...,sI∈RK

✓ 1 2σ2

• y − H(s1 + · · · + sI)
• 2

2

+ X

k∈Ω

ΦU1([L1s1]k) + · · · + X

k∈Ω

ΦUI([LIsI]k) !

Additive white Gaussian noise scenario (AWGN)

pS|Y (s1, . . . , sI|y) / exp ✓ ky H(s1 + · · · + sI)k2 2σ2 ◆ Y

k∈Ω I

Y

i=1

pUi

• [Lisi]k
• Hypotheses

y = Hs + n where n AWGN with variance σ2 s = s1 + · · · + sI (sum of independent sparse processes) Lisi = ui: i.i.d. with pdf pUi and id potential function ΦUi(x)

M

= − log pUi(x)

<latexit sha1_base64="FAcrEiX9R4VCmQ2TzhtN0bXuQ18=">A MUXiclVZtb9s2EJa7t9bJtnb7uC/EYgPpm2Gl29oVC9ChL0uAtsvaJikaZQYlnWw21EtJKrFL6Pfs1+zTgA37J9u3HSnZkeRmSAlbEu4ePne8I4/nZ5xJNRz+3bnw Ycf zJxUvdldVP /v8 pUv9mSaiwB2g5Sn4qVPJXCWwK5i sPLTACNfQ7 /tF9o98/BiFZmrxQsw OYzpOWMQCqlA0utK5 8ncNz8I9NYsS9UEJMi 6ynqk572/IjMik373irsSxbX7TspeuRkAgJIz3tCkh75cf+np+SEqQk5poLRJDAaycYx/XWj53kHLkwPK16cIDfNY+S 68QLUyXN2wi2e2Rd5jFJI8KSEDLAR6KIzKiQ DKRBiDRw6t1wtLPx8WIVR6OWOlyjl+9u4SwARuEA6/0Lgsj0stGenfEcAk0CQkLCa4czTDKSZQngQkOerkzYSVsfXrVgzchRDc9no5JtpD2uqPLa8PB0A6y/OFWH2tONXZGVy79gwsO8hjtBZxKe AOM3WoqVAs4IChzyVkNDi Y9A2w XpoygkUSrwj8Gw0gaOxlLOYh+RMVUT2dYZ4Tt1RiJkJ tWfT8u nXJQa6iO4eaJVmOYQpKh6KcE5USs61IyAQEis8IDQJcV04V+t geOEeauO8md5QyIByCDcH328c6gnwY1CoF5DASZDGMaZHexGNGZ9h+GnOFe5CGc2/u/1un9htLm0m/ZxzUPIuyj3E4KmwsdJxPk5 WGgx9gvt3hgObn97Y1gsYQTMKsxwgIDyj7C6Mz7PoTSDnjT95NTHk6g ZqgqSQ26IJ7IOWhjFaZFd/5RtJiN+fMzV+6elxuX/17cJlzn5PY5pvJ9QoLw/+NuMpiKZOuTOZSFJtqbYDEI4Jq+ORx8h1Oakca9zjnpdUuv5JucCuidTtkwNshZRs60c/vUziLui Wzsw3dmhuqmykNFAd4FrxTY9f0u3YLVrfCPipOEyIbIWIOAFlz303fsnBeI3UrNhELI+1ks7FJp7FAtMmvwIJxDvdNFCv3SdfykiYxFvg68Tl8RVdNOpHQBsPWgucZBKaKm2qY8rIQ1K2w5FlhnsQzpc/39bP2Alnyqol41Ubcx90xV95vK5/VlEvcr2rKJdqnNeXTtnKLcR+EmkOwZuqtNsbeUCZxCjekAK49vIiVuYrHeK/ozcIU9fqEx2lWYKhFjHdni+vBQvWgrdpeqLb qpAlqtKFS9YeYdPCQNSX8GhpF/hvzXxMXVhmkLxFn 4TMm1Bpkt7tI0QS4iTFsJLYxjTJZ/9oxbuaLn4tRB8iWSKMVGtaeihaMfVNDpMzYoytmF5PXl7a27DgC4Pz3z0yTEecLz/SvTzL D4eYQrcJ/YE1OBnpSU0ak2/gH3ybTU/pyh2hKI2CJKMcBC DZtRvj69UL4eiHcTkIW6Gorx9pdKHZE6pdyGQi9M2d+lOJ074SFMKFKW9RiysNpVs5IszmjfogVIEk5i5nCxqXbv2kHeQ4KS/KY C0nGXYWRLK3QEAFAx dTwJ2x8lYTbDaw1ShOeyJtPtNgL1OXWnqkO05sQmwu ZNMe+2pRJIbRgG 61EYkda1K4PU1CRCFeEvaLb7gyXP/Y2Bu6twfCXjbV7d6qu8aLzlfO1s+64zm3n rPl7Di7TtD5rfN758/OXyt/rPy76qxeK EXOtWcL53GWF35D5oke60=</latexit>

Hybrid model: reformulation

18

arg min

s1,...,sI∈RK

✓ 1 2σ2

• y − H(s1 + · · · + sI)
• 2

2

+ X

k∈Ω

ΦU1([L1s1]k) + · · · + X

k∈Ω

ΦUI([LIsI]k) !

<latexit sha1_base64="XGND8S/XYM DbIas67XsTRm5TjY=">A MUXiclVZtbxNHED7TN3DSFtqP/bJqMEogWD7TlhYVCQmIYjVACiQgs Hau1ufl+y9sLsXbLb3e/pr+qlSq/6T9ltnd23n7hxQOMn2e aZ 2ZnZmc3yDmTqtf7p3Xuo48/+fSz8xfaK6uf /HlxUtf7cusECHdCzOeiecBkZSzlO4p jh9ngtKkoDTZ8HRXaN/dkyFZFn6VE1zepiQOGUjFhIFouGl1h0c0JilmnAWp1fLNo6YzDmZSjXlFGEiYoQTlg41foDk0N9EOMqU3LT/Bpilj1/+UqIrmNORWscjQULtl7qPsGRxQl72S4QDFuPfNA5GaFpet7/b5bpju2bJrjmyDYc 9l+CuWDxWHURxu0ryJ 30TUgLRI JDhC4Bg/SmhMgH93zIZ6b+iX6wdAhHaGPnLsh0OAboBZaL283 ywMB8484Ezd5Fs4LHMSUiv6hthAkmiaT P2PDiWq/bsw9afvFnL2ve7NkdXr wLyw7LBKaqpATKQ/8Xq4ONRGKhZwCeyEpODsiMdW2wiXqgChCo0zAJ1XISms4k g5TQJAJkSNZVNnhKfqjETIkax7DQJYZFVyUKjRj4eapXmhaBq6gEYFRypDpq1QxAQNFZ8iEoawroIoiLfG8NQ/1CZ4Y15TyJBwGt3u/tQ/1GPKj6kCvaApfRNmSUIg03hE sanER2RgqtSYzmav7c7 Q6ybS4RQF QcE6VvAVyDBjYFTZXOinijEelFnFQan+z1735/WavXMI Op1hel0AuA/AqsE vKDODURSj5OTAHaiogkDlSM1aOgyUXCqjVc6Kdvzl7LBbNyfnXkW7lm5YfkfxG3SdUbugEMpPyQlAH8fd51BjaksAgntBROr1Egv9uL1XvcHMKlnGnqdc3S57aKSrwsi6OUTk7 xgd7l5J1+bp74WeRdsXT6bkc35o6qbpyD8gD2Aj5xdlWf1i0+ENivGadJkc0QMhsArfmn0zc8nNVJ1YstxMJ s9gsNuU0HpA29RUwM 4QvsniLHzUtryoTpwTUSU+Q6wQqiknENpk2FnwJKchIxyZaZhxNwiqXuDEKs03nGow+oJAP24ukKUv6ogXTcRd6I658m5T+bi XOJ+UVEu0T6sKB82lduMB1SoOQRmpt5uYuhrG emcAoaUlCu8TEcKoKRNIZzRd8uzVCvGuxkeQmpFgna XLdW6juNVWDhWrQVEUsVTNdtORtCy4tjIrqEraWuiB4a+yhdJGrIHoLPJ06ZNKATJZ6tIkQS4g3DQTO7H2gGXNw1MAdLQ+/BoIvkUwgJ6phBhGKZl4jODmZmpYut5E7nvD+ml9zoN3m T8d AwbHM4/h36ShxY/z/AM3EF2x8xADxzl6ESb/Ax9MnHaRzmoLYFILMKJKV0IqS2bEb56tRC+Wg HacRCPWvlRPsLxa7IAieXodC7c+atDMzxGxbRMVHaohYm9ye5s8jyOaO+DxMgzThLmIKLS7tz3T7oCVUwkmMEsxzlcLNAkr2liKqwC08bSwq34zRWY5j2dKLAHdyJtP+dvdBVlGYOpaYUcAmwuvpJMb9tSyWA2jB0+41CKhKUlePD FQg hXBXdFv3gyX /b7Xf9Gt/drf+3Ordmt8bz3jfet +753k3vjrft7Xp7Xtj6vfVH6 /W3yt/rvy36q2ec9BzrZnN1 7tWV35Hwb4fDk=</latexit>

arg min

s∈RIK

1 2σ2

• y − Haugs
• 2

2 +

X

n

ΦUn([u]n) !

s.t.

u = Laugs

<latexit sha1_base64="12EgNxsgzPSjp84RslR1xPWRqaM=">A L83iclVZLbxs3EJbTVyK1jdMe yHq HDSWNAqad0GNRDAiWGjTurasRPYtAXuLrWizX2E5NqSWf6S3ope+4N67p/oscNdSd6HXSgEtFrMfP N7MxwSDfhTKpu9+ FWx98+NH n9y+02x9+tn dxfvfXEg41R4dN+LeSzeukRSziK6r5ji9G0iKAldTt+4Z+tW/+acCsni6LUaJ/Q4JEHEBswjCkT9xX+wz2TCyViqMacIExEgHLKor/FLJDGLdk/01s/GoG8wpwO1jAeCeNoxuoewZEFITnoGYZcF+DeN3QEam5Xsf9MAgwgRSQOTMW Qfu+kh5rfgmka9nUEljtD1tf7/cgsH2mcfY4W1LcmqTnuRw+wYMFQPcDvUuJDYG480rKjOiYXoJrNGjy3y6 b/cWlbqebLVR/cSYvS43J2unfu/Mv9mMvDWmkPE6kPHK6iTrWRCjmcWqaOJU0Id4ZCajOAjCoDSIfDWIBv0ihTFrCkVDKcegCMiRqK s6K7xWZyVCDmTZq+uGplmUHKVq8MOxZlGSKhp5eUCDlCMVI1t45DNBPcXHiHgefFdKFMRbYnjtHGsbvDUvKaRHOPX Oj/2jvWQ8nOqQC9oRC+8OAxJ5Gs8ICHjY58OSMqV0VgOpu/NdrONskaUCKDIT mnSj4FOQYM9G1ewjANYu4bLQLXaOdRt7P63aOuqWE HU8w3Q4A8h/AisG4PKW5G4ikHCcnLuwVRUNmpr1j0dCJIuVUW690ZJrTF1Nhtu7nZ56EOy83fP57cdt0zcntcijl+6QE4P/HXWZQ ypTV0J7wUwxGnblUELv0Id6pdv5HkzKmYZe5xzdb+ZRSdjJgt6/MulZH+gmJzf6Wb3yM8u7YtH4ZkePp46KbnIH5gj2Ar5y9lBf1y0OEGSPCadNUZYhZDcAWnKup694mNdJ0UtWiJmTarFZYMtpPSBt6ytgYMwRvs3iJHzUzHhRmTghokg8R6wQqi0nEGbJyGbBXkI9Rjiy0zDm+SAoeoEzx9gnjHsYfa6rd6sfyKLDMuKwiliH7pgq16vK3YKyxn1YUNZoXxWUr6rKTcZdKtQUAjNTb1Yx9B2M 1s4BQ0pKNf4HA4VwUgUwLmi14wd6kWD7TgxOjvOtqtcz2eq51XV1ky1V X5LFITnV/ztgHXCkZF8RM2al3gXlp7KJ2fVxBdAk+7DBlVIKNaj1YRo a4qCBwHNKA1GJ2zyq4s/rwqyB4jWQEOVEVM4hQVP qw8nJ1NjkufXz4wkfLDklBzrfPNPVRuew eH8y9F7iZfhpxmegNso2zET0Muc nClDX+CPhnl2l8SUGcEIswQuZjSmZBmZbPC09OZ8HQm3Ip85ulJK4famSl2ROzmcukJvTNl3ojBHF8wnw6J0hlqZvJilOQWcTJl1C9gAkQxZyFTcHFptleyhfaogpEcIJjlKIGbBZLskiKqvA6sJpYU7q9RoIYw7elIgTu4E2n iQd3naLSzqHIlgIuAZmufFJM78NSCaC2DJ1epZCKuKZwfNiBCkTwRXBXdKo3w/rLQa/jPO50f32y9Ozp5NZ4u/FV4+vGcsNprDaeNTYbO439hrewtRAvjBbGrbT1e+uP1p859NbCxObLRm 1/voPTSJhYw= </latexit>

Augmented formulation of the problem

Unmixed signal:

s = (s1, . . . , sI) ∈ RK×I

Augmented innovation:

u = (u1, . . . , uI) ∈ RK×I

Augmented system matrix:

Haug = [H · · · H] ∈ RM×KI

Augmented whitening operator:

Laug = diag(L1, . . . , LI) ∈ RKI×KI

<latexit sha1_base64="s j/s5jhr0nY303Jx4f5eK7FCVA=">A NzHiclVfb tw2EN3EvSTbuk3ax74QsQ04wWaxctqmNWogRS6wESd1nNgJYrkGJc1qGVOXkJS9G0Kv/al+Sf+kjx1SF0tap9gIWK8 c3jOcEgOZ72UM6lGo3+uXF367PMv rx2vf/V18vf Hvj5neHMsmEDwd+whPx qMSOIvhQDHF4U0qgEYeh9fe6UPjf30GQrIkfqVmKRxHNIzZmPlUoenk5tW/XZl5Enz9exZGECsIyDgRUcYtgCRjoiZAUpEgY5S7HoQs1iawXLtWXkdZKGCWE+K jIMeDe/ BNO8+s7 a8Ti5CbxOPVPB0RAMCA4BuIBwXBQlA6IP6NxbZ0B58k5erMw4YH9RglkMsJ3UypoBApnRdb98ZDs0ihNhMJvBVOSDonj3Lvd165KUgnploNGl8NYRVRg8FsbEBGXKYhqp2DhpPKO/CjvWzfRrjcmB3HEphDkRLIwpnyTuO8zGpBV9xmRW+vm74kzcINEyYH9Z+c2ksfE3f9TPyWuYhFIspOvlpQXSb kZQYxITmL4+TM5jzfLCTIqm74kTzLrWDWFMwWFpQzaQwRVYJN 0ljHtsn2hURoVmYbx0RYyAuhmYkzAv+f1xLPKsknl6mcT5BQ8zikCQpCKoS0RLabQgR+xowGubr1tWY1G57UjulZKkJcVBswP7JjZXRcGQfMv/ilC8rvfLZO7l5/V8U8TMTrs+plEfOKFXHmgrFfA648BluCtykuCu1zX1O1tBkzwR+YlXs5RaORlLOIg+RmNuJ7PqM8VKfsQg5lm1Vz8P917QcZWr8y7FmcZphbv0ioH GiUqIOdGYQwG+4jNCfR/nlVGzEi2GV86xNsGb4S2H9CmHYGv468axngA/A0yqKyCGcz+JIoqJdsc0YnwWwJhm3Jx5Oa7e+2t4IG2FkQShxMs4ByU30e4iBgtSXR/wEOdahF6unYGpC4NRPocxNaTAjIYIKD4Iawbj8QwKGYykHSenHhZB3I6sLkwGnberUr8uSx1mI784c1XyFuTG6X8St0nXgty2pH5KShD+f9xtBiz9xe1g65LGMzuRuHfgjr47Gv6MQ9qZxr3O cEjamUkn sBqxdDNowG+ZjIR3XuX+jUeVcsn 1c6F4l1JQpBPIjPAvuhdgdfdluc wFZv6UnCZFNkPEHACy4lxO31FYVKSpUtydlUh3sVloltMoEG3WV2DBWCB8k8UyfNK3vKRNjDdqk3iBWDFUs5xIaJNha8HLFHxGOTHVMOF IWiqsHg/17asm9LneXq/O0EWv20j3nYRD3F3VM6HXed+wznH/b hnKN93nA+7zq3GfdAqAqCNVNvdzHwHsuZWTiFG1IA1+4ZXiqC0TjEe0Vv5a oNwfsJm u7R242+V6VLsedV07tWun6wpYrEpfMKf2BPtFBqI5hSdzu8D7YMbj0gXFCpIPpntrQ6YdyHRuj3YRYg5x3kG4SQ hnYvZO+3gTueLXwfB50im BPVGY Rim5eA7w5mZrlRW6D4npyD1ecloAuDk/1rJEzPOB4/xXol6lv8VWGS/AasSemBD0rKMcX3ug305EV3j9SdFsCEVlEYQaojWCXzRjfvauN72rjThw X5dbOdJO7djD r6wS1/ovYr5Cfb62j1nAUyo0hZVD3k8TYsRSVox6sdYAeKEs4gpbFz6a3ftQ16CUqbts78VsLPAbvkDEFD+EJ+ KwF/mMShm C1h6lCOeyJtPOj7bUbTlOHYrMU2ARYX/umqH7oSCWQ2jAMNzoLqaiXN64PU1CRCGeEvaLT7QznXw43hs694ejFxsqDzbJrvNb7oXert95zevd7D3rbvb3eQc9furW0vfRiaX/5+bJa1st5Ab16pRz fa/1LP/1H23JAH4=</latexit>

SLIDE 10

General form of MAP estimator (hybrid)

19

Sparser

Gaussian: pU(x) =

1 √ 2πσ0 e−x2/(2σ2

0)

⇒ ΦU(x) =

1 2σ2

0 x2 + C1

Laplace: pU(x) = λ

2 e−λ|x|

⇒ ΦU(x) = λ|x| + C2

Student: pU(x) =

1 B

• r, 1

2

• ✓

1 x2 + 1 ◆r+ 1

2

⇒ ΦU(x) =

• r + 1

2

• log(1 + x2) + C3
• 4
• 2

2 4 1 2 3 4 5

Potential: ΦU(x) = − log pU(x)

sMAP = argmin 1 2ky Haugsk2

2 + σ2 X n

ΦUn([Laugs]n) !

<latexit sha1_base64="mUqG7BogT80hsHBrtyT4WIO j1s=">A V93icpVjb ty4GZ5sT7vuKds2V70hPH hbMeznk XDVIEyMabNMEmu45z2CDWZEBJlIa2JGpJjg+QmdfoZe+K3vZx+jAF+pGi5NGMk2xdAfZI/4n/+ScZlhlXenv731c+ tGPf/LTn3 8ydrPf/HLX/36 qe/eanEXEbsRSQyIV+FVLGMF+yF5jpjr0rJaB5m7LvwcMfivztiUnFRPNenJZvkNC14wiOqAZpe/VsVhAlRZloFMidPvtw15A4JqExzXpAgY4neDBJ o2pkqrEJzhz5qdlyvw9rLjpPDVHBmZmO34zJH0mgeJpTvAZqnk+rwpBgd8an1YtpYTb3Hedjs8DqVSCTaXEjkDyd6Rtr06v97eG2e8jqy8i/9Hv+2Z1+ ruHQSyiec4KHWVUqf3RdqknFZWaRxkza8FcsZJGhzRl+3gtaM7UpHIeNGQDkJgkQuKv0MRBFzkqmit1moegzKmeqW cBV6E25/r5Nak4kU516yI6oWSeUa0IDYcJOaSRTo7xQuNJIeuJ pROFwjaGsbZBdorhjRM0acFAIxIuZFurbxn WAfMWes1cxS+g80w0PAdnWrTXIfQBTrUxrblfj56NJZaG1ugSYjClFjrmeESezs3CV5SJmsjBd6FyLklPTgSXIWc6kWVtbAkODkrnFrFbNGorpnPLCovef0LKE+nc0O9nCnx485inVc8nUnR2R56KYVA8kY8+Y5IlpzONFlM2d1QlPLW3HzDJOIGkQH/ESaZBKWs54dNJ k0rNw5oVMiH0CaL EGIUT0aeIRtEprpRiGDIyaSyaRAC25gUZsCHgsqYABoTVfMOCLOGc5oh/iFifsi0ckmo7ItIYIJmKRKBbJH gsbk3v0H3+7dJ18+eVYHzhEH9XJI2jynRWxT61jIw6 X2Txj8ig3FyX1wCfwIKNwCSBmw4JCdWp/9CzvMIX5cgRplgqk7izn0VIaWJVQj93ik1AlHtiGpYp5HjLJ4sm5jHG3VCsG iahmIvrowQJaUNah4/A/ELoDgNPgIM 93tRyF1xY/lDxH2G pS WXqWKbZGyK oBWzC1zqyas3VQAmp/wAz0XQV5BVUhz sWuCkA EDAtQj7HYaVELSImWm2v rPfTWL74YjMa3BtvgLdixj2UVoLrQrVls9lGWgc39LnN/ZKxj/tcHrnzGbOFDoeCbqkmhb0wNed1CXnvI0xby1EP2Wsieh+y0kB0P+bqFfO2tD3gBRjtbakS1Z0yDeN1BvD5H7HQ O9bijpMiVJZz0Fl/dHZJdzxGXKgkyEOGUuzKP4pq94d57e8lpHZY+ bQyxGk72EuFXqJ2R8Db2dtkCGqGSP9UTAgA/z1MUTdRAykw1zEHtf8GhSqfGgX6Y+X6Y6Y1RJk Kw5Jh9ZoSmEzGtNnSZnUKJZ+2yJNBLFgSPtj95UAV 6xSxeHNVOgTYgiWmKNrZM1SXaWnEOLIpYrZINvcyr59IEIU83LYd9ubHKAR8srtw roTFu2uVljy0pJItEsdML+SXy0rM67QVbz8uW4dPuIqI3zhYwaOmZnKtyciQi58NzIbY7uIkj1jNuLvIuGs+c 8GKaUIacgzrk/RnN/lreConNFCizpJvTO4JtpckLX19tDURKJs3eDhF5HTk4vJAV8mn8cg/UvrWmMWrUY3TDAxCo2p6ecoPFEPSk xj7uiuGlX48asOBATZJ2v20lrW3iGT0wTtzNeknNwLufgHXIOfoCckBfCleHOtEIBvrnAt6ito/OG4DL9M0vrN8ZmWSBVnQq5ZwG+l9gtBSOuVnwPcZDl/kVlR8JLfL9fAFlR2WvsJezge0XC4NyId4h JyXzvbRp9fd/gCZuRPC4 kM+jIceILk6bPXZw8fKzLDb lO1NO6zmTjY0Lo0rL8Ya6lYQ3FwsJgPfiCylQzfY63MR/kK9lHeYjNRpFpU7tcZSKU x83MBJRV17FDya j6w2HPhYeNr5+LoXZNt7SbtmvtwscC9hxi3WOwdZWSJZXaBelqXbxXyhuM9isoJnUpzWJe1vGi8QhRfL/NMTd2ani0XI5F1x3cvWSux+7ilWeaiF91/2qlflVu+ IzyPVwL4t+74/Y/D6 rVwGh8A4edIPRAvtwtB0yi8yFJybIlRF3k5qyipP0m7IuKDM2KXpAWdk6GfoM8D2FK1ENJtrYkUuTsTitLF3o8VijOG7izTAIhZnU4LYpSWyGYmGw9lPKxo/BY/Qz+cHP06zjWAcT3P12+TdS2O7TkJTZQqnLzWa+4oqaJk+Nbztaw BdMAbKG0hwItUoaV5cC1YSqZfSkUDsC+ktOKDdPhu2Zqx4x1dgJzM04wVjBs1sm PWuxE2r7+w2fIThtDNlwWSse42CnwaFnVBOuPLWMqXchTjz C 4kMJ2GMDOeow5NvbCrIBKe+jh yNdzxBmgkXVMv6XZh4xwo3+daWxoObd+dSdDREU61awtlNj7JTu03F1Is6054dqr6d7N2/evtE U5XGzU7enG/jEhuqDHsbYtuS37YvVjBVeTBFWxTBciHZNb0cqCZnlyKk8JMfoYoM6UZhFItr2P8GuxaJqWUeqOlIfVMZrZC/K5qrJ15mImpSPCT4uSnfPWBP4bn0sdXU8lEO9mK82HO4KRbozPzZW4nId4kFzl9HZb8dJwlOzf3NSgWS0hbBic+Xuwm4Te0ZgAI+RAefQhO cYQNPNrFtEqS9AMtoCO+hnG6A56Zc5IloWediyFJeVP5uZH+mw3LSACNQ2kueoL5+Yf5mRe0f81jP7tjzBw6x7mNSDT/nOY7I6vP+eFif2v0SVf+mHeVOl8odVZAd57LtR3MzszrYa19gTGyib1iHDYmF2u68vH8CmH1fjxT2vacx06v90fJN4+rLy/FwdHO4/fRP/bt3/S3kx73f9 Z7m71R78+9u72Hvd3ei17U+8+V/pXBla1rp9f+fu0f1/5Zk350xfP8t d5rv3rv7TvY3M=</latexit>

General form of MAP estimator (standard)

20

Sparser

sMAP = argmin ⇣

1 2 ky Hsk2 2 + σ2 P n ΦU([Ls]n)

⌘

Gaussian: pU(x) =

1 √ 2πσ0 e−x2/(2σ2

0)

⇒ ΦU(x) =

1 2σ2

0 x2 + C1

Laplace: pU(x) = λ

2 e−λ|x|

⇒ ΦU(x) = λ|x| + C2

Student: pU(x) =

1 B

• r, 1

2

• ✓

1 x2 + 1 ◆r+ 1

2

⇒ ΦU(x) =

• r + 1

2

• log(1 + x2) + C3
• 4
• 2

2 4 1 2 3 4 5

Potential: ΦU(x) = − log pU(x)

SLIDE 11

Proximal operator: pointwise denoiser

21

• 4
• 2

2 4 1 2 3 4 5

• 4
• 2

2 4

• 3
• 2
• 1

1 2 3

σ2ΦU(u)

linear attenuation soft-threshold shrinkage function

≈ `p relaxation for p → 0 `2 minimization `1 minimization

Maximum a posteriori (MAP) estimation

22

Auxiliary innovation variable: u = Ls

Constrained optimization formulation

sMAP = arg min

s∈RK

1 2ky Hsk2

2 + σ2 X n

ΦU

• [u]n
• !

subject to u = Ls

LA(s, u, α) = 1 2 ky Hsk2

2 + σ2 X n

ΦU([u]n) + αT (Ls u) + µ 2 kLs uk2

2

SLIDE 12

Alternating direction method of multipliers (ADMM)

23

Linear inverse problem: Nonlinear denoising:

sk+1 ← arg min

s∈RN LA(s, uk, αk)

• 4
• 2

2 4

• 3
• 2
• 1

1 2 3

Sequential minimization Proximal operator taylored to stochastic model

proxΦU (y; λ) = arg min

u

1 2|y − u|2 + λΦU(u) αk+1 = αk + µ

• Lsk+1 − uk

sk+1 =

• HT H + µLT L

−1 HT y + zk+1

with

zk+1 = LT µuk − αk uk+1 = proxΦU

• Lsk+1 + 1

µαk+1; σ2 µ

• LA(s, u, α) = 1

2 ky Hsk2

2 + σ2 X n

ΦU([u]n) + αT (Ls u) + µ 2 kLs uk2

2

!2 !1 1 2 !2 !1 1 2 !2 !1 1 2 !2 !1 1 2 !2 !1 1 2 !2 !1 1 2

• Deconvolution of fluorescence micrographs

24

Physical model of a diffraction-limited microscope

g(x, y, z) = (h3D ∗ s)(x, y, z)

3-D point spread function (PSF)

h3D(x, y, z) = I0

• pλ

x

M , y M , z M 2

• 2

pλ(x, y, z) = Z

R2 P(ω1, ω2) exp

✓ j2πz ω2

1 + ω2 2

2λf 2 ◆ exp ✓ −j2π xω1 + yω2 λf0 ◆ dω1dω2

Optical parameters

λ: wavelength (emission) M: magnification factor f0: focal length P(ω1, ω2) =

kωk<R0: pupil function

NA = n sin θ = R0/f0: numerical aperture

SLIDE 13

Deconvolution experiments

25

(a) (b) (c)

Figure 10.3 Images used in deconvolution experiments. (a) Stem cells surrounded by goblet

• cells. (b) Nerve cells growing around fibers. (c) Artery cells.

Table 10.2 Deconvolution performance of MAP estimators based on different prior distributions.

Estimation performance (SNR in dB) BSNR (dB) Gaussian Laplace Student’s Stem cells 20 14.43 13.76 11.86 30 15.92 15.77 13.15 40 18.11 18.11 13.83 Nerve cells 20 13.86 15.31 14.01 30 15.89 18.18 15.81 40 18.58 20.57 16.92 Artery cells 20 14.86 15.23 13.48 30 16.59 17.21 14.92 40 18.68 19.61 15.94

L: discrete gradient

3D deconvolution with sparsity constraints

26

Maximum intensity projections of 384×448×260 image stacks; Leica DM 5500 widefield epifluorescence microscope with a 63× oil-immersion objective;

• C. Elegans embryo labeled with Hoechst, Alexa488, Alexa568;

(Vonesch-U. IEEE Trans. Im. Proc. 2009)

SLIDE 14

Computed tomography (straight rays)

27

θ t

(b) (c)

x y θ

r Rθ { s } ( t )

= Z

R2 s(x)δ(t hx, θi)dx

Projection geometry:

x = tθ + rθ⊥ with θ = (cos θ, sin θ)

Radon transform (line integrals)

Rθ{s(x)}(t) = Z

R

s(tθ + rθ⊥)dr

sinogram Equivalent analysis functions:

ηm(x) = δ

• tm hx, θmi
• Table 10.4 Reconstruction results of X-ray computed tomography using different

estimators.

Directions Estimation performance (SNR in dB) Gaussian Laplace Student’s SL Phantom 120 16.8 17.53 18.76 SL Phantom 180 18.13 18.75 20.34 Lung 180 22.49 21.52 21.45 Lung 360 24.38 22.47 22.37

(a) (b)

Figure 10.6 Images used in X-ray tomographic reconstruction experiments. (a) The Shepp-Logan (SL) phantom. (b) Cross section of a human lung.

Computed tomography reconstruction results

28

L: discrete gradient

SLIDE 15

Cryo-electron tomography (real data)

29

High-resolution Fourier-based reconstruction Standard Fourier-based reconstruction High-resolution reconstruction with sparsity

s = s1 + s2

Hybrid Model

30

L1 = D, w1: impulsive noise L2 = D2, w2: Gaussian noise

<latexit sha1_base64="N6lLAhl2r1yH8si4BAIP/sHuSjQ=">A JxHiclVZb 9s2F a7W+zd0u1xL8TiAEORGJK7LVuxA USZCnQhyxd2iKRa1DSkcyZupSXOJ6g/bj9i/2DvW6/YIeS7eqSDC4B2QTPx+875DnkoZdxJpVt/3Xv/nv f/DhR1u9/sef PrZ59sPvnghUy18uPBTnopXHpXAWQIXi kOrzIBNPY4vPRmR8b+8hqEZGnyq1pkMI5plLCQ+VTh0GTbHbjP0mziHLrHaTbYI+4bTQPiksF84gweExZnmkt2DSRJmQTX7Vf4UYl/PVrPQPwI8T9TLSWjSQWfbO/YQ7tspNtxlp0da9nOJg+2ztwg9XUMifI5lfLKsTM1zqlQzOdQ9F0tIaP+jEaQl4svyC4OBSRMBX6JIuVoA0djKRexh8iYq ls28zgrTYzImQoi9rglVbhD+OcJZlWkPiVeqg5USkx20sCJsBXfEGo7+MiNFXoXJ32SvqUQ3A4/HE0zqfAr0HhsgQkMPfTOKZJkLshjRlfB SzVWRuzJc9fu7/V1SRlUShBJPcw5KPsZxFzGYBOX681hHKQ+KXERekTt79vDguz276GAELJY e4iA6kNY3RmPa6hk0JOmn5x6mHgKYoamitSgC+IKzSE3qnBTrP6LFrFR35x46e2G1Lj4d6I2m7UZtc xju+yHwj/H+omgZqC1J7EDMLDWeQkd6cS8wYe5v 28Huc0txlzF3OyaDySeI5FDB4O2NkJMhdGnfKHLyVWe+5YsniTp1HK526SsVfXDnjVd9oPcxvyxPH7In5WXKaZCw5TeaTHed29pbApho1kSoYK412nFlkImkESG5CK/BK2MT5/bX3FS1p8mZU1Hk389SEcslHyivgeQY+o5yYiy3l1fmvq7DkvDC/xDW3mOfl5+31seSyibhsI4 wM1bGo7bxvGbscF/WjB3aU8Y9EGoFwdswP21j4A1eVCY0ChNOAM/daywBAgtLhFUgP0R8cwLWpAJ3U8TkWZvreG06bpuerk1P26aAJWp CzpqJ1h9GYj6Ek46gfZuzHyMTlAFidx0UmzWQsw6iHkL4aYxRNQ4tLtfNvIcFJ7OiOCxJhnWFyLZ70BA+UNsfVcCPgmS E2rzEsCMOuy/bhzL6xeEVIJnD3FHByOWu4o6hW1ywITHXmwJtV FNyoOQuUIfgWzb1dwkKySDVJAEsh1k a/KalKh0ugSQNsZME6bzJhP7KGcvMdTlqEM0p1nk whCJN A+rhkdMpvg ZoDJLgRgkaCZljR8Q3itF8c3c6L0dB5NLR/Ge08+Wn5GtmyvrK+tr6xHOvAemKdWmfWheVbf1p/W/9Y/ ZOerwne7qC3r+3nPOl1Wi9P/4D+qyM7w= </latexit>

SLIDE 16

Forward model: Quasi-random sampling in Fourier domain Denser sampling at low-frequencies Recovery methods:


Fourier Sampling (1D MRI)

31

Hybrid:

min

s1,s2 ky H(s1 + s2)k2 2 + λ1kDs1k1 + λ2kD2s2k2 2

TV:

min

s

ky Hsk2

2 + λkDsk1

Wiener

min

s

ky Hsk2

2 + λkD2sk2 2

<latexit sha1_base64="9zOIb2zmz/tLYoCXePnTncwP+lw=">A MrHicrVZtb9s2EHbcvbSatzXbx30hlnhou8SwFGzZihYokDRIgLbLkjQpYrkGJdE2E+qlJ XY fVP9sf2S/Z1R0m29ZIMLjACtuV7Hj53uiOPdCJGhex2/15p3v s8y+ vP/A+Kr19Tf Plz97lSEMXfJWzdkIX/nYE YDchbS Uj7yJOsO8wcuZc7mj87IpwQcPgRE4j0vfxK BD6mIJpsHqyl+2Q0Y0UBLDlKTXNxb/Y4Z5oh LjP2pw6n3FP2E1m2PiojhqZBTRpDt02Cg7NdIDMwNlP5aCbI/wtMUbWrDPnqUwejnH 8M+EBZyXtLmxjE6mGA0 m7Gc Ehl ArRyFKZnGTGEd2XbPIpO+cXJaD28RXTUmb sljGIQOoT1mfoZJQHh/5uD/D3sj+ tgbVu2CTw5gmf/4NyDB6udTvd KD6g5k/rDXycThYf CP7YVu7JNAugwL0TO7kewrzCV1QdCwY0Ei7F7iEVHp6klQG0weGoYcPoFEqbXEw74QU98Bpo/lWFQxb wV0xYuhqLs1XH8xChaerEc/tZXNIhiSQI3C2gYMyRDpJcs8ignrmRThF0X3ivGEuItKZyYfaWD19NLgHAxI97z u9WX40JuyIScA61vHZD38eQaHuIfcqmHhnimMlE2WI4ezbaRhulW0g oCKoDyNSPAW7DRzYcWmulB+PQuYlio+cRJkb3c72LxvdpMbhZJpzuh0gZB+gFYNxWEwyNxBJOU6GHdjlkvgUoExUs2Hh8ZgRpb2S WLMHpK sna/vHIe7rLa8PqfpK3TtaS2w6CUn5ISoP+Xdl BjomIHQHLC7phopCyxwLWDnmiNrudX2FKOdOw1hlDsGFTN+JDjDlZX0yxtA90l5M7/Wwv/Mz LmkwvdvR1sxR0U3mIOnBXrAXzp6o21aLCQLpV6 pU5RmCOkNgNbM2+UrHpZ1UvS FmLupFpsOtLl1B6Q0vXl0DCWCF9nMQ8fGakuKgtHmBeFl4gVQtXlBME0GWkvOI6ISzFDuhuGLGsERS80OEr0NxyL0PocRx1VX5AG52XGeZWxA6tjBu5UwaMCWNM+L4A12TcF8E0V3KfMIVzOKNAz1X6VQz5AO9OFk7AgOWHKvoJDhVMcjOBcUc8T3dSLE16FUQKp5j56VdXanUO7VehgDh1UIY8GMse8mrc9uB RwouvsFdbBc6Nng+l87IKohvQaZcpkwplUlujVQavMa4rD v0yQjXYnYuK7zLevOrMFhNZAI5kZVpECGv5tWDk5PKaZLl1suOJ/t0zSw5UNnm Y02uoINDudfxj6O3JQ/y3BObqN0x+Sk15nkcIH6z2CdTDL0jwjgVID7KSMzEzI3krRs2nhxMTdezI0HgUd lS9lX5lz4JCHTmYXLleHM+W9EKb 19QjYyxVyp PeTmJshlhNFNUL6EDBCGjPpVwcTHam+lAx0RCSx4h6OUogpsFEvSGICLdDgzDFgRu3sFIjqHbk4kEd3AnUqbpwl2nCOo+FOhSwCUgxconxewmLyQHa 3QsSqFhIthUjg+dEMFoQTBVdGsXgzrD6dWx9zqdP+01l48y +N9xs/NH5sPGqYje3Gi8Z+47DxtuE27zUfN63mVqvTOmn1Wv2M2lzJ53zfKI3W8F+lNpHb</latexit>

Example 1

32

Number of measurements: M = 31 Number of basis functions: K=100

SLIDE 17

Example 1 - Unmixed components

33

Comparison

34

Average performance over 20 signals

SLIDE 18

Example 2

35

Number of measurements: M = 31 Number of basis functions: K=100

s = s1 + s2

L1 = D, w1: impulsive noise L2 = D2, w2: Gaussian noise

<latexit sha1_base64="N6lLAhl2r1yH8si4BAIP/sHuSjQ=">A JxHiclVZb 9s2F a7W+zd0u1xL8TiAEORGJK7LVuxA USZCnQhyxd2iKRa1DSkcyZupSXOJ6g/bj9i/2DvW6/YIeS7eqSDC4B2QTPx+875DnkoZdxJpVt/3Xv/nv f/DhR1u9/sef PrZ59sPvnghUy18uPBTnopXHpXAWQIXi kOrzIBNPY4vPRmR8b+8hqEZGnyq1pkMI5plLCQ+VTh0GTbHbjP0mziHLrHaTbYI+4bTQPiksF84gweExZnmkt2DSRJmQTX7Vf4UYl/PVrPQPwI8T9TLSWjSQWfbO/YQ7tspNtxlp0da9nOJg+2ztwg9XUMifI5lfLKsTM1zqlQzOdQ9F0tIaP+jEaQl4svyC4OBSRMBX6JIuVoA0djKRexh8iYq ls28zgrTYzImQoi9rglVbhD+OcJZlWkPiVeqg5USkx20sCJsBXfEGo7+MiNFXoXJ32SvqUQ3A4/HE0zqfAr0HhsgQkMPfTOKZJkLshjRlfB SzVWRuzJc9fu7/V1SRlUShBJPcw5KPsZxFzGYBOX681hHKQ+KXERekTt79vDguz276GAELJY e4iA6kNY3RmPa6hk0JOmn5x6mHgKYoamitSgC+IKzSE3qnBTrP6LFrFR35x46e2G1Lj4d6I2m7UZtc xju+yHwj/H+omgZqC1J7EDMLDWeQkd6cS8wYe5v 28Huc0txlzF3OyaDySeI5FDB4O2NkJMhdGnfKHLyVWe+5YsniTp1HK526SsVfXDnjVd9oPcxvyxPH7In5WXKaZCw5TeaTHed29pbApho1kSoYK412nFlkImkESG5CK/BK2MT5/bX3FS1p8mZU1Hk389SEcslHyivgeQY+o5yYiy3l1fmvq7DkvDC/xDW3mOfl5+31seSyibhsI4 wM1bGo7bxvGbscF/WjB3aU8Y9EGoFwdswP21j4A1eVCY0ChNOAM/daywBAgtLhFUgP0R8cwLWpAJ3U8TkWZvreG06bpuerk1P26aAJWp CzpqJ1h9GYj6Ek46gfZuzHyMTlAFidx0UmzWQsw6iHkL4aYxRNQ4tLtfNvIcFJ7OiOCxJhnWFyLZ70BA+UNsfVcCPgmS E2rzEsCMOuy/bhzL6xeEVIJnD3FHByOWu4o6hW1ywITHXmwJtV FNyoOQuUIfgWzb1dwkKySDVJAEsh1k a/KalKh0ugSQNsZME6bzJhP7KGcvMdTlqEM0p1nk whCJN A+rhkdMpvg ZoDJLgRgkaCZljR8Q3itF8c3c6L0dB5NLR/Ge08+Wn5GtmyvrK+tr6xHOvAemKdWmfWheVbf1p/W/9Y/ ZOerwne7qC3r+3nPOl1Wi9P/4D+qyM7w= </latexit>

Example 2 - Unmixed components

36

L1 = D, w1: impulsive noise L2 = D2, w2: Gaussian noise

<latexit sha1_base64="N6lLAhl2r1yH8si4BAIP/sHuSjQ=">A JxHiclVZb 9s2F a7W+zd0u1xL8TiAEORGJK7LVuxA USZCnQhyxd2iKRa1DSkcyZupSXOJ6g/bj9i/2DvW6/YIeS7eqSDC4B2QTPx+875DnkoZdxJpVt/3Xv/nv f/DhR1u9/sef PrZ59sPvnghUy18uPBTnopXHpXAWQIXi kOrzIBNPY4vPRmR8b+8hqEZGnyq1pkMI5plLCQ+VTh0GTbHbjP0mziHLrHaTbYI+4bTQPiksF84gweExZnmkt2DSRJmQTX7Vf4UYl/PVrPQPwI8T9TLSWjSQWfbO/YQ7tspNtxlp0da9nOJg+2ztwg9XUMifI5lfLKsTM1zqlQzOdQ9F0tIaP+jEaQl4svyC4OBSRMBX6JIuVoA0djKRexh8iYq ls28zgrTYzImQoi9rglVbhD+OcJZlWkPiVeqg5USkx20sCJsBXfEGo7+MiNFXoXJ32SvqUQ3A4/HE0zqfAr0HhsgQkMPfTOKZJkLshjRlfB SzVWRuzJc9fu7/V1SRlUShBJPcw5KPsZxFzGYBOX681hHKQ+KXERekTt79vDguz276GAELJY e4iA6kNY3RmPa6hk0JOmn5x6mHgKYoamitSgC+IKzSE3qnBTrP6LFrFR35x46e2G1Lj4d6I2m7UZtc xju+yHwj/H+omgZqC1J7EDMLDWeQkd6cS8wYe5v 28Huc0txlzF3OyaDySeI5FDB4O2NkJMhdGnfKHLyVWe+5YsniTp1HK526SsVfXDnjVd9oPcxvyxPH7In5WXKaZCw5TeaTHed29pbApho1kSoYK412nFlkImkESG5CK/BK2MT5/bX3FS1p8mZU1Hk389SEcslHyivgeQY+o5yYiy3l1fmvq7DkvDC/xDW3mOfl5+31seSyibhsI4 wM1bGo7bxvGbscF/WjB3aU8Y9EGoFwdswP21j4A1eVCY0ChNOAM/daywBAgtLhFUgP0R8cwLWpAJ3U8TkWZvreG06bpuerk1P26aAJWp CzpqJ1h9GYj6Ek46gfZuzHyMTlAFidx0UmzWQsw6iHkL4aYxRNQ4tLtfNvIcFJ7OiOCxJhnWFyLZ70BA+UNsfVcCPgmS E2rzEsCMOuy/bhzL6xeEVIJnD3FHByOWu4o6hW1ywITHXmwJtV FNyoOQuUIfgWzb1dwkKySDVJAEsh1k a/KalKh0ugSQNsZME6bzJhP7KGcvMdTlqEM0p1nk whCJN A+rhkdMpvg ZoDJLgRgkaCZljR8Q3itF8c3c6L0dB5NLR/Ge08+Wn5GtmyvrK+tr6xHOvAemKdWmfWheVbf1p/W/9Y/ ZOerwne7qC3r+3nPOl1Wi9P/4D+qyM7w= </latexit>

SLIDE 19

37

CONCLUSION

■ Unifying continuous-domain stochastic model

■ Backward compatibility with classical Gaussian theory ■ Operator-based formulation: Lévy-driven SDEs or SPDEs ■ Gaussian vs. sparse (generalized Poisson, student, SαS)

■ Regularization

■ Sparsification via “operator-like” behavior (whitening) ■ Specific family of id potential functions (typ., non-convex)

■ Conceptual framework for sparse signal recovery

■ Principled approach for the development of algorithms ■ Generalization: hybrid models


• link with sparse encoding, dictionary-based methods

■ Challenges

■ Model identification / learning (self-tuning)

38

Acknowledgments

Many thanks to (former) members of EPFL’s Biomedical Imaging Group

■ Dr. Pouya Tafti ■ Prof. Arash Amini ■ Dr. Julien Fageot ■ Dr. Emrah Bostan ■ Dr. Masih Nilchian ■ Pakshal Bohra ■ Thomas Debarre ■ ....

■ Preprints and demos: http://bigwww.epfl.ch/

■ Prof. Demetri Psaltis ■ Prof. Marco Stampanoni ■ Prof. Carlos-Oscar Sorzano ■ Dr. Arne Seitz ■ ....

and collaborators ...

2

SLIDE 20

39

References

Algorithms and imaging applications Theory of sparse stochastic processes

• M. Unser and P

. Tafti, An Introduction to Sparse Stochastic Processes, Cambridge University Press, 2014; preprint, available at http://www.sparseprocesses.org.

• M. Unser, P

.D. Tafti, ”Stochastic models for sparse and piecewise-smooth signals”, IEEE Trans. Signal Processing, vol. 59, no. 3, pp. 989-1006, March 2011.

• M. Unser, P

. Tafti, and Q. Sun, “A unified formulation of Gaussian vs. sparse stochastic pro- cesses—Part I: Continuous-domain theory,” IEEE Trans. Information Theory, vol. 60, no. 3, pp. 1945-1962, March 2014.

• E. Bostan, U.S. Kamilov, M. Nilchian, M. Unser, “Sparse Stochastic Processes and Discretization
• f Linear Inverse Problems,” IEEE Trans. Image Processing, vol. 22, no. 7, pp. 2699-2710, 2013.
• C. Vonesch, M. Unser, “A Fast Multilevel Algorithm for Wavelet-Regularized Image Restoration,”

IEEE Trans. Image Processing, vol. 18, no. 3, pp. 509-523, March 2009.

• M. Nilchian, C. Vonesch, S. Lefkimmiatis, P

. Modregger, M. Stampanoni, M. Unser, “Constrained Regularized Reconstruction of X-Ray-DPCI Tomograms with Weighted-Norm,” Optics Express, vol. 21, no. 26, pp. 32340-32348, 2013.

• T. Debarre, J. Fageot , H. Gupta, M. Unser, “B-Spline-Based Exact Discretization of Continuous-

Domain Inverse Problems With Generalized TV Regularization”, IEEE Trans. Information Theory, in press.

<latexit sha1_base64="E3+AMIJRWyObcdxsJyIwdjN/L+8=">A RonicpVhtb9vIEdZd3 zq27n92C+DS4wUB5EgJb/EAQKc45fEOTvxWXYSJDLOS3Ikb 3k8naXlnUE87P6Xwr0a/s7OktKsijFBwclEJniDme Z2fm2VGCVHBtPO9fX3z5q1/ 5re/W/mq+fs/ PFPf/569S9vtMxUiOehF K9C5hGwRM8N9wIfJcqZHEg8G1wtWvX316j0lwmZ2ac4kXMBgnv85AZevTj6sr XoADnuQ2VpH3So95nA0UjguAnsoE5p67tYE3xfRv0cx7RqYa06ce3kBPYN/ETJEX+h5DjxuMZ4uKD4az1TAum vlOhy7cJ5oVC04c+GZyFpweXmgWGh MQHd1DLSwJI 3rJrFGh06xvIe/Rq93DnGE7xmuOoaMG1FC6st1uQSBf8FqQpfe04m1stOGYqHELb8zy3WUXdp1hSG5a04NztuvA9i7mQ1y0L5xUX4ZDbpVtsl5fdlCmN0DUyHDJteAgnSoao9QTcHtehQsN/LjcUZB+OCDhTcJjYfUdrTsmIdevRowr+4f7+PpwplmgXDikdOPXIk8GUT3vCZ6vi097c3nbaW7 XIjZ+Z8pm14U3knYpHNYx78ABQYXjTBgukDYPdsRAKm6GMfSlm 6oc4qDTD Ff8ZoguQUtZGqpPJZeP3HFd5OhXfD23Y2 p25DGy7zUnm16 GTPBIXnubn /ofvx+8rXczWP3Y8Wil+AoEDK8KstCxrTP/T4qTAyn6tj/KSsxajASzB hl+6NysJpDl7haMbyiF8hwDNqEpuDW8cludy Kyp6VfFpIBd PrBVuEx0Y70iul4Sba4B+O2NTcfx27bgdlLFhaW76U7Z paOt+X5M24l0TP3Y1n08zy7KPpOl1NFUk7M+AnkJwVThjAWjuPk3WKuJ16nSGmSqiyr/5fTxjwnYtTZaBOjzmZnjtHWlBFQ25y mCW8BPLS3XHhgByKqvZOmGJCUMkdn85qKpzPzrkNDTvZIKZkUuEdsQEBHVDjwTGaoYxmfT5feMcYkWaJ0uUc9I5Xq7vN7XVny9tsWfcZdQA1iz9tl qHz3cO8TnC/hWPY05FpVtwQuFkpHAwsJTova5hc oSmfB6n1V xygpEcz30gJjqsd3zikbO3snu4dwJmNJjGMNI2pIeItWIDFyXk Vz3rudUpKo2H/hsRc65kq+BXb9mZFt9PurHuO/Xw8FYZJjuridui+cuGEpSySqcyE1JbGC+I1lAp ByiDz+W4dYe 7NG7molyay4vj0jbkjKDa oko+6mFizhUnYm3NLhuM6EzRhUBCb4N/wtkgl69jJL0BLYmBGwmJ7RIUeNrqIWlPivuRCkS/MiToj3M VM2aLCR U8Kc+TsmwqMXYmYlwDv8cH3JDNCzr6SvCk8fUMlj2GcTrMfzkvkyrsrBOvznqHvh5goIjBGHy/Lt1nFnjAlCLQL6mBqFGkgRZQVp5nqWF1Ks+c6jx0rIpF J54LZ8 TapHw5NMZtrZkzHV5dIh1Hxra+45JqQeoqzVsze3pVs6evSo9QndT+jciKtAZ0OUakzcyX+5DcQJk6iaIJo/fv3Ac73yguUbf3LzoDG5TmgEedKLZJhZOQgF0/qD76XmIifl46HAotnLaJhg4RVtUV6OJwWsZXYb7FHWJ8pQPq3ZUXvpcRyQJYEe6sU1+/BTax8y0398kfMkzQwmYRWon4nymKHpCSKuMDRiDCwMCS+dQoSj5kFTuWH01N1uX+RDFNdIe9KjcwtHoYxJNaO817d9OY6wz+iAp lL96f3zTU6UcqhrVL5IBN28HlCz3tkQxUwm8+kiIpcDYIi91t2Lmt5xZKNneEqG8 lg+ofmc2DCUSGVRhCUscpWEBzJVUrnw2G1rq4Yypc Gyj39/xdOK8n2si/1mu7Wbdz3UgKI+fsx9k/gu 6w5oUtFZoLEUlSKnNhtq hv8Nnc8d5Neqe8ylakQ8LDCpK0S4sPbN9o2BNwV484wW7dhZntOojG+M05nGmc+SuW/+OBfTO9trG/zT9WJb/fEfkx82mIsfdrKhwf+p70vBLhvjLkgVTKmMRbzbEUfYxsAcptaRcpwH/DODH3lFup+Uzu63fq9H1Kbyok/KCWgm2Jo51yrYVJU/T8fhSenhf2EnpWwIMhPF/nx5H3d4v2ixS5VxnRxd3HxdG5xyf 7ucUlty+4CFCZqQmpYf5i0QZ/IqGyqTFUcApF3rsmtVc0mQ1I8POnZF9/4UimBe2miuFo0dfebGlvcelwtnS4uBTx EzWoqVoBzR3cFTzFA6WEh3c2PcpO1GVJLhZKrGrBYurJYvRgkVPxjhgFtCaU17QRWPstGV/5KR2mtZ0agOa0KWr2dNIv/iTgRlWlZdEaHmVv7MXdGH6nwQ01NDbQ6pBt70Ax7CgmBMLKnTyU9SCGLwxIx4Z+/46rX61BrwPY5lBgnQS0jHJon/YydviLQ3t9DsiYHJEo10dr 7iqVXLds3RiNGJTo4oQ0pGWUiUCY/dgwDNCJG DvqJUQ5q2k4b/uJs Xz pu3S5OX90H7w3fpk7lhp/K3xTePvDb+x1fiu8aJx0jhvhCv/XPn3yn9W/rv6cPXl6g+r3cr0y 8m7/y1UbtWe/8Dc6MdQg= </latexit>

Ground truth

Input Image (GT) Convolved and noisy data

SLIDE 21

Hybrid

SNR = 27.96 dB

arg min

c1,c2∈RN kH(c1 + c2) yk2 2 + λ

⇣ αkrc1k1 + (1 α)k(∆ + βI)c2k2

2

⌘

<latexit sha1_base64="4GTzuT+gynVu/d0/YkA9L8j7NT4=">A Hc3icjVTZTt AFL2hSyjdoH3kZUSKlJRFcVqplfqCKA9UKkuhBAQGa+xMgsGb IcKWf6 fkMf+gftB/S9 15PCFlYPEp85845Z+4yHjvy3CSt13+XJh48fPS4P lk6umz5y9eTs+8aiZhN3bUnhN6YXxgy0R5bqD2Ujf1 E UK+nbntq3z /T+v6FihM3DL6nl5E69mUncNu I1N0WdO/TBl3fDewMrOZObl LIrCaAjTDYS5c5Jt5jn6VJwKcyNbz0V I8WCRtaEWCLWpcZ jZOGWBCmh2G0pDBX3U5VmNKLTmVfyAyk7clc9MQKJoqKqrFUgGsaXSX4mvJShKOsrVJ Cl/yWsEWVqO/L+1Ws6Yr9eU6P2LUMLR Af1shzMTP8GEFoTgQBd8UB AirYHEhIcR2BAHSL0HUOGvhgtl9cV5DCF3C6iFCIkes/xv4OzI+0NcE6aCbMd3MXDX4xMAfPICREXo027CV7vsjJ5b9LOWJNiu8S3rbV89KZwit67eD3kfXmUSwpt+Mg5uJhTxB7KztEqXa4KRS6uZ WiQoQ+slu4HqPtMLNXZ8GchHOn2kpe/8NI8tLc0dgu/L01OxuHr3uyxjt5rKlgQ+e4paOhGX hLbJMxnSQSZnluscd9C9e+UgxQOYP7pDPNQswI2LvM e8qiNFkbH3dt7hWN7hnbzNsbzNO3kJV5iq3YJV2NUqNs6pSxbPqD4ZfMW1Raz96T0 FVfL4fNT9EMNxVfsm EP8ht3qXIda3q3Fmq08d2ECn57mY6TvpzW1ZkPWZPWcx5iZMzj70KfN+p1MqC9izE41/SHox1UJq1+foNKGwNRtsdyC/WY741hPlVjNI4YrXFK4dAJnkJvX4v6SpwKNMZ0lzKLUFvqugz2uKeh+FsajkTpK 4jz3CMIs/GILf4BA2fWkIben8ceHMbw/f0qNFsLBv lo1v7ysrq/oOn4RZmM zZMAHWIF12IY9cEqfSrJ0Vjp/ K8 W54rvymgEyXNeQ0DT3npP9hki40=</latexit>

λ = 10, α = 0.001, β = 0.1

<latexit sha1_base64="3Ugjt8xRK98iev8OmXpUJMp+lmQ=">A Gx3icjVTbThNRFN2gVsQLoI+ TGxIxDTNDJroiwlBHzSRi2ILgRIyl9MyMJ2ZzAUlTR/8Ar/GV3 wK/wD/QvX3j0UOh0oc9LOvq1 9uXMceLATzPT/DM1fePmrcrtmTuzd+/dfzA3v/CwmUZ54q G wVRsuPYqQr8UDUyPwvUTpwou+sEats5fsP+7ROVpH4Ufs5OY7XftTuh3/ZdO4PpYH6pFSDYs43XhmXWjJYdxIesmHXTtKA7Khuo1sF8lY38GO CpYUq6WczWpj+TS3yKCKXcuqSopAy AHZlGLtkU mxbDtUw+2BJIvfkV9mgU2R5RChA3rMf470Pa0NYTOnKmgXewS4JcAadAiMBHiEsi8myH+XJjZehl3Tzg5t1O8Hc3VhTWjQ1gn4c4ir4vjWjJq0yupwUdNsVi4Olez5NIVzty4UFUGh g2lj34E8iuIM/6bAgmldq5t7b4/0okW1l3dWxO/6 szsHq6pm8lZ0C4VS0pmvc0NmwxlN4BlRLYjpAcmV9PeMO7LWhjRlDIL/IhLrSsxAVMfqTYFrDPnIWPbFejdstxe1OxK2X4tYn4lLpMHfbo1Xa0iwOdJ7SgWjcnx59gK+G3h9eg1NJt1w5P4N5qEJ+g1 7mEH/0l2eSh+X9G4eONp4N6mKb6+n8+Qvx ue+Ug42d+XZYytRfxO9HnjWacj3FvIwb3AX8x2lJm5zusbZVobybJdih2wJ3JvFPHcjfE8EkhlTFHhBM/Ces7Fc2VMlZ LpsuVxeC2dV9GZ3zGoeRbKmaidBYXI4+wxiOPSiI35AQVTy1HW3p/LNzcVvGeHheay3Xred36+K 6sqrv8Bl6TE9whix6S v0j apgeq+0w/6Sb8q7ytR5aTydRA6PaUxj2jkqXz7D5DiUTc=</latexit>

s1 (Total variation)

<latexit sha1_base64="sq1Ih dKFx7k4fInB+v8nuvDklU=">A LK3iclVZLbxs3EN6kr2T7ctpjeyAqC0iDRNAqaN0WDRDUiWEDSer6lcBeweDujiTa3EdIri2Z2Et/TS89tH+lpxa9 jf02uHuSt6HXSgEJBDzf xmdoYc0ks4k6rf/ PGzbfefufd927dt / 4MOP l658 mBjFPhw74f81i8 qgEziLYV0x eJUIoKH 4aV3um7wl2cgJIujPTVLYBjSc RGzKcKTc rn6+6z4k81k62Su7uxYpyckYFy9Evj1c6/V4/H6Q9c pJxyrH9vGd2/+6QeynIUTK51TKI6efqKGmQjGfQ2a7qYSE+qd0D oP iNdNAVkFAv8RYrk1hqPhlLOQg+ZIVUT2cSM8UrMWIQcybpXzwszu2o5StXom6FmUZIqiPwioFHKiYqJyRgJmABf8Rmhvo/flVKF8dYU9pyhNsGb5TVA+pRD8Kj37WCoJ8DPQCEuI JzPw5DGgXaHdGQ8VkAI5pylWlXjuZzu2t3SV5BSZBKvJRzUPI7tLvIwYLnudJhOo5 kGkx9jLt3O/31r6 389aHAGzktPvIaH4Ia0ajMdTKNxgJPU4OfVwkykIGUKFqGFnxBUpB2 8wjSz5 OsoWzcL69chrusNn7+G2mbdC2p7XEs5ZukBOn/p1 XUBOQqSdxe+FxyzTR7kTi3oF7+kG/9zUuqWca9zrnZNUuopKvUypg9XLJwPg 1zm51s/apZ9F3hWLZtc7ejh3VHVTOMiO8Cy4l87u6at2i4MC+V+paVKUZ4iYA0A6ztXyDQ/LOql6yQuxcNIsNhubchoPRJv6CmwYS4RvsliGT+xcl9SFE2ypFeElYsVQT lRME9G3gt2E/AZNmjTDWNeNIKqFxbtZOafuKb1eZ7eaX4giw7rjM mYx13x cb4I7FbClfVgBW7IvKuCLJrjJuAdCzSnYM/VmkwOvsZ2ZwinckAK4dvGaUnhR WO8V/SjzDT16oJncZJhqkVInjW1niygJ01oawFtNaGARarEgpa3DbyPGYjqJ2y0doF3YdZj6YKiguQCdbp1yrRBmb 2aJMhWoz BsONQxjTVszeaYN32m5+DQZviUwxJ6qxDCMUzbwGeHMyNcuK3AbF9eQedJyaA10cnvnokjM84Hj/FezdxM/58wyX5C7JT0xJel5Iji7R8HvcJ9MC/TFBOBcQYc4ozA LI+RlM8aTk4XxZGHcigLm63Irh9pZANsi9gq79IXenitvxLjcPWcBTKjSOWux5Ok0KVbEyVxRP8UOEMWchUzhw8XuPsgH2QWFLXlMsJeTBF8WRLILIKD8Hg7blYAPv2isJtjtYarQHb6JtOP4+NapgqYPRaYU+AjIsfpNMX9ISiVQ2ij0Bo1CKupl evDNFQUyg +FZ3mw7A9ORj0nIe9/k+DzuMfykfjLesz6wvr uVYa9Zja9PatvYt3/rZ+sX6zfrd/tX+w/7L/rug3rxRrvnUqg37n/8AoLUY+w= </latexit>

s2 (Sobolev)

<latexit sha1_base64="k+XMAlFui3vmkQWDC8SbUwTPRw =">A LI3iclVZLbxs3EN6kr2T7iJP21gtRWUAaJIJWQeO2aICgTgwbSFLXjp3AXsHg7o4k2txHSK4tmdj/0ksP7V/preilh/6OXjvcXcn7sAuFgARivo/fzM6Q 3oJZ1L1+39fu/7e+x98+NGNm/bHn3z62a2V23f2Z wKH/b8mMfijUclcBbBnmK w5tEA 09Dq+9k3WDvz4FIVkcvVKzBIYhHUdsxHyq0HS08sWq+4LI z3IVsnd3diLOZx+fbTS6f 6+SDtiVNO lY5to9u3/zXDWI/DSFSPqdSHjr9RA01FYr5HDLbTSUk1D+hY9B50Bnpoikgo1jgL1Ikt9Z4NJRyFnrIDKmayCZmjJdixiLkSNa9el6Y2VXLYapG3w41i5JUQeQXAY1STlRMTKZIwAT4is8I9X38rpQqjLem8MoZahO8WV4DpE85BI973w2GegL8FBTiAiI48+MwpFGg3RENGZ8FMKIpV5l25Wg+t7t2l+SVkwSpxEs5ByW/R7uLHCx0nisdpuOYB5kWYy/Tzv1+b+2b+/2sxREwKzn9HhK H9KqwXg8hcINRlKPk1MPN5eCkCFUiBp2RlyRctDGK0wzez7JGsrG/fLKZbjLauPnv5O2SdeS2h7HUr5LSpD+f9p1BTUBmXoStxcewkwT7U4k7h24px/0e49wST3TuNc5J6t2EZV8m1IBqxdLBsYHucrJlX7WLvws8q5YNLva0cO5o6qbwkF2iGfBvXB2T1+2WxwUyP9KTZOiPEPEHADScS6Xb3hY1knVS16IhZNmsdnYlN 4INrUV2D WCJ8k8UyfGLnuqQunFBRFV4iVgzVlBMF82TkvWA3AZ9RTkw3jHnRCKpeWLSTmX/imtbneXqn+YEsOqgzDpqMd wdc3C9Ce5UwJb2Q Vsyb6sgC+b4CbjHg 1p2DP1JtNDrzFdmYKp3BDCuDaPcVLRTAajfFe0Y8z09SrC57HSYapFiF53tR6uoCeNqGtBbTVhAIWqRILWt428B5mIKqfsNHaBd65WY+lC4oKknPU6dYp0wZl2tqjTYZoMc4aD cOYUxbMXsnDd5Ju/k1GLwlMsWcqMYyjFA08xrgzcnULCtyGxTXk7vfcWoOdHF45qNLTvGA4/1XsHcTP+fPM1ySuyQ/MSXpRSE5ukD H3CfTAv0pwThXECEOaMwAy MkJfNGI+PF8bjhXErCpivy60camcBbIvYK+zSF3p7r wR43L3jAUwoUrnrMWSZ9OkWBEnc0X9D tAFHMWMoUPF7v7IB9kFxS25DHBXk4SfFkQyc6BgPJ7OGxXAj74orGaYLeHqUJ3+CbSjuPjW6cKmj4UmVLgIyDH6jfF/AEplUBpo9AbNAqpqJdVrg/TUFEoI/hUdJoPw/Zkf9BzHvb6Pw86T34sH403rC+tr6y7lmOtWU+sTWvb2rN869z6xfrN+t3+1f7D/tP+q6Bev1au+dyqDfuf/wAFkBWV</latexit>

L2

SNR = 27.87 dB

arg min

c∈RN

kHc yk2

2 + λk∆ck2 2

<latexit sha1_base64="7iaOiGC/OqtvcVhm7dTPbpWiROg=">A HB3icjVRLTxNRFD6gVsQHoEs3ExsSH0jaxkSXBFmwsAVBCoGBZmZ6WwbmlZkphkz6A/w17oxb4w9w7T/QH+BW/c7pbaHT8pibds4953zfed25duS5SVoq/ZyYvH zVuH21J3pu/fuP5iZnXtYT8JO7KgtJ/TCeMe2EuW5gdpK3dRTO1GsLN/21LZ9/Jbt2ycqTtw +JCeRmrft9qB23IdK4WqMeuaVtz23aCRmfXM6RqmGxjmxkFW60Kuqzg1zGq2yjJbX/L7VFsalYOK8cIwPURrWmfe5oryUqsPGbg2ZoulxZI8xqhQ1kKR9LMezk1+J5OaFJ DHfJ U ApZI8sSrD2qEwliqDbpwy6GJIrdkVdmga2Ay8FDwvaY/y3sdvT2gB75kwE7SCKh18MpEHzwITwiyFzNEPsHWFm7UXcmXBybqd425rLhzalQ2ivwvU9r4vjWlJq0RupwUVNkWi4OkezdKQrnLlxrqoUDBF0LDdhjyE7guz32RBMIrVzby2x/xJP1vLe0b4d+n1pdTaWr2eyIpE84VRU1TWu6Wx4x1N4DpQpPm0gubKun Eb+oWBjhkDID/KhHzpWYCKGL0hGHPQR84iE+3luN2xuN0rcbWxuNqVuEQ6zN1u0jJtahYbe5 SQ3bcn4zewbaA3h9eg1NJtxw5P715qFx+vagZ tC9M pT6eMzHa0JjhbedSri28t0nvzlNAdnPhROtndlGSNrHr8Tfd541skQ9yZycM7x57MdZmaus/qGmapDWb GYnvs dwbeTx3YzSPGNI4pjB3gqehPePiuTKmSJUx0+XKInBbui/DM+5zKPmW8pkoncV5zyOsUc+jMZ5rcoLyp5a9yzo+Fm7ucv6eHhXqlcUy5PevikvL+g6fosf0BGeoTK9piVZpnbZQ3Q/6Q3/pX+FT4XPhS+Frz3VyQmMe0dBT+PYfodNtig= </latexit> <latexit sha1_base64="7iaOiGC/OqtvcVhm7dTPbpWiROg=">A HB3icjVRLTxNRFD6gVsQHoEs3ExsSH0jaxkSXBFmwsAVBCoGBZmZ6WwbmlZkphkz6A/w17oxb4w9w7T/QH+BW/c7pbaHT8pibds4953zfed25duS5SVoq/ZyYvH zVuH21J3pu/fuP5iZnXtYT8JO7KgtJ/TCeMe2EuW5gdpK3dRTO1GsLN/21LZ9/Jbt2ycqTtw +JCeRmrft9qB23IdK4WqMeuaVtz23aCRmfXM6RqmGxjmxkFW60Kuqzg1zGq2yjJbX/L7VFsalYOK8cIwPURrWmfe5oryUqsPGbg2ZoulxZI8xqhQ1kKR9LMezk1+J5OaFJ DHfJ U ApZI8sSrD2qEwliqDbpwy6GJIrdkVdmga2Ay8FDwvaY/y3sdvT2gB75kwE7SCKh18MpEHzwITwiyFzNEPsHWFm7UXcmXBybqd425rLhzalQ2ivwvU9r4vjWlJq0RupwUVNkWi4OkezdKQrnLlxrqoUDBF0LDdhjyE7guz32RBMIrVzby2x/xJP1vLe0b4d+n1pdTaWr2eyIpE84VRU1TWu6Wx4x1N4DpQpPm0gubKun Eb+oWBjhkDID/KhHzpWYCKGL0hGHPQR84iE+3luN2xuN0rcbWxuNqVuEQ6zN1u0jJtahYbe5 SQ3bcn4zewbaA3h9eg1NJtxw5P715qFx+vagZ tC9M pT6eMzHa0JjhbedSri28t0nvzlNAdnPhROtndlGSNrHr8Tfd541skQ9yZycM7x57MdZmaus/qGmapDWb GYnvs dwbeTx3YzSPGNI4pjB3gqehPePiuTKmSJUx0+XKInBbui/DM+5zKPmW8pkoncV5zyOsUc+jMZ5rcoLyp5a9yzo+Fm7ucv6eHhXqlcUy5PevikvL+g6fosf0BGeoTK9piVZpnbZQ3Q/6Q3/pX+FT4XPhS+Frz3VyQmMe0dBT+PYfodNtig= </latexit> <latexit sha1_base64="7iaOiGC/OqtvcVhm7dTPbpWiROg=">A HB3icjVRLTxNRFD6gVsQHoEs3ExsSH0jaxkSXBFmwsAVBCoGBZmZ6WwbmlZkphkz6A/w17oxb4w9w7T/QH+BW/c7pbaHT8pibds4953zfed25duS5SVoq/ZyYvH zVuH21J3pu/fuP5iZnXtYT8JO7KgtJ/TCeMe2EuW5gdpK3dRTO1GsLN/21LZ9/Jbt2ycqTtw +JCeRmrft9qB23IdK4WqMeuaVtz23aCRmfXM6RqmGxjmxkFW60Kuqzg1zGq2yjJbX/L7VFsalYOK8cIwPURrWmfe5oryUqsPGbg2ZoulxZI8xqhQ1kKR9LMezk1+J5OaFJ DHfJ U ApZI8sSrD2qEwliqDbpwy6GJIrdkVdmga2Ay8FDwvaY/y3sdvT2gB75kwE7SCKh18MpEHzwITwiyFzNEPsHWFm7UXcmXBybqd425rLhzalQ2ivwvU9r4vjWlJq0RupwUVNkWi4OkezdKQrnLlxrqoUDBF0LDdhjyE7guz32RBMIrVzby2x/xJP1vLe0b4d+n1pdTaWr2eyIpE84VRU1TWu6Wx4x1N4DpQpPm0gubKun Eb+oWBjhkDID/KhHzpWYCKGL0hGHPQR84iE+3luN2xuN0rcbWxuNqVuEQ6zN1u0jJtahYbe5 SQ3bcn4zewbaA3h9eg1NJtxw5P715qFx+vagZ tC9M pT6eMzHa0JjhbedSri28t0nvzlNAdnPhROtndlGSNrHr8Tfd541skQ9yZycM7x57MdZmaus/qGmapDWb GYnvs dwbeTx3YzSPGNI4pjB3gqehPePiuTKmSJUx0+XKInBbui/DM+5zKPmW8pkoncV5zyOsUc+jMZ5rcoLyp5a9yzo+Fm7ucv6eHhXqlcUy5PevikvL+g6fosf0BGeoTK9piVZpnbZQ3Q/6Q3/pX+FT4XPhS+Frz3VyQmMe0dBT+PYfodNtig= </latexit> <latexit sha1_base64="7iaOiGC/OqtvcVhm7dTPbpWiROg=">A HB3icjVRLTxNRFD6gVsQHoEs3ExsSH0jaxkSXBFmwsAVBCoGBZmZ6WwbmlZkphkz6A/w17oxb4w9w7T/QH+BW/c7pbaHT8pibds4953zfed25duS5SVoq/ZyYvH zVuH21J3pu/fuP5iZnXtYT8JO7KgtJ/TCeMe2EuW5gdpK3dRTO1GsLN/21LZ9/Jbt2ycqTtw +JCeRmrft9qB23IdK4WqMeuaVtz23aCRmfXM6RqmGxjmxkFW60Kuqzg1zGq2yjJbX/L7VFsalYOK8cIwPURrWmfe5oryUqsPGbg2ZoulxZI8xqhQ1kKR9LMezk1+J5OaFJ DHfJ U ApZI8sSrD2qEwliqDbpwy6GJIrdkVdmga2Ay8FDwvaY/y3sdvT2gB75kwE7SCKh18MpEHzwITwiyFzNEPsHWFm7UXcmXBybqd425rLhzalQ2ivwvU9r4vjWlJq0RupwUVNkWi4OkezdKQrnLlxrqoUDBF0LDdhjyE7guz32RBMIrVzby2x/xJP1vLe0b4d+n1pdTaWr2eyIpE84VRU1TWu6Wx4x1N4DpQpPm0gubKun Eb+oWBjhkDID/KhHzpWYCKGL0hGHPQR84iE+3luN2xuN0rcbWxuNqVuEQ6zN1u0jJtahYbe5 SQ3bcn4zewbaA3h9eg1NJtxw5P715qFx+vagZ tC9M pT6eMzHa0JjhbedSri28t0nvzlNAdnPhROtndlGSNrHr8Tfd541skQ9yZycM7x57MdZmaus/qGmapDWb GYnvs dwbeTx3YzSPGNI4pjB3gqehPePiuTKmSJUx0+XKInBbui/DM+5zKPmW8pkoncV5zyOsUc+jMZ5rcoLyp5a9yzo+Fm7ucv6eHhXqlcUy5PevikvL+g6fosf0BGeoTK9piVZpnbZQ3Q/6Q3/pX+FT4XPhS+Frz3VyQmMe0dBT+PYfodNtig= </latexit>

λ = 0.2

<latexit sha1_base64="Se4HTAjwCfjPH6xagRtOjYwPCbQ=">A GqXicjVTLbtNAFL0NE p4tbBkYxFV4lF doQEG6SqsGB X5SkUZqosp1J6taxje0UV a+gy18C1/BH8BfcO7NJG2ctKlHie+cuefMfYzHiXwvSU3z 1Lh1u07xbvL90r3Hzx89Hhl9Uk9CQexq2pu6Idxw7ET5XuBq Ve6qtGFCu7 /jqwDn9wOsHZypOvD 4mp5Hqt23e4HX9Vw7BdRu+XDt2MZ7w6xUj1bKZsWUx5g1LG2UST+74WrhN7WoQyG5NKA+KQo he2T QnGIVlkUgSsTRmwGJYn64qGVAJ3AC8FDxvoKf57mB1qNMCcNRNhu9jFxy8G06A1cEL4xbB5N0PWB6LM6FXamWhybOd4O1qrDzSlY6CLeGP m/I4l5S69E5y8JBTJAhn52qVgVSFIzcuZ VCIQLGdgfrMWxXmOM6G8J HeurS3rf8WTUZ672ndA/67NzsHo65 8lJ180VS0pXPc0dHwjLvwCqyW+PTA5MyGusc94OsTjBUDML9Lh/pSswAZMfuLcFqTOnIUmaDX85pzec2FvO25vO2FvEQqzNXu0CbtaxUHc+7Skcy4Phl9xto6an98A0 l1XLl/Iz6oXLxjXbN0IPhlbu8kDq+1Lt1oNHFu05lfHuZjpO/nM7kzIeiyetDGcbMWMPvTJ837nUypb2PGNxL+vlop5VZ6yK/a WtqSi7c7kj9VjujTyfqzEbRwxrnlKYO8EloBda3FfmlKk6p7ucWQRtW9dlusdjDSXfUj4SpaO47HmCMet5MsdzR05Q/tSyt6X3x8DNbeXv6VmjXq1YsPfelDc29R2+TM/oOc6QRW9pgz7RLtWQ3Tf6QT/pV/F1ca/YKDZHroUlzXlKU0/R/Q8dykiw</latexit> <latexit sha1_base64="Se4HTAjwCfjPH6xagRtOjYwPCbQ=">A GqXicjVTLbtNAFL0NE p4tbBkYxFV4lF doQEG6SqsGB X5SkUZqosp1J6taxje0UV a+gy18C1/BH8BfcO7NJG2ctKlHie+cuefMfYzHiXwvSU3z 1Lh1u07xbvL90r3Hzx89Hhl9Uk9CQexq2pu6Idxw7ET5XuBq Ve6qtGFCu7 /jqwDn9wOsHZypOvD 4mp5Hqt23e4HX9Vw7BdRu+XDt2MZ7w6xUj1bKZsWUx5g1LG2UST+74WrhN7WoQyG5NKA+KQo he2T QnGIVlkUgSsTRmwGJYn64qGVAJ3AC8FDxvoKf57mB1qNMCcNRNhu9jFxy8G06A1cEL4xbB5N0PWB6LM6FXamWhybOd4O1qrDzSlY6CLeGP m/I4l5S69E5y8JBTJAhn52qVgVSFIzcuZ VCIQLGdgfrMWxXmOM6G8J HeurS3rf8WTUZ672ndA/67NzsHo65 8lJ180VS0pXPc0dHwjLvwCqyW+PTA5MyGusc94OsTjBUDML9Lh/pSswAZMfuLcFqTOnIUmaDX85pzec2FvO25vO2FvEQqzNXu0CbtaxUHc+7Skcy4Phl9xto6an98A0 l1XLl/Iz6oXLxjXbN0IPhlbu8kDq+1Lt1oNHFu05lfHuZjpO/nM7kzIeiyetDGcbMWMPvTJ837nUypb2PGNxL+vlop5VZ6yK/a WtqSi7c7kj9VjujTyfqzEbRwxrnlKYO8EloBda3FfmlKk6p7ucWQRtW9dlusdjDSXfUj4SpaO47HmCMet5MsdzR05Q/tSyt6X3x8DNbeXv6VmjXq1YsPfelDc29R2+TM/oOc6QRW9pgz7RLtWQ3Tf6QT/pV/F1ca/YKDZHroUlzXlKU0/R/Q8dykiw</latexit> <latexit sha1_base64="Se4HTAjwCfjPH6xagRtOjYwPCbQ=">A GqXicjVTLbtNAFL0NE p4tbBkYxFV4lF doQEG6SqsGB X5SkUZqosp1J6taxje0UV a+gy18C1/BH8BfcO7NJG2ctKlHie+cuefMfYzHiXwvSU3z 1Lh1u07xbvL90r3Hzx89Hhl9Uk9CQexq2pu6Idxw7ET5XuBq Ve6qtGFCu7 /jqwDn9wOsHZypOvD 4mp5Hqt23e4HX9Vw7BdRu+XDt2MZ7w6xUj1bKZsWUx5g1LG2UST+74WrhN7WoQyG5NKA+KQo he2T QnGIVlkUgSsTRmwGJYn64qGVAJ3AC8FDxvoKf57mB1qNMCcNRNhu9jFxy8G06A1cEL4xbB5N0PWB6LM6FXamWhybOd4O1qrDzSlY6CLeGP m/I4l5S69E5y8JBTJAhn52qVgVSFIzcuZ VCIQLGdgfrMWxXmOM6G8J HeurS3rf8WTUZ672ndA/67NzsHo65 8lJ180VS0pXPc0dHwjLvwCqyW+PTA5MyGusc94OsTjBUDML9Lh/pSswAZMfuLcFqTOnIUmaDX85pzec2FvO25vO2FvEQqzNXu0CbtaxUHc+7Skcy4Phl9xto6an98A0 l1XLl/Iz6oXLxjXbN0IPhlbu8kDq+1Lt1oNHFu05lfHuZjpO/nM7kzIeiyetDGcbMWMPvTJ837nUypb2PGNxL+vlop5VZ6yK/a WtqSi7c7kj9VjujTyfqzEbRwxrnlKYO8EloBda3FfmlKk6p7ucWQRtW9dlusdjDSXfUj4SpaO47HmCMet5MsdzR05Q/tSyt6X3x8DNbeXv6VmjXq1YsPfelDc29R2+TM/oOc6QRW9pgz7RLtWQ3Tf6QT/pV/F1ca/YKDZHroUlzXlKU0/R/Q8dykiw</latexit> <latexit sha1_base64="Se4HTAjwCfjPH6xagRtOjYwPCbQ=">A GqXicjVTLbtNAFL0NE p4tbBkYxFV4lF doQEG6SqsGB X5SkUZqosp1J6taxje0UV a+gy18C1/BH8BfcO7NJG2ctKlHie+cuefMfYzHiXwvSU3z 1Lh1u07xbvL90r3Hzx89Hhl9Uk9CQexq2pu6Idxw7ET5XuBq Ve6qtGFCu7 /jqwDn9wOsHZypOvD 4mp5Hqt23e4HX9Vw7BdRu+XDt2MZ7w6xUj1bKZsWUx5g1LG2UST+74WrhN7WoQyG5NKA+KQo he2T QnGIVlkUgSsTRmwGJYn64qGVAJ3AC8FDxvoKf57mB1qNMCcNRNhu9jFxy8G06A1cEL4xbB5N0PWB6LM6FXamWhybOd4O1qrDzSlY6CLeGP m/I4l5S69E5y8JBTJAhn52qVgVSFIzcuZ VCIQLGdgfrMWxXmOM6G8J HeurS3rf8WTUZ672ndA/67NzsHo65 8lJ180VS0pXPc0dHwjLvwCqyW+PTA5MyGusc94OsTjBUDML9Lh/pSswAZMfuLcFqTOnIUmaDX85pzec2FvO25vO2FvEQqzNXu0CbtaxUHc+7Skcy4Phl9xto6an98A0 l1XLl/Iz6oXLxjXbN0IPhlbu8kDq+1Lt1oNHFu05lfHuZjpO/nM7kzIeiyetDGcbMWMPvTJ837nUypb2PGNxL+vlop5VZ6yK/a WtqSi7c7kj9VjujTyfqzEbRwxrnlKYO8EloBda3FfmlKk6p7ucWQRtW9dlusdjDSXfUj4SpaO47HmCMet5MsdzR05Q/tSyt6X3x8DNbeXv6VmjXq1YsPfelDc29R2+TM/oOc6QRW9pgz7RLtWQ3Tf6QT/pV/F1ca/YKDZHroUlzXlKU0/R/Q8dykiw</latexit>

SLIDE 22

TV

SNR = 27.97 dB

arg min

c∈RN

kHc yk2

2 + λkrck1

<latexit sha1_base64="nQlripGmx6XpIK/edyf48JeYzGc=">A HBXicjVTLbtNQEJ0WC W8WliysYgq8ShVHCHBsiosuqAPWp WrdvIdm5St37Jdo qK2u+h 1iy4I1H8AfwJotSJyZ3KSNkz58lXjuzJwzr+vrxL6XZtXqz4nJa9dvlG5O3SrfvnP3 v3pmQf1NOokrtp0Iz9Kth07Vb4Xqs3My3y1HSfKDhxfbTlHb9i+dayS1IvCD9lJrPYCux16Lc+1M6ga023LTtqBFzZyq567XcPyQsNa389XupDrKskMazlfYpmtL/h9oi2N2n7NeG5YPqI17VNvK7Qd3+5Deq5mY7pSna/KY4wKphYqpJ+1aGbyO1nUpIhc6lBAikLKIPtkU4q1SyZVKYZuj3LoEkie2BV1qQxsB14KHja0R/hvY7ertSH2zJkK2kU H78ESINmgYngl0DmaIbYO8LM2vO4c+Hk3E7wdjRXAG1GB9Behut7XhXHtWTUotdSg4eaYtFwda5m6UhXOHPjTFUZG LoWG7CnkB2BdnvsyGYVGrn3tpi/yWerOW9q3079PvC6hysQM/krUTyhVPRsq5xVWfDO57CM6As8WkDyZV19Yzb0M8NdMwYAvlRJhRIz0JUxOh1wViDPnIWuWgvxu2Mxe1cilsZi1u5FJdKh7nbTVqkDc3iYM9TasiO+5PTO9jm0PuDK3Aq6ZYr56c3D1XIrxc1xwy650Z5In18q M1wdHCu04VfHu5zpO/nObgzEfCyfauLGNkzeJ3rM8bz od4t5ADu4Z/mK2w8zMdVrfMNPyUJatsdge yL3RhHP3RjNI4E0jikqnOAytKdcPFfGVKg2ZrpcWQxuW/dleMZ9DiXfUjETpbM463mINep5OMZzVU5Q8dSyt6njY+HmNov39KhQr82bkN+/rCws6jt8ih7RY5whk17RAi3RGm2iuh/0h/7Sv9Kn0ufSl9LXnuvkhMY8pKGn9O0/+3Bs+Q= </latexit> <latexit sha1_base64="nQlripGmx6XpIK/edyf48JeYzGc=">A HBXicjVTLbtNQEJ0WC W8WliysYgq8ShVHCHBsiosuqAPWp WrdvIdm5St37Jdo qK2u+h 1iy4I1H8AfwJotSJyZ3KSNkz58lXjuzJwzr+vrxL6XZtXqz4nJa9dvlG5O3SrfvnP3 v3pmQf1NOokrtp0Iz9Kth07Vb4Xqs3My3y1HSfKDhxfbTlHb9i+dayS1IvCD9lJrPYCux16Lc+1M6ga023LTtqBFzZyq567XcPyQsNa389XupDrKskMazlfYpmtL/h9oi2N2n7NeG5YPqI17VNvK7Qd3+5Deq5mY7pSna/KY4wKphYqpJ+1aGbyO1nUpIhc6lBAikLKIPtkU4q1SyZVKYZuj3LoEkie2BV1qQxsB14KHja0R/hvY7ertSH2zJkK2kU H78ESINmgYngl0DmaIbYO8LM2vO4c+Hk3E7wdjRXAG1GB9Behut7XhXHtWTUotdSg4eaYtFwda5m6UhXOHPjTFUZG LoWG7CnkB2BdnvsyGYVGrn3tpi/yWerOW9q3079PvC6hysQM/krUTyhVPRsq5xVWfDO57CM6As8WkDyZV19Yzb0M8NdMwYAvlRJhRIz0JUxOh1wViDPnIWuWgvxu2Mxe1cilsZi1u5FJdKh7nbTVqkDc3iYM9TasiO+5PTO9jm0PuDK3Aq6ZYr56c3D1XIrxc1xwy650Z5In18q M1wdHCu04VfHu5zpO/nObgzEfCyfauLGNkzeJ3rM8bz od4t5ADu4Z/mK2w8zMdVrfMNPyUJatsdge yL3RhHP3RjNI4E0jikqnOAytKdcPFfGVKg2ZrpcWQxuW/dleMZ9DiXfUjETpbM463mINep5OMZzVU5Q8dSyt6njY+HmNov39KhQr82bkN+/rCws6jt8ih7RY5whk17RAi3RGm2iuh/0h/7Sv9Kn0ufSl9LXnuvkhMY8pKGn9O0/+3Bs+Q= </latexit> <latexit sha1_base64="nQlripGmx6XpIK/edyf48JeYzGc=">A HBXicjVTLbtNQEJ0WC W8WliysYgq8ShVHCHBsiosuqAPWp WrdvIdm5St37Jdo qK2u+h 1iy4I1H8AfwJotSJyZ3KSNkz58lXjuzJwzr+vrxL6XZtXqz4nJa9dvlG5O3SrfvnP3 v3pmQf1NOokrtp0Iz9Kth07Vb4Xqs3My3y1HSfKDhxfbTlHb9i+dayS1IvCD9lJrPYCux16Lc+1M6ga023LTtqBFzZyq567XcPyQsNa389XupDrKskMazlfYpmtL/h9oi2N2n7NeG5YPqI17VNvK7Qd3+5Deq5mY7pSna/KY4wKphYqpJ+1aGbyO1nUpIhc6lBAikLKIPtkU4q1SyZVKYZuj3LoEkie2BV1qQxsB14KHja0R/hvY7ertSH2zJkK2kU H78ESINmgYngl0DmaIbYO8LM2vO4c+Hk3E7wdjRXAG1GB9Behut7XhXHtWTUotdSg4eaYtFwda5m6UhXOHPjTFUZG LoWG7CnkB2BdnvsyGYVGrn3tpi/yWerOW9q3079PvC6hysQM/krUTyhVPRsq5xVWfDO57CM6As8WkDyZV19Yzb0M8NdMwYAvlRJhRIz0JUxOh1wViDPnIWuWgvxu2Mxe1cilsZi1u5FJdKh7nbTVqkDc3iYM9TasiO+5PTO9jm0PuDK3Aq6ZYr56c3D1XIrxc1xwy650Z5In18q M1wdHCu04VfHu5zpO/nObgzEfCyfauLGNkzeJ3rM8bz od4t5ADu4Z/mK2w8zMdVrfMNPyUJatsdge yL3RhHP3RjNI4E0jikqnOAytKdcPFfGVKg2ZrpcWQxuW/dleMZ9DiXfUjETpbM463mINep5OMZzVU5Q8dSyt6njY+HmNov39KhQr82bkN+/rCws6jt8ih7RY5whk17RAi3RGm2iuh/0h/7Sv9Kn0ufSl9LXnuvkhMY8pKGn9O0/+3Bs+Q= </latexit> <latexit sha1_base64="nQlripGmx6XpIK/edyf48JeYzGc=">A HBXicjVTLbtNQEJ0WC W8WliysYgq8ShVHCHBsiosuqAPWp WrdvIdm5St37Jdo qK2u+h 1iy4I1H8AfwJotSJyZ3KSNkz58lXjuzJwzr+vrxL6XZtXqz4nJa9dvlG5O3SrfvnP3 v3pmQf1NOokrtp0Iz9Kth07Vb4Xqs3My3y1HSfKDhxfbTlHb9i+dayS1IvCD9lJrPYCux16Lc+1M6ga023LTtqBFzZyq567XcPyQsNa389XupDrKskMazlfYpmtL/h9oi2N2n7NeG5YPqI17VNvK7Qd3+5Deq5mY7pSna/KY4wKphYqpJ+1aGbyO1nUpIhc6lBAikLKIPtkU4q1SyZVKYZuj3LoEkie2BV1qQxsB14KHja0R/hvY7ertSH2zJkK2kU H78ESINmgYngl0DmaIbYO8LM2vO4c+Hk3E7wdjRXAG1GB9Behut7XhXHtWTUotdSg4eaYtFwda5m6UhXOHPjTFUZG LoWG7CnkB2BdnvsyGYVGrn3tpi/yWerOW9q3079PvC6hysQM/krUTyhVPRsq5xVWfDO57CM6As8WkDyZV19Yzb0M8NdMwYAvlRJhRIz0JUxOh1wViDPnIWuWgvxu2Mxe1cilsZi1u5FJdKh7nbTVqkDc3iYM9TasiO+5PTO9jm0PuDK3Aq6ZYr56c3D1XIrxc1xwy650Z5In18q M1wdHCu04VfHu5zpO/nObgzEfCyfauLGNkzeJ3rM8bz od4t5ADu4Z/mK2w8zMdVrfMNPyUJatsdge yL3RhHP3RjNI4E0jikqnOAytKdcPFfGVKg2ZrpcWQxuW/dleMZ9DiXfUjETpbM463mINep5OMZzVU5Q8dSyt6njY+HmNov39KhQr82bkN+/rCws6jt8ih7RY5whk17RAi3RGm2iuh/0h/7Sv9Kn0ufSl9LXnuvkhMY8pKGn9O0/+3Bs+Q= </latexit>

λ = 0.002

<latexit sha1_base64="A /7p3hgKtfz1TGmT1bS36ATv6Q=">A GrXicjVTLbtNAFL0NE p4NIUlG4uoEqCq2BESbJCqwoIFfUBJWrUple1MUreObdlOUWXlS9jCl/AV/AH8BefeTNLEcZt6lPjOmXvO3Md4nMj3ktQ0/y Ubt2+U767eK9y/8HDR0vV5cfNJOzHrmq4oR/G+46dKN8LVCP1Ul/tR7Gye46v9pyz97y+d67ixAuDr+lFpI56djfwOp5rp4COq0stH85t23hnmGumWT+u1vjNjzFrWNqokX52wuXSb2pRm0JyqU89UhRQCtsnmxKMQ7LIpAjYEWXAYlierCsaUAXcPrwUPGygZ/jvYnao0QBz1kyE7WIXH78YTINWwAnhF8Pm3QxZ74syo1dpZ6LJsV3g7WitHtCUToDO4 08b8rjXFLq0FvJwUNOkSCcnatV+lIVjtyYyCqFQgSM7TbWY9iuMEd1NoSTSO5cW1vW/4onozx3tW+f/l2bnYPR0z35IDv5oqloU+e4raPhGXfhJVgt8emCyZkNdI+7wFfHGCsGYH6XDvWkZgEyYvYX4bTGdeQoMkGv5x0U8g7m8rYKeVtzeYlUmKvdpg3a1SoO5tylY5lxfTL6hLV 1P7kBp KquXK+Rn2Q+XiG+6aoQeDK3d5LnV8oXdrQ6ODd5Nq+PYyHSd/Oe3xmQ9Fk9cHMoyZsYLfuT5v3OtkSnsXMbgT+vlop5VZ6zK/a XNqSg7hdyheiz3Rp7P1ZiNI4ZVpBTmTnAF6KUW95U5NaoXdJczi6Bt67pM93ikoeRbykeidBSTnqcYs56nBZ7bcoLyp5a9Lb0/Bm5uK39PzxrN+poF+/Pr2vqGvsMX6Sk9wxmy6A2t0 faoYbc5T/oJ/0qvyo3yq3yt6FraUFzntDU +7+B9gbSVU=</latexit> <latexit sha1_base64="A /7p3hgKtfz1TGmT1bS36ATv6Q=">A GrXicjVTLbtNAFL0NE p4NIUlG4uoEqCq2BESbJCqwoIFfUBJWrUple1MUreObdlOUWXlS9jCl/AV/AH8BefeTNLEcZt6lPjOmXvO3Md4nMj3ktQ0/y Ubt2+U767eK9y/8HDR0vV5cfNJOzHrmq4oR/G+46dKN8LVCP1Ul/tR7Gye46v9pyz97y+d67ixAuDr+lFpI56djfwOp5rp4COq0stH85t23hnmGumWT+u1vjNjzFrWNqokX52wuXSb2pRm0JyqU89UhRQCtsnmxKMQ7LIpAjYEWXAYlierCsaUAXcPrwUPGygZ/jvYnao0QBz1kyE7WIXH78YTINWwAnhF8Pm3QxZ74syo1dpZ6LJsV3g7WitHtCUToDO4 08b8rjXFLq0FvJwUNOkSCcnatV+lIVjtyYyCqFQgSM7TbWY9iuMEd1NoSTSO5cW1vW/4onozx3tW+f/l2bnYPR0z35IDv5oqloU+e4raPhGXfhJVgt8emCyZkNdI+7wFfHGCsGYH6XDvWkZgEyYvYX4bTGdeQoMkGv5x0U8g7m8rYKeVtzeYlUmKvdpg3a1SoO5tylY5lxfTL6hLV 1P7kBp KquXK+Rn2Q+XiG+6aoQeDK3d5LnV8oXdrQ6ODd5Nq+PYyHSd/Oe3xmQ9Fk9cHMoyZsYLfuT5v3OtkSnsXMbgT+vlop5VZ6zK/a XNqSg7hdyheiz3Rp7P1ZiNI4ZVpBTmTnAF6KUW95U5NaoXdJczi6Bt67pM93ikoeRbykeidBSTnqcYs56nBZ7bcoLyp5a9Lb0/Bm5uK39PzxrN+poF+/Pr2vqGvsMX6Sk9wxmy6A2t0 faoYbc5T/oJ/0qvyo3yq3yt6FraUFzntDU +7+B9gbSVU=</latexit> <latexit sha1_base64="A /7p3hgKtfz1TGmT1bS36ATv6Q=">A GrXicjVTLbtNAFL0NE p4NIUlG4uoEqCq2BESbJCqwoIFfUBJWrUple1MUreObdlOUWXlS9jCl/AV/AH8BefeTNLEcZt6lPjOmXvO3Md4nMj3ktQ0/y Ubt2+U767eK9y/8HDR0vV5cfNJOzHrmq4oR/G+46dKN8LVCP1Ul/tR7Gye46v9pyz97y+d67ixAuDr+lFpI56djfwOp5rp4COq0stH85t23hnmGumWT+u1vjNjzFrWNqokX52wuXSb2pRm0JyqU89UhRQCtsnmxKMQ7LIpAjYEWXAYlierCsaUAXcPrwUPGygZ/jvYnao0QBz1kyE7WIXH78YTINWwAnhF8Pm3QxZ74syo1dpZ6LJsV3g7WitHtCUToDO4 08b8rjXFLq0FvJwUNOkSCcnatV+lIVjtyYyCqFQgSM7TbWY9iuMEd1NoSTSO5cW1vW/4onozx3tW+f/l2bnYPR0z35IDv5oqloU+e4raPhGXfhJVgt8emCyZkNdI+7wFfHGCsGYH6XDvWkZgEyYvYX4bTGdeQoMkGv5x0U8g7m8rYKeVtzeYlUmKvdpg3a1SoO5tylY5lxfTL6hLV 1P7kBp KquXK+Rn2Q+XiG+6aoQeDK3d5LnV8oXdrQ6ODd5Nq+PYyHSd/Oe3xmQ9Fk9cHMoyZsYLfuT5v3OtkSnsXMbgT+vlop5VZ6zK/a XNqSg7hdyheiz3Rp7P1ZiNI4ZVpBTmTnAF6KUW95U5NaoXdJczi6Bt67pM93ikoeRbykeidBSTnqcYs56nBZ7bcoLyp5a9Lb0/Bm5uK39PzxrN+poF+/Pr2vqGvsMX6Sk9wxmy6A2t0 faoYbc5T/oJ/0qvyo3yq3yt6FraUFzntDU +7+B9gbSVU=</latexit> <latexit sha1_base64="A /7p3hgKtfz1TGmT1bS36ATv6Q=">A GrXicjVTLbtNAFL0NE p4NIUlG4uoEqCq2BESbJCqwoIFfUBJWrUple1MUreObdlOUWXlS9jCl/AV/AH8BefeTNLEcZt6lPjOmXvO3Md4nMj3ktQ0/y Ubt2+U767eK9y/8HDR0vV5cfNJOzHrmq4oR/G+46dKN8LVCP1Ul/tR7Gye46v9pyz97y+d67ixAuDr+lFpI56djfwOp5rp4COq0stH85t23hnmGumWT+u1vjNjzFrWNqokX52wuXSb2pRm0JyqU89UhRQCtsnmxKMQ7LIpAjYEWXAYlierCsaUAXcPrwUPGygZ/jvYnao0QBz1kyE7WIXH78YTINWwAnhF8Pm3QxZ74syo1dpZ6LJsV3g7WitHtCUToDO4 08b8rjXFLq0FvJwUNOkSCcnatV+lIVjtyYyCqFQgSM7TbWY9iuMEd1NoSTSO5cW1vW/4onozx3tW+f/l2bnYPR0z35IDv5oqloU+e4raPhGXfhJVgt8emCyZkNdI+7wFfHGCsGYH6XDvWkZgEyYvYX4bTGdeQoMkGv5x0U8g7m8rYKeVtzeYlUmKvdpg3a1SoO5tylY5lxfTL6hLV 1P7kBp KquXK+Rn2Q+XiG+6aoQeDK3d5LnV8oXdrQ6ODd5Nq+PYyHSd/Oe3xmQ9Fk9cHMoyZsYLfuT5v3OtkSnsXMbgT+vlop5VZ6zK/a XNqSg7hdyheiz3Rp7P1ZiNI4ZVpBTmTnAF6KUW95U5NaoXdJczi6Bt67pM93ikoeRbykeidBSTnqcYs56nBZ7bcoLyp5a9Lb0/Bm5uK39PzxrN+poF+/Pr2vqGvsMX6Sk9wxmy6A2t0 faoYbc5T/oJ/0qvyo3yq3yt6FraUFzntDU +7+B9gbSVU=</latexit>