Retour sur... la ligne de base BEADS : correction et filtrage - - PowerPoint PPT Presentation

▶
retour sur la ligne de base beads correction et filtrage
SMART_READER_LITE
LIVE PREVIEW

Retour sur... la ligne de base BEADS : correction et filtrage - - PowerPoint PPT Presentation

Jan 20, 2023 36 likes •537 views

I NTRODUCTION BEADS MODEL AND ALGORITHM E VALUATION AND RESULTS C ONCLUSIONS Retour sur... la ligne de base BEADS : correction et filtrage conjoints de mesures analytiques exploitant positivit e et parcimonie X. NING, I. W. SELESNICK

slide-1
SLIDE 1

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Retour sur... la ligne de base BEADS : correction et filtrage conjoints de mesures analytiques exploitant positivit´ e et parcimonie

  • X. NING, I. W. SELESNICK

Polytechnic School of Engineering, New York University

  • L. DUVAL, A. PIRAYRE

IFP Energies nouvelles, Universit´ e Paris-Est

9 octobre 2017

1 / 27

slide-2
SLIDE 2

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Old peaks cast long shadows

2 / 27

slide-3
SLIDE 3

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Old peaks cast long shadows

2 / 27

slide-4
SLIDE 4

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Old peaks cast long shadows

2 / 27

slide-5
SLIDE 5

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Old peaks cast long shadows

2 / 27

slide-6
SLIDE 6

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Old peaks cast long shadows

2 / 27

slide-7
SLIDE 7

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

The quick version

◮ Issue: how to accurately & repeatably quantize peaks?

◮ avoiding separate baseline and noise removal

◮ Question: where is the string behind the bead?

◮ without precise models for: peak, noise, baseline

◮ Answer: use main measurement properties + optimization

◮ sparsity+symmetry, stationarity, smoothness

◮ BEADS: Baseline Estimation And Denoising w/ Sparsity

3 / 27

slide-8
SLIDE 8

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Outline

INTRODUCTION FOREWORD OUTLINE* BACKGROUND BEADS MODEL AND ALGORITHM NOTATIONS COMPOUND SPARSE DERIVATIVE MODELING MAJORIZE-MINIMIZE TYPE OPTIMIZATION EVALUATION AND RESULTS GC: SIMULATED BASELINE AND GAUSSIAN NOISE GC: SIMULATED POISSON NOISE GC: REAL DATA GC×GC: REAL DATA OTHERS CONCLUSIONS

4 / 27

slide-9
SLIDE 9

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Background on background

Figure: Image processing: varying illumination

◮ Background affects quantitative evaluation/comparison ◮ In other domains: (instrumental) bias, (seasonal) trend ◮ In analytical chemistry: drift, continuum, wander, baseline ◮ Very rare cases of parametric modeling (piecewise linear,

polynomial, spline)

5 / 27

slide-10
SLIDE 10

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Background on background

Figure: Econometrics: trends and seasonality

◮ Background affects quantitative evaluation/comparison ◮ In other domains: (instrumental) bias, (seasonal) trend ◮ In analytical chemistry: drift, continuum, wander, baseline ◮ Very rare cases of parametric modeling (piecewise linear,

polynomial, spline)

5 / 27

slide-11
SLIDE 11

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Background on background

Figure: Biomedical: ECG isoelectric line or baseline wander

◮ Background affects quantitative evaluation/comparison ◮ In other domains: (instrumental) bias, (seasonal) trend ◮ In analytical chemistry: drift, continuum, wander, baseline ◮ Very rare cases of parametric modeling (piecewise linear,

polynomial, spline)

5 / 27

slide-12
SLIDE 12

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Background on background

Figure: Gas chromatography: baseline

◮ Background affects quantitative evaluation/comparison ◮ In other domains: (instrumental) bias, (seasonal) trend ◮ In analytical chemistry: drift, continuum, wander, baseline ◮ Very rare cases of parametric modeling (piecewise linear,

polynomial, spline)

5 / 27

slide-13
SLIDE 13

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Background on background

Analytical chemistry, biological data

◮ Signal separation into three main morphological

components

6 / 27

slide-14
SLIDE 14

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Notations and assumptions

Morphological decomposition: y = x + f + w, signals in RN

◮ y: observation (spectrum, analytical data) ◮ x: clean series of peaks (no baseline, no noise) ◮ f: baseline ◮ w: noise

Assumption: without peaks, the baseline can be (approx.) recovered from noise-corrupted data by low-pass filtering

◮ ˆ

f = L(y − ˆ x): L: low-pass filter; H = I − L: high-pass filter

◮ formulated as y − ˆ

x − ˆ f2

2 = H(y − ˆ

x)2

2 ◮ Going further with Di: differentiation operators

7 / 27

slide-15
SLIDE 15

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

An estimate ˆ x can be obtained via: ˆ x = arg min

x

  • F(x) = 1

2H(y − x)2

2 + M

  • i=0

λiRi (Dix)

  • .

8 / 27

slide-16
SLIDE 16

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

Examples of (smooth) sparsity promoting functions for Ri

◮ φA i = |x| ◮ φB i =

  • |x|2 + ǫ

◮ φC i = |x| − ǫ log (|x| + ǫ)

8 / 27

slide-17
SLIDE 17

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

Take the positivity of chromatogram peaks into account: ˆ x = arg min

x

  • F(x) = 1

2H(y − x)2

2

+ λ0

N−1

  • n=0

θǫ(xn; r) +

M

  • i=1

λi

Ni−1

  • n=0

φ ([Dix]n)

  • .

Start from: θ(x; r) =

  • x,

x 0 −rx, x < 0

9 / 27

slide-18
SLIDE 18

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

Take the positivity of chromatogram peaks into account: ˆ x = arg min

x

  • F(x) = 1

2H(y − x)2

2

+ λ0

N−1

  • n=0

θǫ(xn; r) +

M

  • i=1

λi

Ni−1

  • n=0

φ ([Dix]n)

  • .

and majorize it

−5 5 2 4 6 8 10 The majorizer g(x, v) for the penalty function θ(x; r), r = 3 x (s, θr(s)) (v, θr(v)) g(x,v) θr(x)

9 / 27

slide-19
SLIDE 19

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

Take the positivity of chromatogram peaks into account: ˆ x = arg min

x

  • F(x) = 1

2H(y − x)2

2

+ λ0

N−1

  • n=0

θǫ(xn; r) +

M

  • i=1

λi

Ni−1

  • n=0

φ ([Dix]n)

  • .

then smooth it:

−5 5 2 4 6 8 10 The smoothed asymmetric penalty function θε(x; r), r = 3 (−ε, f(−ε)) (ε, f(ε)) x

9 / 27

slide-20
SLIDE 20

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Compound sparse derivative modeling

Take the positivity of chromatogram peaks into account: ˆ x = arg min

x

  • F(x) = 1

2H(y − x)2

2

+ λ0

N−1

  • n=0

θǫ(xn; r) +

M

  • i=1

λi

Ni−1

  • n=0

φ ([Dix]n)

  • .

then majorize it: g0(x, v) =   

1+r 4|v| x2 + 1−r 2 x + |v| 1+r 4 ,

|v| > ǫ

1+r 4ǫ x2 + 1−r 2 x + ǫ 1+r 4 ,

|v| ǫ.

9 / 27

slide-21
SLIDE 21

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Overall principle for Majoration-Minimization-Maximization

Figure: Courtesy Peng Wang1

1https://commons.wikimedia.org/w/index.php?curid=17689902 10 / 27

slide-22
SLIDE 22

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm (short)

Input: y, A, B, λi, i = 0, . . . , M 1. b = BTBA−1y 2. x = y (Initialization) Repeat 3. [Λi]n,n = φ′([Dix]n) [Dix]n , i = 0, . . . , M, 4. M =

M

  • i=0

λiDT

i ΛiDi

5. Q = BTB + ATMA 6. x = AQ−1b Until converged 8. f = y − x − BA−1(y − x) Output: x, f

11 / 27

slide-23
SLIDE 23

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Evaluation 1

1 2000 −10 10 20 30 40 50 Time (sample) 1 2000 −10 10 20 30 40 50 Time (sample) 1 2000 −10 10 20 30 40 50 Time (sample) 1 2000 −10 10 20 30 40 50 Time (sample)

Figure: Simulated chromatograms w/ polynomial+sine baseline

12 / 27

slide-24
SLIDE 24

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Evaluation 1 with Gaussian noise

13 / 27

slide-25
SLIDE 25

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Evaluation 2

1 2000 20 40 60 80 Time (sample) 1 2000 20 40 60 80 Time (sample) 1 2000 10 20 30 40 50 Time (sample) 1 2000 10 20 30 40 Time (sample)

Figure: Simulated chromatograms w/ limited power spectrum noise

14 / 27

slide-26
SLIDE 26

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Evaluation 2 with Gaussian noise

15 / 27

slide-27
SLIDE 27

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Evaluation 3 with Poisson noise

Figure: Simulated chromatograms w/ Poisson noise

16 / 27

slide-28
SLIDE 28

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: mono-dimensional chromatography (data 1)

Figure: Original, superimposed, clean, noise

17 / 27

slide-29
SLIDE 29

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 2)

Figure: Original data

18 / 27

slide-30
SLIDE 30

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 2)

Figure: 2D background (estimated)

18 / 27

slide-31
SLIDE 31

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 2)

Figure: Noise (estimated)

18 / 27

slide-32
SLIDE 32

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 2)

Figure: BEADS corrected data

18 / 27

slide-33
SLIDE 33

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 2)

Figure: Original data (again!)

18 / 27

slide-34
SLIDE 34

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 3)

Figure: Original data

19 / 27

slide-35
SLIDE 35

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 3)

Figure: 2D background (estimated)

19 / 27

slide-36
SLIDE 36

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 3)

Figure: Noise (estimated)

19 / 27

slide-37
SLIDE 37

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 3)

Figure: BEADS corrected data

19 / 27

slide-38
SLIDE 38

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: two-dimensional chromatography (data 3)

Figure: Original data (again!)

19 / 27

slide-39
SLIDE 39

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Results: performance

Figure: Linear cost per sample (almost)

20 / 27

slide-40
SLIDE 40

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Other known uses

◮ A fairly generic model (sparsity, positivity/negativity)

◮ gas chromatography: mono-dimensional and

comprehensive/two-dimensional

◮ Raman spectra: biological and biomedical ◮ MUSE (Multi Unit Spectroscopic Explorer): astronomical

hyperspectral galaxy spectrum

◮ X-ray absorption spectroscopy (XAS), X-ray diffraction

(XRD), and combined XAS/XRD

◮ high-resolution mass spectrometry ◮ postprandial Plasma Glucose (PPG), multichannel

electroencephalogram (EEG) and single-channel electrocardiogram (ECG)

◮ arabic characters 21 / 27

slide-41
SLIDE 41

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Other known uses

◮ A fairly generic model (sparsity, positivity/negativity)

◮ gas chromatography: mono-dimensional and

comprehensive/two-dimensional

◮ Raman spectra: biological and biomedical ◮ MUSE (Multi Unit Spectroscopic Explorer): astronomical

hyperspectral galaxy spectrum

◮ X-ray absorption spectroscopy (XAS), X-ray diffraction

(XRD), and combined XAS/XRD

◮ high-resolution mass spectrometry ◮ postprandial Plasma Glucose (PPG), multichannel

electroencephalogram (EEG) and single-channel electrocardiogram (ECG)

◮ arabic characters 21 / 27

slide-42
SLIDE 42

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Conclusions

◮ Joint Baseline Estimation and Denoising

◮ Little ”hard” modeling ◮ Codes available in Matlab2 and R3 ◮ Interaction between “separative science” and “source

separation”

2http://lc.cx/beads 3http://www.laurent-duval.eu/lcd-publications.html#beads-r-code 22 / 27

slide-43
SLIDE 43

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

Work in progress

◮ Ongoing tests on analytical chemistry data: NIR, NMR, MS ◮ Better documentation and usability ◮ Estimated baseline and noise use? ◮ Novel metrics: errors related to peak quantities ◮ Novel filtering: an update on Savitzky-Golay filters ◮ Novel deconvolution: sparse & positive with norm ratios

23 / 27

slide-44
SLIDE 44

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

More for free: additional references

  • C. Vendeuvre, R. Ruiz-Guerrero, F. Bertoncini, L. Duval, D. Thi´

ebaut, and M.-C. Hennion. Characterisation of middle-distillates by comprehensive two-dimensional gas chromatography (GC × GC): A powerful alternative for performing various standard analysis of middle-distillates.

  • J. Chrom. A, Sep. 2005.
  • C. Vendeuvre, R. Ruiz-Guerrero, F. Bertoncini, L. Duval, and D. Thi´

ebaut. Comprehensive two-dimensional gas chromatography for detailed characterisation of petroleum products. Oil Gas Sci. Tech., Jan.-Feb. 2007.

  • X. Ning, I. W. Selesnick, and L. Duval.

Chromatogram baseline estimation and denoising using sparsity (BEADS).

  • Chemometr. Intell. Lab. Syst., Dec. 2014.
  • A. Repetti, M. Q. Pham, L. Duval, E. Chouzenoux, and J.-C. Pesquet.

Euclid in a taxicab: Sparse blind deconvolution with smoothed ℓ1/ℓ2 regularization. IEEE Signal Process. Lett., May 2015.

  • C. Couprie, L. Duval, M. Moreaud, S. H´

enon, M. Tebib, V. Souchon. BARCHAN: Blob Alignment for Robust CHromatographic ANalysis. Journal of Chromatography A., Feb. 2017.

  • L. Duval, A. Pirayre and I. W. Selesnick.

Peaks, baseline and noise separation. Chapter in Source Separation in Physical-Chemical Sensing, 2018. 24 / 27

slide-45
SLIDE 45

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

We now have a majorizer for F G(x, v) = 1 2H(y − x)2

2 + λ0xT[Γ(v)]x

+ λ0bTx +

M

  • i=1

λi 2 (Dix)T [Λ(Div)] (Dix)

  • + c(v).

Minimizing G(x, v) with respect to x yields x =

  • HTH + 2λ0 Γ(v) +

M

  • i=1

λiDT

i [Λ(Div)] Di

−1 HTHy − λ0b

  • .

with notations c(v) =

  • n
  • φ(vn) − vn

2 φ′(vn)

  • .

25 / 27

slide-46
SLIDE 46

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

We now have a majorizer for F G(x, v) = 1 2H(y − x)2

2 + λ0xT[Γ(v)]x

+ λ0bTx +

M

  • i=1

λi 2 (Dix)T [Λ(Div)] (Dix)

  • + c(v).

Minimizing G(x, v) with respect to x yields x =

  • HTH + 2λ0 Γ(v) +

M

  • i=1

λiDT

i [Λ(Div)] Di

−1 HTHy − λ0b

  • .

with notations [Γ(v)]n,n =   

1+r 4|vn|,

|vn| ǫ

1+r 4ǫ ,

|vn| ǫ

25 / 27

slide-47
SLIDE 47

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

We now have a majorizer for F G(x, v) = 1 2H(y − x)2

2 + λ0xT[Γ(v)]x

+ λ0bTx +

M

  • i=1

λi 2 (Dix)T [Λ(Div)] (Dix)

  • + c(v).

Minimizing G(x, v) with respect to x yields x =

  • HTH + 2λ0 Γ(v) +

M

  • i=1

λiDT

i [Λ(Div)] Di

−1 HTHy − λ0b

  • .

with notations [Λ(v)]n,n = φ′(vn) vn

25 / 27

slide-48
SLIDE 48

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

We now have a majorizer for F G(x, v) = 1 2H(y − x)2

2 + λ0xT[Γ(v)]x

+ λ0bTx +

M

  • i=1

λi 2 (Dix)T [Λ(Div)] (Dix)

  • + c(v).

Minimizing G(x, v) with respect to x yields x =

  • HTH + 2λ0 Γ(v) +

M

  • i=1

λiDT

i [Λ(Div)] Di

−1 HTHy − λ0b

  • .

with notations [b]n = 1 − r 2

25 / 27

slide-49
SLIDE 49

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

Writing filter H = A−1B ≈ BA−1 (banded matrices) we have x = AQ−1 BTBA−1y − λ0ATb

  • where Q is the banded matrix,

Q = BTB + ATMA, and M is the banded matrix, M = 2λ0Γ(v) +

M

  • i=1

λiDT

i [Λ(Div)] Di.

26 / 27

slide-50
SLIDE 50

INTRODUCTION BEADS MODEL AND ALGORITHM EVALUATION AND RESULTS CONCLUSIONS

BEADS Algorithm

Using previous equations, the MM iteration takes the form: M(k) = 2λ0Γ(x(k)) +

M

  • i=1

λiDT

i

  • Λ(Dix(k))
  • Di.

Q(k) = BTB + ATM(k)A x(k+1) = A[Q(k)]−1 BTBA−1y − λ0ATb

  • 27 / 27