Video - - PowerPoint PPT Presentation

▶
video group
SMART_READER_LITE
LIVE PREVIEW

Video - - PowerPoint PPT Presentation

Dec 22, 2022 190 likes •765 views

Video Group CS MSU Graphics & Media Lab Only for Maxus Adaptive

slide-1
SLIDE 1

Восстановление сжатого видео

Моисейцев Алексей Video Group CS MSU Graphics & Media Lab

slide-2
SLIDE 2

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  2

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-3
SLIDE 3

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  3

Введение

slide-4
SLIDE 4

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  4

Введение

slide-5
SLIDE 5

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  6

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-6
SLIDE 6

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  7

Adaptive Fuzzy Post-Filtering

Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.

пикселя значение новое x блока типа от зависит фильтра сила e b a x x x x x x

c b a j N j c j N j c j c

   

   

 

~ , ) , ( ) , ( ) , ( ~

/ ) ( 2 2 1 1

2 2

   

slide-7
SLIDE 7

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  8

Adaptive Fuzzy Post-Filtering

Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.

slide-8
SLIDE 8

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  9

Adaptive Fuzzy Post-Filtering

Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.

PSNR 30.61 dB PSNR 30.93 dB

slide-9
SLIDE 9

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  10

Adaptive Fuzzy Post-Filtering

 Преимущества

 Быстрая работа  Простота в реализации

 Недостатки

 Просто маскировка артефактов  Замыливание изображения

slide-10
SLIDE 10

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  11 11 11

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-11
SLIDE 11

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  12 12

DCT Re-application

1. Сдвинуть изображение по вертикали и горизонтали на (i,j) 2. Сжать изображение 3. Вернуть изображение обратно, т.е. сдвинуть на (-i,-j) 4. Повторить для всех сдвигов из [-3, 4] x [-3, 4] 5. Усреднить результат

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-12
SLIDE 12

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  13 13

DCT Re-application

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-13
SLIDE 13

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  14 14

DCT Re-application

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-14
SLIDE 14

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  15 15

j i,

DCT Re-application

 

 

 

j i j i j i j i

x D Q D y

, , ) , ( ,

ния декодирова и я кодировани оператор

  • сдвига

оператор

  • е

изображени енное восстановл

  • е

изображени сжатое

  • Q

D y x

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-15
SLIDE 15

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  16 16

DCT Re-application

Пример работы

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-16
SLIDE 16

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  17 17

DCT Re-application

Пример работы

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-17
SLIDE 17

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  18 18

DCT Re-application

Пример работы

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.
slide-18
SLIDE 18

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  19

DCT Re-application

 Преимущества

 Заметное уменьшение артефактов  Прост в реализации

 Недостатки

 Желательно знать параметры сжатия  Ориентирован на изображения  Невысокая скорость работы

(~58 умножений на пиксель)

slide-19
SLIDE 19

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  20 20 20

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-20
SLIDE 20

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  21 21

Support Vector Regression

 Support Vector Machine

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

slide-21
SLIDE 21

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  22 22

Support Vector Regression

 

 

l i i i

C w w

1 * 2 1

,  

              , ) ( , ) ( ,

* * i i i i i i i i

y b x w b x w y        

 

i i y

x ,

  • тренировочные данные

) (x f y 

  • моделируемая функция

b x w y   ), ( 

  • вид модели

Минимизация при условии

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

элемента каждого для отклонение , элементов всех для отклонение допустимое

* i

 

i

  

slide-22
SLIDE 22

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  23 23

Support Vector Regression

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

slide-23
SLIDE 23

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  24 24

Support Vector Regression

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

slide-24
SLIDE 24

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  25

Support Vector Regression

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

slide-25
SLIDE 25

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  26

Support Vector Regression

 Преимущества

 Визуальное улучшение на некоторых

типах изображений

 Недостатки

 Зависимость от обучающей выборки  Сложно прогнозируемое поведение

slide-26
SLIDE 26

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  27 27 27

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-27
SLIDE 27

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  28 28

MMRCVQ (Modified Mean-Removed Classified Vector Quantization)

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
slide-28
SLIDE 28

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  29 29

MMRCVQ

Построение кодирующих последовательностей

 Выбор блоков 4x4  Нормировка  Кластеризация

Алгоритм Linde-Buzo-Gray

   

данные исходные , , 1 : данные сжатые , , 1 : ' '       N i x T N i x T

i i

 

i i i i i i

x x z x x z     , ' ' '

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
slide-29
SLIDE 29

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  30 30

MMRCVQ

Linde-Buzo-Gray

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)

Выбор начального разбиения K-means Оценка дисперсии Удвоение числа кластеров

slide-30
SLIDE 30

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  31 31

MMRCVQ

Linde-Buzo-Gray

slide-31
SLIDE 31

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  32 32

MMRCVQ

Linde-Buzo-Gray

slide-32
SLIDE 32

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  33 33

MMRCVQ

Выбор кодирующих последовательностей

Разбиение несжатых данных

Выбор декодирующих последовательностей

 

N i M k l i y x d y x d x R

l i k i i k

... 1 , ... 1 : ) , ' ( ) , ' ( |     

 

M i y C

i

,..., 1 : ' '  

i R x j i

R x y

i j

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
slide-33
SLIDE 33

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  34 34

MMRCVQ

Восстановление

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)

Классификация блока Однородный Текстура Граница Определение направления границы Отсечение плохих результатов Поиск Замена блока

+

slide-34
SLIDE 34

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  35 35

MMRCVQ

Результаты

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
slide-35
SLIDE 35

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  36 36

MMRCVQ

Результаты

Artifact reduction of compressed color images using modified mean-removed classified vector

  • quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston Lo. Journal of the Chinese Institute
  • f Engineers, Vol. 27, No. 5, pp. 747-751 (2004)
slide-36
SLIDE 36

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  37

MMRCVQ

 Преимущества

 Хорошее качество  Приемлемая скорость восстановления

 Недостатки

 Только устранение блочности

slide-37
SLIDE 37

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  38 38 38

Содержание

 Введение  Простые модели

 Adaptive Fuzzy Post-Filtering  DCT Re-application

 Классификация

 Support Vector Regression  Modified Mean-Removed Classified Vector Quantization

 Регуляризация  Заключение

slide-38
SLIDE 38

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  39 39

Regularized Iterative Restoration

) ( ) (

2

f E g f f J    

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999

                                                              

N k N k N k

n n n n n g g g g g f f f f f      

2 1 2 1 2 1

, ,

n Df g  

деградации ллинейно оператор D шум n кадр й наблюдаемы g кадр исходный     f

slide-39
SLIDE 39

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  40 40

Regularized Iterative Restoration

Пространственная регуляризация

 

 

2 2 2 2

) (

l B V l B H l VB l HB s

f Q f Q f Q f Q f E       

 

2 2 2 2

  • therwise

1

  • width/8

1,2,..., l for 8l i if ) , 1 ( ) , ( ) , (

  • therwise

1

  • width/8

1,2,..., l for 8l i if ) , 1 ( ) , ( ) , ( f Q f Q Q f Q f Q Q j i f j i f j i f Q j i f j i f j i f Q

B V B H B VB HB B B H HB

                   

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-40
SLIDE 40

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  41 41

Regularized Iterative Restoration

Временная регуляризация

     

) , ( пикселя движения вектор

  • )

, ( , ) , ( , ) , ( ) , (

) , ( ) , ( 1 1 2 ) , ( ) , (

j i j i n m j i n j m i f j i f f f MFD

j i j i width i height j j i j i k l l k

 

 

   

2

) ( ) , ( ) (

MC l l l t l k l k l t

f f f E f f MFD f E   

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-41
SLIDE 41

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  42 42

Regularized Iterative Restoration

Решение

 Цель  Решение

 

 

 

k i MC i i t B T B S B T B S k i k i

f f f Q Q Q Q I g f f       

   

2 1

1

) ( ) ( ) (

2

f E f E g f f J

t t s s

     

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-42
SLIDE 42

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  43 43

Regularized Iterative Restoration

Многокадровое восстановление

 

) ~ ( ) ~ ( ) ~ ( ,..., , ~

1 2 2 1

f E f E g f f J f f f f

t t s s L l l l T T L T T

            

 

 

 

 

 

     

L l MC l l t L l l B V l B H l VB l HB s

f f f E f Q f Q f Q f Q f E

1 2 1 2 2 2 2 2 1

) ~ ( ) ~ (  

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-43
SLIDE 43

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  44 44

Regularized Iterative Restoration

Многокадровое восстановление

2 2 2 2

~ ~ ~ ~ ~ ~ ) ~ (

2 1

MC t B S B S

f f f Q f Q g f f J         

 

                            

 

k

f f MC t k B T B S B T B S k k

f f f f Q Q Q Q I g f f

~ ~ 1

~ ~ ~ ~ ~ ~ ~

2 1

   

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-44
SLIDE 44

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  45 45

Regularized Iterative Restoration

Примеры работы

# of case 1 spatial only 2 16x16 ME 3 4x4 ME 4 dense 4x4 ME 5 bidirectional 4x4 ME 6 dense bidirectional 4x4 ME Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-45
SLIDE 45

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  46 46

Regularized Iterative Restoration

Примеры работы

dense 4x4 ME

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-46
SLIDE 46

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  47 47

Regularized Iterative Restoration

Примеры работы

dense bidirectional 4x4 ME

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999
slide-47
SLIDE 47

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  48 48

Regularized Iterative Restoration

Оптимизация

   

 

   

t t t b b bt t

b

2 ' inf

2

     

MC t B s

f f f Q g f f J       

2 2

) (

  • цель минимизации

 

 

 

MC t j i B j i s

f f b f Q b g f b f J

j i

     

  

, 2 , 2 *

,

) , (

  • новая цель минимизации

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and

Weisi Lin. 2003.

 

MC t j i B s

f f f Q g f f J

j i

    

  

, 2

,

) (

  • дополнительная переменная
  • идея: добавить вспомогательную переменную, не изменяющую минимум J
slide-48
SLIDE 48

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  49 49

Regularized Iterative Restoration

Оптимизация

Алгоритм:

     

e convergenc Until fixed b with b f J f f f Q b fixed f with b f J b repeat g f tion Initializa

n n n n n B n n n n n

} , min : , min { :

1 * 1 1 * 1    

     

Минимизация может быть выполнена методом градиентного спуска

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and

Weisi Lin. 2003.

slide-49
SLIDE 49

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  50 50

Regularized Iterative Restoration

Оптимизация

 

 

 

 

 

  

     

      j i MC j i n j i t j i n j i n j i B n j i s j i n j i n j i n n

f f b f Q b f f b f J

, 2 , 1 , , 1 , 2 1 , 1 , , 2 1 , , 1 *

) , (   

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and

Weisi Lin. 2003.

 

   

 

 

mc j i t n j i s t n j i t s mc j i t n j i s t t n j i s t n j i n j i t n j i s t n j i s n j i t n j i

f b b f b b f f b b x f

1 , 1 , 1 , , 1 , 1 , 1 , , 1 , 1 , , 1 ,

2 1 1 2 1 1 1 2 1 1 1 1

        

                                

slide-50
SLIDE 50

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  51 51

Regularized Iterative Restoration

Пример работы

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and

Weisi Lin. 2003.

slide-51
SLIDE 51

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  52

Regularized Iterative Restoration

Пример работы

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi
  • Lin. 2003.
slide-52
SLIDE 52

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  53 53

Regularized Iterative Restoration

Пример работы

A Coding Artifacts Removal Algorithm Based On Spatial And Temporal

  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and

Weisi Lin. 2003.

slide-53
SLIDE 53

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  54

Regularized Iterative Restoration

 Преимущества

 Высокое качество восстановления

 Недостатки

 Очень медленная работа  Сложен в реализации

slide-54
SLIDE 54

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  55

Заключение

 Рассмотрены методы

 Adaptive Fuzzy Post-Filtering  DCT Re-application  Support Vector Regression  Modified Mean-Removed Classified Vector Quantization  Регуляризация

slide-55
SLIDE 55

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  56

Литература

  • A Coding Artifacts Removal Algorithm Based On Spatial And Temporal
  • Regularization. Susu Yao, Genan Feng, Xiao Lin, Keng Pang Lim and Weisi Lin.

2003.

Artifact reduction of compressed color images using modified mean-removed classified vector quantization. Jim Zone-Chang Lai, Yi-Ching Liaw, and Winston

  • Lo. Journal of the Chinese Institute of Engineers, Vol. 27, No. 5, pp. 747-751

(2004)

Compressed Video Enhancement using Regularized Iterative Restoration

  • Algorithms. Passant Vatsalya Karunaratne. Evanston, Illinois. 1999

Compression Artifact Reduction Using Support Vector Regression. Sanjeev Kumar, Truong Nguyen, Mainak Biswas. ICIP 2006

Adaptive Fuzzy Post-Filtering for Highly Compressed Video. Hao-Song Kong, Yao Nie, Anthony Vetro, Huifang Sun, Kenneth E. Barner. ICIP 2004.

Enhancement of JPEG-Compressed Images by Re-application of JPEG. Aria

  • Nosratinia. Department of Electrical Engineering, University of Texas at Dallas,
  • Richardson. 2002.

A Tutorial on Support Vector Regression. Alex J. Smola† and Bernhard Sch¨olkop. 2003

Application Of The Motion Vector Constraint To The Regularized Enhancement Of Compressed Video. C. Andrew Segall And Aggelos K. Katsaggelos. 2001

slide-56
SLIDE 56

CS MSU Graphics & Media Lab (Video Group) Only for Maxus  57 57

Вопросы

?

57