On Evaluation of Video Quality Metrics: an HDR Dataset for Computer - - PowerPoint PPT Presentation
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin adk*, Tun O. Aydn, Karol Myszkowski, and Hans-Peter Seidel Outline Quality Assessment In Computer Graphics Proposed Dataset
January 26, 2011
– In Computer Graphics
– Reference-test video pairs – LDR-LDR, HDR-HDR, LDR-HDR
– 4 VQMs
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
Reference Test
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
MSE = 280 MSE = 280 MSE = 280 ! Random Noise Blur ~15% Decreased Luminance Reference
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
Probability of Detection: ~15% Decreased Luminance Random Noise Distortion Map Distortion Map
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
– Usually only informal studies – Objective metrics help in this – Objective metrics can be applied in algorithms
– LDR, HDR, LDR/HDR – Localized distortion maps – Specific artifacts
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
No temporal filtering With temporal filtering [Herzog et al. 2010] Predicted distortion map
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
High quality Low quality Predicted distortion map
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
HDR video LDR video Tone mapping
Medium Compression High Compression
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
[Fattal et al. 2002] Detail Amplification Detail Loss
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
No suitable dataset available
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
(1) Show videos side-by-side
(2) Subjects mark regions where they detect differences
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
Average distortion
DRIVQM
[Aydin et al.’10]
PDM
[Winkler’05]
HDRVDP
[Mantiuk al.‘05]
DRIVDP
[Aydin et al.’08]
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
Stimulus DRIVQM PDM HDRVDP DRIVDP 1
0.765
0.591 0.488
2
0.883 0.686 0.673 0.859
3
0.843 0.886 0.0769 0.865
4
0.815 0.0205 0.211
5
0.844 0.565 0.803 0.689
6
0.761
0.709 0.299
7
0.879 0.155 0.882 0.924
8
0.733 0.109 0.339 0.393
9
0.753 0.368 0.473 0.617
Average 0.809 0.257 0.528 0.563
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
– Random pixel noise filtered with a Gaussian.
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
DRIVQM PDM [Winkler 2005]
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
DRIVQM [Aydin et al. 2010] PDM [Winkler 2005] HDRVDP [Mantiuk et al 2005] DRIVDP [Aydin et al 2008]
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
DRIVQM [Aydin et al. 2010] PDM [Winkler 2005] HDRVDP [Mantiuk et al 2005] DRIVDP [Aydin et al 2008]
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
January 26, 2011
On Evaluation of Video Quality Metrics: an HDR Dataset for Computer Graphics Applications Martin Čadík, mcadik@mpii.de SPIE HVEI’11, San Francisco
<http://www.mpi-inf.mpg.de/resources/hdr/quality>