TSBK01 J RGEN A HLBERG - History - How many samples/pixels/bits? - - PDF document

TSBK01 IMAGE CODING AND DATA

COMPRESSION

...and a little bit of speech & audio coding ...video coding is included as well

JÖRGEN AHLBERG

ahlberg@isy.liu.se

• PhD in Image Coding
• Research Scientist at the Swedish

Defence Research Agency (FOI)

• Co-founder of Visage Technologies
• I have no office at the university, so

don’t try to find me there!

TODAY

• 1. Overview of the course
• 2. Introduction to image coding:
• Purpose
• History
• How many samples/pixels/bits?
• 3. A fundamental difference: Lossy vs lossless

coding

• 4. Models for image & audio coding and the coding

methods they imply

PART 1: OVERVIEW OF THE COURSE

Course website

http://www.icg.isy.liu.se/courses/tsbk01

Course components

• 12 lectures
• 8 problem-solving classes
• 2 computer-aided classes
• 2 laborations
• A written exam

Lectures

• 1. Introduction
• 2. Basic Information Theory
• 3. Source Coding Theory, Huffman Coding
• 4. Arithmetic Coding, Lempel-Ziv Coding,

Lossless Image Coding

• 5. Coding of Analog Sources, Scalar Quantization
• 6. Vector Quantization
• 7. Predictive Coding
• 8. Transform Coding
• 9. Subband and Wavelet Coding

10.Video Coding 11.Speech and Audio Coding 12.Fractal Coding. MPEG-4 Coding

Problem-solving classes/lessons

• 1. Entropy, Markov sources
• 2. Source Coding
• 3. Rate-Distortion, Scalar Quantization
• 4. Vector Quantization
• 5. Predictive Coding
• 6. Transform Coding
• 7. Subband and Wavelet Coding
• 8. Miscellaneous

Computer-aided classes/lessons

• 1. Scalar and Vector Quantization
• 2. To be determined

Laborations

Mandatory!

• 1. To be determined.
• 2. To be determined.

Exam

Written exam Saturday December 20, 14-18. Mandatory.

Literature

• 1. K. Sayood, Introduction to Data Compression.
• 2. Package containing Exercises, Laborations, and

a Table & Formula collection.

Teachers

English lectures: Jörgen Ahlberg, ahlberg@isy.liu.se Laborations, lessons: Harald Nautsch, harna@isy.liu.se Peter Johansson, pejoh@isy.liu.se Examiner, Swedish lectures: Robert Forchheimer, robert@isy.liu.se

PART 2: INTRODUCTION TO IMAGE CODING

Purpose

A compact digital representation of still or moving images.

Constraints

• Good image quality
• Robust to channel errors
• Real time performance
• Cheap
• r

a u d i

• !

! !

Image coding Telecommunications Image processing Computer graphics Digital signal processing

IMAGES: HOW MANY PIXELS?

128 128 256 65 536 512 262 144 640 480 307 200 720 576 414 720 512 256 Search image for archives Ultrasound images “Classic” standard format for image processing systems VGA TV (digital studio standard) NTSC 352 101 376 288 CIF 176 144 QCIF 320 240 76 800 Low-end computer graphics 1152 HDTV 2 359 296 2048 X-ray, aerial/sattelite/consumer photos: > 1 Mpixels

HOW MANY BITS PER PIXEL?

Bits per pixel Image type Examples 1 binary fax 4 simple computer PDA graphics 8 grayscale telephoto, ultrasound palette colour computer graphics 12 high contrast X-ray 5+6+5 = 16 “high colour”

• ld digital photography

8 · 3 = 24 “true colour” digital photography (RGB) computer graphics 8 · 4 = 32 true colour + alpha computer graphics (RGBA) 14 · 4 multispectral remote sensing 14 · (many) hyperspectral remote sensing

Lenna, 512x512, 8bpp Lenna, 128x128 Lenna, 512x512, 3bpp Lenna, 512x512, 0.5 bpp (JPEG)

AUDIO: HOW MANY BITS PER SAMPLE

AND PER SECOND? CD-quality

• 16 bits per sample
• 44100 samples per second
• Two channels
• → 1.4 Mbit/s
• Often used as reference (“non-

compressed audio”). Be careful!

• Modern compression: 64 kbit/s with

good quality

Digital telephony

• 8 bits per sample
• 8000 samples per second
• → 64 kbit/s
• Modern compression: 4 kbit/s with

good quality

BITS PER SECOND

Still images

Fax 2.4 – 14.4 kbit/s Telephoto 4.8 – 128 kbit/s Teleradiology 64 – 128 kbit/s

Moving images

Video telephony 4 – 128 kbit/s Video conferencing 64 – 383 kbit/s Multimedia < 1.5 Mbits/s Digital TV 3 – 6 Mbits/s HDTV < 22 Mbits/s

Audio

Digital speech 4 – 16 kbit/s Music 64 – 256 kbits/s

VIDEO: HOW MANY BITS PER SECOND?

EXAMPLE APPLICATION: FAX (TELE FACSIMILE)

Generation 1 (1966)

• FM or AM modulation
• 6 minutes @ 96 lines/inch
• 4 minutes @ 64 lines/inch

Generation 2 Single sideband etc gives 3 (2) minutes. Generation 3 (1970)

• Digital communication, runlength

coding

• <1 minute @ 4.8 kbit/s
• Standardised 1980

Generation 4 (1981)

• 16 times better quality
• Colour

PART 3: LOSSY VS LOSSLESS CODING

in practise the same as perception-based coding vs data compression compare to coding of analog sources vs coding of digital sources

Fundamental difference!

LOSSLESS CODING

(Data compression, Entropy coding) Take some digital data, i.e., bits, and represent it using fewer bits in a way that you can reconstruct the original data exactly. Limited by the entropy of the data (source coding theorem). Based on information theory. Some methods

• Huffman coding
• Lempel-Ziv coding
• Arithmetic coding

Some applications

• PNG
• GIF
• Zip
• Lossless versions of JPEG,

JPEG2K, WMA.

LOSSY CODING

Take some digital or analog data and represent it using as few bits as possible in a way that you can reconstruct the original data as well as possible by some measure. Determine what kind of distortion and how much distortion you can accept. No limit in how much you can compress, as long as you can accept more distortion. Some methods

• Transform coding
• Wavelet coding
• Predictive coding
• Psychoacoustic/visual coding

Some applications

• Still images: JPEG, JPEG2K, ...
• Moving images: MPEG-1/2/4,

WMV, ...

• Audio: MP3, WMA, RA, ...
• Speech: GSM (AMR), MPEG-4, ...

A COMMON SCHEME

• g

• r

PART 4: SIGNAL MODELS FOR

IMAGE AND AUDIO CODING Object models

• Illumination
• Objects
• Projections

Signal models

• Deterministic models
• Statistical models

Perception models

• Time/space-frequency models
• Psycho-acoustic/visual models

A MODEL IMPLIES A CODING METHOD

Object models

• Object-based coding
• Semantic coding

Deterministic models

• Interpolation coding
• Contour coding

Statistical models

• Huffman coding
• Predictive coding
• Transform coding
• Vector quantization

Perception models

• Time/space-frequency coding
• Psycho-acoustic/visual masking

A DETERMINISTIC MODEL

The image constists of edges and surfaces:

• Coding: Store the size and position
• f the edges.
• Decoding: Interpolate the values

between the edges. (Interpolative coding, L.D.Davidsson, 1967)

STATISTICAL MODELS

Symbol probablities

a 0.5 b 0.25 c 0.125 d 0.125

Markov models

S3 S1 S2

P11 P13 P31 P12 P21 P33 P32 P23 P22

VARIABLE LENGTH CODING

a 00 b 01 c 10 d 11

• > 2 bits per symbol

Huffman coding

a 1 b 01 c 001 d 000

• > 1.75 bits per symbol

Arithmetic coding

Controlable code word lengths

Universal coding

Adaptive code word lengths PSYCHO-ACOUSTIC MODELS

Strong tones will mask weaker ones

PSYCHO-ACOUSTIC MODELS

Hearing threshold 40 30 20 10 2 4 6 8 10 12 kHz dB

PSYCHO-ACOUSTIC TIME-FREQUENCY

MODELS

Better resolution in lower frequencis: Remap!

VARIOUS MODELS AND METHODS

• 1D Markov models: predictive

coding

• 2D Markov models: Transform

coding

• The font model: Dictionary coding
• Vision based models

Not available in this set of slides: Use Robert’s.

MODEL-BASED CODING

Semantic/object-based coding, coding through animation

SUMMARY

• Image & audio coding - Purpose,

constraints, applications.

• History of image transmission.
• Pixels per image, samples per

second, bits per pixel/sample, bits per second.

• Lossy vs lossless coding
• Signal models → coding methods
• Deterministic models
• Interpolation coding, contour coding
• Statistical models
• Predictive coding, Transform coding,

Vector quantization

• Perception models
• Dual-mode coders, Masking, Time/

Space-Frequency coding

• Object/scene models
• Object-based/Semantic coding
• Self-similarity models
• Fractal coding, IFS