Arithmetic Coding Mathias Winther Madsen mathias.winther@gmail.com - - PowerPoint PPT Presentation

Arithmetic Coding

Mathias Winther Madsen mathias.winther@gmail.com

Institute for Logic, Language, and Computation University of Amsterdam

March 2015

Binary Intervals

Definition

A binary interval is an interval of the form k 2m , k + 1 2m

• .

1/2

1 Yes

1/4 1/2

Yes

1/3

1 No

1/4 3/4

No

Binary Interval Names

Binary intervals have names.

1

1/4 3/8

E.g., [1/4, 3/8] ∼ 010.

Binary Interval Names

1/2 3/4

∼ 10

3/8 1/2

∼ 001

1/2

1

∼ ?

1/4

∼ ?

Binary Interval Names

1/2 3/4

∼ 10

3/8 1/2

∼ 001

1/2

1

∼ 1

1/4

∼ 00

Shannon-Fano-Elias Coding

Intervals can be approximated by inner binary intervals. 0.2 0.7

1/4 1/2

E.g., [0.2, 0.7] ≈ [1/4, 1/2] ∼ 01.

Shannon-Fano-Elias Coding

A distribution is a set of intervals

“a” “b” “c” 00 10 111

which can be approximated. Robert Mario Fano: “The Transmission of Information.”

• Tech. Rep. 65, Research Laboratory of Electronics, MIT, 1949.

Shannon-Fano-Elias Coding [0, .4] [.4, 1] 00 1 [0, .3] [.3, .6] [.6, 1] 00 010 11 [0, .6] [.6, .8] [.8, 1] ? ? ?

Shannon-Fano-Elias Coding [0, .4] [.4, 1] 00 1 [0, .3] [.3, .6] [.6, 1] 00 010 11 [0, .6] [.6, .8] [.8, 1] 101 111

Arithmetic Coding

A conditional distribution is a subdivision of an interval.

S T W

A H O

E A O

Arithmetic Coding

• 1. Initialize your “zoom” as A = [0, 1].
• 2. For each character x in the text:

2.1 Divide A according to your predictive distribution; 2.2 Replace A by the subinterval representing x.

• 3. Approximate A by an inner binary interval B;
• 4. Output the codeword for B.