Arithmetic Coding Mathias Winther Madsen mathias.winther@gmail.com Institute for Logic, Language, and Computation University of Amsterdam March 2015
Binary Intervals Definition A binary interval is an interval of the form � k 2 m , k + 1 � . 2 m 1 / 2 1 Yes 1 / 4 1 / 2 Yes 1 / 3 1 No 1 / 4 3 / 4 No
Binary Interval Names Binary intervals have names . 0 1 0 1 / 4 3 / 8 E.g., [ 1 / 4 , 3 / 8 ] ∼ 010 .
Binary Interval Names 1 / 2 3 / 4 10 ∼ 3 / 8 1 / 2 001 ∼ 1 / 2 1 ? ∼ 0 1 / 4 ? ∼
Binary Interval Names 1 / 2 3 / 4 10 ∼ 3 / 8 1 / 2 001 ∼ 1 / 2 1 ∼ 1 0 1 / 4 ∼ 00
Shannon-Fano-Elias Coding Intervals can be approximated by inner binary intervals . 0.2 0.7 1 / 4 1 / 2 E.g., [ 0 . 2 , 0 . 7 ] ≈ [ 1 / 4 , 1 / 2 ] ∼ 01 .
Shannon-Fano-Elias Coding A distribution is a set of intervals “ a ” “ b ” “ c ” 00 10 111 which can be approximated. Robert Mario Fano: “The Transmission of Information.” Tech. Rep. 65, Research Laboratory of Electronics, MIT, 1949.
Shannon-Fano-Elias Coding [ 0 , . 4 ] [ . 4 , 1 ] 00 1 [ 0 , . 3 ] [ . 3 , . 6 ] [ . 6 , 1 ] 00 010 11 [ 0 , . 6 ] [ . 6 , . 8 ] [ . 8 , 1 ] ? ? ?
Shannon-Fano-Elias Coding [ 0 , . 4 ] [ . 4 , 1 ] 00 1 [ 0 , . 3 ] [ . 3 , . 6 ] [ . 6 , 1 ] 00 010 11 [ 0 , . 6 ] [ . 6 , . 8 ] [ . 8 , 1 ] 0 101 111
Arithmetic Coding A conditional distribution is a subdivision of an interval. S T W A H O E A O
Arithmetic Coding 1. Initialize your “zoom” as A = [ 0 , 1 ] . 2. For each character x in the text: 2.1 Divide A according to your predictive distribution; 2.2 Replace A by the subinterval representing x . 3. Approximate A by an inner binary interval B ; 4. Output the codeword for B .
Recommend
More recommend
