Codebook and Marker Sequence Design for Synchronization-Correcting - - PowerPoint PPT Presentation

Outline Introduction System Description Bound Results Closure

Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Victor Buttigieg1 Johann A. Briffa2

1Department of Communications and Computer Engineering

University of Malta Msida MSD2080, Malta

2Department of Computing

University of Surrey Guildford GU2 7XH, England

3 August 2011

1 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Overview

1

Introduction and motivation

2

System Description

3

Bound on SEC Code Performance

4

Results

5

Conclusions

2 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Introduction

System Description

Concatenated code for insertion/deletion/substitution errors

Inner code used to correct synchronization errors Outer code used to correct remaining substitution errors

Similar scheme to Davey & MacKay (2001) and Ratzer (2005)

Motivation

Practical applications are emerging that require synchronization correcting codes

Image watermarking Bit-patterned magnetic media

Synchronization correcting codes still not very well understood

3 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Synchronization-Error Channel

Binary Substitution-Insertion-Deletion (BSID) Channel

ti ti+1 Pd

i

P

s

P 1

s

P  1

i d

P P  

For every transmitted bit:

We can have zero or more insertions Followed by one of the following

Deletion Or a correct transmission Or a substitution

4 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

General System Overview

Information Source q-ary Encoder BSID Channel Soft-input q-ary Decoder Information Sink Inner Codec Outer Codec

+

SEC Encoder Marker Sequence Inner Decoder d c m r f 5 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Outer Code

q-ary Outer Code

Objective – to correct residual substitution errors Input and output symbols are in Fq Takes K input symbols and emits N symbol blocks Pre-requisite – Soft input decoder available.

6 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

Synchronization and Error Correcting (SEC) Code

Maps each q-ary outer code symbol to a fixed length binary codeword Main objective – to correct for synchronization errors at bit level

Marker Sequence

Available at both encoder and decoder Added modulo-2 to the output from the SEC encoder Main objective – to maintain symbol level synchronization. Minimizes symbol insertion/deletion at the output of inner decoder

7 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

SEC Codes

SEC code has parameters (n, q, dlmin)

Fixed length n bits q codewords Minimum Levenshtein distance dlmin (which is maximized)

N input symbols result in a frame of Nn bits dlmin determines number of correctable substitution, deletion and insertion errors t = dlmin − 1 2

• 8 / 25

Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

Offset Codes

Adding a modification vector to a code maintains its Hamming distance structure This is in general not true in the case of Levenshtein distance Most modification vectors will decrease dlmin An Allowed Modification Vector (AMV) is a vector that retains the minimum Levenshtein distance of the code The result is an Offset Code

9 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Marker Sequence

Error effects

Any errors in a codeword that are not corrected by the SEC code may results in

the codeword decoded incorrectly but decoder remaining synchronized at codeword boundaries the decoder losing synchronization by a few bits at both boundaries the decoder deleting or inserting an entire codeword, maintaining synchronization at subsequent codeword boundaries

Marker Sequence role

Outer code may be able to correct first two types of errors Third type may result in an uncorrectable long burst of errors Marker sequence is introduced to address this problem

10 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Marker Sequence

Objective of Marker Sequence

Required to combat insertion/deletion of complete codewords If decoder inserts/deletes complete codeword, marker sequence would be out of sync This results in a larger number of perceived errors by the decoder Marker sequence improves synchronization at the codeword level

Previous Work

Gallager (1961) used a pseudo-random sequence together with convolutional codes Davey & MacKay (2001) also used a pseudo-random sequence together with a sparse code

11 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Marker Sequence

Choice of Marker Sequence

Pseudo-random sequence would destroy SEC code properties Instead, a sequence of Allowed Modification Vectors are used Simulation results have shown that a sequential use of AMVs leads to adequate performance Performance at high error rates is improved if a larger number of AMVs is used

12 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

SEC Code Construction

Two different construction techniques were used

SEC codes with parameters (n, M, 3) using the Varshamov-Tenegolts construction (Varshamov, 1965; Levenshtein, 1966) SEC codes with larger dlminusing Simulated Annealing adopting similar techniques as given by Gamal et al. (1987) for error-correcting codes

Construction of good (n, q, dlmin) SEC codes with a maximum number of AMVs is still an open problem

13 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

Inner Decoder

Decoder uses the symbol-level algorithm given by [Briffa et al. 2010] APPs for symbol d ∈ Fq at block index 0 ≤ ι < N obtained using Lι(d) =

• x1,x2

αι(x1)βι+1(x2)γι(d, x1, x2 − x1).

14 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Inner Code

Inner Decoder

The forward and backward metrics are respectively defined by αι(x) = Pr

• r|nι+x

, ςnι = x

• βι(x)

= Pr

• r|ρ

nι+x | ςnι = x

• where ςnι represents the assumed channel drift at the beginning of

the codeword at block index ι. The joint probability of the received and transmitted sequence segments corresponding to block index ι is defined as γι(d, x, δx) = Pr

• r|n(ι+1)+x+δx

nι+x

, f|n(ι+1)

nι

• .

15 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Comparison with Previous Work

Davey-MacKay and derivatives

System similar to the Davey-MacKay (2001) construction

DM construction uses a sparse code with minimum Hamming weight codewords instead of SEC code Synchronization is entirely achieved through pseudo-random marker sequence Sparse code ensures that marker sequence is not changed much Principal disadvantages

Sparse code is a liability in the synchronization process Difficult to distinguish between sparse codewords

Briffa et al. (2010) improved the inner decoder by taking into consideration the sparse codebook used (symbol-level decoder)

This symbol-level decoder is used in the current work

16 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Bound on SEC Code Performance

Assumption

Decoder is synchronized on the codeword boundaries Symbol-error probability bounded by considering the effect of errors

• n individual codewords

Probability of having ni, nd and ns Errors

The probability of having an error pattern with, respectively, ni, nd and ns insertion, deletion and substitution errors is Pr {ni, nd, ns} = Pni

i

· Pnd

d

· (PtPs)ns · (Pt(1 − Ps))n−nd −ns where Pt = 1 − Pi − Pd is the probability of transmitting a bit

17 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Bound on SEC Code Performance

Number of error patterns having ni, nd and ns Errors

n+ni −1

ni

• ways of inserting ni bits

n

nd

• ways of deleting nd bits

n−nd

ns

• ways of substituting ns bits.

The total combinations of having ni, nd and ns errors is given by Cni ,nd ,ns = n + ni − 1 ni n nd n − nd ns

• 18 / 25

Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Bound on SEC Code Performance

Symbol Error Probability

The probability of having ni, nd and ns errors anywhere within an n-bit codeword is given by P(E) = Cni ,nd ,ns Pr {ni, nd, ns} Assume that the code can correct up to t-errors and no more Probability of symbol error, PS(E), is bounded by PS(E) ≤

• ni +nd +ns>t

Cni ,nd ,ns Pr {ni, nd, ns}

19 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Bound on SEC Code Performance

Approximate Bounds

If Pi = Pd = Ps =: p ≪ 1, bound may be approximated by PS(E) ≤

• ni +nd +ns=t+1

Cni ,nd ,nspt+1 If Ps = 0 and Pi = Pd =: p ≪ 1, bound may be approximated by PS(E) ≤

• ni +nd =t+1

n + ni − 1 ni n nd

• pt+1

Performance at low error rates mainly depends on dlmin

20 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Effect of Min. Levenshtein Distance

10−4 10−3 10−2 10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−5 10−4 10−3 10−2 10−1 100 Symbol Error Rate

SEC Codes (no marker) and DM Code (random marker), N =50

(7,8), sparse, R =0.43 (7,8,3), R =0.43 (16,64,5), R =0.38 (24,128,7), R =0.29

Summary Results clearly show that codes with a larger dlmin have a steeper slope, and therefore better performance.

21 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Effect of Min. Levenshtein Distance

10−4 10−3 10−2 10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−5 10−4 10−3 10−2 10−1 100 Symbol Error Rate

SEC Codes (no marker) and DM Code (random marker), N =50

(7,8), sparse, R =0.43 (7,8,3), R =0.43 (16,64,5), R =0.38 (24,128,7), R =0.29

Summary Bound is not tight since decoder can correct some error patterns with more than t-errors

21 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

(8, 16, 3) SEC Code, N = 667

10−3 10−2 10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−4 10−3 10−2 10−1 100 Symbol Error Rate

(8,16,3) SEC Code, N =667

no marker 2 AMVs 8 AMVs SERh SERl

Summary With Marker Sequence there is improved performance at high error rates

22 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

(8, 16, 3) SEC Code, N = 667

10−3 10−2 10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−4 10−3 10−2 10−1 100 Symbol Error Rate

(8,16,3) SEC Code, N =667

no marker 2 AMVs 8 AMVs SERh SERl

Summary A more varied marker sequence also improves performance

22 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

(8, 16, 3) SEC Code, N = 667

10−3 10−2 10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−4 10−3 10−2 10−1 100 Symbol Error Rate

(8,16,3) SEC Code, N =667

no marker 2 AMVs 8 AMVs SERh SERl

Summary At low error rates, marker sequence is not really required

22 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Concatenated Systems

10−1 2 ×10−2 2 ×10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−4 10−3 10−2 10−1 100 Frame Error Rate

Concatenated Systems

(6,8), LDPC (1000,100), R =0.050 (7,8), turbo (855,171), R =0.086 (9,4,5), LDPC (667,500), R =0.167 (9,4,5), turbo (750,250), R =0.074

Summary (9,4,5) SEC code with 40 AMVs in concatenation with non-binary (667,600) LDPC and (750,250) Turbo codes

23 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Concatenated Systems

10−1 2 ×10−2 2 ×10−1 Channel Error Event Probability (Pd =Pi; Ps =0) 10−4 10−3 10−2 10−1 100 Frame Error Rate

Concatenated Systems

(6,8), LDPC (1000,100), R =0.050 (7,8), turbo (855,171), R =0.086 (9,4,5), LDPC (667,500), R =0.167 (9,4,5), turbo (750,250), R =0.074

Summary At a similar

• verall code rate

and block size

• ur new scheme

performs significantly better than the best published results

23 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Conclusions

Contributions

Presented novel construction consisting of a SEC code together with a matched Marker Sequence Performance of SEC code depends on the Minimum Levenshtein Distance

Advantages

Significant improvement in performance achieved without increasing complexity compared to similar systems Construction allows us to directly control the trade-off between code rate, performance and complexity

24 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes

Outline Introduction System Description Bound Results Closure

Conclusions

Future Work

Iterative decoding

Use feedback from outer decoding How do we co-design inner and outer code?

Inner code optimization

Better construction methods for the SEC code Maximize the number of AMVs for given SEC code parameters

25 / 25 Codebook and Marker Sequence Design for Synchronization-Correcting Codes