Decoding of Linear Codes Arman Fazeli Alexander Vardy Hanwen Yao - - PowerPoint PPT Presentation

Hardness of Successive-Cancellation Decoding of Linear Codes

Virtual Presentation at International Symposium on Information Theory 2020

Arman Fazeli afazelic@ucsd.edu Alexander Vardy avardy@ucsd.edu Hanwen Yao hwyao@ucsd.edu

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

• Gained Interest in successive-cancellation decoding since

the advent of polar coding from nearly a decade ago.

• It can be accomplished with decoding complexity .
• The complexity is remained unknown for other families of

codes.

Motivation

• Gained Interest in successive-cancellation decoding since

the advent of polar coding from nearly a decade ago.

• It can be accomplished with decoding complexity .
• The complexity is remained unknown for other families of

codes.

• Secondary motivation:

– SC decoding of large-kernel polar codes

Motivation

Conventional Polar Codes

Conventional Polar Codes

Large-Kernel Polar Codes

Large-Kernel Polar Codes

Naïve decoding complexity is given by .

• Gained Interest in successive-cancellation decoding since

the advent of polar coding from nearly a decade ago.

• It can be accomplished with decoding complexity .
• The complexity is remained unknown for other families of

codes.

• Secondary motivation:

– SC decoding of large-kernel polar codes

• Is it possible to SC decode arbitrary linear codes efficiently?

Motivation

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

• A problem is said to be NP-hard if every
• ther problem in NP can be reduced to it

by a deterministic Turing machine in a polynomial time.

• A common strategy to prove NP-hardness

is by the polynomial reduction from a known NP-hard problem.

• A few NP-hard problems in coding theory:

– Maximum-likelihood Decoding (MLD) – Computation of the weight distribution

NP-hardness

Picture is in courtesy of Wikipedia.

• ML decoding is equivalent to minimum distance decoding

for common communication channels such as binary symmetric channel (BSC) with flip probability .

• MLD-Linear is shown to be NP-Hard for the class of binary

linear codes by Berlekamp, McEliece, and van Tilborg.

• It can be properly formulated as

ML Decoding

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

The channel model for the i-th synthesized channel, i.e. successive-cancellation decoding at step i, is given as follows. : uncoded information vector : transmitted codeword : received vector

Successive-Cancellation Decoding

• The choice of the generator matrix matters! It can even

change the hardness of the problem.

• For a fixed generator matrix, the SCD-Linear decision

problem is formally stated as follows.

SCD-Linear

• But there are many generator matrices that can generate the

same code! A natural question follows:

– How hard is the SCD-Linear problem if we relax the constraint on G and let the solution choose from the ensemble of all generator matrices for the given linear code?

• This problem can be formally stated as follows.

Ensemble SCD-Linear

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

• Let be a generator matrix.
• At step i, the values of are known.
• The output of the channel is given by .
• For a given frozen vector , define
• The decision problem is SCD-Linear is equivalent to

SCD-Linear is NP-hard

• Given that all the codewords in (for ) are

transmitted equally likely, we have where denotes the Hamming distance.

• So, the decision problem in SCD-Linear can be further

simplified as where .

SCD-Linear is NP-hard (continued)

• Now, set . For any , we have
• This means that both sides of

are dominated by the terms with the smallest powers.

SCD-Linear is NP-hard (continued)

• Hence, SC-Distance, formally stated as bellow, is a special

case of SCD-Linear.

• The proof goes by showing that MLD-Linear problem can be

polynomially reduced to the SC-Distance problem.

SCD-Linear is NP-hard (continued)

SC-Distance NP-hard

• Let be a generator matrix of the given binary code C

and y be the received vector. Further let denote the rows in .

• The goal is to invoke the algorithmic solution for SCD-

Distance (oracle) k times in order to find a nearest codeword

• f y in C.
• Define
• In step 1, we ask the SC-Distance oracle to find out about

where a tie is broken with a coin flip.

• This allows us to reduce the search size by half.

SC-Distance NP-hard (continued)

• Repeat the same method in each step. In step i,

– We start from a linear shift of denoted by , which consists at least one of the closest codewords to y. – Ask the SC-Distance oracle about which allows us to cut the search size by half once again. – Depending on the oracle’s response, update the shift vector as

• After k steps, the shift vector is the closest codeword to y.

Ensemble SC-Distance is NP-hard

• Each step is there to reduce the search size in half.
• The choice of the generator matrix has no significant role.
• We can achieve the same if the SC-Distance problem is relaxed

to make the comparison for a generator matrix of its choice.

• Hence, the resulting computational problem is also NP-hard.
• Corollary: The Ensemble SCD-Linear problem is also NP-Hard.

• Motivation
• Background
• Problem Definition
• SCD-Linear is NP-hard
• Connections to Polar Codes

Outline

Extension to Matrices

• We showed that one cannot devise an SC decoding algorithm

that runs efficiently for an arbitrary generator matrix.

• Note that it is possible to add rows on the top of any

generator matrix and the SC decoding of the original matrix will be contained within the SC decoding of the extended square matrix.

• But this statement does not remain true if the additional rows

are placed after or within the original rows.

• This is the reason why the SCD-Linear for is not

equivalent to the SC decoding of an -polar code.

Polar Codes With Dynamically Frozen Values

Originally proposed in and later generalized in it was shown that one can improve the distance properties of polar codes by setting the values of frozen bits dynamically as a linear combination of the previously decoded bits. That is where ‘s are fixed and known to both encoder and decoder.

Polar Codes With Dynamically Frozen Values

• This can be achieved by precoding the length-n vector

with a non-singular upper-triangular matrix .

• The encoding relation is hence given by

where serves as an input to the polar encoder.

• Given that there is a one-to-one relation between and ,

the same SC decoder that is capable of computing is also capable of computing which extends the number of efficiently SC-decodable matrices by a factor of

Linear Codes Are Polar Codes with Dynamically Frozen Values

• Let be a generator matrix of the given binary code
• C. Then there exists a unique matrix such that

The matrix is not necessarily upper-triangular. It is not even necessarily a square matrix. We can apply a set of elementary row operations on both sides of to arrive at where is in the reduced row echelon form.

• is also a generator matrix of C. In other words, every

linear code can be encoded as a polar code with dynamically frozen values.

• A remark not to miss is that while is a generator matrix of

the code, SC decoding of the corresponding polar code is not equivalent to SC decoding of C.