Q-ary Asymmetric LOCO Codes: Constrained Codes Supporting Flash - - PowerPoint PPT Presentation

Q-ary Asymmetric LOCO Codes: Constrained Codes Supporting Flash Evolution

Ahmed Hareedy, Beyza Dabak, and Robert Calderbank Duke University International Symposium on Information Theory June 2020

2

Presentation Outline

➢ Motivation and technical vision ➢ Introduction and history ➢ Q-ary asymmetric LOCO codes ➢ Rates and reconfigurability ➢ Conclusion and future directions

➢ Modern applications (IoT) require storage densities to grow rapidly.

❑ Data storage is a story where density increases as a result of advances in

physics/architectures and innovations in signal processing. ➢ Data storage types:

❑ Magnetic, non-volatile (HDD). ❑ Electronic, non-volatile (Flash). ❑ Resistive, non-volatile (3D XPt). ❑ Electronic, volatile (DRAM).

➢ The cold-warm-hot axis. ➢ Densities approach 10 Tbpsi!

❑ With the vertical NAND

(3D NAND), Flash devices are already winning!

3

Storage Densities Are Rapidly Growing

MR head Giant MR head Tunneling MR head NAND SLC NAND MLC V-NAND TLC V-NAND QLC TDMR

➢ The Flash cell is a MOSFET but with a floating gate (FG).

❑ Programming is performed via applying very

high positive voltage to the gate (NPN).

❑ Electrons tunnel into the FG. ❑ The charge level in the FG controls threshold.

➢ Advances in physics enabled more than two charge levels per cell (SLC vs. M/T/Q/P-LC). ➢ How to read Word 3 in NAND Flash:

❑ Apply ON voltage on all

word lines except 3, and read voltage on 3.

4

Understanding Flash Operation

G Very high +ve voltage P

………………………………

FG N N

• -

➢ Inter-cell interference (ICI):

❑ Parasitic capacitances result in charge propagation (101 in SLC).

➢ Programming errors:

❑ Failed programming/erasing operations result in asymmetric errors.

➢ Other sources of error:

❑ Charge leakage over time and Gaussian electronic noise.

➢ What about magnetic recording devices?

❑ Inter-symbol interference (ISI), inter-track interference (in TDMR), jitter

• r timing problems, and Gaussian noise.

5

Sources of Error in Flash Devices

High Low High MLC Flash

➢ Data storage devices operate at very low frame error rates. ➢ Our technical vision is:

❑ To devise efficient coding techniques that exploit the advances in

physics to significantly improve performance. ➢ Mitigating interference:

❑ Constrained codes prevent error-prone patterns from being written. ❑ LOCO codes forbid these patterns with minimal redundancy. ❑ LOCO codes can be easily reconfigured as the device ages.

➢ Handling other sources of error:

❑ Graph-based (LDPC) codes correct the errors after reading. ❑ Examples are spatially-coupled and multi-dimensional LDPC codes.

➢ These techniques result in significant lifetime and density gains!

6

How Can We Take Full Advantage?

7

Presentation Outline

➢ Motivation and technical vision ➢ Introduction and history ➢ Q-ary asymmetric LOCO codes ➢ Rates and reconfigurability ➢ Conclusion and future directions

➢ Constrained codes impose restrictions on written (transmitted) data.

❑ The set of forbidden patterns can be symmetric or asymmetric. ❑ The rate is (# of information bits)/(# of coded bits or symbols).

➢ The universe of constrained sequences is represented by an FSTD. The capacity is the highest achievable rate. ➢ Example: 𝒯1 = {010, 101} constraint.

❑ The adjacency matrix of this FSTD is: ❑ The capacity is log2(𝜇max),

which is 0.6942 here.

8

Introduction to Constrained Codes

𝑇1 𝑇2 𝑇3 𝑇4 𝑇1 𝑇2 𝑇3 𝑇4

9

History and Our LOCO Codes

1948 Shannon: FSTD of constrained codes [R1] 1970 Tang and Bahl: RLL codes [R2] Cover: Enumerative source coding [R3] 1980 Adler, Coppersmith, and Hassner: State splitting and merging to produce FSMs [R4] 1990 Karabed, Immink, Siegel, and Wolf: Optimization of FSMs [R5, R6] 2000 Constrained codes based on lexicographic indexing [R7, R8] LOCO codes are 𝒯𝑦-constrained (symmetric) codes: Capacity-achieving, simple, and reconfigurable [H1] Franaszek 𝒯𝑦 ≜ {010, 101, 0110, 1001, … , 0𝟐𝑦0, 1𝟏𝑦1} 2019

Claude Shannon Kees Immink

10

Presentation Outline

➢ Motivation and technical vision ➢ Introduction and history ➢ Q-ary asymmetric LOCO codes ➢ Rates and reconfigurability ➢ Conclusion and future directions

➢ Consider Flash devices with 𝑟 levels per cell:

❑ SLC (𝑟 = 2), MLC (𝑟 = 4), TLC (𝑟 = 8), QLC (𝑟 = 16), PLC (𝑟 = 32). ❑ Symbols in GF(𝑟) = {0, 1, 𝛽, … , 𝛽𝑟−2} are written as charge (threshold)

levels in {0, 1, 2, … , 𝑟 − 1}. ➢ What should we forbid?

❑ Patterns resulting in max charge at the outer

cells but less at the inner ones [R9].

❑ Let 𝜀 be in GF(𝑟)\{𝛽𝑟−2}. The set of

forbidden patterns is: 𝒭𝑦

𝑟 ≜ 𝛽𝑟−2𝜺𝑧𝛽𝑟−2, ∀𝜺𝑧 ∈ GF(𝑟)\{𝛽𝑟−2} 𝑧 | 1 ≤ 𝑧 ≤ 𝑦 .

❑ If 𝑟 = 2 (binary), 𝜺𝑧 has to be only 𝟏𝑧. Then,

𝒭𝑦

2 = 101,1𝟏21, … , 1𝟏𝑦1 .

❑ The codes are 𝑟-ary asymmetric LOCO (QA-LOCO) codes. ❑ Handling 𝑦 > 1 can increase the lifetime and reduce the time to market.

11

Device Physics Determine Patterns to Forbid

𝑟 − 1 < 𝑟 − 1 𝑟 − 1

➢ A QA-LOCO code 𝒭𝒟𝑛,𝑦

𝑟

is defined by:

❑ Each codeword 𝐝 ∈ 𝒭𝒟𝑛,𝑦

𝑟

has symbols in GF(𝑟) and is of length 𝑛.

❑ Codewords in 𝒭𝒟𝑛,𝑦

𝑟

are ordered lexicographically.

❑ Each codeword 𝐝 ∈ 𝒭𝒟𝑛,𝑦

𝑟

does not contain any pattern in 𝒭𝑦

𝑟, 𝑦 ≥ 1.

❑ All codewords satisfying the above properties are included.

➢ Codewords in 𝒭𝒟𝑛,𝑦

𝑟 , 𝑛 ≥ 2, are partitioned into three groups:

❑ Group 1: Codewords starting with 𝜀, ∀𝜀, from the left. ❑ Group 2: Codewords starting with 𝛽𝑟−2𝛽𝑟−2 from the left. ❑ Group 3: Codewords starting with 𝛽𝑟−2𝜺𝑦+1, ∀𝜺𝑦+1, from the left.

➢ What QA-LOCO codes offer [H2]:

❑ They mitigate ICI, and they are capacity-achieving. ❑ They have simple encoding-decoding, and they are reconfigurable.

12

Formal Definition and Group Structure

13

QA-LOCO Codes with 𝑟 = 2 and 𝑦 = 1

Index 𝑕(𝐝) Codewords of the code 𝒭𝒟𝑛,1

2

𝑛 = 1 𝑛 = 2 𝑛 = 3 𝑛 = 4 00 000 0000 Group 1 1 1 01 001 0001 2 10 010 0010 3 11 011 0011 4 100 0100 5 110 0110 6 111 0111 7 1000 Group 3 8 1001 9 1100 Group 2 10 1110 11 1111

➢ Theorem: Let 𝑂𝑟(𝑛, 𝑦) be the cardinality of 𝒭𝒟𝑛,𝑦

𝑟 . Define:

𝑂𝑟 𝑛, 𝑦 ≜ 𝑟 − 1 𝑛, 𝑛 ≤ 0, and 𝑂𝑟 1, 𝑦 ≜ 𝑟. Then, 𝑂𝑟(𝑛, 𝑦), 𝑛 ≥ 2, is recursively given by: 𝑂𝑟 𝑛, 𝑦 = 𝑟𝑂𝑟 𝑛 − 1, 𝑦 − 𝑟 − 1 𝑂𝑟 𝑛 − 2, 𝑦 + 𝑟 − 1 𝑦+1𝑂𝑟(𝑛 − 𝑦 − 2, 𝑦). ➢ Example: For 𝑟 = 2 and 𝑦 = 1: 𝑂2 𝑛, 1 = 2𝑂2 𝑛 − 1, 1 − 𝑂2 𝑛 − 2, 1 + 𝑂2(𝑛 − 3,1).

❑ 𝑂2 −1,1 ≜ 1, 𝑂2 0, 1 ≜ 1, 𝑂2 1, 1 ≜ 2. ❑ 𝑂2 2, 1 = 2𝑂2 1, 1 − 𝑂2 0, 1 + 𝑂2 −1,1 = 4. ❑ 𝑂2 3, 1 = 2𝑂2 2, 1 − 𝑂2 1, 1 + 𝑂2 0,1 = 7. ❑ 𝑂2 4, 1 = 2𝑂2 3, 1 − 𝑂2 2, 1 + 𝑂2 1,1 = 12. ❑ The numbers are consistent with the table.

14

Enumerating the Codewords

➢ Theorem: The index of a QA-LOCO codeword 𝐝 ≜ 𝑑𝑛−1𝑑𝑛−2 … 𝑑0 ∈ 𝒭𝒟𝑛,𝑦

𝑟 ,

𝑛 ≥ 2, is given by the rule: 𝑕 𝐝 = σ𝑗=0

𝑛−1 𝑏𝑗 𝑟 − 1 𝛿𝑗𝑂𝑟 𝑗 − 𝛿𝑗, 𝑦 ,

where 𝑏𝑗 ≜ gflog𝛽(𝑑𝑗) + 1 is the level equivalent of 𝑑𝑗, and 𝛿𝑗 = 𝑦 − 𝑙𝑗 + 1; 𝑙𝑗 is the distance to the closest 𝛽𝑟−2 symbol.

❑ For example, if 𝑑𝑗 = 𝛽𝑟−2, 𝑏𝑗 = 𝑟 − 1. ❑ For example, if 𝑑𝑗+1𝑑𝑗 = 𝛽𝑟−2𝑑𝑗, 𝑙𝑗 = 1 and 𝛿𝑗 = 𝑦.

➢ The theorem gives the encoding-decoding rule of QA-LOCO codes.

❑ Encoding: Mapping from index to codeword. ❑ Decoding: Demapping from codeword to index.

➢ The same hardware can support multiple constraints by updating 𝑂’s!

❑ Runtime operations are only subtractions and additions.

15

Codeword to Index and Vice Versa

➢ For the binary case (𝑟 = 2): 𝑕 𝐝 = σ𝑗=0

𝑛−1 𝑏𝑗𝑂2 𝑗 − 𝑏𝑗+1𝑦, 𝑦 .

➢ Example: 𝑟 = 2, 𝑛 = 4, and 𝑦 = 1:

❑ 𝑕(𝐝 = 0111) = σ𝑗=0

3

𝑏𝑗𝑂2 𝑗 − 𝑏𝑗+1, 1 = 𝑂2 2, 1 + 𝑂2 0, 1 + 𝑂2 −1, 1 = 4 + 1 + 1 = 6.

❑ 𝑕 𝐝 = 1110

= σ𝑗=0

3

𝑏𝑗𝑂2 𝑗 − 𝑏𝑗+1, 1 = 𝑂2 3, 1 + 𝑂2 1, 1 + 𝑂2(0, 1) = 7 + 2 + 1 = 10. Both are consistent with the indices in the table.

16

Example of QA-LOCO Rule

Index 𝑕(𝐝) Codewords of the code 𝒭𝒟4,1

2

0000 Group 1 1 0001 2 0010 3 0011 4 0100 5 0110 6 0111 7 1000 Group 3 8 1001 9 1100 Group 2 10 1110 11 1111

17

Presentation Outline

➢ Motivation and technical vision ➢ Introduction and history ➢ Q-ary asymmetric LOCO codes ➢ Rates and reconfigurability ➢ Conclusion and future directions

➢ Bridging is needed to prevent forbidden patterns across codewords.

❑ Consider 𝒭𝒟4,1

2 : 1110— 1000 leads to 11101000.

❑ If the RMS in a codeword and the LMS in the next codeword are both

𝛽𝑟−2’s, bridge with 𝑦 consecutive 𝛽𝑟−2’s.

❑ Otherwise, bridge with 𝑦 consecutive 0’s: 111001000.

➢ Self-clocking is needed to maintain calibration of the system.

❑ Just remove the all 0’s and the all 𝛽𝑟−2’s from the code 𝒭𝒟𝑛,𝑦

𝑟 .

➢ The rate of a self-clocked QA-LOCO code is: 𝑆QA−LOCO

c

= 𝑡c 𝑛 + 𝑦 = log2 𝑂𝑟 𝑛, 𝑦 − 2 𝑛 + 𝑦 .

❑ This rate is in information bits per coded symbol. ❑ To normalize this rate, divide by log2𝑟.

18

Rates of QA-LOCO Codes

➢ QA-LOCO codes are capacity-achieving.

❑ All except two codewords are included, and 𝑦 does not depend on 𝑛.

➢ Examples on QA-LOCO normalized rates (𝑦 = 1):

❑ Exploiting physics: Less than 3% redundancy suffices for ICI mitigation! ❑ Achieved at low complexity and limited error propagation.

19

Data Protection Almost for Free!

𝑟 = 2 (SLC) 𝑟 = 4 (MLC) 𝑟 = 8 (TLC) 𝑟 = 16 (QLC) 𝑛 Norm rate 𝑛 Norm rate 𝑛 Norm rate 𝑛 Norm rate 17 0.7778 26 0.9260 26 0.9506 27 0.9554 44 0.8000 49 0.9500 44 0.9704 45 0.9728 76 0.8052 77 0.9552 71 0.9769 66 0.9813 113 0.8070 97 0.9592 103 0.9840 111 0.9888 Capacity 0.8114 Capacity 0.9687 Capacity 0.9939 Capacity 0.9987

➢ QA-LOCO codes can be easily reconfigured.

❑ As the device ages, the set of patterns to forbid becomes bigger (𝑦 > 1). ❑ Reconfiguration is as easy as reprogramming an adder! ❑ A small number of multiplexers pick the appropriate cardinalities.

➢ Comparisons versus other techniques:

❑ It is quite complicated to design capacity-achieving non-binary

constrained code based on FSMs.

❑ The 𝑟-ary LO-RLL codes in [R2] do not exploit Flash physics. ❑ QA-LOCO codes offer lower complexity than the codes in [R10]. ❑ QA-LOCO codes handle any 𝑦, unlike the codes in [R10] and [R11]. ❑ Non-binary constrained codes are significantly more efficient than their

binary counterparts in terms of rate.

20

Reconfigurability and Comparisons

➢ We simulated three setups in an industry-recommended MR system. ➢ Setup 3 (LDPC + LOCO on parity bits only) achieves [H1]:

➢ About 20% (16%) density gain compared with Setup 1 (Setup 2). ➢ Investing the additional redundancy via LOCO is more beneficial! ➢ Even the error floor performance in Setup 3 is better. ➢ We plan on extending this idea to Flash systems.

21

Protecting Parity Bits Achieves Great Gains

The diffusion of more reliable information provides the LDPC decoder with a better channel.

Overall length 4270 bits Overall rate 0.645

22

Presentation Outline

➢ Motivation and technical vision ➢ Introduction and history ➢ Q-ary asymmetric LOCO codes ➢ Rates and reconfigurability ➢ Conclusion and future directions

➢ Conclusion:

❑ Storage densities are rapidly growing. Data require high protection. ❑ QA-LOCO codes exploit physics of Flash devices to fortify them with

minimal redundancy.

❑ As the device ages, QA-LOCO codes can be reconfigured in order to

contribute to extending lifetime.

❑ Machine learning can help with reconfiguring QA-LOCO codes.

➢ Future directions:

❑ Two-dimensional constrained codes for TDMR devices. ❑ Combining multi-dimensional LOCO codes with multi-dimensional LDPC

codes for modern devices.

23

Conclusion and Future Directions

➢ [H1] A. Hareedy and R. Calderbank, “LOCO codes: Lexicographically-ordered constrained codes,” IEEE Trans. Inf. Theory, 2019. ➢ [H2] A. Hareedy, B. Dabak, and R. Calderbank, “Managing device lifecycle: Reconfigurable constrained codes for M/T/Q/P-LC Flash memories,” ArXiv and submitted to IEEE Trans.

• Inf. Theory, 2020.

➢ B. Dabak, A. Hareedy, and R. Calderbank, “Non-Binary Constrained Codes for Two- Dimensional Magnetic Recording,” ArXiv and submitted to IEEE Trans. Magn., 2020. ➢ A. Hareedy and R. Calderbank, “A new family of constrained codes with applications in data storage,” in Proc. IEEE ITW, 2019. ➢ A. Hareedy and R. Calderbank, “Asymmetric LOCO codes: Constrained codes for Flash memories,” in Proc. Allerton, 2019. ➢ A. Hareedy, B. Dabak, and R. Calderbank, “Q-ary asymmetric LOCO codes: Constrained codes supporting Flash evolution,” accepted at IEEE ISIT, 2020.

24

Our Related Work

➢ [R1] C. E. Shannon, “A mathematical theory of communication,” Bell Sys. Tech. J., 1948. ➢ [R2] D. T. Tang and R. L. Bahl, “Block codes for a class of constrained noiseless channels,”

• Inf. and Control, 1970.

➢ [R3] T. Cover, “Enumerative source encoding,” IEEE Trans. Inf. Theory, 1973. ➢ [R4] R. Adler, D. Coppersmith, and M. Hassner, “Algorithms for sliding block codes–An application of symbolic dynamics to information theory,” IEEE Trans. Inf. Theory, 1983. ➢ [R5] R. Karabed and P. H. Siegel, “Coding for higher-order partial-response channels,” in

• Proc. SPIE 2605, 1995.

➢ [R6] K. A. S. Immink, P. H. Siegel, and J. K. Wolf, “Codes for digital recorders,” IEEE Trans.

• Inf. Theory, 1998.

25

References (1 of 2)

➢ [R7] K. A. S. Immink, “A practical method for approaching the channel capacity of constrained channels,” IEEE Trans. Inf. Theory, 1997. ➢ [R8] V. Braun and K. A. S. Immink, “An enumerative coding technique for DC-free runlength-limited sequences,” IEEE Trans. Commun., 2000. ➢ [R9] V. Taranalli, H. Uchikawa, and P. H. Siegel, “Error analysis and inter-cell interference mitigation in multi-level cell flash memories,” in Proc. IEEE ICC, 2015. ➢ [R10] Y. M. Chee, J. Chrisnata, H. M. Kiah, S. Ling, T. T. Nguyen, and V. K. Vu, “Capacity- achieving codes that mitigate intercell interference and charge leakage in Flash memories,” IEEE Trans. Inf. Theory, 2019. ➢ [R11] M. Qin, E. Yaakobi, and P. H. Siegel, “Constrained codes that mitigate inter-cell interference in read/write cycles for flash memories,” IEEE J. Sel. Areas Commun., 2014.

26

References (2 of 2)

Thank You!