Integer-Forcing for Channels, Sources and ADCs Or Ordentlich Tel - - PowerPoint PPT Presentation

Integer-Forcing for Channels, Sources and ADCs

Or Ordentlich Tel Aviv University

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N) ML decoding is difficult

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N) ML decoding is difficult Successive decoding/onion peeling only works for unbalanced channel gains

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N) ML decoding is difficult Successive decoding/onion peeling only works for unbalanced channel gains Traditional linear equalizers (ZF/MMSE) are useless: more variables than observations

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N)

Integer-forcing approach:

An almost linear solution: First decode two linear combinations of x1 and x2 Then recover x1 and x2

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N)

Observation

Since x1 ∈ 2R − PAM ⊂ Z and x2 ∈ 2R − PAM ⊂ Z, then a1x1 + a2x2 ∈ Z for any two integers a1 and a2

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Motivating Example

y = x1 + √ 2x2 + z x1 ∈ 2R − PAM x2 ∈ 2R − PAM z ∼ N(0, N)

Observation

Since x1 ∈ 2R − PAM ⊂ Z and x2 ∈ 2R − PAM ⊂ Z, then a1x1 + a2x2 ∈ Z for any two integers a1 and a2

Decoding a linear combination with integer coefficients is easy

Just scale y such that both channel coefficients are close to integers, and round the result to the nearest integer

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2 Assume for the example that x1 = 2, x2 = 1 and z = 1/4 y = x1 + √ 2x2 +z

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2 Assume for the example that x1 = 2, x2 = 1 and z = 1/4 y = x1 + √ 2x2 +z Step 1: choose a scalar α such that α[1 √ 2] ≈ [1 1] and compute ˜ y = αy

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2 Assume for the example that x1 = 2, x2 = 1 and z = 1/4 y = x1 + √ 2x2 +z Step 1: choose a scalar α such that α[1 √ 2] ≈ [1 1] and compute ˜ y = αy αy = αx1 +α √ 2x2+αz

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2 Assume for the example that x1 = 2, x2 = 1 and z = 1/4 y = x1 + √ 2x2 +z Step 1: choose a scalar α such that α[1 √ 2] ≈ [1 1] and compute ˜ y = αy Step 2: Quantize ˜ y = αy to the nearest integer αy = αx1 +α √ 2x2+αz

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Decoding an Equation

y = 1x1 + √ 2x2 + z desired equation is v = 1x1 + 1x2 Assume for the example that x1 = 2, x2 = 1 and z = 1/4 y = x1 + √ 2x2 +z Step 1: choose a scalar α such that α[1 √ 2] ≈ [1 1] and compute ˜ y = αy Step 2: Quantize ˜ y = αy to the nearest integer αy = αx1 +α √ 2x2+αz ˆ v

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

We have αy = a1x1 + a2x2 + αy − a1x1 − a2x2 = v + (α · 1 − 1)x1 + (α √ 2 − 1)x2 + αz = v + zeff where zeff =

self noise

• (α · 1 − 1)x1 + (α

√ 2 − 1)x2 +

Gaussian noise

• αz

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

We have αy = a1x1 + a2x2 + αy − a1x1 − a2x2 = v + (α · 1 − 1)x1 + (α √ 2 − 1)x2 + αz = v + zeff where zeff =

self noise

• (α · 1 − 1)x1 + (α

√ 2 − 1)x2 +

Gaussian noise

• αz

Definition: effective variance

σ2

eff(α, a) = E(z2 eff) = αh − a2P + α2N

h is the channel coefficients vector a is the integer coefficients vector P is the average transmission power of all users

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

σ2

eff(α, a) = αh − a2P + α2N

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

σ2

eff(α, a) = αh − a2P + α2N

Lemma : error probability

For an appropriately scaled and shifted 2R − PAM constellation the error probability in decoding v is upper bounded by Pr(ˆ v = v) < 2 exp

• −3

222( 1

2 log(P/σ2 eff(α,a))−R)

• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

σ2

eff(α, a) = αh − a2P + α2N

Lemma : error probability

For an appropriately scaled and shifted 2R − PAM constellation the error probability in decoding v is upper bounded by Pr(ˆ v = v) < 2 exp

• −3

222( 1

2 log(P/σ2 eff(α,a))−R)

• The optimal α is the MMSE coefficient for estimating aTx from y. With

this choice we have σ2

eff(a) = PaT

• I + P

N hTh −1 a

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

σ2

eff(α, a) = αh − a2P + α2N

Lemma : error probability

For an appropriately scaled and shifted 2R − PAM constellation the error probability in decoding v is upper bounded by Pr(ˆ v = v) < 2 exp

• −3

222( 1

2 log(P/σ2 eff(α,a))−R)

• The effective SNR for decoding v = aTx is

SNReff(a) P σ2

eff(a) =

• aT

I + SNRhTh −1 a −1

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Error Probability for Decoding an Equation

σ2

eff(α, a) = αh − a2P + α2N

Lemma : error probability

For an appropriately scaled and shifted 2R − PAM constellation the error probability in decoding v is upper bounded by Pr(ˆ v = v) < 2 exp

• −3

222( 1

2 log(P/σ2 eff(α,a))−R)

• The effective SNR for decoding v = aTx is

SNReff(a) P σ2

eff(a) =

• aT

I + SNRhTh −1 a −1 The receiver does not care about the linear combinations themselves = ⇒Can choose a1, a2 = arg maxa1,a2 min (SNReff(a1), SNReff(a2))

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Performance Gain w.r.t. Linear Equalization

For the channel y = 1x1 + √ 2x2 + z with SNR = 10dB the best equations are a1 = [1 1] and a2 = [1 2]. For these choices we have SNReff(a1) = 8.34 and SNReff(a2) = 3.67

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Performance Gain w.r.t. Linear Equalization

For the channel y = 1x1 + √ 2x2 + z with SNR = 10dB the best equations are a1 = [1 1] and a2 = [1 2]. For these choices we have SNReff(a1) = 8.34 and SNReff(a2) = 3.67 The linear MMSE equalizer is equivalent to IF with a1 = [1 0] and a2 = [0 1]. For these suboptimal choices we have SNReff(a1) = 1.47 and SNReff(a2) = 2.82

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Performance Gain w.r.t. Linear Equalization

For the channel y = 1x1 + √ 2x2 + z with SNR = 10dB the best equations are a1 = [1 1] and a2 = [1 2]. For these choices we have SNReff(a1) = 8.34 and SNReff(a2) = 3.67 The linear MMSE equalizer is equivalent to IF with a1 = [1 0] and a2 = [0 1]. For these suboptimal choices we have SNReff(a1) = 1.47 and SNReff(a2) = 2.82 The IF approach improves upon ZF/MMSE by allowing the receiver to decode linear combinations with coefficients that match the channel gains

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Performance Gain w.r.t. Linear Equalization

For the channel y = 1x1 + √ 2x2 + z with SNR = 10dB the best equations are a1 = [1 1] and a2 = [1 2]. For these choices we have SNReff(a1) = 8.34 and SNReff(a2) = 3.67 The linear MMSE equalizer is equivalent to IF with a1 = [1 0] and a2 = [0 1]. For these suboptimal choices we have SNReff(a1) = 1.47 and SNReff(a2) = 2.82 The IF approach improves upon ZF/MMSE by allowing the receiver to decode linear combinations with coefficients that match the channel gains How large is the improvement? How large is the gap from ML decoding?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Some Comments

For uncoded PAM transmission IF is (almost) the same as lattice reduction aided MIMO equalization (Yao & Wornell 02)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Some Comments

For uncoded PAM transmission IF is (almost) the same as lattice reduction aided MIMO equalization (Yao & Wornell 02) Linear/lattice codes are closed under integer-valued linear combinations − →Can use them instead of PAM

Theorem (Nazer-Gastpar11IT)

If all users transmit from the same capacity achieving nested-lattice code with rate R, then v = a1x1 + a2x2 can be decoded with a vanishing error probability if R < Rcomp(a) 1 2 log (SNReff(a))

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Some Comments

If the receiver can decode a full-rank set of integer-linear combinations, it can decode all codewords

Theorem : (Zhan et al. ISIT2010)

For two linearly independent integer vectors a1 and a2 the IF receiver can decode both messages if both users transmitted from the same lattice code with rate R < min 1 2 log (SNReff(a1)) , 1 2 log (SNReff(a2))

• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Applications for Channel Coding Problems

Joint work with Uri Erez (TAU) and Bobak Nazer (BU)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

Consider the MAC channel y = h1x1 + h2x2 + z where xi2 ≤ nSNR and z is AWGN with zero mean and unit variance

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

Consider the MAC channel y = h1x1 + h2x2 + z where xi2 ≤ nSNR and z is AWGN with zero mean and unit variance Capacity region is achieved using i.i.d. Gaussian codebooks

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

Consider the MAC channel y = h1x1 + h2x2 + z where xi2 ≤ nSNR and z is AWGN with zero mean and unit variance Capacity region is achieved using i.i.d. Gaussian codebooks

Strange question

What is the maximal achievable symmetric rate R, if both users transmit from the same linear/lattice code?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

Consider the MAC channel y = h1x1 + h2x2 + z where xi2 ≤ nSNR and z is AWGN with zero mean and unit variance Capacity region is achieved using i.i.d. Gaussian codebooks

Strange question

What is the maximal achievable symmetric rate R, if both users transmit from the same linear/lattice code?

Strange question, strange answer

R depends on the “rationality” of the channel gains

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

1 2

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes

1 √ 2

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Analyzing a random ensemble of linear codes with ML decoding is very complicated, but... IF is applicable when all users transmit from the same linear code − → The IF rate is a lower bound on R

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Analyzing a random ensemble of linear codes with ML decoding is very complicated, but... IF is applicable when all users transmit from the same linear code − → The IF rate is a lower bound on R But how does the IF rate behave as a function of the channel gains?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Consider the channel y = 1x1 + gx2 + z at SNR = 40dB

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Consider the channel y = 1x1 + gx2 + z at SNR = 40dB The IF receiver has to decode two linearly independent equations with inte- ger coefficients

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Consider the channel y = 1x1 + gx2 + z at SNR = 40dB Normalized computation rate for best coefficient vector a1

1 1.2 1.4 1.6 1.8 2 0.2 0.4 0.6 0.8 1 1.2 1.4 g Normalized Computation Rate First Equation

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Consider the channel y = 1x1 + gx2 + z at SNR = 40dB Normalized computation rate for second best coefficient vector a2

1 1.2 1.4 1.6 1.8 2 0.2 0.4 0.6 0.8 1 1.2 1.4 g Normalized Computation Rate First Equation Second Equation

The red curve is a lower bound on R

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Lower Bound

Consider the channel y = 1x1 + gx2 + z at SNR = 40dB Normalized sum of computation rates

1 1.2 1.4 1.6 1.8 2 0.2 0.4 0.6 0.8 1 1.2 1.4 g Normalized Computation Rate First Equation Second Equation Sum

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Rate Region

Using Minkowski’s second theorem we get the following result

Theorem

The sum of optimal computation rates is lower bounded by Rcomp,1 + Rcomp,2 ≥ 1 2 log

• 1 + h2SNR
• − 1 bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Rate Region

Using Minkowski’s second theorem we get the following result

Theorem

The sum of optimal computation rates is lower bounded by Rcomp,1 + Rcomp,2 ≥ 1 2 log

• 1 + h2SNR
• − 1 bits

With nested lattice codes each computation rate can be associated with an individual rate for one of the users (not trivial)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Rate Region

Using Minkowski’s second theorem we get the following result

Theorem

The sum of optimal computation rates is lower bounded by Rcomp,1 + Rcomp,2 ≥ 1 2 log

• 1 + h2SNR
• − 1 bits

With nested lattice codes each computation rate can be associated with an individual rate for one of the users (not trivial) The sum rate with nested lattice codes is at most 1 bit smaller than the MAC sum capacity

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Rate Region

Using Minkowski’s second theorem we get the following result

Theorem

The sum of optimal computation rates is lower bounded by Rcomp,1 + Rcomp,2 ≥ 1 2 log

• 1 + h2SNR
• − 1 bits

With nested lattice codes each computation rate can be associated with an individual rate for one of the users (not trivial) The sum rate with nested lattice codes is at most 1 bit smaller than the MAC sum capacity

Theorem

With successive IF the sum of 2 optimal computation rates equals the sum-capacity of the MAC

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Gaussian MAC with Nested Linear Codes - IF Rate Region

Gaussian two-user MAC y = 1x1 + √ 2x2 + z at SNR = 15dB R1 2.51 1.85 1.44 0.28 R2 3.00 1.85 1.44 0.77

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

w1 E1 x1 1 g g w2 E2 x2 1 g g

. . . . . .

wK EK xK 1 g g z1 y1 z2 y2 zK yK D1 ˆ w1 D2 ˆ w2 DK ˆ wK yk = xk + g

• m=k

xm + zk

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

w1 E1 x1 1 g g w2 E2 x2 1 g g

. . . . . .

wK EK xK 1 g g z1 y1 z2 y2 zK yK D1 ˆ w1 D2 ˆ w2 DK ˆ wK yk = xk + g

• m=k

xm + zk What is the symmetric capacity Csym of this channel?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

If all users transmit from the same nested lattice codebook we get xint,k =

• m=k

xm ∈ Λ

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

If all users transmit from the same nested lattice codebook we get xint,k =

• m=k

xm ∈ Λ Same lattice = ⇒ Interference alignment

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

If all users transmit from the same nested lattice codebook we get xint,k =

• m=k

xm ∈ Λ Same lattice = ⇒ Interference alignment Each receiver sees an effective 2-user MAC yk = xk + gxint,k + zk, xk, xint,k ∈ Λ

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Application - Symmetric Gaussian K-User IC

If all users transmit from the same nested lattice codebook we get xint,k =

• m=k

xm ∈ Λ Same lattice = ⇒ Interference alignment Each receiver sees an effective 2-user MAC yk = xk + gxint,k + zk, xk, xint,k ∈ Λ Can apply our MAC results and obtain lower bound on Csym

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

10

1

1 2 3 4 5 3−user IC @ SNR=30dB g symmetric rate[bits/channel use]

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

10

1

1 2 3 4 5 3−user IC @ SNR=30dB g symmetric rate[bits/channel use]

Question

For γ > 0 bits, what is the fraction of channel gains g for which upper bound − lower bound > γ bits?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Outage Set

10

1

1 2 3 4 5 3−user IC @ SNR=30dB g symmetric rate[bits/channel use]

Strong interference regime

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Outage Set

10

1

1 2 3 4 5 3−user IC @ SNR=30dB g symmetric rate[bits/channel use]

γ = 0.25 bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Outage Set

48% outage for γ = 0.25 bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Outage Set

22% outage for γ = 0.5 bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Outage Set

11% outage for γ = 0.75 bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

Theorem - inner bound for the strong interference regime

The fraction of channel gains for which upper bound − lower bound > 3 + γ 2 bits is smaller than 2−γ

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

Theorem - inner bound for the strong interference regime

The fraction of channel gains for which upper bound − lower bound > 3 + γ 2 bits is smaller than 2−γ

Etkin and E. Ordentlich 09

The DoF of the symmetric Gaussian K-user IC is discontinuous at rational values of g

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

Theorem - inner bound for the strong interference regime

The fraction of channel gains for which upper bound − lower bound > 3 + γ 2 bits is smaller than 2−γ

Etkin and E. Ordentlich 09

The DoF of the symmetric Gaussian K-user IC is discontinuous at rational values of g It appears the the notches in the achievable rate region are inherent to the problem

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

• Approx. Symmetric Capacity: Strong Interference Regime

Theorem - inner bound for the strong interference regime

The fraction of channel gains for which upper bound − lower bound > 3 + γ 2 bits is smaller than 2−γ

Etkin and E. Ordentlich 09

The DoF of the symmetric Gaussian K-user IC is discontinuous at rational values of g It appears the the notches in the achievable rate region are inherent to the problem

What about weak interference (|g| < 1)?

Can use a lattice Han-Kobayashi scheme with IF decoding

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

The Symmetric Gaussian K-User IC: New Inner Bounds

10

−1

10 10

1

0.5 1 1.5 2 2.5 3 3.5

SNR=20dB

g Symmetric Rate

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

The Symmetric Gaussian K-User IC: New Inner Bounds

10

−2

10 10

2

1 2 3 4 5 6

SNR=35dB

g Symmetric Rate

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

The Symmetric Gaussian K-User IC: New Inner Bounds

10

−2

10 10

2

2 4 6 8

SNR=50dB

g Symmetric Rate

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

The Symmetric Gaussian K-User IC: New Inner Bounds

10

−2

10 10

2

2 4 6 8 10

SNR=65dB

g Symmetric Rate

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Applications for Source Coding Problems

Joint work with Uri Erez (TAU)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Distributed Lossy Compression

x1 E1 R1 . . . xK EK RK D (ˆ x1, d1) . . . (ˆ xK, dK)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Distributed Lossy Compression

x1 E1 R1 . . . xK EK RK D (ˆ x1, d1) . . . (ˆ xK, dK) The fundamental limits were studied. Inner and outer bounds exist, and they even sometimes agree

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Distributed Lossy Compression

x1 E1 R1 . . . xK EK RK D (ˆ x1, d1) . . . (ˆ xK, dK) The fundamental limits were studied. Inner and outer bounds exist, and they even sometimes agree However, in some applications the encoders/decoder are required to be extremely simple

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Distributed Lossy Compression

x1 E1 R1 . . . xK EK RK D (ˆ x1, d1) . . . (ˆ xK, dK) We restrict attention to One-shot compression - block length is 1 Jointly Gaussian source x ∼ N (0, Kxx) Symmetric rates and distortions - R1 = · · · = RK = R and d1 = · · · = dK = d MSE distortion measure: E(xk − ˆ xk)2 ≤ d

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Basic Idea

Rather than solving the problem x1 E1 R . . . xK EK R D ˆ x1 . . . ˆ xK

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Basic Idea

First solve x1 E1 R . . . xK EK R D

• K

m=1 a1mxm

. . .

• K

m=1 aKmxm

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Basic Idea

First solve x1 E1 R . . . xK EK R D

• K

m=1 a1mxm

. . .

• K

m=1 aKmxm

This can be done with low complexity if all coefficients are integers and R is proportional to maxk

• Var

K

m=1 akmxm

• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Basic Idea

First solve x1 E1 R . . . xK EK R D

• K

m=1 a1mxm

. . .

• K

m=1 aKmxm

This can be done with low complexity if all coefficients are integers and R is proportional to maxk

• Var

K

m=1 akmxm

• If x1, . . . , xK are sufficiently correlated we can find K linearly

independent integer-valued coefficient vectors with small variance

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Basic Idea

First solve x1 E1 R . . . xK EK R D

• K

m=1 a1mxm

. . .

• K

m=1 aKmxm

This can be done with low complexity if all coefficients are integers and R is proportional to maxk

• Var

K

m=1 akmxm

• If x1, . . . , xK are sufficiently correlated we can find K linearly

independent integer-valued coefficient vectors with small variance Invert equation to get ˆ x1, . . . , ˆ xK

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Each signal can be written as xk = x∗

k + Mk∆

where x∗

k ∈

• − ∆

2 , ∆ 2

• and Mk ∈ Z

−3∆ −2∆ −∆ ∆ 2∆ 3∆

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Each signal can be written as xk = x∗

k + Mk∆

where x∗

k ∈

• − ∆

2 , ∆ 2

• and Mk ∈ Z

−3∆ −2∆ −∆ ∆ 2∆ 3∆

Simple modulo property

For any set of integers a1, . . . , aK K

• k=1

akxk ∗ = K

• k=1

akx∗

k

∗ .

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

K

• k=1

akxk ∗ = K

• k=1

akx∗

k

∗ If we know that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• , then

K

• k=1

akxk = K

• k=1

akx∗

k

∗

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

K

• k=1

akxk ∗ = K

• k=1

akx∗

k

∗ If we know that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• , then

K

• k=1

akxk = K

• k=1

akx∗

k

∗ It suffices to compress x∗

1, . . . , x∗ K for estimating K k=1 akxk

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

K

• k=1

akxk ∗ = K

• k=1

akx∗

k

∗ If we know that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• , then

K

• k=1

akxk = K

• k=1

akx∗

k

∗ It suffices to compress x∗

1, . . . , x∗ K for estimating K k=1 akxk

If x is a Gaussian vector, a can be chosen to minimize the variance of aTx. This maximizes the probability that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

K

• k=1

akxk ∗ = K

• k=1

akx∗

k

∗ If we know that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• , then

K

• k=1

akxk = K

• k=1

akx∗

k

∗ It suffices to compress x∗

1, . . . , x∗ K for estimating K k=1 akxk

If x is a Gaussian vector, a can be chosen to minimize the variance of aTx. This maximizes the probability that K

k=1 akxk ∈

• − ∆

2 , ∆ 2

• After estimating K linearly independent combinations, one can estimate

each of the individual signals

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing encoder

Quantizes xk to the nearest point in the lattice Λf = √ 12dZ and sends the index of this point modulo the lattice Λ = 2R√ 12dZ − → The equivalent signal is ˜ xk = [xk + dk]∗ where dk ∼ Unif

• −

√ 12d 2

,

√ 12d 2

• and [·]∗ is with respect to ∆ = 2R√

12d

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing encoder

Quantizes xk to the nearest point in the lattice Λf = √ 12dZ and sends the index of this point modulo the lattice Λ = 2R√ 12dZ − → The equivalent signal is ˜ xk = [xk + dk]∗ where dk ∼ Unif

• −

√ 12d 2

,

√ 12d 2

• and [·]∗ is with respect to ∆ = 2R√

12d −3∆ −2∆ −∆ ∆ 2∆ 3∆ xk

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing encoder

Quantizes xk to the nearest point in the lattice Λf = √ 12dZ and sends the index of this point modulo the lattice Λ = 2R√ 12dZ − → The equivalent signal is ˜ xk = [xk + dk]∗ where dk ∼ Unif

• −

√ 12d 2

,

√ 12d 2

• and [·]∗ is with respect to ∆ = 2R√

12d −3∆ −2∆ −∆ ∆ 2∆ 3∆ xk QΛf (xk)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing encoder

Quantizes xk to the nearest point in the lattice Λf = √ 12dZ and sends the index of this point modulo the lattice Λ = 2R√ 12dZ − → The equivalent signal is ˜ xk = [xk + dk]∗ where dk ∼ Unif

• −

√ 12d 2

,

√ 12d 2

• and [·]∗ is with respect to ∆ = 2R√

12d −3∆ −2∆ −∆ ∆ 2∆ 3∆ xk QΛf (xk) ˜ xk

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing encoder

Quantizes xk to the nearest point in the lattice Λf = √ 12dZ and sends the index of this point modulo the lattice Λ = 2R√ 12dZ − → The equivalent signal is ˜ xk = [xk + dk]∗ where dk ∼ Unif

• −

√ 12d 2

,

√ 12d 2

• and [·]∗ is with respect to ∆ = 2R√

12d −3∆ −2∆ −∆ ∆ 2∆ 3∆ xk QΛf (xk) ˜ xk modulo = one dimensional binning

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing decoder

Computes K estimates for integer linear combinations

• aT

mx =

K

• k=1

amk˜ xk ∗ = K

• k=1

amk(xk + dk) ∗ , m = 1, . . . , K

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding

Integer-forcing decoder

Computes K estimates for integer linear combinations

• aT

mx =

K

• k=1

amk˜ xk ∗ = K

• k=1

amk(xk + dk) ∗ , m = 1, . . . , K If K

• k=1

amk(xk + dk) ∗ =

K

• k=1

amk(xk + dk), m = 1, . . . , K the decoder can invert the equations and get xk + dk for k = 1, . . . , K

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

The decoder succeeds if

K

• m=1
• K
• k=1

amk(xk + dk)

• < ∆

2

• =

K

• m=1
• K
• k=1

amk(xk + dk)

• ≥ ∆

2

• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

The decoder succeeds if

K

• m=1
• K
• k=1

amk(xk + dk)

• < ∆

2

• =

K

• m=1
• K
• k=1

amk(xk + dk)

• ≥ ∆

2

• E(wm) = 0 and σ2

wm = aT m (Kxx + dI) am

Chernoff’s bound gives Pr (|wm| > τ) ≤ 2e

−

τ2 2σ2 wm Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

The decoder succeeds if

K

• m=1
• K
• k=1

amk(xk + dk)

• < ∆

2

• =

K

• m=1
• K
• k=1

amk(xk + dk)

• ≥ ∆

2

• E(wm) = 0 and σ2

wm = aT m (Kxx + dI) am

Chernoff’s bound gives Pr (|wm| > τ) ≤ 2e

−

τ2 2σ2 wm

Substituting ∆ = 2R√ 12d and applying the union bound gives Pe ≤ 2K exp   −3 22

2

• R− 1

2 log

• maxm=1,...,K aT

m(Kxx+dI)am d



 

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

Pe ≤ 2K exp   −3 22

2

• R− 1

2 log

• maxm=1,...,K aT

m(Kxx+dI)am d



 

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

Pe ≤ 2K exp   −3 22

2

• R− 1

2 log

• maxm=1,...,K aT

m(Kxx+dI)am d



  Let RIF(A, d) 1 2 log

• max

m=1,...,K aT m

• I + 1

d Kxx

• am
• Or Ordentlich

Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

Pe ≤ 2K exp   −3 22

2

• R− 1

2 log

• maxm=1,...,K aT

m(Kxx+dI)am d



  Let RIF(A, d) 1 2 log

• max

m=1,...,K aT m

• I + 1

d Kxx

• am
• Theorem

Let R = RIF(A, d) + δ such that 2R is a positive integer. IF source coding produces estimates with average MSE distortion d for all x1, . . . , xK with probability greater than 1 − 2K exp

• − 3

222δ

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Error Probability

Pe ≤ 2K exp   −3 22

2

• R− 1

2 log

• maxm=1,...,K aT

m(Kxx+dI)am d



  Let RIF(A, d) 1 2 log

• max

m=1,...,K aT m

• I + 1

d Kxx

• am
• Theorem

Let R = RIF(A, d) + δ such that 2R is a positive integer. IF source coding produces estimates with average MSE distortion d for all x1, . . . , xK with probability greater than 1 − 2K exp

• − 3

222δ

But... is RIF(A, d) any good?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Example

Jointly Gaussian vector x ∼ N (0, Kxx) with Kxx = I + SNRHHT and all entries of H are i.i.d. N(0, 1)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Example

Jointly Gaussian vector x ∼ N (0, Kxx) with Kxx = I + SNRHHT and all entries of H are i.i.d. N(0, 1) Models compress-and-forward for Rayleigh fading MIMO channel w1 Tx 1 s1 . . . wK Tx K sK H x1 z1

Relay 1 R0

. . . xK zK

Relay K R0

CP ˆ w1 . . . ˆ wK Distributed SC

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding - Example

x ∼ N (0, Kxx), Kxx = I + SNRHHT , SNR = 20dB and H ∈ R8×2

−40 −30 −20 −10 10 20 2 4 6 8 10 E(R)[bits] d[dB] Naive compression Successive Wyner−Ziv coding RIF(d) Berger−Tung Benchmark

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Another Look

x1 E1 R . . . xK EK R D (ˆ x1, d) . . . (ˆ xK, d) So what does Ek boils down to?

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Another Look

x1 E1 R . . . xK EK R D (ˆ x1, d) . . . (ˆ xK, d) So what does Ek boils down to? Just rounding to the nearest integer and then reducing modulo 2R (up to scaling)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Integer-Forcing Source Coding: Another Look

x1 E1 R . . . xK EK R D (ˆ x1, d) . . . (ˆ xK, d) So what does Ek boils down to? Just rounding to the nearest integer and then reducing modulo 2R (up to scaling) For an analog signal, the rounding operation is implemented by an Analog-to-Digital converter (ADC)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Modulo ADC for Spatially Correlated Signals

The rounding operation requires many bits that get thrown away eventually

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Modulo ADC for Spatially Correlated Signals

The rounding operation requires many bits that get thrown away eventually If 2R is an odd integer the order can be switched: first reduce modulo 2R and then round to the nearest integer − →The ADC only needs to produce R bits

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Modulo ADC for Spatially Correlated Signals

The rounding operation requires many bits that get thrown away eventually If 2R is an odd integer the order can be switched: first reduce modulo 2R and then round to the nearest integer − →The ADC only needs to produce R bits For an ADC producing less bits = less power consumption

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Modulo ADC for Spatially Correlated Signals

x modΛ . . . . . . QΛf (·) Modulo ADC

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Modulo ADC for Spatially Correlated Signals

x modΛ . . . . . . QΛf (·) Modulo ADC Possible application: The MIMO receiver w1 Tx 1 s1 . . . wK Tx KsK H x1 z1

ADC 1 R0

. . . xK zK

ADC K R0

Decoder ˆ w1 . . . ˆ wK Rx

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Extensions and Summary

More results

Precoded integer-forcing universally achieves the MIMO capacity to within a constant gap (Ordentlich-Erez13) Cyclic-coded integer-forcing equalization for ISI channels (Ordentlich-Erez12IT)

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Extensions and Summary

More results

Precoded integer-forcing universally achieves the MIMO capacity to within a constant gap (Ordentlich-Erez13) Cyclic-coded integer-forcing equalization for ISI channels (Ordentlich-Erez12IT)

Summary

Integer-forcing can often achieve symmetric rates close to the MAC symmetric capacity Integer-forcing can be applied for approximating the Gaussian symmetric K-user IC symmetric capacity We presented a new low-complexity distributed lossy compression scheme based on integer-forcing Potentially, the encoder in this scheme can be implemented by ADCs

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs

Thanks for your attention!

Or Ordentlich Integer-Forcing for Channels, Sources and ADCs