Computability of the Zero-Error capacity with Kolmogorov Oracle - - PowerPoint PPT Presentation

▶
computability of the zero error capacity with kolmogorov
SMART_READER_LITE
LIVE PREVIEW

Computability of the Zero-Error capacity with Kolmogorov Oracle - - PowerPoint PPT Presentation

Mar 08, 2024 9 likes •211 views

Computability of the Zero-Error capacity with Kolmogorov Oracle Holger Boche 1 and Christian Deppe 2 Technical University of Munich 1 Chair of Theoretical Information Technology 2 Institute for Communications Engineering ISIT 21-26 June 2020

slide-1
SLIDE 1

Computability of the Zero-Error capacity with Kolmogorov Oracle

Holger Boche1 and Christian Deppe2 Technical University of Munich

1 Chair of Theoretical Information Technology 2 Institute for Communications Engineering

ISIT 21-26 June 2020

Work was supported by Germany’s Excellence Strategy - EXC 2092 CASA - 390781972 (Boche) and BMBF through grant 16KIS1005 (Deppe)

slide-2
SLIDE 2

Content

  • Motivation - Status Quo
  • Zero-Error Capacity
  • Shannon Capacity
  • Computability
  • Kolmogorov Oracle
  • Computability of C0 with Kolmogorov Oracle
  • Zuiddam’s Characterization of C0 and Algorithmic Computability

Holger Boche (TUM) 2

slide-3
SLIDE 3

Motivation - Status Zero-Error Capacity C0

  • Introduced by Shannon in 1956, no formula for "computing" C0 is known today.
  • "Folklore theorem": Computation of C0 is a "hard problem"

(To compute the independence number of the strong product of the confusability graph is NP hard, but this result implies nothing about Turing computability of C0 or Θ). BUT:

  • It is unknown if C0 or the Shannon capacity for graphs Θ is Turing computable.
  • It is even unknown if Θ(G) is a computable number for all graphs G.

Plan: Investigate Turing computability in the framework of Turing with Oracle methods and analyze:

  • Turing computability of Θ.
  • Algorithmic behavior of Zuiddams recent characterization of Θ.

Holger Boche (TUM) 3

slide-4
SLIDE 4

Zero-Error Capacity

Definition

A discrete memoryless channel (DMC) is a triple (X, Y, W), where X is the finite input alphabet, Y is the finite output alphabet, and W(y|x) with x ∈ X , y ∈ Y is a stochastic matrix. The probability for a sequence yn ∈ Yn to be received if xn ∈ X n was sent is defined by W n(yn|xn) =

n

  • j=1

W(yj|xj).

Holger Boche (TUM) 4

slide-5
SLIDE 5

Zero-Error Capacity

  • Two sequences xn and x′n of size n of input variables are distinguishable by a receiver if the vectors

W n(·|xn) and W n(·|x′n) are orthogonal.

  • That means if W n(yn|xn) > 0 then W n(yn|x′n) = 0 and if W n(yn|x′n) > 0 then W n(yn|xn) = 0.
  • We denote by M(W, n) the maximum cardinality of a set of mutually orthogonal vectors among the

W n(·|xn) with xn ∈ X n.

The zero-error capacity of W is:

C0(W) = lim inf

n→∞

log2 M(W, n)

n

Holger Boche (TUM) 5

slide-6
SLIDE 6

Shannon Capacity of a Graph

  • Shannon introduced a simple graph GW.
  • In this graph two letters/vertices x and x′ are connected, if one could be confused with the other.
  • Therefore, the size of the maximum independent set α(GW) is the maximum number of 1-letter

messages which can be sent without danger of confusion.

  • The definition is extended to words of length n by α(G⊠n

W ).

Theorem (Shannon 1956)

2C0(W) = Θ(GW) = lim

n→∞ α(G⊠n W )

1 n.

No "computable" formula for Θ(GW) is known. Question: Is there an algorithm, that takes G and error µ as inputs and computes the number Θ(G) with precision µ?

Holger Boche (TUM) 6

slide-7
SLIDE 7

Ahlswede’s Question

In his paper 1970 Rudolf Ahlswede wrote “One would like to have a “reasonable” formula for C0, which does not “depend on an infinite product space.” Such a formula is unknown. An answer as: for given d there exists a k = k(d) such that N(nk, 0) = (N(k, 0))n” could be considered “reasonable”.” N(n, 0) denotes the maximal N for for which a zero-error code for n exists. Thus, the definition of N(n, 0) matches our definition of M(W, n) for a fixed channel W. If Ahlswede’s “resonable formula” were correct, we would of course have achieved Turing computability

  • immediately. Ahlswede’s question about a reasonable formula can be interpreted in the weakest form as a

question about Turing computablility.

Holger Boche (TUM) 7

slide-8
SLIDE 8

Turing Computability

  • The concept of a Turing machine is a mathematical model of an abstract machine that manipulates

symbols on a strip of tape according to certain given rules.

  • It can simulate any given algorithm and therewith provides a simple but very powerful model of

computation.

  • Turing machines have no limitations on computational complexity,unlimited computing capacity and

storage, and execute programs completely error-free.

  • They provide fundamental performance limits for digital computers and they are the ideal concept to

decide whether or not a function (here the zero-error capacity) is effectively computable.

Holger Boche (TUM) 8

slide-9
SLIDE 9

Computability

We would like to make statements about the computability of the zero-error capacity. This capacity is generally a real number. Therefore, we first define when a real number is computable.

Definition

A sequence of rational numbers {rn}n∈N0 is called a computable sequence if there exist recursive functions a, b, s : N0 → N0 with b(n) = 0 for all n ∈ N0 and rn = (−1)s(n)a(n) b(n), n ∈ N0. A real number x is said to be computable if there exists a computable sequence of rational numbers

{rn}n∈N0 such that |x − rn| < 2−n for all n ∈ N0. We denote the set of computable real numbers by Rc.

Holger Boche (TUM) 9

slide-10
SLIDE 10

Algorithmic Computability

  • We have a representation for C0(W) and Θ(G) as the limit of a monotonic convergent sequence1.
  • We do not have an effective estimate of the rate of convergence.
  • If we want to calculate the first bits of the binary representation of the number Θ(G) even for a fixed

graph G, this is not possible with only the result of Shannon.

  • An approach would now be, e.g. for the decimal representation, to derive the best possible lower bound

for Θ(G) from achievability part and derive a good upper bound for Θ(G) from an approach for the inverse part.

  • If the two bounds match for the first L decimal places, then we have determined Θ(G) for the first L

decimal places.

1 Note: There are well known and simple examples of computable monotonic increasing and boundes sequences of rational numbers such that the limit points are not computable numbers.

Holger Boche (TUM) 10

slide-11
SLIDE 11

Semi-Decidable

Definition

A subset G1 ⊂ G is called semi-decidable, if there is a Turing machine TM1 with the state “stop”, such that TM1(G) stops if and only if G ∈ G1. Therefore TM1 has only one stop state. If TM1 does not stop, it computes forever. A subset G1 ⊂ G is called decidable, if G1 and Gc

1 are semi decidable.

Lemma

Let λ ∈ Rc, λ ≥ 0. Then the set G(λ) :=

  • G ∈ G : Θ(G) > λ
  • is semi decidable.

Holger Boche (TUM) 11

slide-12
SLIDE 12

Kolmogorov Oracle

Definition

Let i be an enumeration of simple graphs and uN0 an optimal recursive listing of the set of natural numbers (Schorr). The Kolmogorov oracle OK,G(·) is a function from N0 to the power set of the set of graphs that produces a list OK,G(n) :=

  • G : CuG(G) ≤ n
  • for each n ∈ N0, where CuG(G) := min{k : i(uN0(k)) = G}.

According to our definition of graphs and the set G with the listing i, this is the same as the listing OK,N0 of the natural numbers k with CuN0(k) ≤ n.

Holger Boche (TUM) 12

slide-13
SLIDE 13

Computability with Kolmogorov Oracle

  • We say that TM can use the oracle OK,G if for every n ∈ N0, on input n the Turing machine gets the list

OK,G(n).

  • With TM(OK,G) we denote a Turing Machine that has access to the Oracle OK,G.

Theorem

Let λ ∈ Rc, λ > 0, then the set G(λ) is decidable with a Turing machine TM∗(OK,G). This means there exists a Turing machine TM∗(OK,G), such that the set G(λ) is computable with this Turing machine with oracle.

Holger Boche (TUM) 13

slide-14
SLIDE 14

Alon’s Question

Corollary

Let λ ∈ Rc, λ ≥ 0. Then, the set {G : Θ(G) ≤ λ} is semi-decidable for Turning machines with oracle OK,N0,

  • racle OK,G, respectively.
  • 1. Noga Alon has asked 2006 if the set {G : Θ(G) ≤ λ} is semi-decidable. We gave a positive answer to

this question if we can include the oracle.

  • 2. We do not know if C0 is computable concerning TM(OK,G).

Holger Boche (TUM) 14

slide-15
SLIDE 15

Effective Converse

Theorem

The Shannon capacity and thus the function Θ is Turing computable if and only if there is a computable sequence {FN}N∈N0 of computable functions FN : G → Rc, so that the following conditions apply:

  • 1. For all N ∈ N0 holds FN(G) ≥ Θ(G) for all G ∈ G.
  • 2. limN→∞ FN(G) = Θ(G) for all G ∈ G.

Holger Boche (TUM) 15

slide-16
SLIDE 16

Zuiddam’s Characterization

  • In order to prove the computability of C0 or Θ, we need computable converses in the sense of previous

Theorem.

  • Therefore, the characterization of Zuiddam use the functions from the asymptotic spectrum of graphs is

interesting.

  • We examine this approach with regard to computability.

Holger Boche (TUM) 16

slide-17
SLIDE 17

Zuiddam’s Characterization

  • The result of Zuiddam is a new dual characterisation of the Shannon capacity of graphs.
  • This characterisation is obtained by applying Strassen’s theory of asymptotic spectra.
  • We denote by

∼ the asymptotic preorder on Graphs induced by the Strassen preorder.

  • X(G) = X(G,

∼) denotes the asymptotic spectrum of (S, ) (the set of -monotone semiring

homomorphisms from S to R≥0).

Theorem (Zuiddam 2019)

G is a collection of graphs which is closed under the disjoint union ⊔ and the strong graph product ⊠, and

which contains the graph with a single vertex, K1. Then we have

  • 1. G

∼ H iff ∀φ ∈ X(G) : φ(G) ≤ φ(H)

  • 2. Θ(G) = minφ∈X(G) φ(G).

Holger Boche (TUM) 17

slide-18
SLIDE 18

Zuiddam’s Characterization

  • If all φ ∈ X(G) have the property that they can be computed as functions φ : G → Rc and if we find a

recursive subset such that the minimization over this subset gives the Shannon capacity, then we could immediately prove that for G ∈ G always Θ(G) ∈ Rc applies, which is still open up to now.

  • The proof of Zuiddam’s Theorem is not constructive. The Zorn lemma is needed.

Holger Boche (TUM) 18

slide-19
SLIDE 19

Computability of

∼

Our goal is to use a powerful oracle, such that with the help of the oracle there exists a Turing machine which computes the binary relation

∼.

Definition

Let φk, k ∈ N0, the list of partial recursive Functions. φk is called total, if the domain of φk equals N0. Let Tot = {k ∈ N0 : φk is total function}. Then OTot is defined as the following oracle. In the calculation step l, TM asks the oracle if k ∈ OTot is satisfied. TM receives in one calculation step the answer yes or no. The Turing machine uses this answer for the next computation, etc. New queries can always be made to the

  • racle.

Theorem

  • ∼ is as a binary operation decidable by a Turing machine TM(·, OTot).

Holger Boche (TUM) 19

slide-20
SLIDE 20

Conclusions

  • It is unknown if the zero-error capacity is computable in general, even the weakest requirement that

Θ(G) ∈ Rc for all G ∈ G is unknown to be true.

  • We show that in general the zero-error capacity is semi computable with the help of a Kolmogorov

Oracle.

  • We show that C0 and Θ are computable functions if and only if there is a computable sequence of

computable functions of upper bounds, i.e. the converse exist in the sense of information theory, which pointwise converge to C0 or Θ.

  • We examine Zuiddam’s characterization of C0 and Θ in terms of algorithmic computability.
  • The full paper is available on arXiv.org > cs > arXiv:2001.11442

Holger Boche (TUM) 20