[PDF] - F ACTOR graph is a popular graphical model to represent where the PDF Document

SLIDE 1

FIRST YEAR REPORT 1

Generalization of Factor Graphs and Belief Propagation for Quantum Information Science

Michael X. CAO

Abstract—Factor graph is a useful tool to represent probability

systems. In this article, we propose a new factor graph model

in representing quantum systems. Belief propagation algorithm is also generalized to this new model, and is justified via the method of loop calculus. Index Terms—Factor Graph, Quantum Probabilities, Belief Propagation, Loop Calculus

I. INTRODUCTION

F

ACTOR graph is a popular graphical model to represent factorizations [1], [2], and has been proven practically useful in describing probability systems. Famous applications include Ising model [3], LDPC codes [4] and Turbo codes. Recent studies have generalized factor graph to represent quantum probabilities [5], [6]. Related research also include [7], where bipartite factor graph is proposed to represent quantum systems (i.e., quantum measures). In this paper, we propose a new factor graph model, namely quantum (normal) factor graph (QFG or QNFG), to represent quantum systems. The concerned global function is in form

f

g (x, x′; y) =

a∈F

fa (x∂a, x′

∂a; yδa)

(1) where for any fixed yδa, induced function f yδa

a

(x∂a, x′

∂a)

is Hermitian positive semi-definite (PSD), i.e., the matrix [f yδa

a

]x∂a,x′

∂a is PSD. Whereas, in [5], it is required for each

fa to be decomposable, i.e., fa (x∂a, x′

∂a; yδa) = ˜

fa (x∂a; yδa) ˜ fa (x′

∂a; yδa)

(2) In this sense, our model is more general. The partition sum problem, which is ubiquitous in many problems modeled by classical factor graphs, is also consid- ered for QFGs. In particular, we consider a generalized version

f belief propagation (BP) algorithm for QFGs. The major

approaches applied here include loop calculus [8], [9] and cluster variation methods [10]. The rest of this article is organized as follows. Section II gives a tour from the classical factor graphs to the quantum factor graphs as our proposal. Section III provides several examples in elementary quantum mechanics described by

QNFGs. Section IV introduces the sum-product algorithm and

the belief propagation for QFGs. The derivation of the loop calculus for belief propagation for QFGs is also contained in this section. Such derivation is based on Mori’s idea [7], [8]. Section V shows our partial result by applying cluster variation method to QFG. Section VI concludes the paper. In this article, for finite alphabet X, we denote Lh (X) the inner product space of Hermitian linear operators acting on X, p uA uB bB bA qB z x z′ y

Fig. 1: The factor graph for the factorization g (x, y, z, z′) =

p (x) uA (x, z) uB (x, z′) bA (y, z′) hB (y, z) q (y) where the inner product between A, B ∈ Lh (X) is defined as A, BLh(X) = Tr (AB) and the partial inner product between A ∈ Lh (X) and B ∈ Lh (X × Y) is defined as A, BLh(X) (y, y′) = A, B (y, y′)Lh(X) . We also denote L+

h (X) the subspace of Hermitian positive

semi-definite linear operators (PSD operators for short) acting

n X.
II. ON FACTOR GRAPHS

Instead of giving an abstract and generic definition at the beginning, firstly we would like to make an exploration from the classical [2], [11] factor graph to the recent model [5] for quantum probabilities. Our proposal of quantum factor graphs is shown right afterwards. An abstract and more general notion which is related to [8] is shown in the subsection II-D. Readers familiar with the classical factor graphs may want to jump to subsection II-C directly.

A. Classical Factor Graphs

Classically, a factor graph describes a factorization of a global function. Following is a clear example. Example 1. The graph in Figure 1 depicts following factorization g (x, y, z, z′) = p (x) uA (x, z) uB (x, z′) bA (y, z′) hB (y, z) q (y) where squares represents factors and circles represents vari-

ables. An edge is drawn between a variable node and a factor

node if this variable is an argument of this factor. Another practical example is the probability represented by a hidden Markov model. Example 2 (A hidden Markov model). Consider the factor graph in Figure 2. Here, the global function is

SLIDE 2

FIRST YEAR REPORT 2

x0 x1 x2 x3 y1 y2 y3

Fig. 2: Normal factor graph for a hidden Markov model of

length 3 p = uA uB bB bA = q

x z z′ y

Fig. 3: Normal factor graph for the first example

p (y1, . . . , y3, x0, . . . , x3) = p0 (x0)

3

k=1

pk (yk, xk|xk−1) . In this case, x0, x1, . . . are the hidden variables. Note that in above factor graph, the nodes for the variables are omitted, since each variable is only connected to at most two factors. A factor graph with variables represented by edges directly is called a normal factor graph [1], [11], [12]. The notion

f normal factor graph is not only generic (as shown in next

example), but also provide extra benefits by introducing the concept of “opening/closing the box” [1], [5], [6]. Example 3. This example shows the conversion of a standard factor graph into a normal factor graph by adding extra “equal” factor nodes. Figure 3 depicts the corresponding normal factor graph for Example 1, where two “equal” factor nodes are added to make duplications of variables x and y, respectively. Now, consider the dashed box in Figure 2. We define the exterior function of such a box as the product of all factors inside the box summing over all internal variables. For this example, we have the exterior function as pY1,Y2,Y3|X0(y1, y2, y3|x0) =

x1,x2,x3

p(x0)p(y1, x1|x0)p(y2, x2|x1)p(y3, x3|x2). Note that the exterior function is always a functions of the variables crossing the box boundary. By replacing the box with a factor corresponding to its exterior function, the resultant factor graph yields a new factorization with less

variables. In this case, the marginals with respect to (w.r.t.)

the remaining variables will keep unchanged. Such operation is often refereed to as “closing-the-box”, whereas its reverse is called “opening-the-box” [1], [5], [6]. The operation of “closing-the-box” is closely related to the idea of sum-product algorithm [2] where a sequence of “closing-the-box” operation are taken in order. However, as presented in next subsection, such operation is not limited to traditional probability models. p (x) = U U H BH B =

X Y

pY |X (y|x)

Fig. 4: Factor graph for an elementary quantum system
B. Factor Graphs for Quantum Probabilities

Factor graphs can be used to represent quantum probabilities if more general factors are allowed [5], [6]. Following is an example as a modification of Example 3. Example 4. Consider the factor graph in Figure 4. Here, some

f the factors are given in matrix form. The dots at the end of

each edges are used to specify which variable is performing as the first index of the matrix. In this case, the global function g (x, y, ˜ x, ˜ x′) p (x) U (˜ x, x) U H (x, ˜ x′) BH (y, ˜ x) B (˜ x′, y) = p (x) U (˜ x, x) B (˜ x′, y) U (˜ x′, x) B (˜ x, y) (3) where U and B are both complex unitary matrices. Though we have complex-valued functions as factors in this cases, the marginal functions can still be real-nonnegative, due to the symmetric structure. For example, by closing the box in Figure 4, we have following exterior function pY |X(y|x) =

˜

x,˜ x′

U (˜ x, x) B (˜ x′, y) U (˜ x′, x) B (˜ x, y) =

˜

x

U (˜ x, x) B (˜ x, y) ·

˜

x′

U (˜ x′, x) B (˜ x′, y) =

˜

x

U (˜ x, x) B (˜ x, y)

2

which exactly describes the conditional probability of variable Y given X in a quantum system with one-time unitary evolution and a single projective measurement [13]. Consider the factor graphs constructed in a symmetric manner where complex functions always appear in conjugate pairs, as in Example 4. In this case, any factorization with constrain in equation (2) is representable, in particular, a variety of a quantum systems are included [5], [6].

C. Quantum Factor Graphs (QFGs)

1) Redraw of example 4: Example 5. Consider the factor graph in Figure 5 as a redraw

f Figure 4, where each factor here is given by a matrix with

the combination of the upper-edge variables as the first index. The dot notation in this case is used in a different manner: It is applied to clarify the order between upper (lower)-edge variables in nesting-up of the first (second) index. In this case, the factors in red and blue can be respectively written as ˆ U

(˜

x, x), (˜ x′, x′)

U (˜

x, x) · U (˜ x′, x′) ˆ B

(˜

x, ˜ y), (˜ x′, ˜ y′)

B (˜

x, ˜ y) · B (˜ x′, ˜ y′) .

SLIDE 3

FIRST YEAR REPORT 3 diag (p(x))

ˆ U ˆ B

Iy

X X′ ˜ X ˜ X′ ˜ Y ˜ Y ′ Y f1 f2

Fig. 5: Quantum normal factor graph for elementary

quantum system Moreover, the global function is g (x, x′, ˜ x, ˜ x′, ˜ y, ˜ y′, y) =      p(x)U (˜ x, x) B (˜ x′, y) U (˜ x′, x) B (˜ x, y) if x = x′ and, ˜ y = ˜ y′ = y

therwise

which is identical to the global function in equation (3). In addition, we have the exterior functions f1, f2 of the colored boxes in Figure 5 as follows. f1

(x, ˜

y) , (x′, ˜ y′)

=
˜

x,˜ x′

U (˜ x, x) B (˜ x, ˜ y) · U (˜ x′, x′)B (˜ x′, ˜ y′) =

˜

x

U (˜ x, x) B (˜ x, ˜ y) ·

˜

x′

U (˜ x′, x′) B (˜ x′, ˜ y′) =

BHU
˜

y,x ·

BHU
˜

y′,x′

f2 (y) =

x

p(x)f1

(x, y) , (x, y)
=
x

p(x)

˜

x

U (˜ x, x) B (˜ x, y)

2

Here, f2 is the probability measure on Y given the distribution

f X by p(x). One can view f1 as an generalization of

p (Y |X) into matrix form. In particular, note that diag (f1) = p (Y |X). Therefore, we have successfully described a quantum system using a simpler graph with compatible closing-the-box

perations. Indeed, such techniques can be made generic, and

we name such graphical models as quantum factor graphs (QFG). 2) Quantum (Normal) Factor Graph (QFG or QNFG): Definition 1. A quantum factor graph or QFG is a bipartite graph between a set of factor nodes and a set of variable nodes, which describes a factorization in a quantum system, as in equation (1). Here, each factor node stands for a factor, and each variable node stands for one (with all adjacent edges drawn in single line) or a pair of (with all adjacent edges drawn in double line) variables. For each factor (indexed by a ∈ F) fa (x∂a, x′

∂a; yδa)

with the set of double edges indicated by ∂a and the set of single edges indicated by δa, fa is a PSD operator over X∂a, given yδa fixed arbitrarily. Here, the Cartesian product X∂a =

×

i∈∂a Xi is the alphabet for both variables x∂a and x′ ∂a.

A QFG is said to be normal (QNFG) if all variable nodes have degree at most 2. In such cases, variable nodes are often drawn as edges (as in Example 5). In both cases, we refer the function to be factorized as the global function, i.e., the left-hand side of equation (1). In this article, we mainly focus

n QNFGs.

Lemma 2. In a QNFG, let fab be the exterior function of a pair of adjacent factors fa, fb with common variable pair (xi, x′

i), then

fab = Tr

f

x∂a\{i},x′

∂a\{i};yδa

a

· f

x∂b\{i},x′

∂b\{i};yδb

b

= f yδa

a

, f yδb

b

L+

h (Xi) .

Here, the bold letters fa, fb stands for the matrix representation of the (induced) PSD operator.

Proof. Above results can be obtained by direct computation.

Thus, the proof is omitted. Above lemma provides some justification of QNFG model, since the (partial) inner product of two PSD operators is a still PSD operator (over different alphabet). Hence, by eliminating all pairs of double-edge variables in a QNFG, it would eventually result in a real nonnegative number. 3) Conversion into QNFG: A factor graph describing a quantum system in style of Figure 4 can be systematically parsed into a QNFG via following converting methods. TABLE I: Conversion into QNFG

Classical factor graph QNFG squeeze U UH ˆ U ˆ U

(˜

x, x), (˜ x′, x′)

U (˜

x, x) · U (˜ x′, x′) = vec (U) vec (U)H equality = X X′ I X X′ I is the identity matrix, i.e., I (x, x′) = δ (x, x′) merge p (x) = X X′

diag (p(x))

X X′ diag (p) (x, x′) =

p (x)

if x = x′

therwise

parametrize = ˜ Y ˜ Y ′ Y Iy ˜ Y ˜ Y ′ Y Iy (˜ y, ˜ y′) =

1

if ˜ y = ˜ y′ = y

therwise

SLIDE 4

FIRST YEAR REPORT 4 diag (p(x))

ˆ U1 ˆ U2 · · · ˆ Un ˆ B

Iy

X X′ Y

ρ1 ρ2 ρn

Fig. 6: Unitary Evolution over time in n steps followed by a single projective measure

diag (p(x0))

ˆ U0 ˆ A1 ˆ U1 ˆ A2 I

X0 X′ X1 X′

1

˜ X1 ˜ X′

1

X2 X′

2

˜ X2 ˜ X′

2

Y1 Y2

Fig. 7: A Two-Measure Quantum System

diag (p(x0))

ˆ U0 ˆ B1 Iy ˆ BH

1

ˆ U1 ˆ B2 Iy ˆ BH

2

I

X0 X′ Y1 Y2

Fig. 8: A Two-Measure Quantum System with projective measure with 1-dimension eigenspace

diag (p(x0))

ˆ U0 ˆ A1 ˆ U1 ˆ A2 ˆ U2 I

X0 X′ X1 X′

1

˜ X1 ˜ X′

1

X2 X′

2

˜ X2 ˜ X′

2

X3 X′

3

W1 W ′

1

W2 W ′

2

Y1 Y2

Fig. 9: A Quantum System with partial measurement
D. A Unifying Model*

As illustrated by Lemma 2, the closing-the-box operation can be viewed as a sequence of (partial) inner product opera-

tions. Interestingly, this is also true for classical normal factor

graphs, where the inner products are taken on vector level. This motivates us to define following unifying notation as a generalization of both classical and quantum factor graphs. Let V, F be two finite sets indexing a set of alphabets (Xi)i∈V and a set of factors (fa)a∈F. Each factor fa is an element in a inner product space L (X∂a) arisen from the set X∂a = ×

i∈∂a

Xi where ∂a ⊂ V for each a ∈ F. For classical factor graph, L (X∂a) is the set of nonnegative functions on X∂a, i.e. L (X∂a) = Lf (X∂a)

f : X∂a −

→ R+ . In this case, we define the factor graph with factors (fa ∈ L (X∂a))a∈F and (hi ∈ L (Xi))a∈F as a bipartite graph G = (F, V, E) between finite sets F and V with edges given by E = {(a, i) ∈ F × V : i ∈ ∂a}. In addition, we denote ∂i = {a ∈ F : i ∈ ∂a}. For the following of this article, we assume Xi = X ∀i, where X is a finite set/alphabet with a specific fixing element 0 ∈ X. Given a factor graph G defined above, the global function defined by G is given by g

a∈F

fa

i∈V

hi. (4) The product operator ⊗ defined on the family of inner product space (L (Xs))s⊂V is defined in such a way that fa ⊗ fb ∈ L (X∂a∪∂b) , fa ⊗ fb, ga ⊗ gbL(X∂a∪∂b) = fa, gaL(X∂a) · fb, gbL(X∂b) for arbitrary fa, ga ∈ L (X∂a) and fb, gb ∈ L (X∂b). Note that for QFGs, we take L (X∂a) = L+

h (X∂a), the inner product

space of PSD operators on X∂a. In this case, fa ⊗ fb stands for Kronecker product between fa and fb if ∂a ∩ ∂b = φ;

SLIDE 5

FIRST YEAR REPORT 5

but stands for Hadamard product when ∂a = ∂b. By above notation, the partition sum of G is defined as Z (G) =

a∈F

fa,

i∈V

hi

L(XV)

. Remark 3. The format of the global function as in (4) where a “hidden” factor hi is appended for each i ∈ V has its benefits in both practice and theory. In fact, the factors (hi)i∈V can always be taken as 1 (a multiplicative fixing element) to yield a simplified global function similar to (1). On the other hand, however, the global function in equation (1) does not lose it generality, since each “hidden” factor hi can be understand as ha with ∂a = {i}. In the remaining part of this article, we use ⊗ as a short-hand notation to denote the product of Hermitian linear

perators without writing out the variables that these operators

act on. In later case, the common product operator is used, i.e.,

a∈F

fa

(x∂a, x′

∂a) ≡

a∈F

fa (x∂a, x′

∂a) .

III. EXAMPLES

In this section, we would like to present several examples

f QNFGs, especially the QNFGs representing elementary

quantum mechanics. Some of the examples presented here are related to the examples in Loeliger and Vontobel’s recent work [5], [6]. Example 6. The QNFG in Figure 6 describes a quantum system with unitary evolutions over time in n steps followed by a single projective measure. Consider the exterior functions

f the red boxes in the figure, we have

ρk (xk, x′

k)

=

xk−1,x′

k−1

ˆ Uk

(xk, xk−1) ,
x′

k, x′ k−1

ρk−1

xk−1, x′

k−1

=
ˆ

Uk, ρk−1

Lh(Xk−1)

which is exactly the Schr¨

dinger representation for this sys-
tem. Whereas the exterior functions of the blue boxes corre-

spond to the Heisenberg representation. Example 7. Figure 7 describes a QNFG for a quantum system with two measures. Here, we require

yk
ˆ

Ayk

k , δ ˆ Xk, ˆ X′

k

LH( ˆ

Xk) = δXk,Xk.

Example 8. The QNFG in Figure 8 is a special case of last ex-

ample. Here, both measurements are projective measurements

with one-dimensional eigenspaces. Example 9. The QNFG in Figure 9 depicts a quantum system with two partial measurements. Here each factor ˆ Uk is a PSD

perator on corresponding alphabet. For instance, ˆ

U1 can be

rganized into a PSD matrix as

ˆ U1

(x2, w2, ˜

x1, w1) , (x′

2, w′ 2, ˜

x′

1, w′ 1)

.

Note that this QNFG contains cycles. 1 2 3 4 5 a b c d e f

Fig. 10: Sum-Product Algorithm on a normal QGF with no

cycles Algorithm 1 Sum-Product/Message-Passing Algorithm Require: Acyclic QFG G = (F, V, E), root r ∈ F Ensure: Partition sum Z (G) =

x∂r,x∂r
i∈∂r

mi→r (xi, x′

i)

1: for all a ∈ F a leaf factor do 2:

ma→i (xi, x′

i) = fa (xi, x′ i) where {i} = ∂a

3: end for 4: while ∃mi→a or ma→i undefined do 5:

if all {mb→i}b∈∂i\{a} defined then

6:

mi→a ←

b∈∂i\{a}

mb→i (xi, x′

i);

7:

end if

8:

if all {mj→a}j∈∂a\{i} defined then

9:

ma→i ←

j∈∂a\{i} mj→a, fa
Lh(X∂a\{i});

10:

end if

11: end while

IV. SUM-PRODUCT ALGORITHM AND

BELIEF PROPAGATION FOR QFG The sum-product algorithm (SPA) for acyclic QFGs and its extension to QFGs with cycles, namely belief propagation (BP) algorithm, are the main topics of this section. Similar to the sum-product algorithm for classical factor graphs [2], one can also “shink” a tree QGF to a root factor via a sequence

f closing-the-box operations from the leaf factors. Following

example illustrates this idea. Example 10. By taking a sequence of “closing-the-box” operations for each adjacent factors, the QNFG in Figure 10 can be “shrunk” to a null graph with no factors which represents a nonnegative constant. The numbers in each box indicate the sequence of these “closing-the-box” operations. The constant

btained in the end is exactly the partition sum.

This idea can be formally described as Algorithm 1. For simplicity, we assume the global function is in format as in equation (1), but with no single-edge variables. Note that, in this algorithm, we assume all the leaf nodes are

factors. This assumption is valid since one can always append

a dummy factor to the leaf variables. Such assumption also gains us additional control, as shown in following example.

SLIDE 6

FIRST YEAR REPORT 6

1 ˜ g ˜ g

δy,y′ Y Y ′ Y Y ′

˜ g

Y Y ′

Fig. 11: Control of the leaf variable node/open edges

Example 11. Consider a QNFG with open edge (Y, Y ′) in Figure 11. By appending all-one factor (left-hand side) and δy,y′ (Y, Y ′) δ (Y, y) δ (Y ′, y′) (right-hand side), one shall have following partition sums, respectively: Z (GLHS) =

y,y′

˜ g (y, y′) Z (GRHS) = ˜ g (y, y′) which can be viewed as the exterior functions of the outer and inner boxes in above figure, respectively. This phenomenon is more explicitly described in next proposition. Proposition 4. Let mi→a, ma→i be the messages in the SPAs

f a QGF with global function g (x, x′), then for each i ∈ V,

gi (xi, x′

i)

xV\{i},x′

V\{i}

g (x, x′) =

a∈∂i

ma→i (xi, x′

i) (5)

Z (G) =

xi,x′

i

a∈∂i

ma→i (xi, x′

i)

(6)

Proof. Omitted.

It is direct to justify SPA for QGFs without cycles. Though there is no evidence the idea of SPA would still make sense for graph with cycles, following describes a heuristic extension

f this algorithm for general QFGs. In the remaining part of

this section, we assume the global function is in format as in equation (4), or more precisely, g (x, x′) =

a∈F

fa (x∂a, x′

∂a)

i∈V

hi (xi, x′

i) .

(7) Here, without loss of generality, we assume our QFG has no

pen or single edges, i.e., all variables appears in pairs.

Definition 5 (Belief Propagation for QFG). For a factor graph G = {F, V, E} describing a quantum probability. Given initial messages m(0)

a→i, m(0) i→a ∈ L+ h , for all (i, a) ∈ E, we have

following update rules for belief propagation (BP) algorithm. m(t+1)

a→i ∝

j∈∂a\{i}

m(t)

j→a, fa

Lh(X∂a\{i})

(8) m(t+1)

i→a ∝ hi ·

b∈∂i\{a}

m(t)

b→i

(9) The messages are said to be fixed-point messages when above equations holds without time-stamp superscripts. In this case, we define the estimated partition sum for i ∈ V, a ∈ F, (i, a) ∈ E as Zi

hi,
a∈∂i

m(t)

a→i

, Za
i∈∂a

m(t)

i→a, fa

,

Zi,a ma→i, mi→a ,

respectively. Moreover, the Bethe approximation of the parti-

tion sum is defined as ZBethe

a∈F

Za

i∈V

Zi

(i,a)∈E

Zi,a . (10) If the concerned QFG is a tree, it is easy to check that Zi = Za = Zi,a = Z, given the initial messages set properly, and “=” taken in (8), (9). In this case, ZBethe = Z|F|+|V|−|E| = Z. In the remaining of this section, we justify the BP algorithm for QFGs from the aspect of loop calculus [7], [8], [9]. Theorem 6 (Holant Theorem). Consider QFG described

above. Let

ˆ fa (y∂a, y′

∂a)

x∂a,x′

∂a

i∈∂a

ˆ φi,a (yi, y′

i; xi, x′ i) · fa (x∂a, x′ ∂a)

=

i∈∂a

ˆ φi,a (yi, y′

i) , fa

Lh(X∂a)

(11) ˆ hi

yi,∂a, y′

i,∂a

x,x′

hi (x, x′) ·

a∈∂i

φi,a

x, x′; yi,a, y′

i,a

=
hi,
a∈∂i

φi,a

yi,a, y′

i,a

Lh(Xi)

(12) where {ˆ φi,a}(i,a)∈E and {φi,a}(i,a)∈E are two classes of transformations such that

y,y′

φi,a (x, x′; y, y′) ˆ φi,a (y, y′; z, z′) = δ (x, z) δ (x′, z′) . (13) Then, Z (G)

x,x′
a∈F

fa (x∂a, x′

∂a)

i∈V

hi (xi, x′

i)

=

y,y′
a∈F

ˆ fa

y∂a,a, y′

∂a,a

i∈V

ˆ hi

yi,∂a, y′

i,∂a

. (14)
Proof. Proof can be done via direct computation, and thus is
mitted.

Now, following the idea of [7], [9], consider following additional constrains for all (i, a) ∈ E ˆ fa

y∂a,a, y′

∂a,a

= 0 if wH
y∂a,a ⊙ y′

∂a,a

= 1

(15) ˆ hi

yi,∂a, y′

i,∂a

= 0 if wH
yi,∂a ⊙ y′

i,∂a

= 1

(16)

SLIDE 7

FIRST YEAR REPORT 7

Here, ⊙ stands for Hadamard product between vectors, and wH is the Hamming weight function on vectors. It is not hard to see that condition (15) can be rewritten as

x∂a,x′

∂a

 

j∈∂a\{i}

ˆ φj,a

0, 0; xj, x′

j

· fa (x∂a, x′

∂a)

  · ˆ φj,a (yi, y′

i; xi, x′ i) = 0

(17) for any j ∈ ∂a and any

yj, y′

j

= (0, 0). In other words,
x∂a\{i},x′

∂a\{i}

j∈∂a\{i}

ˆ φj,a

0, 0; xj, x′

j

· fa (x∂a, x′

∂a)

is orthogonal to ˆ φj,a (yi, y′

i) for all nonzero

yj, y′

j

. There-

fore, considering (13), we have φi,a (x, x′; 0, 0) = 1 ˆ fa (0, 0) ·

x∂a\{i},x′

∂a\{i}

j∈∂a\{i}

ˆ φj,a

0, 0; xj, x′

j

· fa (x∂a, x′

∂a)

= 1 ˆ fa (0, 0) ·

j∈∂a\{i}

ˆ φj,a (0, 0) , fa

Lh(X∂a\{i})

. (18) Similarly, we have ˆ φi,a (0, 0; x, x′) = 1 ˆ hi (0, 0) · hi (x, x′)

a∈∂i

φi,a (x, x′; 0, 0) . (19) Compare equations (8), (9) and (18), (19). We have φi,a (0, 0; x, x′) = ci,ama→i (x, x′) ˆ φi,a (0, 0; x, x′) = ˆ ci,ami→a (x, x′) as a pair of fixed-point solution to BP algorithm. Here, ci,aˆ ci,a = ma→i, mi→a = 1 Zi,a . In addition, we also have ˆ fa (0, 0) = Za

i∈∂a

ˆ ci,a, ˆ hi (0, 0) = Zi

a∈∂i

ci,a. Therefore, ZBethe =

a∈F

ˆ fa (0, 0)

i∈V

ˆ hi (0, 0) . (20) Now, we can derive the loop calculus for BP algorithm on QFGs. Theorem 7 (Loop Calculus). Consider the Bethe Approxima- tion obtained as above, we have Z = ZBethe

E⊂E′

K (E) where the extended loop set is defined as E′

E ⊂ E :

di (E) = 1 ∀i ∈ V, da (E) = 1 ∀i ∈ F

where K (E) is some function depending on E, and K (φ) = 1.

Proof. Z (G) =

y,y′
a∈F

ˆ fa

y∂a,a, y′

∂a,a i∈V

ˆ hi

yi,∂a, y′

i,∂a

= ZBethe
y,y′
a∈F

ˆ fa

y∂a,a, y′

∂a,a

ˆ

fa (0, 0)

i∈V

ˆ hi

yi,∂a, y′

i,∂a

ˆ

hi (0, 0) = ZBethe

y,y′
a∈F

i∈∂a

ˆ φi,a

yi,a, y′

i,a; X, X′

ˆ φi,a (0, 0; X, X′)

ba

·

i∈V

a∈∂i

φi,a

X, X′; yi,a, y′

i,a

φi,a (X, X′; 0, 0)
bi

= ZBethe

E⊂E′
(y,y′)=0
a∈F
i∈∂a,(i,a)∈E

ˆ φi,a

yi,a, y′

i,a; X, X′

ˆ φi,a (0, 0; X, X′)

ba

·

i∈V
a∈∂i,(i,a)∈E

φi,a

X, X′; yi,a, y′

i,a

φi,a (X, X′; 0, 0)
bi

= ZBethe

E⊂E′

K (E) . Here, ba (x∂a, x′

∂a) 1

Za fa (x∂a, x′

∂a)

i∈∂a

mi→a (xi, x′

i) , (21)

bi (xi, x′

i) 1

Zi hi (xi, x′

i)

a∈∂i

ma→i (xi, x′

i) .

(22) From above theorem, it is easy to see that the Bethe approximation is exact for acyclic QFGs, since in this case E′ = {φ}. Moreover, it also suggest that the Bethe approximation is close for a QFG with a relatively smaller number of cycles.

V. VARIATIONAL APPROACH ON

QUANTUM FACTOR GRAPHS Though we have justified the BP algorithm form QFGs via the approach of loop calculus, it is still unclear whether the marginal beliefs given by the BP algorithm, as defined in equations (21) and (22), can recovery or estimate the marginal quantum probabilities. This section contains our partial result

n generalization of the variational approach [10] to QFGs.

Similar to Section IV, in this section we still assume the concerned QFG G = (F, V, E) has no leaf variables and the global function is in form of equation (7). Due to space limitation, the proofs in this section are all omitted. Definition 8. For a QGF as described above. The Helmholtz free energy and the Gibbs free energy w.r.t. belief b(x, x′) are defined as following, respectively. FH − ln Z (G) FGibbs (b(x, x′)) −

a∈F
˜

ba, Ln (fa)

−
i∈V
˜

bi, Ln (hi)

+ b, Ln (b)

SLIDE 8

FIRST YEAR REPORT 8

Here, the belief b(x, x′) is a normalized PSD operator over “global” alphabet XV, i.e., its matrix form b is a |X||V|-sized square PSD matrix, and sum of all its entries is 1. The induced

˜

ba

a∈F and
˜

bi

a∈V are PSD operators on X∂a and Xi,

respectively, with ˜ ba 1, bLh(XV\{∂a}) , ˜ bi 1, bLh(XV\{i}) . (23) Here, Ln (·) is performed on matrix level, i.e., Ln

UΛU H

Udiag

{ln Λk,k}k
U H

for any PSD matrix UΛU H, where U is unitary and Λ is a non-negative diagonal matrix. Theorem 9. We have following relationship between Gibbs free energy and Helmholtz free energy FGibbs(b(x, x′)) = FH + D(b p) (24) where D(b p) of two normalized PSD operator is defined as D(b p) b, Ln (b)Lh − b, Ln (p)Lh (25) and the quantum probability p is the normalized global function, i.e., p (x, x′) =

1 Z(G)g (x, x′).

With above definition, we have following immediate result Lemma 10 (Non-negativity of relative entropy). The relative entropy D(b p) is always nonnegative, i.e., D(b p) 0. (26) Moreover, equality holds if and only if b = p. Theorem 9 indicates that the problem to calculate the partition sum can be remodeled as the minimization problem

f the Gibbs free energy. Moreover, the Lemma 10 ensures the

recovery of the quantum probability p at the minimizing point. However, it is still #P-hard to calculate FGibbs. Nevertheless, we give following approximation to the Gibbs free energy. Definition 11. For above QGF, the Bethe free energy w.r.t. marginal beliefs (ba)a∈F , (bi)i∈V is defined as FBethe

(ba)a∈F , (bi)i∈V
−
a∈F

ba, Ln (fa) −

i∈V

bi, Ln (fi) +

a∈F

ba, Ln (ba) −

i∈V

(di − 1) bi, Ln (bi) . Here, {ba}a∈F and {bi}i∈V are some given normalized PSD

perators on {X∂a}a∈F and {Xi}i∈V, respectively.

Lemma 12. Given an acyclic QFG, for any marginal quantum belief {ba}a∈F and {bi}a∈V with additional constrain bi (xi, x′

i) =

x∂a\{i},x′

∂a\{i}

ba (x∂a, x′

∂a)

∀ (i, a) ∈ E (27) there exists a normalized Hermitian linear operator b with equation (23) holds by replacing ˜ ba = ba, ˜ bi = bi. More specifically, we can take b (x, x′) =

a∈F

ba (x∂a, x∂a′)

i∈∂a

bi (xi, xi)

i∈V

bi (xi, x′

i) .

Conjecture 13. The operator b defined in Lemma 12 is positive semi-definite. In such case, min

b

FGibbs(b) = min

{ba}a∈F,{bi}a∈V

FBethe

(ba)a∈F , (bi)i∈V
s.t. Equation (27) holds.

The optimization on the right-hand side is called the “constrained minimization problem of Bethe free energy”. Conjecture 14. The stationary condition of the constrained minimization problem of Bethe free energy is equivalent to equations (21) and (22).

VI. CONCLUSION

In this article, we proposed quantum factor graph (QFG) as a simpler but more generic model to represent quantum

probabilities. We also generalized the belief propagation (BP)

algorithm for QFGs. In particular, we justified the generalized BP algorithm via the method of loop calculus. However, it is still an open problem whether the BP algorithm for QFG can recover or estimate the marginal quantum probabilities. Section V showed our attempts, and the current difficulties as well. In addition to this open problem, another potential topic is on the belief propagation for the generic factor graph defined in subsection II-D. REFERENCES

[1] H.-A. Loeliger, “An introduction to factor graphs,” IEEE Signal Process. Mag., vol. 21, no. 1, pp. 28–41, 2004. [2] F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498–519, 2001. [3] M. Molkaraie and H.-A. Loeliger, “Partition function of the Ising model via factor graph duality,” in Proc. IEEE Int. Symp. Inf. Theory, 2013,

pp. 2304–2308.

[4] R. G. Gallager, “Low-density parity-check codes,” IRE* Trans. Inf. Theory, vol. 8, no. 1, pp. 21–28, 1962. [5] H.-A. Loeliger and P. O. Vontobel, “Factor graphs for quantum probabilities,” 2015. [Online]. Available: http://arxiv.org/abs/1508.00689 [6] ——, “A factor-graph representation of probabilities in quantum mechanics,” in Proc. IEEE Int. Symp. Inf. Theory, 2012, pp. 656–660. [7] [8] R. Mori, “Loop calculus for nonbinary alphabets using concepts from information geometry,” IEEE Trans. Inf. Theory, vol. 61, no. 4, pp. 1887– 1904, 2015. [9] V. Y. Chernyak and M. Chertkov, “Loop calculus and belief propagation for q-ary alphabet: Loop tower,” in Proc. IEEE Int. Symp. Inf. Theory. IEEE, 2007, pp. 316–320. [10] J. S. Yedidia, W. T. Freeman, and Y. Weiss, “Constructing free-energy approximations and generalized belief propagation algorithms,” IEEE

Trans. Inf. Theory, vol. 51, no. 7, pp. 2282–2312, 2005.

[11] G. D. Forney Jr, “Codes on graphs: normal realizations,” IEEE Trans.

Inf. Theory, vol. 47, no. 2, pp. 520–548, 2001.

[12] A. Al-Bashabsheh, Y. Mao, and P. O. Vontobel, “Normal factor graphs: A diagrammatic approach to linear algebra,” in Proc. IEEE Int. Symp.

Inf. Theory.

IEEE, 2011, pp. 2178–2182. [13] M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information. The Edinburgh Building, Cambridge CB2 8RU, UK: Cambridge University Press, 2011.