Classical Factor Graphs Modeling Factor Graph representing Factorization In general, a factor graph for factorization � g ( x ) = f a ( x ∂ a ) a ∈F is a bipartite graph G = ( F , V , E ) between F and V with edge set E = { ( i , a ) ∈ F × V : i ∈ ∂ a } . z u A p u A u B q B b B b B b A y x p = = q u B b A x z z ′ y z ′ A standard factor graph A normal factor graph Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 4
Classical Factor Graphs “Closing-the-box” Operation Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 5
Classical Factor Graphs “Closing-the-box” Operation Normal Factor Graph and “Closing-the-box” Operation Traditionally, we assume all the factors to be nonnegative. Thus, any marginal function is still a measure . x 0 x 1 x 2 x 3 p 0 p 1 p 2 p 3 y 1 y 2 y 3 Normal Factor Graph for a hidden Markov model of length 3: p ( y 1 , . . . , y 3 , x 0 , . . . , x 3 ) = p 0 ( x 0 ) � 3 k =1 p k ( y k , x k | x k − 1 ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 5
Classical Factor Graphs “Closing-the-box” Operation Normal Factor Graph and “Closing-the-box” Operation Traditionally, we assume all the factors to be nonnegative. Thus, any marginal function is still a measure . x 0 x 1 x 2 x 3 p 0 p 1 p 2 p 3 y 1 y 2 y 3 Normal Factor Graph for a hidden Markov model of length 3: p ( y 1 , . . . , y 3 , x 0 , . . . , x 3 ) = p 0 ( x 0 ) � 3 k =1 p k ( y k , x k | x k − 1 ) Exterior Function of above dashed box: p Y 1 , Y 2 , Y 3 | X 0 ( y 1 , y 2 , y 3 | x 0 ) = � p 1 ( y 1 , x 1 | x 0 ) p 2 ( y 2 , x 2 | x 1 ) p 3 ( y 3 , x 3 | x 2 ) . x 1 , x 2 , x 3 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 5
Classical Factor Graphs “Closing-the-box” Operation Normal Factor Graph and “Closing-the-box” Operation Traditionally, we assume all the factors to be nonnegative. Thus, any marginal function is still a measure . x 0 p 0 p Y 1 , Y 2 , Y 3 | X 0 ( y 1 , y 2 , y 3 | x 0 ) y 1 y 2 y 3 Normal Factor Graph for a hidden Markov model of length 3: p ( y 1 , . . . , y 3 , x 0 , . . . , x 3 ) = p 0 ( x 0 ) � 3 k =1 p k ( y k , x k | x k − 1 ) Exterior Function of above dashed box: p Y 1 , Y 2 , Y 3 | X 0 ( y 1 , y 2 , y 3 | x 0 ) = � p 1 ( y 1 , x 1 | x 0 ) p 2 ( y 2 , x 2 | x 1 ) p 3 ( y 3 , x 3 | x 2 ) . x 1 , x 2 , x 3 “Closing-the-box” Operation: Replacing the box with a factor corresponding to its exterior function Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 5
Quantum Factor Graphs A Motivating Example Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs A Motivating Example Factor Graph representing Quantum Probabilities Factor graphs can be used to represent quantum probabilities if more general factors are allowed [Loeliger and Vontobel, 2015, Loeliger and Vontobel, 2012]. X Y = U B H = p ( x ) B U H Factor graph for an elementary quantum system The global function: x , x ) U H ( x , ˜ x ′ ) B H ( y , ˜ x ′ ) � p ( x ) U (˜ x ′ , y ) g ( x , y , ˜ x , ˜ x ) B (˜ x ′ , y ) U (˜ = p ( x ) U (˜ x , x ) B (˜ x ′ , x ) B (˜ x , y ) . Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs A Motivating Example Factor Graph representing Quantum Probabilities Factor graphs can be used to represent quantum probabilities if more general factors are allowed [Loeliger and Vontobel, 2015, Loeliger and Vontobel, 2012]. X Y = U B H = p ( x ) B U H p Y | X ( y | x ) Factor graph for an elementary quantum system The exterior function of the dashed box: � x ′ , y ) U (˜ p Y | X ( y | x ) = U (˜ x , x ) B (˜ x ′ , x ) B (˜ x , y ) x ′ x , ˜ ˜ 2 � � � � � = U (˜ x , x ) B (˜ x , y ) . � � � � � ˜ � x Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs A Motivating Example Factor Graph representing Quantum Probabilities Factor graphs can be used to represent quantum probabilities if more general factors are allowed [Loeliger and Vontobel, 2015, Loeliger and Vontobel, 2012]. X Y = U B H = p ( x ) B U H Factor graph for an elementary quantum system x 0 x 1 x 2 x 3 p 0 p 1 p 2 p 3 y 1 y 2 y 3 Normal Factor Graph for a hidden Markov model of length 3: p ( y 1 , . . . , y 3 , x 0 , . . . , x 3 ) = p 0 ( x 0 ) � 3 k =1 p k ( y k , x k | x k − 1 ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs A Motivating Example Factor Graph representing Quantum Probabilities Factor graphs can be used to represent quantum probabilities if more general factors are allowed [Loeliger and Vontobel, 2015, Loeliger and Vontobel, 2012]. X Y = U B H = p ( x ) B U H Factor graph for an elementary quantum system X Y = B H = ˆ U p ( x ) B Redraw of above Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs A Motivating Example Factor Graph representing Quantum Probabilities Factor graphs can be used to represent quantum probabilities if more general factors are allowed [Loeliger and Vontobel, 2015, Loeliger and Vontobel, 2012]. X Y = U B H = p ( x ) B U H Factor graph for an elementary quantum system X Y = = ˆ ˆ U B p ( x ) Redraw of above Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 6
Quantum Factor Graphs Quantum Factor Graph Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
✶ ✶ Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example In this case, we have Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example In this case, we have Global function: g ( x , x ′ , ˜ x ′ , ˜ y ′ , y ) = p ( x ) U (˜ x ′ , y ) U (˜ x ′ , x ) B (˜ x , ˜ y , ˜ x , x ) B (˜ x , y ) y ′ = y } · ✶ { x = x ′ } ✶ { ˜ y = ˜ Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ f 1 diag ( p ( x )) I y Redraw of last example In this case, we have Global function: g ( x , x ′ , ˜ x ′ , ˜ y ′ , y ) = p ( x ) U (˜ x ′ , y ) U (˜ x ′ , x ) B (˜ x , ˜ y , ˜ x , x ) B (˜ x , y ) y ′ = y } · ✶ { x = x ′ } ✶ { ˜ y = ˜ � ˆ x ′ , x ′ ) · ˆ y ) , ( x ′ , ˜ y ′ ) x ′ , ˜ y ′ ) � � Exterior function: f 1 ( x , ˜ = U (˜ x , x ; ˜ B (˜ x , ˜ y ; ˜ x ′ x , ˜ ˜ Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ f 1 diag ( p ( x )) I y Redraw of last example In this case, we have Global function: g ( x , x ′ , ˜ x ′ , ˜ y ′ , y ) = p ( x ) U (˜ x ′ , y ) U (˜ x ′ , x ) B (˜ x , ˜ y , ˜ x , x ) B (˜ x , y ) y ′ = y } · ✶ { x = x ′ } ✶ { ˜ y = ˜ � � U , ˆ ˆ y ) , ( x ′ , ˜ y ′ ) � � Exterior function: f 1 ( x , ˜ = B L H ( ˜ X ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ f 1 diag ( p ( x )) I y f 2 Redraw of last example In this case, we have Global function: g ( x , x ′ , ˜ x ′ , ˜ y ′ , y ) = p ( x ) U (˜ x ′ , y ) U (˜ x ′ , x ) B (˜ x , ˜ y , ˜ x , x ) B (˜ x , y ) y ′ = y } · ✶ { x = x ′ } ✶ { ˜ y = ˜ � � U , ˆ ˆ y ) , ( x ′ , ˜ y ′ ) � � Exterior function: f 1 ( x , ˜ = B L H ( ˜ X ) � � � Exterior function: f 2 ( y ) = p ( x ) f 1 ( x , y ) , ( x , y ) x Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ f 1 diag ( p ( x )) I y f 2 Redraw of last example In this case, we have Global function: g ( x , x ′ , ˜ x ′ , ˜ y ′ , y ) = p ( x ) U (˜ x ′ , y ) U (˜ x ′ , x ) B (˜ x , ˜ y , ˜ x , x ) B (˜ x , y ) y ′ = y } · ✶ { x = x ′ } ✶ { ˜ y = ˜ � � U , ˆ ˆ y ) , ( x ′ , ˜ y ′ ) � � Exterior function: f 1 ( x , ˜ = B L H ( ˜ X ) � � ˆ Exterior function: f 2 ( y ) = P ⊗ I y , f 1 L H ( X⊗ ˜ Y ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example Definition 1 (Quantum Normal Factor Graph) A quantum normal factor graph or QNFG is a normal factor graph where each variable edge may stands for one or a pair of variables. For each factor (indexed by a ∈ F ) f a ( x ∂ a , x ′ ∂ a ; y δ a ) is a PSD operator over X ∂ a , given y δ a fixed arbitrarily. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example Definition 1 (Quantum Normal Factor Graph) A quantum normal factor graph or QNFG is a normal factor graph where each variable edge may stands for one or a pair of variables. For each factor (indexed by a ∈ F ) f a ( x ∂ a , x ′ ∂ a ; y δ a ) is a PSD operator over X ∂ a , given y δ a fixed arbitrarily. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example Definition 1 (Quantum Normal Factor Graph) A quantum normal factor graph or QNFG is a normal factor graph where each variable edge may stands for one or a pair of variables. For each factor (indexed by a ∈ F ) f a ( x ∂ a , x ′ ∂ a ; y δ a ) is a PSD operator over X ∂ a , given y δ a fixed arbitrarily. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example Definition 1 (Quantum Normal Factor Graph) A quantum normal factor graph or QNFG is a normal factor graph where each variable edge may stands for one or a pair of variables. For each factor (indexed by a ∈ F ) f a ( x ∂ a , x ′ ∂ a ; y δ a ) is a PSD operator over X ∂ a , given y δ a fixed arbitrarily. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Quantum Factor Graph Quantum Normal Factor Graph (QNFG) as a simplified model ˜ ˜ X X Y Y ˆ ˆ ˆ P U B X ′ ˜ ˜ X ′ Y ′ diag ( p ( x )) I y Redraw of last example Definition 1 (Quantum Normal Factor Graph) A quantum normal factor graph or QNFG is a normal factor graph where each variable edge may stands for one or a pair of variables. For each factor (indexed by a ∈ F ) f a ( x ∂ a , x ′ ∂ a ; y δ a ) is a PSD operator over X ∂ a , given y δ a fixed arbitrarily. We can defined quantum factor graph (QFG) similarly, allowing some variable nodes to have degree higher than 2. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 7
Quantum Factor Graphs Construction of a QNFG Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 8
Quantum Factor Graphs Construction of a QNFG Conversion into QNFG: Squeeze Classical factor graph Quantum Normal Factor Graph U ˆ U U H ˆ x ′ , x ′ )) � U (˜ x ′ , x ′ ) U ((˜ x , x ) , (˜ x , x ) · U (˜ = vec ( U ) vec ( U ) H Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 8
Quantum Factor Graphs Construction of a QNFG Conversion into QNFG: Equality Classical factor graph Quantum Normal Factor Graph X X = X ′ I X ′ I is the identity matrix, i.e., I ( x , x ′ ) = δ ( x , x ′ ). Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 8
Quantum Factor Graphs Construction of a QNFG Conversion into QNFG: Merge Classical factor graph Quantum Normal Factor Graph X X = X ′ diag ( p ( x )) p ( x ) X ′ � if x = x ′ p ( x ) diag ( p ) ( x , x ′ ) = 0 otherwise Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 8
Quantum Factor Graphs Construction of a QNFG Conversion into QNFG: Parametrize Classical factor graph Quantum Normal Factor Graph ˜ Y ˜ Y Y Y = ˜ Y ′ I y ˜ Y ′ � y ′ = y 1 if ˜ y = ˜ y ′ ) = I y (˜ y , ˜ 0 otherwise Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 8
Quantum Factor Graphs Several Examples Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y Unitary Evolution over time in n steps followed by a single projective measure Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 Unitary Evolution over time in n steps followed by a single projective measure � � � ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) 1 ) = U 1 x , x ′ Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 ρ 2 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ � � ρ 2 ( x 2 , x ′ � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ρ ( x 1 , x ′ 2 ) = 1 ) 1 ) U 2 x 1 , x ′ 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 ρ 2 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ � � � � ρ 2 ( x 2 , x ′ � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ρ ( x 1 , x ′ ˆ 2 ) = 1 ) 1 ) = U 2 U 2 , ρ 1 L H ( X 1 ) x 1 , x ′ 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 ρ 2 ρ n Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ � � � � ρ 2 ( x 2 , x ′ � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ρ ( x 1 , x ′ ˆ 2 ) = 1 ) 1 ) = U 2 U 2 , ρ 1 L H ( X 1 ) x 1 , x ′ 1 � � � ρ n ( x n , x ′ � ˆ x ′ n , x ′ x n − 1 , x ′ � � � n ) = ( x n , x n − 1 ) , U n − 1 ρ n − 1 n − 1 x n − 1 , x ′ n − 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 ρ 2 ρ n Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ � � � � ρ 2 ( x 2 , x ′ � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ρ ( x 1 , x ′ ˆ 2 ) = 1 ) 1 ) = U 2 U 2 , ρ 1 L H ( X 1 ) x 1 , x ′ 1 � � � ρ n ( x n , x ′ � ˆ x ′ n , x ′ x n − 1 , x ′ � � � n ) = ( x n , x n − 1 ) , U n − 1 ρ n − 1 n − 1 x n − 1 , x ′ n − 1 � � ˆ = U n , ρ n − 1 L H ( X n − 1 ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ρ 1 ρ 2 ρ n Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ρ 1 ( x 1 , x ′ ( x 1 , x ) , ( x ′ 1 , x ′ ) p ( x , x ′ ) δ ( x , x ′ ) = 1 ) = U 1 U 1 , diag ( p ) L H ( X ) x , x ′ � � � � ρ 2 ( x 2 , x ′ � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ρ ( x 1 , x ′ ˆ 2 ) = 1 ) 1 ) = U 2 U 2 , ρ 1 L H ( X 1 ) x 1 , x ′ 1 � � � ρ n ( x n , x ′ � ˆ x ′ n , x ′ x n − 1 , x ′ � � � n ) = ( x n , x n − 1 ) , U n − 1 ρ n − 1 n − 1 x n − 1 , x ′ n − 1 � � ˆ = U n , ρ n − 1 L H ( X n − 1 ) Schr¨ odinger representation. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n Unitary Evolution over time in n steps followed by a single projective measure � � � ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) y Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) n − 1 n − 1 x n , x ′ n Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 ϕ 1 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n � � � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ϕ 2 ( x 2 , x ′ ϕ 1 = U 2 1 ) 2 ) x 2 , x ′ 2 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 ϕ 1 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n � � � � � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ϕ 2 ( x 2 , x ′ ˆ ϕ 1 = U 2 1 ) 2 ) = U 2 , ϕ 2 L H ( ˜ X 2 ) x 2 , x ′ 2 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 ϕ 1 ϕ 0 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n � � � � � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ϕ 2 ( x 2 , x ′ ˆ ϕ 1 = U 2 1 ) 2 ) = U 2 , ϕ 2 L H ( ˜ X 2 ) x 2 , x ′ 2 � � � ˆ ( x 1 , x ) , ( x ′ 1 , x ′ ) ϕ 1 ( x 1 , x ′ ϕ 0 = U 1 1 ) x 1 , x ′ 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 ϕ 1 ϕ 0 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n � � � � � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ϕ 2 ( x 2 , x ′ ˆ ϕ 1 = U 2 1 ) 2 ) = U 2 , ϕ 2 L H ( ˜ X 2 ) x 2 , x ′ 2 � � � � � ˆ ( x 1 , x ) , ( x ′ 1 , x ′ ) ϕ 1 ( x 1 , x ′ ˆ ϕ 0 = U 1 1 ) = U 1 , ϕ 1 L H ( ˜ X 1 ) x 1 , x ′ 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 1 ˜ X 1 X n X Y Y · · · X ′ X ′ X ′ ˜ Y ′ 1 n ˆ ˆ ˆ ˆ U 1 U 2 U n B diag ( p ( x )) I y ϕ n ϕ n − 1 ϕ 1 ϕ 0 Unitary Evolution over time in n steps followed by a single projective measure � � � � � ˆ ˆ ϕ n ( x n , x ′ ( x n , y ) , ( x ′ n ) = B n , y ) = B , I y L H ( ˜ Y ) y � � � � � � ˆ ˆ x n − 1 , x ′ x ′ n , x ′ ϕ ( x n , x ′ � � � ϕ n − 1 = U n − 1 ( x n , x n − 1 ) , n ) = U n , ϕ n n − 1 n − 1 L H ( ˜ X n ) x n , x ′ n � � � � � ˆ ( x 2 , x 1 ) , ( x ′ 2 , x ′ ϕ 2 ( x 2 , x ′ ˆ ϕ 1 = U 2 1 ) 2 ) = U 2 , ϕ 2 L H ( ˜ X 2 ) x 2 , x ′ 2 � � � � � ˆ ( x 1 , x ) , ( x ′ 1 , x ′ ) ϕ 1 ( x 1 , x ′ ˆ ϕ 0 = U 1 1 ) = U 1 , ϕ 1 L H ( ˜ X 1 ) x 1 , x ′ 1 Heisenberg representation. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 9
Quantum Factor Graphs Several Examples Example 2 Y 1 Y 2 ˜ ˜ X 0 X 1 X 1 X 2 X 2 X ′ X ′ ˜ X ′ ˜ X ′ X ′ 0 1 2 1 2 ˆ ˆ ˆ ˆ I diag ( p ( x 0 )) U 0 A 1 U 1 A 2 A Two-Measurement Quantum System Here, we assume � � � � ˆ x k , x ′ = δ ( x k , x ′ A k (˜ x k , x k ) , (˜ k ) k ) y k x k Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 10
Quantum Factor Graphs Several Examples Example 2 Y 1 Y 2 ˜ ˜ X 0 X 1 X 1 X 2 X 2 X ′ X ′ ˜ X ′ ˜ X ′ X ′ 0 1 2 1 2 ˆ ˆ ˆ ˆ I diag ( p ( x 0 )) U 0 A 1 U 1 A 2 A Two-Measurement Quantum System Here, we assume � � � � ˆ x k , x ′ = δ ( x k , x ′ A k (˜ x k , x k ) , (˜ k ) k ) y k x k � � � A y k ˆ or, equivalently k , δ ˆ X ) = δ X k , X ′ X k , ˆ X ′ L H ( ˆ k k y k Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 10
Quantum Factor Graphs Several Examples Example 2 Y 1 Y 2 ˜ ˜ X 0 X 1 X 1 X 2 X 2 X ′ X ′ ˜ X ′ ˜ X ′ X ′ 0 1 2 1 2 ˆ ˆ ˆ ˆ I diag ( p ( x 0 )) U 0 A 1 U 1 A 2 A Two-Measurement Quantum System Here, we assume � � � � ˆ x k , x ′ = δ ( x k , x ′ A k (˜ x k , x k ) , (˜ k ) k ) y k x k � � � A y k ˆ or, equivalently k , δ ˆ X ) = δ X k , X ′ X k , ˆ X ′ L H ( ˆ k k y k Y k A special example: I y k ˆ ˆ B H B k k Projective Measurement with 1-dim Eigenspaces Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 10
Quantum Factor Graphs Several Examples Example 3 Y 1 Y 2 ˜ ˜ X 1 X 1 X 2 X 2 ˆ ˆ A 1 A 2 X 0 X 3 X ′ ˜ X ′ ˜ X ′ X ′ 1 2 1 2 W 1 W 2 X ′ X ′ 0 3 I W ′ W ′ diag ( p ( x 0 )) 1 2 ˆ ˆ ˆ U 0 U 1 U 2 A Quantum System with partial measurement Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 11
Quantum Factor Graphs Several Examples Example 3 Y 1 Y 2 ˜ ˜ X 1 X 1 X 2 X 2 ˆ ˆ A 1 A 2 X 0 X 3 X ′ ˜ X ′ ˜ X ′ X ′ 1 2 1 2 W 1 W 2 X ′ X ′ 0 3 I W ′ W ′ diag ( p ( x 0 )) 1 2 ˆ ˆ ˆ U 0 U 1 U 2 A Quantum System with partial measurement X 0 = X 1 × W 1 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 11
Quantum Factor Graphs Several Examples Example 3 Y 1 Y 2 ˜ ˜ X 1 X 1 X 2 X 2 ˆ ˆ A 1 A 2 X 0 X 3 X ′ ˜ X ′ ˜ X ′ X ′ 1 2 1 2 W 1 W 2 X ′ X ′ 0 3 I W ′ W ′ diag ( p ( x 0 )) 1 2 ˆ ˆ ˆ U 0 U 1 U 2 A Quantum System with partial measurement X 0 = X 1 × W 1 This QFG contains cycles. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 11
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a x 4 x 5 c b x 1 x 2 x 3 e d f Sum-Product Algorithm on a normal factor graph with no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 c b 1 x 1 x 2 x 3 e d f Sum-Product Algorithm on a normal factor graph with no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 c bd x 2 x 3 e f Sum-Product Algorithm on a normal factor graph with no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 c bd 2 f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 x 3 e f x 2 Sum-Product Algorithm on a normal factor graph with no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 ce bd f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 3 f x 2 Sum-Product Algorithm on a normal factor graph with no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 ce bd 3 f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 3 f x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 bd cef f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a 4 f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 4 x 5 x 1 bd cef f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles f abd ( x 5 ) = � f a ( x 4 , x 5 ) f bd ( x 4 ) x 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x abd f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 5 x 1 cef f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles f abd ( x 5 ) = � f a ( x 4 , x 5 ) f bd ( x 4 ) x 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x abd 5 f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 5 x 1 cef f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles f abd ( x 5 ) = � f a ( x 4 , x 5 ) f bd ( x 4 ) x 4 Z = f abcdef = � f abd ( x 5 ) f cef ( x 5 ) x 5 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x abcdef f bd ( x 4 ) = � f b ( x 1 , x 4 ) f d ( x 1 ) x 1 f ce ( x 3 , x 5 ) = � f c ( x 2 , x 3 , x 5 ) f e ( x 2 ) x 2 f cef ( x 5 ) = � f ce ( x 3 , x 5 ) f f ( x 3 ) Sum-Product Algorithm on a normal factor graph with x 3 no cycles f abd ( x 5 ) = � f a ( x 4 , x 5 ) f bd ( x 4 ) x 4 Z = f abcdef = � f abd ( x 5 ) f cef ( x 5 ) x 5 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � g ( x ) x a = � f b , f d � x 1 f bd x 4 x 5 c b f ce = � f c , f e � x 2 x 1 x 2 x 3 e d f f cef = � f ce , f f � x 3 Sum-Product Algorithm on a normal factor graph with no cycles f abd = � f a , f bd � x 4 Z = f abcdef = � f abd , f cef � x 5 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm for Acyclic Factor Graphs Target: Calculate Z ( G ) � � x , x ′ g ( x , x ′ ) a x ′ x ′ 4 5 = � f b , f d � L H ( X 1 ) f bd x 4 x 5 c b x ′ x ′ x ′ 1 2 3 f ce = � f c , f e � L H ( X 2 ) x 1 x 2 x 3 e d f f cef = � f ce , f f � L H ( X 3 ) Sum-Product Algorithm on a quantum normal factor graph with no cycles f abd = � f a , f bd � L H ( X 4 ) Z = f abcdef = � f abd , f cef � L H ( X 5 ) Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 12
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm: General Rules More generally, we are applying following two rules: a m i → a PSD i m a 1 → i m a 2 → i PSD PSD a 1 a 2 � m b → i ( x i , x ′ m i → a ← i ) b ∈ ∂ i \{ a } Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 13
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm: General Rules More generally, we are applying following two rules: a i m i → a m a → i PSD PSD i a m a 1 → i m a 2 → i m i 1 → a m i 2 → a PSD PSD PSD PSD a 1 a 2 i 1 i 2 � � � m b → i ( x i , x ′ � m i → a ← i ) m a → i ← m j → a , f a b ∈ ∂ i \{ a } j ∈ ∂ a \{ i } ∂ a \{ i } Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 13
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Sum-Product Algorithm: General Rules More generally, we are applying following two rules: a i m i → a m a → i PSD PSD i a m a 1 → i m a 2 → i m i 1 → a m i 2 → a PSD PSD PSD PSD a 1 a 2 i 1 i 2 � � � m b → i ( x i , x ′ � m i → a ← i ) m a → i ← m j → a , f a b ∈ ∂ i \{ a } j ∈ ∂ a \{ i } ∂ a \{ i } with initialization at the leaf factors m a → i ( x i , x ′ i ) = f a ( x i , x ′ i ) where { i } = ∂ a Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 13
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm For general QFGs with cycles Definition 2 (General Sum-Product / Belief Propagation Algorithm for QFG) For a general QFG G = {F , V , E} with global funciton g ( x , x ′ ) = � f a ( x ∂ a , x ′ � h i ( x i , x ′ ∂ a ) i ) . (1) a ∈F i ∈V Update rules for belief propagation (BP) algorithm: � � m ( t +1) � m ( t ) ∝ j → a , f a (2) a → i j ∈ ∂ a \{ i } L h ( X ∂ a \{ i } ) m ( t +1) m ( t ) � i → a ∝ h i · (3) b → i b ∈ ∂ i \{ a } The messages are said to be fixed-point messages when above equations holds without time-stamp superscripts. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 14
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm For general QFGs with cycles Definition 2 (General Sum-Product / Belief Propagation Algorithm for QFG) For a general QFG G = {F , V , E} with global funciton g ( x , x ′ ) = � f a ( x ∂ a , x ′ � h i ( x i , x ′ ∂ a ) i ) . (1) a ∈F i ∈V Update rules for belief propagation (BP) algorithm: � � � m a → i ∝ m j → a , f a (2) j ∈ ∂ a \{ i } L h ( X ∂ a \{ i } ) � m i → a ∝ h i · m b → i (3) b ∈ ∂ i \{ a } The messages are said to be fixed-point messages when above equations holds without time-stamp superscripts. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 14
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Loop Calculus for SP / BP Algorithms Theorem 3 ( Loop Calculus [Chertkov and Chernyak, 2006, Mori, 2015a, Mori, 2015b]) At BP-fixed point, we have �� � � � � � Z � = Z Bethe 1 + K ( E ) f a , h i a ∈F i ∈V E ⊂E ′ L ( X V ) where the extended loop set is defined as E ′ � { E ⊂ E\ { φ } : d i ( E ) � = 1 ∀ i ∈ V , d a ( E ) � = 1 ∀ i ∈ F} where K ( E ) is some function depending on E, and K ( φ ) = 1 , and � � � � � � m ( t ) m ( t ) � � � h i , � Z a Z i i → a , f a a → i a ∈F i ∈V a ∈F i ∈ ∂ a i ∈V a ∈ ∂ i Z Bethe � = . � � Z i , a � m a → i , m i → a � ( i , a ) ∈E ( i , a ) ∈E Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 15
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Loop Calculus for BP Algorithms Interpretation Bethe Approximation is exact for acyclic QFG; Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 15
Problem of Calculating the Partition Sum Sum-Product / Belief Propagation Algorithm Loop Calculus for BP Algorithms Interpretation Bethe Approximation is exact for acyclic QFG; Bethe Approximation is close to the exact value for QFGs with small number of cycles. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 15
Problem of Calculating the Partition Sum Exploration on Variational Approach Outline Classical Factor Graphs 1 Modeling “Closing-the-box” Operation Quantum Factor Graphs 2 A Motivating Example Quantum Factor Graph Construction of a QNFG Several Examples Problem of Calculating the Partition Sum 3 Sum-Product / Belief Propagation Algorithm Exploration on Variational Approach End Matters 4 Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Problem of Calculating the Partition Sum Exploration on Variational Approach Variational Approach for Classic Factor Graphs Target: Calculate Z ( G ) � � g ( x ), where g ( x ) = � f a ( x a ) � h i ( x i ) x a ∈F i ∈V I: Calculate F H � − ln Z ( G ); Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Problem of Calculating the Partition Sum Exploration on Variational Approach Variational Approach for Classic Factor Graphs Target: Calculate Z ( G ) � � g ( x ), where g ( x ) = � f a ( x a ) � h i ( x i ) x a ∈F i ∈V I: Calculate F H � − ln Z ( G ); II: Minimize F Gibbs ( b ) over all possible global probability function b ( x ); � � b is a probability function F Gibbs ( b ) � − min b ( x ) ln f a ( x ∂ a ) a ∈F x � � − b ( x ) ln h i ( x i ) i ∈V x � + b ( x ) ln b ( x ) x = F H + D ( b � p ) � F H . Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Problem of Calculating the Partition Sum Exploration on Variational Approach Variational Approach for Classic Factor Graphs Target: Calculate Z ( G ) � � g ( x ), where g ( x ) = � f a ( x a ) � h i ( x i ) x a ∈F i ∈V I: Calculate F H � − ln Z ( G ); II: Minimize F Gibbs ( b ) over all possible global probability function b ( x ); � � III: Minimize F Bethe { b a } a ∈F , { b i } i ∈V over all valid marginal probability functions { b a } a ∈F , { b i } i ∈V ; � � � � � � � − ( b a ) a ∈F , ( b i ) i ∈V b a ( x ∂ a ) ln f a ( x ∂ a ) − b i ( x i ) ln h i ( x i ) F Bethe a ∈F x ∂ a i ∈V x i � � + b a ( x ∂ a ) ln b a ( x ∂ a ) a ∈F x ∂ a � � − ( d i − 1) b i ( x i ) ln b i ( x i ) i ∈V x i = F Gibbs for acyclic factor graphs. Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Problem of Calculating the Partition Sum Exploration on Variational Approach Variational Approach for Classic Factor Graphs Target: Calculate Z ( G ) � � g ( x ), where g ( x ) = � f a ( x a ) � h i ( x i ) x a ∈F i ∈V I: Calculate F H � − ln Z ( G ); II: Minimize F Gibbs ( b ) over all possible global probability function b ( x ); � � III: Minimize F Bethe { b a } a ∈F , { b i } i ∈V over all valid marginal probability functions { b a } a ∈F , { b i } i ∈V ; IV: Study the Stationary Condition of above optimization problem, which turned out to be equivalent to � � min ( b a ) a ∈F , ( b i ) i ∈V b a , b i probability functions F Bethe � s . t . b a ( x a ) = b i ( x i ) ∀ ( i , a ) ∈ E , ∀ x i ∈ X i x ∂ a \{ i } Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Problem of Calculating the Partition Sum Exploration on Variational Approach Variational Approach for Classic Factor Graphs Target: Calculate Z ( G ) � � g ( x ), where g ( x ) = � f a ( x a ) � h i ( x i ) x a ∈F i ∈V I: Calculate F H � − ln Z ( G ); II: Minimize F Gibbs ( b ) over all possible global probability function b ( x ); � � III: Minimize F Bethe { b a } a ∈F , { b i } i ∈V over all valid marginal probability functions { b a } a ∈F , { b i } i ∈V ; IV: Study the Stationary Condition of above optimization problem, which turned out to be equivalent to � b a ∝ f a · m i → a i ∈ ∂ a � b i ∝ h 1 · m a → i a ∈ ∂ i � b a ( x a ) = b i ( x i ) ∀ ( i , a ) ∈ E , ∀ x i ∈ X i x ∂ a \{ i } Michael X. CAO (IE@CUHK) Quantum Factor Graph September 25, 2015 16
Recommend
More recommend
