Arithmetic Circuits of the Noisy-Or Models Jirka Vomlel and Petr - PowerPoint PPT Presentation

Arithmetic Circuits of the Noisy-Or Models Jirka Vomlel and Petr Savick´ y Academy of Sciences of the Czech Republic PGM’08, Hirtshals, Denmark y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 1 / 21

Contents BN2O models. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 2 / 21

Contents BN2O models. Methods exploiting the local structure of noisy-or models. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 2 / 21

Contents BN2O models. Methods exploiting the local structure of noisy-or models. Arithmetic circuits - a measure of inference complexity. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 2 / 21

Contents BN2O models. Methods exploiting the local structure of noisy-or models. Arithmetic circuits - a measure of inference complexity. Results of experiments. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 2 / 21

BN2O model Bipartite graph with two levels of Boolean variables: X j , j = 1 , . . . , x and Y i , i = 1 , . . . , y . X 1 X 2 X 3 X 4 Y 1 Y 2 Y 3 Y 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 3 / 21

BN2O model Bipartite graph with two levels of Boolean variables: X j , j = 1 , . . . , x and Y i , i = 1 , . . . , y . X 1 X 2 X 3 X 4 Y 1 Y 2 Y 3 Y 4 CPT of Y i is a noisy-or gate: n � ( p i , j ) x j , P ( Y i = 0 | X Pa ( i ) = x Pa ( i ) ) = j ∈ Pa ( i ) y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 3 / 21

BN2O model Bipartite graph with two levels of Boolean variables: X j , j = 1 , . . . , x and Y i , i = 1 , . . . , y . X 1 X 2 X 3 X 4 Y 1 Y 2 Y 3 Y 4 CPT of Y i is a noisy-or gate: n � ( p i , j ) x j , P ( Y i = 0 | X Pa ( i ) = x Pa ( i ) ) = j ∈ Pa ( i ) where p i , j is the inhibition probability for the parent X j of Y i . y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 3 / 21

BN2O model Bipartite graph with two levels of Boolean variables: X j , j = 1 , . . . , x and Y i , i = 1 , . . . , y . X 1 X 2 X 3 X 4 Y 1 Y 2 Y 3 Y 4 CPT of Y i is a noisy-or gate: n � ( p i , j ) x j , P ( Y i = 0 | X Pa ( i ) = x Pa ( i ) ) = j ∈ Pa ( i ) where p i , j is the inhibition probability for the parent X j of Y i . Y i is false only if all its parents with value true are inhibited. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 3 / 21

Compilation of a noisy-or gate - the standard BN approach Lauritzen and Spiegelhalter (1988), Jensen et al. (1990), Shafer and Shenoy (1990) X 1 X 2 Y X 3 X 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 4 / 21

Compilation of a noisy-or gate - the standard BN approach Lauritzen and Spiegelhalter (1988), Jensen et al. (1990), Shafer and Shenoy (1990) X 1 X 1 X 2 X 2 Y Y X 3 X 3 X 4 X 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 4 / 21

Compilation of a noisy-or gate - the standard BN approach Lauritzen and Spiegelhalter (1988), Jensen et al. (1990), Shafer and Shenoy (1990) X 1 X 1 X 2 X 2 Y Y X 3 X 3 X 4 X 4 The total table size is 2 5 = 32. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 4 / 21

Compilation of a noisy-or gate - parent divorcing Olesen et al. (1989) X 1 X 2 Y X 3 X 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 5 / 21

Compilation of a noisy-or gate - parent divorcing Olesen et al. (1989) X 1 X 1 X 2 X 2 Y X 3 X 3 X 4 X 4 A 1 A 3 Y y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 5 / 21

Compilation of a noisy-or gate - parent divorcing Olesen et al. (1989) X 1 X 1 X 1 X 2 X 2 X 2 Y X 3 X 3 X 3 X 4 X 4 A 1 A 3 Y X 4 A 1 A 3 Y y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 5 / 21

Compilation of a noisy-or gate - parent divorcing Olesen et al. (1989) X 1 X 1 X 1 X 2 X 2 X 2 Y X 3 X 3 X 3 X 4 X 4 A 1 A 3 Y X 4 A 1 A 3 Y The total table size is 3 · 2 3 = 24. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 5 / 21

Rank-one decomposition D´ ıez and Gal´ an (2003), Vomlel (2002), Savick´ y and Vomlel (2007) n p x j � � P ( Y i = y i | X Pa ( i ) = x Pa ( i ) ) = (1 − 2 y i ) i , j + y i 1 i =1 j ∈ Pa ( i ) y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 6 / 21

Rank-one decomposition D´ ıez and Gal´ an (2003), Vomlel (2002), Savick´ y and Vomlel (2007) n p x j � � P ( Y i = y i | X Pa ( i ) = x Pa ( i ) ) = (1 − 2 y i ) i , j + y i 1 i =1 j ∈ Pa ( i ) 1 � � = ξ ( b i , y i ) · ϕ i , j ( b i , x j ) b i =0 j ∈ Pa ( i ) y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 6 / 21

Rank-one decomposition D´ ıez and Gal´ an (2003), Vomlel (2002), Savick´ y and Vomlel (2007) n p x j � � P ( Y i = y i | X Pa ( i ) = x Pa ( i ) ) = (1 − 2 y i ) i , j + y i 1 i =1 j ∈ Pa ( i ) 1 � � = ξ ( b i , y i ) · ϕ i , j ( b i , x j ) b i =0 j ∈ Pa ( i ) 1 p 1 X 1 1 1 1 p 2 X 2 1 1 1 − 1 B Y 0 1 1 p 3 X 3 1 1 1 p 4 X 4 1 1 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 6 / 21

Correspondence to tensor rank-one decomposition Savick´ y and Vomlel (2007) A decomposition of a conditional probability table P ( Y | X 1 , . . . , X n ) using the auxiliary variable B � � P ( Y i | X Pa ( i ) ) = ξ ( B , Y i ) · ϕ i , j ( B , X j ) B j ∈ Pa ( i ) that has the minimal number of states of B y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 7 / 21

Correspondence to tensor rank-one decomposition Savick´ y and Vomlel (2007) A decomposition of a conditional probability table P ( Y | X 1 , . . . , X n ) using the auxiliary variable B � � P ( Y i | X Pa ( i ) ) = ξ ( B , Y i ) · ϕ i , j ( B , X j ) B j ∈ Pa ( i ) that has the minimal number of states of B is in fact a (minimal) tensor rank-one decomposition of tensor P ( Y i | X Pa ( i ) ). y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 7 / 21

Correspondence to tensor rank-one decomposition Savick´ y and Vomlel (2007) A decomposition of a conditional probability table P ( Y | X 1 , . . . , X n ) using the auxiliary variable B � � P ( Y i | X Pa ( i ) ) = ξ ( B , Y i ) · ϕ i , j ( B , X j ) B j ∈ Pa ( i ) that has the minimal number of states of B is in fact a (minimal) tensor rank-one decomposition of tensor P ( Y i | X Pa ( i ) ). Definition (Tensor of rank one) A tensor has rank one if it is the outer product of vectors. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 7 / 21

Compilation of a noisy-or gate - rank-one decomposition X 1 X 2 Y X 3 X 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 8 / 21

Compilation of a noisy-or gate - rank-one decomposition X 1 X 1 X 2 X 2 Y B Y X 3 X 3 X 4 X 4 y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 8 / 21

Compilation of a noisy-or gate - rank-one decomposition X 1 X 1 X 2 X 2 Y B Y X 3 X 3 X 4 X 4 The total table size is 5 · 2 2 = 20. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 8 / 21

Comparisons for the noisy-or gate 1e+06 the standard BN approach parent divorcing rank-one decomposition 100000 the total table size 10000 1000 100 10 4 6 8 10 12 14 |Pa(i)| y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 9 / 21

Arithmetic circuits Definition (Arithmetic circuit (AC)) An AC is a rooted, directed acyclic graph whose leaf nodes correspond to its inputs and whose other nodes are labeled with multiplication and addition operations. The root node corresponds to the output of the AC. y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 10 / 21

Arithmetic circuits Definition (Arithmetic circuit (AC)) An AC is a rooted, directed acyclic graph whose leaf nodes correspond to its inputs and whose other nodes are labeled with multiplication and addition operations. The root node corresponds to the output of the AC. Circuit inputs: y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 10 / 21

Arithmetic circuits Definition (Arithmetic circuit (AC)) An AC is a rooted, directed acyclic graph whose leaf nodes correspond to its inputs and whose other nodes are labeled with multiplication and addition operations. The root node corresponds to the output of the AC. Circuit inputs: BN parameters θ x | u = P ( X = x | U = u ) y (AV ˇ J. Vomlel and P. Savick´ CR) ACs of BN2O PGM’08, Hirtshals, Denmark 10 / 21