Smoothing Structured Decomposable Circuits Andy Shih 1 Guy Van den - - PowerPoint PPT Presentation

▶

Nov 25, 2023 277 likes •385 views

Smoothing Structured Decomposable Circuits Andy Shih 1 Guy Van den Broeck 2 Paul Beame 3 Antoine Amarilli 4 1 Stanford University 2 University of California, Los Angeles 3 University of Washington 4 LTCI, Tlcom Paris, IP Paris NeurIPS 2019

SLIDE 1

Smoothing Structured Decomposable Circuits

Andy Shih1 Guy Van den Broeck2 Paul Beame3 Antoine Amarilli4

1Stanford University 2University of California, Los Angeles 3University of Washington 4LTCI, Télécom Paris, IP Paris

NeurIPS 2019

SLIDE 2

Probabilistic Circuits

Tractable computation graph, encoding a distribution. SOTA for: ▶ Inference algorithms for PGMs / probabilistic programs ▶ Discrete density estimation Exact likelihoods and partition function! Gaining popularity: Tractable Probabilistic Models: (UAI19 / AAAI20 tutorial)

Smoothing Structured Decomposable Circuits 1/ 8

SLIDE 3

Tractability

Difgerent combination of properties leads to difgerent families of circuits SPN AC PSDD Decomposability ✓ ✓ ✓(S) Determinism ✗ ✓ ✓ Smoothness ✓ ✓ ✓ Pr(evid) ✓ ✓ ✓ Marginal ✓ ✓ ✓ MPE ✗ ✓ ✓ Marginal MAP ✗ ✗ ✓∗ Expectation ✗ ✗ ✓∗ ...with difgerent tractability properties.

Smoothing Structured Decomposable Circuits 2/ 8

SLIDE 4

Smoothness

Defjnition A circuit is smooth if for every pair of children c1 and c2 of a ⊕-gate, varsc1 = varsc2.

!" !# !$ (a) A circuit.

!" !#

!$ -!$

!" -!"
!$
!# -!#

(b) A smooth circuit.

Figure: Two equivalent circuits computing (x0 ⊗ x1) ⊕ x2. The left one is not smooth and the right one is smooth.

Smoothing Structured Decomposable Circuits 3/ 8

SLIDE 5

Smoothing a Circuit: Prior Work

▶ Go to each gate O(m) and fjll in each variable O(n) ▶ Quadratic Complexity O(nm) ▶ Problematic when n ≥ 1, 000 and m ≥ 1, 000, 000 Our near-linear smoothing algorithm: O(m · α(m, n))

Smoothing Structured Decomposable Circuits 4/ 8

SLIDE 6

Smoothing a Circuit: Our Work

Key Insight: missing variables for each gate form two intervals.

A

Figure: A \ B forms two intervals

We need to fjll in 2m intervals.

Smoothing Structured Decomposable Circuits 5/ 8

SLIDE 7

Semigroup Range Sum

Theorem Given n variables defjned over a semigroup and m intervals, the sum of all intervals can be computed using O(m · α(m, n)) additions [Chazelle and Rosenberg 1989]. α(m, n) is the inverse Ackermann function, which grows very slowly. *The original theorem only bounds the number of additions. We bound the number of computations.

Smoothing Structured Decomposable Circuits 6/ 8

SLIDE 8

Takeaways

▶ Probabilistic circuits can encode complex distributions. ▶ They can compute exact likelihoods, marginals, and more

▶ But only if they are smooth.

▶ Best smoothing algorithm was quadratic. ▶ We propose a near linear time smoothing algorithm.

Smoothing Structured Decomposable Circuits 7/ 8

SLIDE 9

Thanks! Poster: East Exhibition Hall B+C #182, 10:45AM Code: https://github.com/AndyShih12/SSDC Contact: andyshih@cs.stanford.edu

Smoothing Structured Decomposable Circuits 8/ 8