SLIDE 1
Classical AI Methods:
Hungry? $25? Restaurant? Sleep?
Clear Modeling Assumption
…
Neural Networks:
“Black Box” Good performance on Image Classification
1
Learning Logistic Circuits Yitao Liang, Guy Van den Broeck January - - PowerPoint PPT Presentation
Learning Logistic Circuits Yitao Liang, Guy Van den Broeck January - - PowerPoint PPT Presentation
Learning Logistic Circuits Yitao Liang, Guy Van den Broeck January 31, 2019 Which model to choose Neural Networks: Classical AI Methods: Hungry? $25? Sleep? Restaurant? Black Box Clear Modeling Assumption Good performance on
SLIDE 2
SLIDE 3
Starting Point: Probabilistic Circuits
A promising synthesis of the two
3
SPN
State-of-the-art on Density Estimation !"($)
2
SLIDE 4
4
What if we only want to learn a classifier !" # $)
3
SLIDE 5
Logical Circuits
4
A ¬A B ¬B C ¬C D ¬D
Input:
A B C D 1 1
1 1 1 1 1
Bottom-up Evaluation
0 = 1 AND 0 1 1 1 1 1 1 1 1
SLIDE 6
Logical -> Probabilistic Circuits
5
Red Parameters: Conditional Probabilities
0.9 0.1 0.2 0.8 0.6 0.4 0.1 0.9 0.3 0.7 0.1 0.9 0.8 0.2 A ¬A B ¬B C ¬C D ¬D
SLIDE 7
0.9 0.1 0.2 0.8 0.6 0.4 0.1 0.9 0.3 0.7 0.1 0.9 0.8 0.2 A ¬A B ¬B C ¬C D ¬D
Logical -> Probabilistic Circuits
6
Input:
A B C D Pr(A, B, C, D) 1 1 ?
1 1 1 1
Multiply the parameters bottom-up
1
0.1 0.8 0.0 0.3
0.01 0.24 0.00 0.194 0.096 0.096
!"($, &, ', () = +. +-.
0.1= 0.1*1 + 0.9*0
0.24= 0.8*0.3
SLIDE 8
Evaluate Logistic Circuits
7
Input:
A B C D Pr(Y | A, B, C, D) 1 1 ?
Multiply the parameters bottom-up Logistic function on final output
−2.6 −5.8 −1 3 4 2.3 −0.5 0.3 1.5 2.8 −4 1 3.9 4 A ¬A B ¬B C ¬C D ¬D
1 1 1 1
!" # = % &, (, ), *) =
% %,-./(1%.3) = 4. 563
SLIDE 9
Are logistic circuits amenable to tractable learning
8
SLIDE 10
Special Case: Logistic Regression
9
What about other logistic circuits in more general forms?
Pr # = 1 &, (, ), * = 1 ) 1 + ex p( − & ∗ 34 − ¬& ∗ 3¬4 − ( ∗ 36 − ⋯
Logistic Regression
SLIDE 11
Parameter Learning
“Hot” wires are active features Pr($ = 1 ∣ ( = 0, + = 1, , = 1, - = 0)
10
SLIDE 12
Due to decomposability and determinism, reduce to logistic regression
Features associated with each wire “Global Circuit Flow”
Parameter Learning
11
Convex Parameter learning
SLIDE 13
14
Similar to LearnPsdd
Calculate Variance Execute the best Generate candidate
- perations
Structure Learning
Split nodes to reduce variance of gradients
12
SLIDE 14
15
Comparable Accuracy with Neural Nets
13
SLIDE 15
16
Significantly Smaller in Size
14
SLIDE 16
17
Better Data Efficiency
15
SLIDE 17
18
Probabilities become log-odds
Probabilistic -> Logistic Circuits
Discriminative Counterparts
16
SLIDE 18
19
This is the feature that contributes the most to this image’s classification probability feature value : 0.925 feature weight : 3.489 feature interpretation: curvy lines and hallow center
What do Features Mean
17
SLIDE 19
Conclusion
18
- Synthesis of symbolic AI and statistical learning
- Discriminative counterparts of probabilistic circuits
- Convex parameter learning
- Simple heuristic for structure learning
- Good performance
- Easy to interpret
Logistic circuits:
SLIDE 20
Thanks
https://github.com/UCLA-StarAI/LogisticCircuit