A Semantic Loss Function for Deep Learning with Symbolic Knowledge - PowerPoint PPT Presentation

A Semantic Loss Function for Deep Learning with Symbolic Knowledge Jingyi Xu, Zilu Zhang , Tal Friedman , Yitao Liang, Guy Van den Broeck

Goal : Constrain neural network outputs using logic 1

Multiclass Classification 0.8 0.3 0.9 2 2

Multiclass Classification 𝒚 𝟐 ¬𝒚 𝟑 ¬𝒚 𝟒 ∨ 0.8 0.3 0.9 Want exactly one class: ¬𝒚 𝟐 𝒚 𝟑 ¬𝒚 𝟒 ∨ ¬𝒚 𝟐 ¬𝒚 𝟑 𝒚 𝟒 3 3

Multiclass Classification 𝒚 𝟐 ¬𝒚 𝟑 ¬𝒚 𝟒 T F T ∨ 0.8 0.3 0.9 Want exactly one class: ¬𝒚 𝟐 𝒚 𝟑 ¬𝒚 𝟒 ∨ ¬𝒚 𝟐 ¬𝒚 𝟑 𝒚 𝟒 No information gained! 4 4

Why is mixing so difficult? Deep Learning Logic • Continuous • Discrete • Smooth • Symbolic • Differentiable • Strong semantics 5 5

Multiclass Classification 𝒚 𝟐 ¬𝒚 𝟑 ¬𝒚 𝟒 ∨ 0.8 0.3 0.9 Want exactly one class: ¬𝒚 𝟐 𝒚 𝟑 ¬𝒚 𝟒 ∨ ¬𝒚 𝟐 ¬𝒚 𝟑 𝒚 𝟒 Probability constraint is satisfied 6 6

Use a probabilistic interpretation! 7

Multiclass Classification 𝒚 𝟐 ¬𝒚 𝟑 ¬𝒚 𝟒 ∨ 0.8 0.3 0.9 Want exactly one class: ¬𝒚 𝟐 𝒚 𝟑 ¬𝒚 𝟒 ∨ ¬𝒚 𝟐 ¬𝒚 𝟑 𝒚 𝟒 Probability constraint is satisfied 𝒚 𝟐 𝟐 − 𝒚 𝟑 𝟐 − 𝒚 𝟒 + 𝟐 − 𝒚 𝟐 𝒚 𝟑 𝟐 − 𝒚 𝟒 + 𝟐 − 𝒚 𝟐 𝟐 − 𝒚 𝟑 𝒚 𝟒 = 𝟏. 𝟐𝟗𝟗 8 8

Semantic Loss • Continuous, smooth, easily differentiable function • Represents how close outputs are to satisfying the constraint • Axiomatically respects semantics of logic, maintains precise meaning – independent of syntax 9

How do we compute semantic loss? 10

Logical Circuits • In general: #P-hard • Linear in size of circuit = - log( ) L(α, p ) = L( , p ) 11

Supervised Learning • Predict shortest paths • Add semantic loss representing paths Is output Is output Does output a path? the true shortest path? have true edges? 12

Semi-Supervised Learning • Unlabeled data must have some label 13

Semi-Supervised Learning • Unlabeled data must have some label • Exactly-one constraint increases confidence 14

15

Main Takeaway • Deep learning and logic can be combined by using a probabilistic approach • Maintain precise meaning while fitting into the deep learning framework 16

Thanks! 17