Exploiting Innocuousness in Bayesian Networks 28 th Australasian - - PowerPoint PPT Presentation

INSTITUTE OF INFORMATION SYSTEMS

Exploiting Innocuousness in Bayesian Networks

28th Australasian Joint Conference on Artificial Intelligence

Alexander Motzek❻ Ralf Möller❻

❻Universität zu Lübeck

Institute of Information Systems Ratzeburger Allee 160, 23562 Lübeck, Germany {motzek,moeller}@ifis.uni-luebeck.de

December, 4th 2015

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Introduction

▲ (Dynamic) Bayesian Networks. ▲ New form of independence - Innocuousness. ▲ New form of DBNs - Activator DBNs. ▲ Formalize and Exploit.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Bayesian Networks

▲ Syntactically defined by a graph B. ▲ Local semantics as specifications of local CPD ▲ Global semantic as the joint probability

P❼Ñ X➁ ▼

X❃Ñ X

P❼X❙parents❼X➁➁

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Bayesian Networks

▲ Graph encodes guaranteed independencies. ▲ Not dependencies! ▲ Actual dependencies encoded/specified in CPDs.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Independencies - A Gedankenexperiment

▲ Multiple causes can cause one effect. ▲ Our hand is exposed to various risks in a blackbox. ▲ Exposures can cause Harm.

H

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Independencies - A Gedankenexperiment

▲ E.g., exposures to Sand, Bunsen burner, O2 ▲ P❼H❙S, B, O2➁ ▲ ✔sand is present or not ✥s present,

a ✔burner is turned on, ✔o2 is present

▲ Exposures might be part of a much deeper

probabilistic process.

H B S O2

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Causal Independence

▲ Classic. Sand is completely irrelevant, i.e., independent. ▲ Investigated by Zhang and Poole, et al.

(i) change graph. (ii) P❼H❙S, B, O2➁ P❼H❙B, O2➁ specifiable in a local CPD by ➛h, b, o2 ✂ P❼h❙ ✔ s, b, o2➁ P❼h❙✥s, b, o2➁

H B S O2

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Context-Specific Independence

▲ A Bunsen burner only works / can only causes harm if ✔o2 is present. ▲ Investigated by Boutelier et al. ▲ P❼H❙S, B, ✥o2➁ P❼H❙S, ✥o2➁ specifiable in local CPD by

➛s, h, b, o2 ✂ P❼h❙s, ✔b, ✥o2➁ P❼h❙s, ✥b, ✥o2➁ ➜s, h, b, o2 ✂ P❼h❙s, ✔b, ✔o2➁ ① P❼h❙s, ✥b, ✔o2➁

H B S ✥o2

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Innocuousness

▲ Allegedly, Burner is only relevant if it is turned on ✔b. ▲ A turned off ✥burner is completely irrelevant, could have been left out. ▲ Very commonly found in Noisy-OR Assumptions.

‘‘A false dependence does not cause any harm’’.

▲ How to formalize?

The relevant context ✥b is the ‘‘irrelevant’’ random variable to be removed...

▲ P❼H❙S, ✥b, O2➁ P❼H❙S, O2➁ not specifiable/expressible? H ✥b S O2 ???

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Why formalize innocuousness?

Expressing Innocuousness is interesting:

▲ More expressive and causal specifications ▲ Removing a link is always good (computation time...) ▲ Can actually be formalized with and are beneficial for ADBNs.

IJCAI’15

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

ADBNs - Running example

▲ Regulatory compliance of employees. ▲ A ‘‘creduluous’’ employee might manipulate documents. ▲ A credulous employee might (undeliberately) influence other employees. ▲ Might become credulous too, etc. ▲ Influences occur through exchanged messages. ▲ Track probabilistic credulousness-state over time. ▲ Employees: Claire, Don and Eearl.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Problem as a DBN

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

DC

▲ Say, only Claire influences Don,

influences Earl.

▲ i.e. C influences E indirectly. ▲ Typical DBN. ✓ ▲ Problem correctly represented. ✓

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Problem as a DBN

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

DC

M1

CE

M2

CE

▲ Let’s add some more influences. ▲ Claire can also influence Earl directly. ▲ Typical DBN. ✓ ▲ Problem correctly represented. ✓

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Problem as a DBN

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

DC

M1

CE

M2

CE

M1

DC

M1

EC

M1

ED

M2

DC

M2

EC

M2

ED

▲ Say, everybody can influence everybody. ▲ ‘‘A BN is a DAG’’. ▲ Not a DBN.✗ ▲ Problem correctly represented. ✓?

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Problem as a DBN

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

DC

M1

CE

M2

CE

M1

DC

M1

EC

M1

ED

M2

DC

M2

EC

M2

ED

▲ Resolve cycles over time. ▲ ‘‘Diagonal’’ inter-state dependencies. ▲ Common DBN . ✓ ▲ Problem correctly represented.✗ ?

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

DBN Restrictions

▲ ‘‘Diagonal’’ encodes ‘‘incubation time’’:

t: Receive Message. t ✔ 1: Read and become influenced. a) Enforces infinitesimal resolution of time (e.g., seconds) ✗ High computation cost.

Observations not available this fine (e.g., only daily)? Computation too costly? Transition only known hourly?

b) Indirect influences not considerable. ✗ Does not explain the world.

C0 D0 E0 C1 D1 E1 C2 D2 E2 C3 D3 E3 C4 D4 E4 C5 D5 E5 C0 D0 E0 C1 D1 E1 C2 D2 E2

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Classic DBNs spread indirect effects over time

C0 D0 E0 C1 D1 E1 C2 D2 E2 C0 D0 E0 C1 D1 E1 C2 D2 E2

I.e., observations that require anticipations of indirect effects are not supported.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Classic DBNs spread indirect effects over time

C0 D0 E0 C1 D1 E1 C2 D2 E2 C0 D0 E0 C1 D1 E1 C2 D2 E2

I.e., observations that require anticipations of indirect effects are not supported.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Intuitive Design

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

CE

M2

CE

M1

DC

M1

EC

M1

ED

M2

DC

M2

EC

M2

ED

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Activator Random Variables

▲ Random variables Mt XY representing exchanged messages are special ▲ Mt XY have activator nature, i.e., are Activator Random Variables

➛x, x➐ ❃ dom❼X➁, ➛y ❃ dom❼Y➁, ➛Ñ z ❃ dom❼Ñ Z➁ ✂ P❼y❙x, ✥aXY,Ñ z➁ P❼y❙x➐, ✥aXY,Ñ z➁ P❼y❙❻, ✥aXY,Ñ z➁

❻ wildcard, Ñ z further dependencies

➜x, x❻ ❃ dom❼X➁, ➜y ❃ dom❼Y➁, ➜Ñ z ❃ dom❼Ñ Z➁ P❼y❙x, aXY,Ñ z➁ ① P❼y❙x❻, aXY,Ñ z➁

▲ Hint: O2 is an activator for Bunsen

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Activator Dynamic Bayesian Networks

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

CE

M2

CE

M1

DC

M1

EC

M1

ED

M2

DC

M2

EC

M2

ED

▲ Is an Activator Dynamic Bayesian Network ▲ We show: Semantically a (D)BN, despite

being based on a cylic graph!

▲ Straight forward semantic as joint probability

as usual.

▲

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Activator Dynamic Bayesian Networks

C0 D0 E0 C1 D1 E1 C2 D2 E2

M1

CD

M1

DE

M2

CD

M2

DE

M1

CE

M2

CE

M1

DC

M1

EC

M1

ED

M2

DC

M2

EC

M2

ED

▲ Is an Activator Dynamic Bayesian Network ▲ We show: Semantically a (D)BN, despite

being based on a cylic graph!

▲ Straight forward semantic as joint probability

as usual.

▲ Under some restrictions...

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Restrictions Comparison

Cyclic ADBN

▲ No cyclic Mt XY observations allowed. ▲ Activator set must form DAG.

‘‘Diagonal’’ DBN

▲ No ‘‘interlocking’’ Mt XY obs. allowed. ▲ must form bipartite graph.

#DAG >> #Bipartite

C0 D0 E0 C1 D1 E1 C2 D2 E2

m1 CD m1 DE M2 CD M2 DE M1 CE M2 CE ✥m1 DC ✥m1 EC ✥m1 ED M2 DC M2 EC M2 ED

C0 D0 E0 C1 D1 E1 C2 D2 E2

m1 CD m1 DE M2 CD M2 DE M1 DC M1 CE M2 CE ✥m1 DC ✥m1 EC ✥m1 ED M2 DC M2 EC M2 ED

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

ADBN Contributions

▲ Bayesian networks can syntactically be based on cyclic graphs. ▲ Cyclic structures shall not represent feedback-loops. ▲ ADBNs are well-defined, if activators are observed acyclic. ▲ A required structure can not be known in advance and

is changing at every timestep.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Formalizing Innocuousness in ADBNs - Recap

▲ Allegedly, Burner is only relevant if it is turned on ✔b. ▲ Given ✥b, we could have removed the dependence. ▲ Problem: The relevant context ✥b is the

‘‘irrelevant’’ random variable to be removed...

H ✥b S O2 ??? ▲ in ADBNs P❼H❙S, ✥b, O2➁ P❼H❙S, O2➁ is now specifiable!

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Formalizing Innocuousness in ADBNs

▲ O2 is an activator for Bunsen burner. ▲ Given ✥o2, Burner becomes completely irrelevant. (Activator Nature) ▲ But also, given ✥burner, O2 becomes irrelevant. ▲ I.e., given ✥burner, it is like no oxygen is present, i.e., Burner is irrelevant. ▲ This is innocuousness. ▲ We can actually formalize P❼H❙S, ✥b, O2➁ P❼H❙S, O2➁ by

➛s, h, b, o2 ✂ P❼h❙s, ✥b, ✔o2➁ P❼h❙s, ✥b, ✥o2➁ P❼h❙s, ✔b, ✥o2

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Formalizing Innocuousness in ADBNs Formally

▲ P❼X❙Y, AYX, Ñ

Z➁

▲ We say, AYX can stand in multiple innocuousness context φAYX ❃ Ñ

φAYX

▲ This expresses, P❼X❙➌φAYX ❷y➑, y, AYX, Ñ

Z➁ P❼X❙➌φAYX❷y➑, AYX, Ñ Z➁

Definition (Activator Innocuousness)

Let ΦAYX be the vector of random variables used in a context φAYX associated with

• AYX. Every innocuousness context φAYX ❃ Ñ

φAYX is then defined to hold ➛x ❃ dom❼X➁, ➛Ñ z ❃ dom❼Ñ Z➁ ✂ P❼x❙φAYX , ✔aYX,Ñ z➁ P❼x❙φAYX , ✥aYX,Ñ z➁ (1) P❼x❙➌φAYX ❷y ❃ dom❼Y➁➑, y, ✥aYX,Ñ z➁ P❼x❙➌φAYX ❷y➑, ❻, ✥aYX,Ñ z➁ , (2) with remaining arbitrary dependencies of X on other random variables Ñ Z and Ñ z as an arbitrary instantiation of those, excluding AYX and ΦAYX .

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Exploiting Innocuousness in ADBNs

▲ In ADBNs, Activators must ‘‘break open cycles’’. ▲ Only DAG observations were allowed. ▲ Innocuousness can formalize ‘‘a false dependent can be left out’’. ▲ Can break open cycles, too! (Proof see paper)

Way more observations beyond DAGs allowed.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Exploiting Innocuousness in ADBNs - Example

▲ Noisy OR assumption for local CPDs in regulatory compliance domain. ▲ If employee C is compliant ✥c, he does not influence any other employees. ▲ Can ‘‘break open cycles’’. ▲ Refines activator acyclicity constraint A.

Even cyclic message exchanges are well-defined.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

ADBN and Innocuousness Contributions

▲ Bayesian networks can syntactically be based on cyclic graphs. ▲ Well-defined beyond DAG activator observations. ▲ Innocuousness properties exist frequently. ▲ Increase causal expressiveness of CPDs.

ADBNs provide ✓ Free choice of time granularity. ✓ More expressive CPD specifications. ✓ BN as world-representing first-class declaration.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15

INSTITUTE OF INFORMATION SYSTEMS

Thank you

Fin and fin of conference.

EXPLOITING INNOCUOUSNESS IN BAYESIAN NETWORKS, AI’15