USING DOMAIN KNO WLEDGE IN MA CHINE LEARNING Inductiv e vs - - PDF document

▶
using domain kno wledge in ma chine learning inductiv e
SMART_READER_LITE
LIVE PREVIEW

USING DOMAIN KNO WLEDGE IN MA CHINE LEARNING Inductiv e vs - - PDF document

Jan 11, 2023 304 likes •540 views

USING DOMAIN KNO WLEDGE IN MA CHINE LEARNING Inductiv e vs Analytical Learning Carolina Ruiz Motiv ation Inductiv e Learning induction Examples

slide-1
SLIDE 1 USING DOMAIN KNO WLEDGE IN MA CHINE LEARNING Inductiv e vs Analytical Learning Carolina Ruiz
slide-2
SLIDE 2 Motiv ation Inductiv e Learning induction Examples
  • hypothesis
  • Prior
Knowledge generalization
  • Statistical
reasoning is used to iden tify features that empirically distinguish M from K examples eg decision trees NNs ILP
  • GAs
  • F
undamen tal b
  • unds
  • n
accuracy dep end
  • n
n um b er
  • f
training examples Analytical Learning deduction Examples
  • hypothesis
  • Prior
Knowledge generalization
  • Logical
reasoning is used to iden tify features that distinguish M from K examples
  • W
  • rks
w ell ev en when training examples are scarce Inductiv e
  • Analytical
Learning
  • Com
bines the b est
  • f
b
  • th
w
  • rlds
slide-3
SLIDE 3 Learning L e arning is to impr
  • ve
with exp erienc e at some task Learning from Examples Inductiv e Learning
  • Giv
en A set
  • f
training examples hx
  • f
x
  • i
  • hx
n
  • f
x n i
  • Find
A h yp
  • thesis
h dened
  • v
er the whole space
  • f
instances that coincides with f
  • v
er the training data ie hx i
  • f
x i
  • for
all
  • i
  • n
Learning from Examples and Prior Kno wledge
  • Giv
en A set
  • f
training examples hx
  • f
x
  • i
  • hx
n
  • f
x n i and a set B
  • f
rules expressing prior kno wledge
  • Find
A h yp
  • thesis
h dened
  • v
er the whole space
  • f
instances st B
  • h
  • x
i
  • f
x i
  • for
all
  • i
  • n
slide-4
SLIDE 4 P art
  • Motiv
ation Inductiv e Learning induction Examples
  • hypothesis
  • Prior
Knowledge generalization
  • Statistical
reasoning is used to iden tify features that empirically distinguish M from K examples eg decision trees NNs ILP
  • GAs
  • F
undamen tal b
  • unds
  • n
accuracy dep end
  • n
n um b er
  • f
training examples
  • When
the
  • utput
h yp
  • thesis
is a set
  • f
rules then this learning is called Inductive L
  • gic
Pr
  • gr
am ming ILP
  • ILP
uses prior kno wledge to augmen t the input description
  • f
instances whic h increases the com plexit y
  • f
the h yp
  • thesis
space
slide-5
SLIDE 5 Inductiv e Logic Programming INDUCTIVE LOGIC PROGRAMMING
  • INDUCTIVE
MACHINE LEARNING COMPUTATIONAL LOGIC F rom Inductiv e Mac hine Learning inherits its goals
  • to
dev elop to
  • ls
and tec hniques to induce h yp
  • the
ses from
  • bserv
ations examples
  • r
  • to
syn thesize new kno wledge from exp erience F rom Computational Logic inherits
  • abilit
y to
  • v
ercome limitations
  • f
classical induc tiv e learners
  • use
  • f
limited kno wledge represen tation formal ism eg prop
  • sitional
logic
  • inabilit
y to use substan tial domain kno wledge during learning pro cess
  • theory
  • rien
tation and tec hniques
  • in
terest in prop erties
  • f
rules con v ergence sound ness and completeness
  • f
algorithms computa tional complexit y
slide-6
SLIDE 6 EXAMPLE tak en from Luc De Raedt and Nada La vrac
  • Giv
en
  • a
set
  • f
examples D M
  • f
f l iestw eety
  • g
K
  • a
bac kground theory B bir dtwe ety bir doliver Find
  • a
h yp
  • thesis
h that explains the examples D wrt B
  • h
  • f
f l iesX
  • bir
dX
  • g
slide-7
SLIDE 7 EXAMPLE tak en from Luc De Raedt and Nada La vrac
  • Giv
en
  • a
set
  • f
examples D M
  • sor
t
  • sor
t
  • K
  • sor
t
  • sor
t
  • a
bac kground theory B B
  • per
mutation sor ted Find
  • a
h yp
  • thesis
h that satises the language bias and explains the examples D wrt B
  • h
  • fsor
tX
  • Y
  • per
mutationX
  • Y
sor tedY g
slide-8
SLIDE 8 Sequen tial Co v ering T emplate Giv en a set
  • f
examples M
  • K
  • le
arne drules
  • fg
  • while
there are examples still to b e explained do a rule
  • Learnonerule
b remo v e from examples those explained b y rule c le arne drules
  • le
arne drules
  • rule
  • Output
le arne drules
slide-9
SLIDE 9 ILP implemen tations
  • f
Learnonerule General to Sp ecic Searc h top do wn eg F OIL
  • Start
with the most general clause eg
  • sor
tX
  • Y
  • sp
ecialize the rule b y adding the b est p
  • ssible
lit eral to the b
  • dy
  • f
the rule eg
  • sor
tX
  • Y
  • per
mutationX
  • Y
  • un
til the h yp
  • thesis
en tails no K examples en suring that h yp
  • thesis
en tails at least a threshold n um b er
  • f
M examples Sp ecic to General Searc h b
  • ttom
up eg GOLEM
  • Start
with the most sp ecic clause that implies a giv en example
  • generalize
  • un
til the h yp
  • thesis
cannot b e further generalized without implying negativ e examples
slide-10
SLIDE 10 Searc hing for the b est literal
  • F
  • il
Quinlan
  • Candidate
Literals All literals that can b e constructed using predicates and terms that app ear in B andor in the examples Ev aluation F unction F
  • ils
informationbased no tion
  • f
gain Note This searc h can b e computationally explosiv e Imp
  • rtan
t to prune the searc h space Matthew Berub es MQP advised b y C Ruiz and S Alv arez prunes the searc h space b y using typ e in formation making F OIL applicable to some large do mains
slide-11
SLIDE 11 Summarizing Inductiv e Learning
  • Uses
statistical inference to compute a h yp
  • thesis
that t the data
  • Uses
bac kground kno wledge to augmen t the input de scription
  • f
instances whic h ma y increase the com plexit y
  • f
the h yp
  • thesis
space
  • Requires
little bac kground kno wledge
  • Requires
large amoun ts
  • f
training data
slide-12
SLIDE 12 P art I I Motiv ation Analytical Learning deduction Examples
  • hypothesis
  • Prior
Knowledge generalization
  • Logical
reasoning is used to iden tify features that distinguish M from K examples
  • W
  • rks
w ell ev en when training examples are scarce
  • Analytical
Learning Metho d ExplanationBased Learning EBL
  • EBL
uses prior kno wledge to reduce the complex it y
  • f
the h yp
  • thesis
space and to impro v e the ac curacy
  • f
the
  • utput
h yp
  • thesis
  • Prior
Kno wledge is assumed to b e correct and com plete
slide-13
SLIDE 13 Analytical Learning The Analytical Learning T ask
  • Giv
en A set D
  • f
training examples hx i
  • f
x i i and Bac kground Kno wledge B represen ted as a set
  • f
rules
  • Horn
clauses
  • Find
a h yp
  • thesis
h suc h that
  • D
  • B
  • h
  • for
all
  • i
  • n
h
  • x
i
  • f
x i
  • P
art
  • ab
  • v
e reduces the size
  • f
the h yp
  • thesis
space and mak es the
  • utput
h yp
  • thesis
more accurate Assumptions The bac kground kno wledge is Correct eac h
  • f
its assertions is a truthful statemen t ab
  • ut
the w
  • rld
Complete ev ery instance that satises the target concept can b e pro v en b y the domain kno wledge to satisfy it Are these reasonable assumptions Wh y b
  • ther
learn ing if the bac kground kno wledge is already complete
slide-14
SLIDE 14 Analytical Learning
  • Motiv
ation
  • Chess
example tak en from Mitc hells b
  • k
  • Man
y features are presen t Prior kno wledge helps us tell apart relev an t features from irrelev an t
  • nes
  • F
  • r
the c hess domain it is easy to write do wn rules that completely describ e all legal mo v es but it is v ery hard to write do wn rules that completely describ e an
  • ptimal
mo v e strategy
slide-15
SLIDE 15 EBL An Analytical Learning Metho d EBL ExplanationBased Learning
  • Prior
kno wledge is used to construct an explanationpro
  • f
  • f
eac h example whic h is expressed as a Horn rule
  • this
explanation is used to distinguish b et w een the relev an t features
  • f
the training example and the ir relev an t
  • nes
  • the
explanation rule is generalized to the exten t p
  • s
sible and added to the curren t h yp
  • thesis
set
  • f
rules
slide-16
SLIDE 16 PrologEBG An EBL System KedarCab elli
  • McCart
y
  • represen
tativ e
  • f
sev eral EBL algorithms
  • guaran
teed to
  • utput
a h yp
  • thesis
that is correct and co v ers the p
  • sitiv
e training examples
  • sequen
tial co v ering algorithm
  • hyp
  • thesis
  • fg
  • F
  • r
eac h p
  • sitiv
e training example not explained b y hyp
  • thesis
do a Explain using the bac kground kno wledge wh y the example satises the target concept uses Prologs bac ktrac king b Analyze the explanation to determine the most general conditions under whic h the explanation holds uses regression c Rene the hyp
  • thesis
b y adding this explana tion to it
  • Output
hyp
  • thesis
slide-17
SLIDE 17 PrologEBG Example tak en from Mitc hells b
  • k
slide-18
SLIDE 18 EBL Dieren t P ersp ectiv es
  • Using
the bac kground kno wlede to general ize the training examples rationallogical generalization
  • Using
the training examples to reform ulate the bac kground kno wledge reform ulating B in to a more
  • p
erational form rules in the h yp
  • thesis
classify an instance in a single inference step
  • know
le dge c
  • mpilation
  • Using
B and D to restate what the learner already kno ws dierence b et w een what
  • ne
kno ws in principle and what
  • ne
can ecien tly do in practice
slide-19
SLIDE 19 Summarizing Analytical Learning
  • Uses
logical deduction to compute a h yp
  • thesis
that t the data and the bac kground kno wledge
  • Uses
bac kground kno wledge to augmen t the informa tion pro vided b y the training data whic h decreases the size
  • f
the h yp
  • thesis
space
  • Requires
correct and complete bac kground kno wledge
  • Do
es w ell
  • n
scarce data
slide-20
SLIDE 20 P AR T I I I Motiv ation Inductiv e
  • Analytical
Learning induction
  • deduction
Examples
  • hypothesis
  • Prior
Knowledge generalization The Learning T ask
  • Giv
en A set D
  • f
training examples hx i
  • f
x i i and Bac kground Kno wledge B represen ted as a set
  • f
rules
  • Horn
clauses
  • Find
a h yp
  • thesis
h that b est ts D and B
  • Assumptions
Data Ma y con tain errors ma y b e scarce Prior Kno wledge If a v ailable it ma y b e incorrect
  • r
incomplete Goal Do b etter than inductiv e
  • r
analytical learning alone
slide-21
SLIDE 21 Inductiv e
  • Analytical
Searc h for h
  • Use
B to deriv e an initial h yp
  • thesis
from whic h to b egin the searc h eg KBANN Sha vlik
  • T
  • w
ell
  • Use
B to rene the h yp
  • thesis
searc h space eg EBNN Mitc hell
  • Thrun
  • Matt
Berub es MQP
  • Use
B to alter the a v ailable searc h steps eg F OCL P azzani et al