Com bining Inductiv e and Analytical Learning [Read Ch. 12] - - PDF document

▶
com bining inductiv e and analytical learning read ch 12
SMART_READER_LITE
LIVE PREVIEW

Com bining Inductiv e and Analytical Learning [Read Ch. 12] - - PDF document

Oct 16, 2022 420 likes •609 views

Com bining Inductiv e and Analytical Learning [Read Ch. 12] [Suggested exercises: 12.1, 12.2, 12.6, 12.7, 12.8] Wh y com bine inductiv e and analytical learning? KBANN: Prior kno wledge to initial i ze the

slide-1
SLIDE 1 Com bining Inductiv e and Analytical Learning [Read Ch. 12] [Suggested exercises: 12.1, 12.2, 12.6, 12.7, 12.8]
  • Wh
y com bine inductiv e and analytical learning?
  • KBANN:
Prior kno wledge to initial i ze the h yp
  • thesis
  • T
angetProp, EBNN: Prior kno wledge alters searc h
  • b
jectiv e
  • F
OCL: Prior kno wledge alters searc h
  • p
erators 1 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-2
SLIDE 2 Inductiv e and Analytical Learning Inductiv e learning Analytical learning Hyp
  • thesis
ts data Hyp
  • thesis
ts domain theory Statistical inference Deductiv e inference Requires littl e prior kno wledge Learns from scarce data Syn tactic inductiv e bias Bias is domain theory 2 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-3
SLIDE 3 What W e W
  • uld
Lik e

Analytical learning Inductive learning Plentiful data No prior knowledge Perfect prior knowledge Scarce data

General purp
  • se
learning metho d:
  • No
domain theory ! learn as w ell as inductiv e metho ds
  • P
erfect domain theory ! learn as w ell as Pr
  • log-EBG
  • Accomo
date arbitrary and unkno wn errors in domain theory
  • Accomo
date arbitrary and unkno wn errors in training data 3 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-4
SLIDE 4 Domain theory: Cup Stable, Liftable, Op enV essel Stable BottomIsFlat Liftable Graspable, Ligh t Graspable HasHandle Op enV essel HasConca vit y , Conca vit yP
  • in
tsUp T raining examples: Cups Non-Cups BottomIsFlat p p p p p p p p Conca vit yP
  • in
ts Up p p p p p p p Exp ensiv e p p p p F ragile p p p p p p HandleOnT
  • p
p p HandleOnSide p p p HasConca vit y p p p p p p p p p HasHandle p p p p p Ligh t p p p p p p p p MadeOfCeramic p p p p MadeOfP ap er p p MadeOfSt yrofoam p p p p 4 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-5
SLIDE 5 KBANN KBANN (data D , domain theory B ) 1. Create a feedforw ard net w
  • rk
h equiv ale n t to B 2. Use Ba ckpr
  • p
to tune h to t D 5 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-6
SLIDE 6 Neural Net Equiv alen t to Domain Theory

HasHandle HandleOnTop HandleOnSide BottomIsFlat HasConcavity ConcavityPointsUp Light MadeOfCeramic MadeOfPaper MadeOfStyrofoam Expensive Fragile Cup Stable Liftable OpenVessel Graspable

6 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-7
SLIDE 7 Creating Net w
  • rk
Equiv alen t to Do- main Theory Create
  • ne
unit p er horn clause rule (i.e., an AND unit)
  • Connect
unit inputs to corresp
  • nding
clause an teceden ts
  • F
  • r
eac h non-negated an teceden t, corresp
  • nding
input w eigh t w W , where W is some constan t
  • F
  • r
eac h negated an teceden t, input w eigh t w W
  • Threshold
w eigh t w (n
  • :5)W
, where n is n um b er
  • f
non-negated an teceden ts Finally , add man y additional connections with near-zero w eigh ts Lif tabl e Gr aspabl e; :H eav y 7 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-8
SLIDE 8 Result
  • f
rening the net w
  • rk

HasHandle HandleOnTop HandleOnSide BottomIsFlat HasConcavity ConcavityPointsUp Light MadeOfCeramic MadeOfPaper MadeOfStyrofoam Expensive Fragile Stable Liftable Open-Vessel

Large positive weight Large negative weight Negligible weight

Cup Graspable

8 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-9
SLIDE 9 KBANN Results Classifyi ng promoter regions in DNA lea v e
  • ne
  • ut
testing:
  • Bac
kpropagation: error rate 8/106
  • KBANN:
4/106 Similar impro v emen ts
  • n
  • ther
classicati
  • n,
con trol tasks. 9 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-10
SLIDE 10 Hyp
  • thesis
space searc h in KBANN

Hypothesis Space Hypotheses that fit training data equally well Initial hypothesis for KBANN Initial hypothesis for BACKPROPAGATION

10 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-11
SLIDE 11 EBNN Key idea:
  • Previously
learned appro ximate domain theory
  • Domain
theory represen ted b y collecti
  • n
  • f
neural net w
  • rks
  • Learn
target function as another neural net w
  • rk
11 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-12
SLIDE 12

BottomIsFlat ConcavityPointsUp Expensive Fragile HandleOnTop HandleOnSide HasConcavity HasHandle Light MadeOfCeramic MadeOfPaper MadeOfStyrofoam Cup BottomIsFlat ConcavityPointsUp Expensive Fragile HandleOnTop HandleOnSide HasConcavity HasHandle Light MadeOfCeramic MadeOfPaper MadeOfStyrofoam = T = T = T = T = F = T = T = T = T = T = F = F Graspable Stable OpenVessel Liftable Cup

Target network:

0.2 0.8 Training derivatives Cup target

Explanation of training example in terms of domain theory:

Cup = T

12 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-13
SLIDE 13 Mo died Ob jectiv e for Gradien t Descen t E = X i 2 6 6 6 6 4 (f (x i )
  • ^
f (x i )) 2 +
  • i
X j B B B @ @ A(x) @ x j
  • @
^ f (x) @ x j 1 C C C A 2 (x=x i ) 3 7 7 7 7 5 where
  • i
  • 1
  • jA(x
i )
  • f
(x i )j c
  • f
(x) is target function
  • ^
f (x) is neural net appro ximation to f (x)
  • A(x)
is domain theory appro ximation to f (x) 13 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-14
SLIDE 14

x x f x x x

1 2 3

f(x) g h f(x )

1

f(x )

2

f(x )

3

x

14 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-15
SLIDE 15 Hyp
  • thesis
Space Searc h in EBNN

Hypotheses that maximize fit to data Hypothesis Space BACKPROPAGATION Search TANGENTPROP Search Hypotheses that maximize fit to data and prior knowledge

15 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-16
SLIDE 16 Searc h in F OCL

... [2+,3–] [2+,3–] [2+,4–] [4+,2–] Generated by the domain theory Cup HasHandle Cup HasHandle Cup Fragile Cup BottomIsFlat, Light, HasConcavity, ConcavityPointsUp Cup [2+,0–] [0+,2–] [4+,0–] ... Cup BottomIsFlat, Light, HasConcavity, ConcavityPointsUp HandleOnTop BottomIsFlat, Light, HasConcavity, ConcavityPointsUp, HandleOnTop BottomIsFlat, Light, HasConcavity, ConcavityPointsUp, HandleOnSide Cup Cup

16 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997
slide-17
SLIDE 17 F OCL Results Recognizing legal c hess endgame p
  • sitions:
  • 30
p
  • sitiv
e, 30 negativ e examples
  • F
OIL: 86%
  • F
OCL: 94% (using domain theory with 76% accuracy) NYNEX telephone net w
  • rk
diagnosis
  • 500
training examples
  • F
OIL: 90%
  • F
OCL: 98% (using domain theory with 95% accuracy) 17 lecture slides for textb
  • k
Machine L e arning, c T. Mitc hell, McGra w Hill, 1997