Learning from Snapshot Examples Jacob Beal MIT CSAIL April, 2005 - - PowerPoint PPT Presentation

▶
learning from snapshot examples
SMART_READER_LITE
LIVE PREVIEW

Learning from Snapshot Examples Jacob Beal MIT CSAIL April, 2005 - - PowerPoint PPT Presentation

Dec 16, 2023 285 likes •699 views

Learning from Snapshot Examples Jacob Beal MIT CSAIL April, 2005 Associating a Lemon Mind Learner Associating a Lemon Mind Learner Associating a Lemon Mind Learner Space is cluttered with objects Associating a Lemon Mind Learner

slide-1
SLIDE 1

Learning from Snapshot Examples

Jacob Beal MIT CSAIL April, 2005

slide-2
SLIDE 2

Mind

Associating a Lemon

Learner

slide-3
SLIDE 3

Mind

Associating a Lemon

Learner

slide-4
SLIDE 4

Mind

Associating a Lemon

Learner

  • Space is cluttered with objects
slide-5
SLIDE 5

Mind

Associating a Lemon

Learner

  • Space is cluttered with objects
slide-6
SLIDE 6

Mind

Associating a Lemon

Learner

  • Time may be skewed externally or internally
slide-7
SLIDE 7

Mind

Associating a Lemon

Learner

  • Time may be skewed externally or internally
slide-8
SLIDE 8

Mind

Associating a Lemon

Learner

  • Time may be skewed externally or internally
slide-9
SLIDE 9

Mind

Associating a Lemon

Learner

  • Time may be skewed externally or internally
slide-10
SLIDE 10

Mind

Associating a Lemon

Learner

  • Time may be skewed externally or internally
slide-11
SLIDE 11

Mind

Snapshot Learning Framework

Learner Snapshot Mechanism Learning Mechanism

Targets, Examples Target Models

Perceptual Channels

  • Bootstrapping feedback cycle

– better model → better examples → better model

slide-12
SLIDE 12

Mind

Snapshot Learning Framework

Learner Snapshot Mechanism Learning Mechanism

Targets, Examples Target Models

Perceptual Channels

  • What are the targets?
  • How can it choose good examples?
slide-13
SLIDE 13

Targets

“Lemon” would be best, settle for its components

  • Each percept is a target
  • Learn each target independently

This means we'll learn each association several times

slide-14
SLIDE 14

Examples from Samples

Input is DT sampling of evolving perceptual state

  • Incrementally select examples from samples
  • Can only learn about things coextensive in time

Solvable by buffering w. short term memory

slide-15
SLIDE 15

Relevance of a Sample

  • Create a relevance measure for each channel

– High-relevance should indicate useful content

Color Relevance Measure

slide-16
SLIDE 16

Sparseness Assumptions

At the right level of abstraction, the world is sparse

  • Percepts are sparse across time

most of life doesn't involve lemons

  • Percepts are sparse at each sample

most of life doesn't appear when the lemon does

slide-17
SLIDE 17

Sparseness→ Irrelevant periods

Lots of irrelevant periods → lots of relevant periods

Time

slide-18
SLIDE 18

Be choosy!

Many chances → take only the best

– a few good >> many iffy – avoid overfitting from closely correlated examples

Relevance peaks?

Time

slide-19
SLIDE 19

Are peaks a good idea?

Color Time Relevance

0 1 0 1 0 1

Shape Smell

Consider the relevance measures as signals:

Sum

0 1 2

Projecting to a single measure loses a lot of info...

slide-20
SLIDE 20

Top-Cliff Heuristic

  • Generalizing “peak” to multiple dimensions

– Some channel's relevance is falling – No channel's relevance is rising – All relevant channels have risen since their last drop

(channels recently co-active with currently active channels) Color Time Relevance

0 1 0 1 0 1

Shape Smell Color Time Relevance

0 1 0 1 0 1

Shape Smell

slide-21
SLIDE 21

Top-Cliff Examples

Time Relevance

snapshot snapshot 1 2 3 4 5 6

0 1 0 1 0 1

Color Shape Smell

slide-22
SLIDE 22

Experiment: Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-23
SLIDE 23

Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-24
SLIDE 24

Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-25
SLIDE 25

Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-26
SLIDE 26

Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-27
SLIDE 27

Learning from Examples

  • Sequence of randomly generated examples
  • Transition between examples in random order

Mind Learner Snapshot Mechanism Learning Mechanism

slide-28
SLIDE 28

Applying Snapshot Learning

  • Target Model: {possible associate, confidence}
  • Modified Hebbian Learning
  • Relevance = # of possible associates present
  • Extra virtual channel for target percept

– Relevance 1 if present, 0 if absent – Determines if example is positive or negative

slide-29
SLIDE 29

Modified Hebbian Learning

  • Initial set: percepts from first relevant period

– Late entry is possible but difficult

  • Examples adjust confidence levels

– Positive Example: +1 if present, -1 if absent – Negative Example: -1 if present, 0 if absent – Confidence < P → prune out associate!

  • Same channel as target are harder to prune
  • If no associates, restart
slide-30
SLIDE 30

Experimental Parameters

  • 50 features
  • 2 channels
  • 1 percept/feature/channel = 100 targets
  • Randomly generated examples, 2-6 features/exa
  • Random transition between examples
slide-31
SLIDE 31

Top-Cliff vs. Controls

  • 10 trials of 1000 examples each
slide-32
SLIDE 32

Predictable Variation w. Parameters

slide-33
SLIDE 33

Resilient to Adverse Conditions

slide-34
SLIDE 34

...much more than the controls...

slide-35
SLIDE 35

Experiment: Learning w/o a Teacher

What if there's no teacher providing examples?

– A teacher guarantees there are associations... – ... but world has lots of structure!

  • Without a teacher, the system will still find

targets and examples. Will they teach it anything?

slide-36
SLIDE 36

4-Way Intersection Model

  • 5 locations (N,S,E,W,Center)
  • 11 types of vehicle (Sedan, SUV, etc.)

– Cars arrive randomly, with random exit goals. – Arrive moving, but queue up if blocked. – Moving or starting moving takes 1 second. – Left turns only when clear.

  • 6 lights (NS-red, EW-green, etc.)

– 60 second cycle: 27 green, 3 yellow, 30 red – Go on green, maybe yellow, right on red when clear.

slide-37
SLIDE 37

Intersection Percepts

  • 6 channels: N, S, E, W, Center, Light

– Cardinal directions: type of 1st in queue, exiting cars – Center: types of cars there – Light: two active lights

  • Distinguishable copy of previous percepts
  • Random transitions, as before

(L NS_GREEN EW_RED PREV_NS_GREEN PREV_EW_RED) (N) (S PREV_CONVERTIBLE) (C CONVERTIBLE) (E SEDAN PREV_SEDAN) (W COMPACT PREV_COMPACT)

slide-38
SLIDE 38

What does it learn?

  • After 16 light cycles:

– Lights don't depend on cars – Stoplight state transitions (97% perfect)

EW_GREEN = PREV_NS_RED, PREV_EW_GREEN, PREV_NS_YELLOW, NS_RED EW_YELLOW = PREV_EW_YELLOW, NS_RED, PREV_EW_GREEN, PREV_NS_RED EW_RED = NS_YELLOW, PREV_EW_RED, PREV_NS_GREEN, NS_GREEN NS_GREEN = PREV_EW_RED, PREV_NS_GREEN, EW_RED, PREV_EW_YELLOW NS_YELLOW = PREV_NS_YELLOW, EW_RED, PREV_NS_GREEN, PREV_EW_RED NS_RED = PREV_NS_RED, PREV_EW_GREEN, EW_GREEN, PREV_NS_YELLOW PREV_EW_GREEN = PREV_NS_RED, NS_RED, EW_GREEN PREV_EW_YELLOW = PREV_NS_GREEN, PREV_NS_RED, NS_GREEN EW_RED PREV_EW_RED = PREV_NS_YELLOW, NS_YELLOW, EW_RED, NS_GREEN, PREV_NS_GREEN PREV_NS_GREEN = PREV_NS_YELLOW, NS_YELLOW, PREV_EW_RED, EW_RED, NS_GREEN PREV_NS_YELLOW = EW_GREEN, NS_RED, PREV_EW_RED, NS_YELLOW PREV_NS_RED = PREV_EW_RED, EW_RED, PREV_EW_YELLOW, NS_GREEN

slide-39
SLIDE 39

Reconstructed FSM

slide-40
SLIDE 40

Summary

  • Snapshot learning simplifies a hard problem

– Top-Cliff finds sparse examples incrementally – Feedback improves quality of examples over time – It's easier to find good examples for single targets

  • Snapshot learning works for sequences of

examples or a predictably evolving state

  • Pretending there's a teacher helps learn!