[PPT] - Abstraction by Structure Carl Henrik Ek, Danica Kragic { chek, danik PowerPoint Presentation

SLIDE 1

Introduction Structural Representations Structural Models Conclusion References

Abstraction by Structure

Carl Henrik Ek, Danica Kragic {chek, danik}@csc.kth.se

Royal Institute of Technology

April 18, 2012

Ek, Kragic KTH Abstraction by Structure

SLIDE 2

Introduction Structural Representations Structural Models Conclusion References

Centre for Autonomous Systems

Head of group, Prof. Danica Kragic, 8 Senior staff
12+ Post-docs/Researchers
20+ PhD students
To celebrate our 20th ICRA is coming to town in 2016

Ek, Kragic KTH Abstraction by Structure

SLIDE 3

Introduction Structural Representations Structural Models Conclusion References

Centre for Autonomous Systems

Head of group, Prof. Danica Kragic, 8 Senior staff
12+ Post-docs/Researchers
20+ PhD students
To celebrate our 20th ICRA is coming to town in 2016

Ek, Kragic KTH Abstraction by Structure

SLIDE 4

Introduction Structural Representations Structural Models Conclusion References

Collaborators

Hedvig Kjellstr¨

m, Niklas Bergstr¨
m, Marianna Madry, Florian

Pokorny, Renaud Detry, Dan Song, Javier Romero, Martin Hjelm, Andrea Baisero, Guoliang Luo, Andreas Damianou, Neil Lawrence, Neill Campbell, Colin Dalton, Alexander Davies

Ek, Kragic KTH Abstraction by Structure

SLIDE 5

Introduction Structural Representations Structural Models Conclusion References

Introduction Structural Representations Structural Models Conclusion

Ek, Kragic KTH Abstraction by Structure

SLIDE 6

Introduction Structural Representations Structural Models Conclusion References

Abstraction

Ek, Kragic KTH Abstraction by Structure

SLIDE 7

Introduction Structural Representations Structural Models Conclusion References

Abstraction

1 2 3 4

Ek, Kragic KTH Abstraction by Structure

SLIDE 8

Introduction Structural Representations Structural Models Conclusion References

Abstraction

1 2 3 4

P(Y|X) Ek, Kragic KTH Abstraction by Structure

SLIDE 9

Introduction Structural Representations Structural Models Conclusion References

Abstraction

1 2 3 4

P(Y|X) Ek, Kragic KTH Abstraction by Structure

SLIDE 10

Introduction Structural Representations Structural Models Conclusion References

Abstraction

1 2 3 4

P(Y|X) Ek, Kragic KTH Abstraction by Structure

SLIDE 11

Introduction Structural Representations Structural Models Conclusion References

Abstraction

1 2 3 4

P(Y|X) Ek, Kragic KTH Abstraction by Structure

SLIDE 12

Introduction Structural Representations Structural Models Conclusion References

Representation

Internal representation

◮ basis for reasoning ◮ unobservable

External representation

◮ Facilitates

communication

◮ Agreed/negotiated ◮ Aware Ek, Kragic KTH Abstraction by Structure

SLIDE 13

Introduction Structural Representations Structural Models Conclusion References

“Grammar”

Facilitates reasoning

◮ Rules of generalisation ◮ Ex Triangle inequality

Preferential Representation

◮ “Simple” structurally Ek, Kragic KTH Abstraction by Structure

SLIDE 14

Introduction Structural Representations Structural Models Conclusion References

Representation

Ek, Kragic KTH Abstraction by Structure

SLIDE 15

Introduction Structural Representations Structural Models Conclusion References

Representation

Representation result of capturing
Over-interpretation on
Same yes, similar no

◮ success of NN, RBF Ek, Kragic KTH Abstraction by Structure

SLIDE 16

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 17

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 18

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 19

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 20

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 21

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 22

Introduction Structural Representations Structural Models Conclusion References

Scenario

Ek, Kragic KTH Abstraction by Structure

SLIDE 23

Introduction Structural Representations Structural Models Conclusion References

Scenario

1 2 3 4

Ek, Kragic KTH Abstraction by Structure

SLIDE 24

Introduction Structural Representations Structural Models Conclusion References

Scenario

1 2 3 4

Ek, Kragic KTH Abstraction by Structure

SLIDE 25

Introduction Structural Representations Structural Models Conclusion References

Scenario

1 2 3 4

Ek, Kragic KTH Abstraction by Structure

SLIDE 26

Introduction Structural Representations Structural Models Conclusion References

External

Information

Sensory data

◮ images ◮ depth ◮ . . .

Language/Grammar

“Mathematical”

◮ similarity ◮ integration/derivation ◮ generalisation Ek, Kragic KTH Abstraction by Structure

SLIDE 27

Introduction Structural Representations Structural Models Conclusion References

Motivation

What to represent?

Task dependent
Very rich sensory domain

◮ Generalisation not

discrimination

Generalising variance?

◮ structure? ◮ appearance? Ek, Kragic KTH Abstraction by Structure

SLIDE 28

Introduction Structural Representations Structural Models Conclusion References

The Importance of Structure1

Variance Generalisation Discrimination

1Ek and Kragic [2011] Ek, Kragic KTH Abstraction by Structure

SLIDE 29

Introduction Structural Representations Structural Models Conclusion References

The Importance of Structure1

1Ek and Kragic [2011] Ek, Kragic KTH Abstraction by Structure

SLIDE 30

Introduction Structural Representations Structural Models Conclusion References

The Importance of Structure1

1Ek and Kragic [2011] Ek, Kragic KTH Abstraction by Structure

SLIDE 31

Introduction Structural Representations Structural Models Conclusion References

The Importance of Structure1

1Ek and Kragic [2011] Ek, Kragic KTH Abstraction by Structure

SLIDE 32

Introduction Structural Representations Structural Models Conclusion References

Current approach

Ek, Kragic KTH Abstraction by Structure

SLIDE 33

Introduction Structural Representations Structural Models Conclusion References

Current approach

Apperance

438 − 5, Alastair Cook Gray Nicolls bat

Ek, Kragic KTH Abstraction by Structure

SLIDE 34

Introduction Structural Representations Structural Models Conclusion References

Current approach

Structure

Summer, Field, Outdoor

Ek, Kragic KTH Abstraction by Structure

SLIDE 35

Introduction Structural Representations Structural Models Conclusion References

Current approach

Apperance “Current” Structure

Worst of both worlds

Ek, Kragic KTH Abstraction by Structure

SLIDE 36

Introduction Structural Representations Structural Models Conclusion References

Introduction Structural Representations Structural Models Conclusion

Ek, Kragic KTH Abstraction by Structure

SLIDE 37

Introduction Structural Representations Structural Models Conclusion References

Action Representation

2

2Aksoy et al. [2010] Ek, Kragic KTH Abstraction by Structure

SLIDE 38

Introduction Structural Representations Structural Models Conclusion References

Text analysis

“No” appearance problem
All information in ordering (1D Structure)

Ek, Kragic KTH Abstraction by Structure

SLIDE 39

Introduction Structural Representations Structural Models Conclusion References

String Feature Space3

AA NN AN AT NT . . . Φ(ANT) = [ λ2 λ3 λ2 . . . Φ(ANNT) = [ λ2 λ2+λ3 λ4 λ2+λ3 . . . Φ(AATN) = [ λ2 λ3+λ4 λ2+λ3 . . .

ANNT TOOT TN NA OTTN AOOTTOOA ANNT TOOTNA OTTN

Infinite dimensional representation space
Kernel finite dimensional
Inner-product efficiently computed

3Lodhi et al. [2002] Ek, Kragic KTH Abstraction by Structure

SLIDE 40

Introduction Structural Representations Structural Models Conclusion References

Action Representation

4

4Aksoy et al. [2010] Ek, Kragic KTH Abstraction by Structure

SLIDE 41

Introduction Structural Representations Structural Models Conclusion References

Action Representation

A N T O T N A O O T N N N N A O T T T O A AN NT TO OT TN NA OT TN AO OT TO 0A · · ·

Ek, Kragic KTH Abstraction by Structure

SLIDE 42

Introduction Structural Representations Structural Models Conclusion References

Action Representation

4

4Luo et al. [2011] Ek, Kragic KTH Abstraction by Structure

SLIDE 43

Introduction Structural Representations Structural Models Conclusion References

Action Representation

99.9% 98.0% 94.6% 94.4% 97.5% 81.8% 49.8% 37.9% 94.8 1.6 0.4 3.2 0.4 99.2 0.4 2.0 1.2 96.8 0.8 99.3 75.2 4.0 18.8 2.0 6.0 81.2 12.8 10.0 6.8 81.2 2.0 1.6 0.4 8.4 89.6 47.6 7.2 11.6 33.6 34.8 34.0 10.0 21.2 40.8 6.4 30.0 22.8 10.0 1.2 1.2 87.6 57.2 3.6 0.8 38.4 53.6 10.8 6.4 29.2 61.6 0.8 8.0 29.6 22.8 0.8 0.8 75.6 99.7 0.3 100 0.1 99.9 100 99.3 0.6 0.1 99.7 0.3 1.6 1.4 97 1.3 2.7 0.1 95.9 91.7 2.8 4.4 1.1 1.3 96.7 1.6 0.4 4.2 2.4 93.3 0.1 1.7 1.7 0.1 96.5 90.2 2.2 7.1 0.4 0.4 96.4 2.9 0.3 6.6 1.2 92.1 0.3 0.5 0.7 98.8

Ek, Kragic KTH Abstraction by Structure

SLIDE 44

Introduction Structural Representations Structural Models Conclusion References

Objects4

Feature Representation

Local representation
Encode order
Distribution of order

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 45

Introduction Structural Representations Structural Models Conclusion References

Objects4

Feature Representation

Local representation
Encode order
Distribution of order

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 46

Introduction Structural Representations Structural Models Conclusion References

Objects4

Feature Representation

Local representation
Encode order
Distribution of order

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 47

Introduction Structural Representations Structural Models Conclusion References

Objects4

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 48

Introduction Structural Representations Structural Models Conclusion References

Objects4

distance histogram between points of type 1 and 2 (marked as 1-2) etc. Images are best viewed in color.

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 49

Introduction Structural Representations Structural Models Conclusion References

Objects4

Test Setting

Synthetic
Real
Real & different pose,scale
Synthetic & full,partial

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 50

Introduction Structural Representations Structural Models Conclusion References

Objects4

Test Setting

Synthetic
Real
Real & different pose,scale
Synthetic & full,partial

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 51

Introduction Structural Representations Structural Models Conclusion References

Objects4

Test Setting

Synthetic
Real
Real & different pose,scale
Synthetic & full,partial

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 52

Introduction Structural Representations Structural Models Conclusion References

Objects4

Test Setting

Synthetic
Real
Real & different pose,scale
Synthetic & full,partial

4In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

SLIDE 53

Introduction Structural Representations Structural Models Conclusion References

Grasping5

Pre-segmentation of objects
Exploit structure in joint object

and grasp space

Part based generalisation

−0.1 0.17 0.21 0.2 0.19 0.16 0.15 0.3 0.2 0.18 0.1 −0.2

5Detry et al. [2012] Ek, Kragic KTH Abstraction by Structure

SLIDE 54

Introduction Structural Representations Structural Models Conclusion References

Grasping5

5Detry et al. [2012] Ek, Kragic KTH Abstraction by Structure

SLIDE 55

Introduction Structural Representations Structural Models Conclusion References

Introduction Structural Representations Structural Models Conclusion

Ek, Kragic KTH Abstraction by Structure

SLIDE 56

Introduction Structural Representations Structural Models Conclusion References

Re-representations

Preference: low-dimensional
Linearity
Observed data Y ∈ ℜN·D
Underlying intrinsic representation X ∈ ℜN·q
Generative mapping: yi = f(xi)

Ek, Kragic KTH Abstraction by Structure

SLIDE 57

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 58

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0.5 !0.4 !0.3 !0.2 !0.1 0.1 0.2 0.3 0.4 0.5 !1.5 !1 !0.5 0.5 1 1.5 Linear Kernel

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 59

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0.5 !0.4 !0.3 !0.2 !0.1 0.1 0.2 0.3 0.4 0.5 !2 !1.5 !1 !0.5 0.5 1 1.5 2 2.5 RBF Kernel width=1

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 60

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0.5 !0.4 !0.3 !0.2 !0.1 0.1 0.2 0.3 0.4 0.5 !3 !2.5 !2 !1.5 !1 !0.5 0.5 1 1.5 2 RBF Kernel width=1e!1

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 61

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0"# !0"4 !0"3 !0"2 !0"1 0"1 0"2 0"3 0"4 0"# !3 !2 !1 1 2 3 ()* ,e./e0 1i3t561e!2

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 62

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0.5 !0.4 !0.3 !0.2 !0.1 0.1 0.2 0.3 0.4 0.5 !2.5 !2 !1.5 !1 !0.5 0.5 1 1.5 2 MLP Kernel

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 63

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Prior

!0.5 !0.4 !0.3 !0.2 !0.1 0.1 0.2 0.3 0.4 0.5 !6 !5 !4 !3 !2 !1 1 2 3 4 RBF, Linear, Noise

Distribution over infinite objects: functions.

Ek, Kragic KTH Abstraction by Structure

SLIDE 64

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Posterior

Combine prior with observed data y∗|X∗, X, y ∼N(K(X∗, X)K(X, X)−1y, , K(X∗, X∗) − K(X∗, X)K(X, X)−1K(X, X∗))

Ek, Kragic KTH Abstraction by Structure

SLIDE 65

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Posterior

Combine prior with observed data y∗|X∗, X, y ∼N(K(X∗, X)K(X, X)−1y, , K(X∗, X∗) − K(X∗, X)K(X, X)−1K(X, X∗))

Ek, Kragic KTH Abstraction by Structure

SLIDE 66

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Posterior

Combine prior with observed data y∗|X∗, X, y ∼N(K(X∗, X)K(X, X)−1y, , K(X∗, X∗) − K(X∗, X)K(X, X)−1K(X, X∗))

Ek, Kragic KTH Abstraction by Structure

SLIDE 67

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Posterior

Combine prior with observed data y∗|X∗, X, y ∼N(K(X∗, X)K(X, X)−1y, , K(X∗, X∗) − K(X∗, X)K(X, X)−1K(X, X∗))

Ek, Kragic KTH Abstraction by Structure

SLIDE 68

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 69

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 70

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 71

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 72

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 73

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 74

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 75

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 76

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 77

Introduction Structural Representations Structural Models Conclusion References

Gaussian Processes: Marginal Likelihood6

1.5 1 0.5 0.5 1 1.5 2 1.5 1 0.5 0.5 1

10

1

10 10

1

12 11 10 9 8 7 6 5 4 loglikelihood length scale

−1 2tr

yT(K + β−1I)−1y
data−fit

− 1 2log

det(K + β−1I)
complexity

−N 2 log2π

6Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

SLIDE 78

Introduction Structural Representations Structural Models Conclusion References

Re-representation

GP-LVMa

aLawrence [2005]

Occam’s Razor

◮ Dimensionality ◮ Co-variance function

Sufficiently regularises

problem

Y X θY

Ek, Kragic KTH Abstraction by Structure

SLIDE 79

Introduction Structural Representations Structural Models Conclusion References

Shared Representations7

GP-LVM

Fully shared

◮ not CCA style

Shared/Private

Y Z X θY θZ 7Ek [2008]Salzmann et al. [2010] Ek, Kragic KTH Abstraction by Structure

SLIDE 80

Introduction Structural Representations Structural Models Conclusion References

Shared Representations7

GP-LVM

Fully shared

◮ not CCA style

Shared/Private

Y Z X XY XZ θY θZ

7Ek [2008]Salzmann et al. [2010] Ek, Kragic KTH Abstraction by Structure

SLIDE 81

Introduction Structural Representations Structural Models Conclusion References

Factorized Variance8

Bayesian GP-LVMa

◮ Prior on X ◮ ARD

k

xi, xj
= (σY

ard)2e− 1

2

Q

q=1 wY q (xi,q−xj,q) 2 aTitsias and Lawrence [2010]

Y Z X θY WY θZ WZ

8In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

SLIDE 82

Introduction Structural Representations Structural Models Conclusion References

Factorized Variance8

8In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

SLIDE 83

Introduction Structural Representations Structural Models Conclusion References

Factorized Variance8

1 2 3 4 5 6 7 8 9 10 11 12 13 14 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 1 2 3 4 5 6 7 8 9 10 11 12 13 14 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08

8In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

SLIDE 84

Introduction Structural Representations Structural Models Conclusion References

Factorized Variance8

8In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

SLIDE 85

Introduction Structural Representations Structural Models Conclusion References

Factorized Variance8

                                   

8In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

SLIDE 86

Introduction Structural Representations Structural Models Conclusion References

Factorised Density9

Dimensionality Reduction I

Conditional dependency structures,

p(X) =

i

p(xi|π(xi), θi, S)

Learning,

◮ Parameters: θi ◮ Structure: S Priors? Carnality

Heuristics for discrete data

9Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

SLIDE 87

Introduction Structural Representations Structural Models Conclusion References

Factorised Density9

Dimensionality Reduction II

Very ill-defined
Re-representation

◮ “a mapping and configuration”

Prefer “clustered”

re-representation

Y X Y XW XB T

9Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

SLIDE 88

Introduction Structural Representations Structural Models Conclusion References

Factorised Density9

Objective

p(Y, X, U|θ) =

p(Y|f, θ)p(f|fU, X, θ)p(fU|U|θ)p(X)p(U|θ)dfdfU

9Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

SLIDE 89

Introduction Structural Representations Structural Models Conclusion References

Factorised Density9

9Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

SLIDE 90

Introduction Structural Representations Structural Models Conclusion References

Factorised Density9

T O1 O2 O3 O4 C1 C2 C3 C4 C5 C6 C7 A1 A2 A3 A4 A5 9Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

SLIDE 91

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Hand Over

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 92

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Pouring

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 93

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Tool-Use

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 94

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Hand Over

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 95

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Pouring

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 96

Introduction Structural Representations Structural Models Conclusion References

Factorised Density10

Glass Hammer Knife

Tool-Use

10Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

SLIDE 97

Introduction Structural Representations Structural Models Conclusion References

Structural Density11

Topology Respecting

Structural properties
Geometrical notion irrelevant
Topological information
Barcodesa

aCarlsson [2009] 11In submission: Pokorny, Kjellstr¨

m

Ek, Kragic KTH Abstraction by Structure

SLIDE 98

Introduction Structural Representations Structural Models Conclusion References

Structural Density11

Topology Respecting

Structural properties
Geometrical notion irrelevant
Topological information
Barcodesa

aCarlsson [2009]

−10 10 20 30 40 0.1 0.2 −10 10 20 30 40 0.1 0.2

11In submission: Pokorny, Kjellstr¨

m

Ek, Kragic KTH Abstraction by Structure

SLIDE 99

Introduction Structural Representations Structural Models Conclusion References

Structural Density11

Topology Respecting

Structural properties
Geometrical notion irrelevant
Topological information
Barcodesa

aCarlsson [2009]

−10 10 20 30 40 0.1 0.2 −10 10 20 30 40 0.1 0.2

11In submission: Pokorny, Kjellstr¨

m

Ek, Kragic KTH Abstraction by Structure

SLIDE 100

Introduction Structural Representations Structural Models Conclusion References

Structural Density11

Topology Respecting

Structural properties
Geometrical notion irrelevant
Topological information
Barcodesa

aCarlsson [2009]

−150 −100 −50 50 100 150 200 250 −150 −100 −50 50 100 150 200 250

11In submission: Pokorny, Kjellstr¨

m

Ek, Kragic KTH Abstraction by Structure

SLIDE 101

Introduction Structural Representations Structural Models Conclusion References

Introduction Structural Representations Structural Models Conclusion

Ek, Kragic KTH Abstraction by Structure

SLIDE 102

Introduction Structural Representations Structural Models Conclusion References

Conclusions

Generalisation not discrimination
Less is sometimes more
Model relevance

Ek, Kragic KTH Abstraction by Structure

SLIDE 103

Introduction Structural Representations Structural Models Conclusion References

Future Work

Multidimensional structure
Different generalisations
Latent space priors
New kernels

◮ know the characteristics of the space Ek, Kragic KTH Abstraction by Structure

SLIDE 104

Introduction Structural Representations Structural Models Conclusion References

e.o.f.

Ek, Kragic KTH Abstraction by Structure

SLIDE 105

Introduction Structural Representations Structural Models Conclusion References

References I

E. Aksoy, A. Abramov, F

. W¨

rg¨
tter, and B. Dellen. Categorizing

Object-Action Relations from Semantic Scene Graphs. In IEEE International conference on robotics and automation, 2010.

G. Carlsson. Topology and data. American Mathematical Society,

2009.

R. Detry, C. H. Ek, M. Pronobis, J. Piater, and D. Kragic.

Generalizing Grasps Across Partly Similar Objects. In IEEE International conference on robotics and automation, 2012.

C. H. Ek. GP-LVM for Data Consolidation. Neural Information

Processing Systems: Workshop on Learning from multiple sources, 2008.

Ek, Kragic KTH Abstraction by Structure

SLIDE 106

Introduction Structural Representations Structural Models Conclusion References

References II

C. H. Ek and D. Kragic. The importance of structure. International

Symposium on Robotic Research, 2011.

C. H. Ek, D. Song, and D. Kragic. Learning Conditional Structures

in Graphical Models from a Large Set of Observation Streams through efficient Discretisation. In IEEE International Conference on Robotics and Automation, Workshop on Manipulation under Uncertainty, 2011.

N. D. Lawrence. Probabilistic non-linear principal component

analysis with Gaussian process latent variable models. The Journal of Machine Learning Research, 2005.

H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and
C. Watkins. Text classification using string kernels. The Journal
f Machine Learning Research, 2002.

Ek, Kragic KTH Abstraction by Structure

SLIDE 107

Introduction Structural Representations Structural Models Conclusion References

References III

G. Luo, N. Bergstr¨
m, C. H. Ek, and D. Kragic. Representing

actions with Kernels. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011.

M. Salzmann, C. H. Ek, R. Urtasun, and T. Darrell. Factorized

Orthogonal Latent Spaces. International Conference on Artificial Intelligence and Statistics, 2010.

D. Song, C. H. Ek, K. Huebner, and D. Kragic.

Embodiment-specific representation of robot grasping using graphical models and latent-space discretization. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011a.

Ek, Kragic KTH Abstraction by Structure

SLIDE 108

Introduction Structural Representations Structural Models Conclusion References

References IV

D. Song, C. H. Ek, K. Huebner, and D. Kragic. Multivariate

discretization for bayesian network structure learning in robot

grasping. In IEEE International conference on robotics and

automation, 2011b.

M. Titsias and N. D. Lawrence. Bayesian Gaussian Process

Latent Variable Model. In International Conference on Artificial Intelligence and Statistics, 2010.

Ek, Kragic KTH Abstraction by Structure