motion autonomy IN CONSTRAINED ENVIRONMENTS Requires specific - - PDF document

27/02/2015 1

Advisors: Franck Multon Julien Pettré

Motion planning and synthesis for arbitrary creatures in constrained environments

Steve Tonneau

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

motion autonomy IN CONSTRAINED ENVIRONMENTS

Requires specific locomotion skills Motion results from contact interactions

2

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Motion results from relevant contact sequence

3

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

State of the art application: assassin’s creed 3

4

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

"Can you climb ON EVERY TREE?"

“It has to be a tree that splits into two trunks”

5

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

MOTION CAPTURE: one motion, one session

New geometries? Non humanoid characters?

6

(Beyond: 2 souls)

27/02/2015 2

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Motion Autonomy as a motion planning problem

7

A virtual creature R The workspace W Start and goal postures A collision free motion

Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Motion Autonomy as a motion planning problem

8

A virtual creature R The environment W Start and goal postures A relevant contact sequence

Introduction Introduction Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Related work

Robotics solutions to the motion planning problem

Configuration space approach Sampling based motion planners

Towards more natural looking motions

Motion editing techniques Model based approaches

9

• Conf. space
• Conf. space

motion planners motion planners motion editing motion editing motion analysis motion analysis Introduction Introduction Related work EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion Overview Overview

• Conf. space

motion planners motion planners motion editing motion editing motion analysis motion analysis

10

.. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . . . . . . . . . .

x y z

.. .. . . .. . . . . .. .. . . .. .

(Lozano-Peréz83)

THe configuration space (Lozano-Peréz 83)

q =

• …

∈ ℝN Introduction Introduction Related work EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion Overview Overview

• Conf. space

motion planners motion planners motion editing motion editing motion analysis motion analysis

11

.. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . .. .. . . .. . . . . . . . . . .

x y z

.. .. . . .. . . . . .. .. . . .. .

q =

• …

∈ ℝN

(Lozano-Peréz83)

THe configuration space (Lozano-Peréz 83)

Configuration space Configuration space 3D environment 3D environment Introduction Introduction Related work EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion Overview Overview

• Conf. space

motion planners motion planners motion editing motion editing motion analysis motion analysis

12

Configuration space Configuration space

THe configuration space (Lozano-Peréz 83)

3D environment 3D environment

27/02/2015 3

Introduction Introduction

• Conf. space
• Conf. space

motion planners motion editing motion editing motion analysis motion analysis Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Sampling based motion planning

Probabilistic Road Maps (Kavrakiet al. 96) Rapidly exploring Random Trees (Lavalle98)

13

x x

Introduction Introduction

• Conf. space
• Conf. space

motion planners motion editing motion editing motion analysis motion analysis Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Contact before motion approaches

1 step planning in the contact space

(Bretlet al. 03) (Escandeet al. 09) (Bouyarmaneet al. 09)

PERFORMANce Quality of Motion

14

Introduction Introduction

• Conf. space
• Conf. space

motion planners motion editing motion editing motion analysis motion analysis Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

TOWARDS More natural looking SOLUTIONS

Trajectory optimization (CIO)

(Mordatchet al. 12)

Local minima PERFORMANce

15

• Conf. space
• Conf. space

motion planners motion planners motion editing motion analysis motion analysis Introduction Introduction Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Combining Motion planning and Motion Editing

Sampling + Motion capture

(Pettréet al. 03) (Choi et al. 03) (Huang et al. 11 )

16

…

• Conf. space
• Conf. space

motion planners motion planners motion editing motion analysis motion analysis Introduction Introduction Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Combining Motion planning and Motion Editing

Warping

(Witkinet Popovic95) (Gleicher97), (Al-ashqaret al. 13)

Blending

(Boulic et al. 96 ????) (Kovaret Gleicher03) (Ménardaiset al. 04)

Concatenating

(Kovaret Gleicher02) (Shum al. 09)

17

w1 + w2 =

• Conf. space
• Conf. space

motion planners motion planners motion editing motion analysis motion analysis Introduction Introduction Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Motion editing LIMITATIONS

UNNATURAL DEFORMATIONS

(Safonovaet Hodgins, 07) (Reitsmaet Pollard, 07)

FIXED CONTACT SEQUENCEs

18

JohanssenUnity 3d

27/02/2015 4

Introduction Introduction Related work EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion Overview Overview

• Conf. space
• Conf. space

motion planners motion planners motion editing motion editing motion analysis

FROM MOTION ANALYSIS TO MODELS

Dynamic models and controllers

(Coroset al. 11) (Mordatchet al. 13) (Wampleret al. 14)

Motion invariants for reaching / manipulating

(Kallmannet al. 10) (Sreenivasaet al. 12)

19

Introduction Introduction Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

“A trade off between naturalness and control”

(Van Welbergenet al. 10)

Related work SUMMARY

20

Unknown Constrained environments Quality of motion Performance Arbitrary creatures Motion editing + +++ +++ NO Model based ++ ++ +++ YES Sampling based +++ + + YES Sampling based +++ ? ? YES Introduction Introduction Related work Related work Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Contribution 1: A new Criterion for task efficiency

Considering locomotion tasks in constrained environments How to generate a task efficient contact configuration? EFORT: The Extended FORce Transmission ratio

21

Introduction Introduction Related work Related work Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Contribution 1: A new Criterion for task efficiency

22

EFORT EFORT

Introduction Introduction Related work Related work Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Contribution 2: A sampling based planner for EFORT

How to compute relevant contact sequences for locomotion tasks in constrained environments?

23

Introduction Introduction Related work Related work Overview EFORT EFORT RB-PRM RB-PRM Conclusion Conclusion

Contribution 2: A sampling based planner for EFORT

How to compute relevant contact sequences for locomotion tasks in constrained environments? RB-PRM: Reachability Based PRM

24

EFORT RB-PRM RB-PRM EFORT EFORT ANIMATION ANIMATION

27/02/2015 5

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

EFORT

25

EFORT EFORT

Task efficient contact configurations for arbitrary virtual creatures Tonneau, Pettréet Multon, Graphics interface ’14 conference Using task efficient contact configurations to animate creatures in arbitrary environments Tonneau, Pettréet Multon, Computer and Graphics journal, vol41, 2014 Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

THE IMPORTANCE OF FORCE EXERTION

26

Which contacts to stand up?

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

THE IMPORTANCE OF FORCE EXERTION

Random stable configuration More natural configuration More efficient configuration for force exertion

27

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

EFORT: PROBLEM STATEMENT

Given: How to rapidly compute:

28

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Velocity and force ellipsoids

(Yoshikawa 84)

THE Manipulability ellipsoid

29

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

EFORT: THE EXTENDED force transmission ratio

, ! ! " #$

%

(Chiu 87)

&'()* , +* q, . ./

• 30

27/02/2015 6

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Method pipeline

Offline phase: sampling reachable workspace

(Herdaet al. 04), (Matthew et al. 14), (Haeringet al. 14)

31

X X

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Method pipeline

Offline phase: sampling reachable workspace

(Herdaet al. 04), (Matthew et al. 14), (Haeringet al. 14)

32

Configuration space Configuration space 3D environment 3D environment Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Method pipeline

Offline phase: sampling in the reachable workspace

(Herdaet al. 04), (Matthew et al. 14), (Haeringet al. 14)

33

Configuration spaces Configuration spaces 3D environment 3D environment Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Method pipeline

Online phase

34

? ? EFORT 3D environment 3D environment Configuration space Configuration space Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

ILLUSTRATION

35

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Results: pushing

1 10000 5 5 6 5 1 10000 samples Average numberof contact candidates: 142 Average time: 5 5 Worst time: 6 5

36

27/02/2015 7

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Results: pulling

1 10000 5 5 6 5 1 10000 samples Average numberof contact candidates: 142 Average time: 5 5 Worst time: 6 5

37

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Results: pulling

1 10000 5 5 6 5 1 10000 samples Average numberof contact candidates: 142 Average time: 5 5 Worst time: 6 5

38

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Results: Insect crawling

1 1000 7 5 5 4 5 1 1000 samples Average num contact candidates: 20 Average time: 7 5 5 Worst time: 4 5

39

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

Results: Real time computation of contact configurations

1 100 000 7 5 5 4 5 1 100 000 samples Average num contact candidates: 26 Average time: 7 5 5 Worst time: 4 5

40

Introduction Introduction Related work Related work Overview Overview EFORT RB-PRM RB-PRM Conclusion Conclusion

CONCLUSION ON EFORT

A method for synthesizing task efficient contact configurations Desirable properties Arbitrary creatures and environments Real time generation Task efficiency Limitations / future work Additional criterion for global posture Handling joint torque limitations

41

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RB-PRM

42

EFORT EFORT EFORT RB-PRM RB-PRM EFORT EFORT ANIMATION ANIMATION

Reachability based planning for task efficient contact sequences in constrained environments Tonneau, Pettré, Park, Manocha, Mansard and Multon, RSS 2015 conference, submitted

27/02/2015 8

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

The reachability condition

Sampling root configurations for contact creation

43

V V X X

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

The reachability condition

(Pignonet al. 91, 92)

9)(: Limb Range Of Motion 9*;< Abstract enclosed shape

44

V V

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Sampling in the reachability space

(Yamato et al. 98)

45

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Graph GENERATION

46

X X V V X X V V

3D environment 3D environment Configuration space Configuration space Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Path REQUEST

47

3D environment 3D environment Configuration space Configuration space Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Path REQUEST

48

3D environment 3D environment Configuration space Configuration space

27/02/2015 9

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Path REQUEST

49

3D environment 3D environment Configuration space Configuration space Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

CONTACT SEQUENCE GENERATION

50

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Rule 1: maintain existing contacts

51

X X X X

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Rule 2: create contact when losing balance

(Qiuet al. 11)

52 ! !

EFORT EFORT

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

TRAJECTORY EXTENSION WITH EFORT

! ! ! ! ! 53

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

TRAJECTORY EXTENSION WITH EFORT

54

27/02/2015 10

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Results

55

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

56

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

57

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

58

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

59

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

60

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10

27/02/2015 11

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

61

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

62

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

63

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

64

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Truck EGRESS

65

1 10000 73 10 1 10000 samples Offline generation time: 73 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

66

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10

27/02/2015 12

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

67

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

68

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

69

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

70

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

71

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

72

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10

27/02/2015 13

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

73

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

74

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

75

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

76

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

77

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

78

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10

27/02/2015 14

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

79

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

80

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

81

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

82

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

83

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: Crawling

84

1 10000 5 10 1 10000 samples Offline generation time: 5 Online computation time: 10

27/02/2015 15

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

RESULTS: STANDING UP

85

1 10000 5 7 1 1 10000 samples Offline generation time: 5 Online computation time: 7 1 Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Standing up: basic interpolation

86

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM Conclusion Conclusion

Conclusion on RB-PRM

A motion planning framework for task efficient contact configurations Desirable properties

Completeness of RB-PRM Performance Task efficiency

Limitations / future work

Global optimality of the sequence (number of contacts) Better task formulation Contact not limited to end effectors

87

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion

GENERAL Conclusion

How to generate a task efficient contact configuration in an unknown environment? EFORT

How to compute relevant contact sequences for locomotion tasks in constrained environments?

RB-PRM

88

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion

GENERAL Conclusion

Enhanced autonomy of motion for virtual characters Limitations:

Guaranteeing balanced trajectories Global optimality of the approach

89

Sampling based +++ ++ +++ YES Unknown Constrained environments Quality of motion Performance Arbitrary creatures Sampling based +++ + + YES Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion

FUTURE RESEARCH and applications

Experiments with EFORT Proposing a global animation framework (UNC) Towards Robotics applications

90

ANIMATION ANIMATION

27/02/2015 16

Introduction Introduction Related work Related work Overview Overview EFORT EFORT RB-PRM RB-PRM Conclusion

Thank you for your attention

Task efficient contact configurations for arbitrary virtual creatures

Tonneau, Pettréet Multon, Graphics interface ’14 conference

Using task efficient contact configurations to animate creatures in arbitrary environments

Tonneau, Pettréet Multon, Computer and Graphics journal, vol41, 2014

Reachability based planning for task efficient contact sequences in constrained environments

Tonneau, Pettré, Park, Manocha, Mansard et Multon, RSS 2015 conference, submitted

91