rs rrt r - PowerPoint PPT Presentation

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ❚♦✇❛r❞s ❝♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✐♥❦❡❞ ▼✉❧t✐✲❈♦♠♣♦♥❡♥t ❘♦❜♦t✐❝ ❙②st❡♠s ❇♦r❥❛ ❋❡r♥❛♥❞❡③✲●❛✉♥❛✱ ❏♦s❡ ▼❛♥✉❡❧ ▲♦♣❡③✲●✉❡❞❡✱ ▼❛♥✉❡❧ ●r❛ñ❛ ❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮ ❍❆■❙ ✷✵✶✶✱ ❲r♦❝❧❛✇ ✭P♦❧❛♥❞✮ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✶ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ❖✉t❧✐♥❡ ■♥tr♦❞✉❝t✐♦♥ ✶ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ✷ ❊①♣❡r✐♠❡♥ts ✸ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✷ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ▲✐♥❦❡❞ ▼✉❧t✐❝♦♠♣♦♥❡♥t ❘♦❜♦t✐❝ ❙②st❡♠s ❉❡✜♥✐t✐♦♥✿ ❣r♦✉♣ ♦❢ r♦❜♦t✐❝ ✉♥✐ts ♣❤②s✐❝❛❧❧②✲❧✐♥❦❡❞ ❜② ❛ ♥♦♥✲r✐❣✐❞ ❡❧❡♠❡♥t✳ P❤②s✐❝❛❧ ❧✐♥❦ ✐♥tr♦❞✉❝❡s ♥❡✇ ♥♦♥✲❧✐♥❡❛r ❞②♥❛♠✐❝s ❛♥❞ ♣❤②s✐❝❛❧ ❝♦♥str❛✐♥ts ✐♥ t❤❡ s②st❡♠✳ ❚r❛❞✐t✐♦♥❛❧ ❝♦♥tr♦❧ t❡❝❤♥✐q✉❡s ❛r❡ ♥♦t ❛♣♣r♦♣r✐❛t❡ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✸ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ▼✉❧t✐✲❆❣❡♥t ❘❡✐♥❢♦r❝❡♠❡♥t ▲❡❛r♥✐♥❣ ❘❡✐♥❢♦r❝❡♠❡♥t ▲❡❛r♥✐♥❣ ✭❘▲✮ ❙❡t ♦❢ ❛❧❣♦r✐t❤♠s t❤❛t ❧❡❛r♥ ❜② ❡①♣❧♦r✐♥❣ t❤❡ st❛t❡ s♣❛❝❡ S t❛❦✐♥❣ ❛❝t✐♦♥s ❢r♦♠ s❡t A ❆ r❡✇❛r❞ ❢✉♥❝t✐♦♥ q✉❛❧✐✜❡s ❤♦✇ ❣♦♦❞ t❤❡ ♦❜s❡r✈❡❞ st❛t❡ ✐s ✭ R : S → R ✮ ●♦❛❧✿ ♠❛①✐♠✐③❡ t❤❡ ❛❝❝✉♠✉❧❛t❡❞ r❡✇❛r❞s ♦✈❡r t✐♠❡ ◗✲▲❡❛r♥✐♥❣ ❊st✐♠❛t❡s t❤❡ r❡✇❛r❞s t♦ ❜❡ ♦❜t❛✐♥❡❞ ❛❢t❡r t❛❦✐♥❣ ❛❝t✐♦♥ a ✐♥ st❛t❡ s ❜② ❧♦♦❦✐♥❣ ♦♥❡ st❡♣ ❛❤❡❛❞✿ � �� Q ( s ′ , a ′ ) − Q ( s , a ) � Q ( s , a ) ← Q ( s , a )+ α r + γ ∗ max a ′ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✹ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ▼✉❧t✐✲❆❣❡♥t ❘❡✐♥❢♦r❝❡♠❡♥t ▲❡❛r♥✐♥❣ ▼❛✐♥ ❘▲ ❞r❛✇❜❛❝❦✿ ❡①♣♦♥❡♥t✐❛❧ ❣r♦✇t❤ ♦❢ t❤❡ st❛t❡✲❛❝t✐♦♥ s♣❛❝❡ ✭ | S × A | ✮ ▼✉❧t✐✲❆❣❡♥t ❘❡✐♥❢♦r❝❡♠❡♥t ▲❡❛r♥✐♥❣ ✭▼❆❘▲✮ ♠❛❦❡s ✐t ❡✈❡♥ ✇♦rs❡✿ | S × A n | ▲✲▼❈❘❙ ♣r❡s❡♥t ❛♥ ❛❞❞✐t✐♦♥❛❧ ♣r♦❜❧❡♠✿ ♣❤②s✐❝❛❧ ❝♦♥str❛✐♥ts✳ ❙♦♠❡ st❛t❡s ❢♦r❝❡ s✐♠✉❧❛t✐♦♥ t♦ st♦♣ ❛♥❞ st❛rt ♦✈❡r ❊①❛♠♣❧❡s✿ ♣❤②s✐❝❛❧✲❧✐♥❦ str❡t❝❤❡❞ ❜❡②♦♥❞ ✐ts ♥♦♠✐♥❛❧ ❧❡♥❣t❤ ❝♦❧❧✐s✐♦♥ ❜❡t✇❡❡♥ r♦❜♦t✐❝ ✉♥✐ts ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✺ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts Pr♦❜❧❡♠ ❙t❛t❡♠❡♥t A set of n linked robots (each of them represented as P i ) must carry the tip of a hose from a starting configuration to the goal ❆✈❛✐❧❛❜❧❡ ❛❝t✐♦♥s✿ ❯♣✱ ❉♦✇♥✱ ▲❡❢t✱ ❘✐❣❤t✱ ❯♣✲▲❡❢t✱ ❯♣✲❘✐❣❤t✱ ❉♦✇♥✲▲❡❢t✱ ❉♦✇♥✲❘✐❣❤t ❛♥❞ ◆♦♥❡ ❙✐♠♣❧❡ ❤♦s❡ ♠♦❞❡❧✿ ❧✐♥❡ s❡❣♠❡♥t ❚❡r♠✐♥❛t✐♦♥ ❝♦♥❞✐t✐♦♥s✿ ❆ r♦❜♦t st❡♣s ♦✈❡r t❤❡ ❤♦s❡ ❍♦s❡ s❡❣♠❡♥ts ❛r❡ str❡t❝❤❡❞ ♦✈❡r ♥♦♠✐♥❛❧ ❧❡♥❣t❤ ❆ r♦❜♦t ❣❡ts ♦✉t ♦❢ t❤❡ s✐♠✉❧❛t✐♦♥ ✇♦r❧❞ ❚✇♦ r♦❜♦ts ❝♦❧✐❞❡ ❉❡❝❡♥tr❛❧✐③❡❞ ❝♦♥tr♦❧ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✻ ✴ ✶✺

■♥tr♦❞✉❝t✐♦♥ P❛r❛❞✐❣♠❛t✐❝ ❛♣♣❧✐❝❛t✐♦♥✿ ❍♦s❡ tr❛♥s♣♦rt❛t✐♦♥ ❊①♣❡r✐♠❡♥ts ❊①❛♠♣❧❡ ❋✐❣✉r❡✿ ❆♥ ❡①❛♠♣❧❡ ♦❢ ❛♥ ✐♥✐t✐❛❧ ❝♦♥✜❣✉r❛t✐♦♥ ❤tt♣✿✴✴✇✇✇✳❡❤✉✳❡s✴❝❝✇✐♥t❝♦ ✭❈♦♠♣✉t❛t✐♦♥❛❧ ■♥t❡❧❧✐❣❡♥❝❡ ●r♦✉♣ ❯♥✐✈❡rs✐t② ♦❢ t❤❡ ❇❛sq✉❡ ❈♦✉♥tr② ✭❯P❱✴❊❍❯✮✮ ❈♦♥❝✉rr❡♥t ◗✲▲❡❛r♥✐♥❣ ♦♥ ▲✲▼❈❘❙ ❍❆■❙ ✷✵✶✶ ✼ ✴ ✶✺