Speaking Swarmish Physical Human-Robot Interface Design for Large - - PowerPoint PPT Presentation

▶
speaking swarmish
SMART_READER_LITE
LIVE PREVIEW

Speaking Swarmish Physical Human-Robot Interface Design for Large - - PowerPoint PPT Presentation

Jun 25, 2023 262 likes •632 views

Speaking Swarmish Physical Human-Robot Interface Design for Large Swarms of Autonomous Mobile Robots James McLurkin 1 , Jennifer Smith 2 , James Frankel 3 , David Sotkowitz 4 , David Blau 5 , Brian Schmidt 6 1, 5, 6 2, 3, 4 James McLurkin

slide-1
SLIDE 1

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Speaking Swarmish

Physical

∧Human-Robot Interface Design for Large

Swarms of Autonomous Mobile Robots

James McLurkin1, Jennifer Smith2, James Frankel3, David Sotkowitz4, David Blau5, Brian Schmidt6

1, 5, 6 2, 3, 4

slide-2
SLIDE 2

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

My Position

  • My Axe: As a community, we don’t spend enough time

working on (or give enough respect to) user and development interfaces for robotic systems

  • As the field matures, popular interfaces will

inevitability arise. We can either design thoughtful interfaces (good) or let them arise spontaneously (bad)

  • This work describes a set of simple user interfaces that

use Physical I/O to interface an unaided user to large* groups of robots

  • Oooo! A new Acronym:

HMRI = Human Multi-Robot Interface

Now, we’re ready to get some work done!

*Large ≈ 100. Your definition may vary.

slide-3
SLIDE 3

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

The HMRI Problem

  • Requirements

๐Get information to all the robots (programs, commands) ๐Get information about what all the robots are doing

(internal state)

๐Get information from all the robots (mission sensors)

  • Some Terms:

๐Virtual I/O: Interface with robots through computer ๐Physical I/O: Interface with robots directly ๐Development Interface: Interface for developers ๐User Interface: Interface for users

slide-4
SLIDE 4

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Related Work

Daily, Cho, Martin, Payton, 2002 “World Embedded Interfaces”

Collett and MacDonald, 2006, “Developer Oriented Visualisation of a Robot Program” StarCraft, Blizzard Entertainment, 1999 Kismet, Breazeal et. al., 1998-200X

slide-5
SLIDE 5

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

HRI Design Space

slide-6
SLIDE 6

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

This Work

  • Physical Infrastructure for hands-free operation
  • Centralized Control Software for development,

debugging and operation

  • Physical I/O for in situ interaction

๐ in situ = Allow the user to watch the robots while

reading telemetry about individual and group behaviors

๐Displays on robots are useless on multi-robot systems ๐The Human perceptual system can process a vast amount

  • f multi-modal real-time data

๐Allow the trained, but unaided user to “read” the swarm

slide-7
SLIDE 7

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Physical Infrastructure for HMRI

  • Problem: The robots outnumber the humans

๐Anything you have to do to one robot, you have to do to

all of them. This does not scale well.

  • Solution: Hands-Free Design

๐The SwarmBot has dedicated hardware for the physical

user interface

๐Basic Swarm-level tasks must be hands-free:

Remote Power On/Off Autonomous Charging Remote Programming Efficient Transportation Software for Centralized Command, Control, and Data Collection

slide-8
SLIDE 8

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

SwarmBot Hardware

Behavior LEDs Radio Charging Contacts Camera IR Inter-Robot Communications 1.1 Watt Audio System Bump Skirt

  • range = for physical UI

grey = for autonomous

  • peration
slide-9
SLIDE 9

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

The Swarm “Extrastructure”

slide-10
SLIDE 10

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Docking

slide-11
SLIDE 11

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Long-Range Navigation

slide-12
SLIDE 12

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Very Long-Range Navigation

slide-13
SLIDE 13

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Remote Downloading System

  • [diagram of remote downloading procedure goes here]

(waive hands furiously)

  • Status tones are separated by 25ms, and are still clear

and easy to disambiguate

slide-14
SLIDE 14

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

SwarmCraft GUI

  • Virtual I/O: Real-Time data collection and visualization
  • Necessary, but not sufficient
slide-15
SLIDE 15

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

SwarmCraft GUI

slide-16
SLIDE 16

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

SwarmCraft GUI

slide-17
SLIDE 17

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Group Exercise: Conversational Swarmish

Three different color lights: red, green, blue + Two blinking patterns: sine and square + Many frequencies . = 108 patterns than can each be read in ≤ 500ms

slide-18
SLIDE 18

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Clumping

slide-19
SLIDE 19

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Orientation

slide-20
SLIDE 20

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Navigation

slide-21
SLIDE 21

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Dispersing

slide-22
SLIDE 22

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Dispersion

slide-23
SLIDE 23

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

BubbleLine

slide-24
SLIDE 24

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

BubbleLine: Flip

slide-25
SLIDE 25

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

DTA: Extreme Comms-Line

slide-26
SLIDE 26

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

DTA: Extreme Comms

slide-27
SLIDE 27

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

DTA: Card Dealers (Clumping)

slide-28
SLIDE 28

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

DTA: Tree-Recolor

slide-29
SLIDE 29

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Application: “Directed Dispersion”

  • FrontierBot
  • WallBot
  • InteriorBot
  • ChargingBot
  • GuideBot
slide-30
SLIDE 30

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Directed Dispersion: Big Room Explore

  • Blue=Frontiers

Red=Interior Green=Charging White=Guides

slide-31
SLIDE 31

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Directed Dispersion: Big Room Return

slide-32
SLIDE 32

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Directed Dispersion: Complex Rooms

slide-33
SLIDE 33

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Limitations of Swarmish

  • Have not found a good way to represent continuous

quantities

  • Current lexicon has about 108 usable patterns
  • Sounds need to be carefully designed to work together

⇒ lots of trial and error

slide-34
SLIDE 34

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Future Work

  • Careful analysis of the human perceptual system and its

bandwidth limitations

  • How to optimally leverage multi-modal input

๐Rich non-speech and non-text interfaces

  • Simple Augmented Reality Devices

๐Hand-held tools to query robots: Tricoder-like device

slide-35
SLIDE 35

James McLurkin AAAI Spring Symposium 2006 Speaking Swarmish

Conclusion

HMRI ⇒ Usability ⇒ Efficiency ⇒ ((Debug Cycles)/Second)++ ⇒ ((Ideas Tested)/Second)++ ⇒ (Software Quality)++ ⇒ (Publications/Second)++

Questions?