PALADYN The construction of humanoid robotic head Krzysztof Luks, - - PowerPoint PPT Presentation

▶
paladyn
SMART_READER_LITE
LIVE PREVIEW

PALADYN The construction of humanoid robotic head Krzysztof Luks, - - PowerPoint PPT Presentation

Apr 08, 2024 233 likes •381 views

PALADYN The construction of humanoid robotic head Krzysztof Luks, Piotr Ka zmierczak { kluks,piotr } @ai.pjwstk.edu.pl Polish-Japanese Institute of Information Technology. PJIIT Robotics and Multiagent Systems Lab. Poland. 1 the

slide-1
SLIDE 1

PALADYN

The construction of humanoid robotic head

Krzysztof Luks, Piotr Ka´ zmierczak {kluks,piotr}@ai.pjwstk.edu.pl

Polish-Japanese Institute of Information Technology. PJIIT Robotics and Multiagent Systems Lab. Poland. µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 1

slide-2
SLIDE 2

PRESENTATION OUTLINE

❶ HARDWARE OVERVIEW ❷ SOFTWARE OVERVIEW ❹ DRIVER ARCHITECTURE ❸ DATA ACQUISITION SCHEME ❺ BINAURAL LOCALISATION ❻ BINAURAL SYSTEM DESIGN ❼ MATCHING EYE MOVEMENTS ❽ VOLUNTARY EYE MOVEMENTS ❾ PREATTENTIVE CUES ❿ FUTURE WORK

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 2

slide-3
SLIDE 3

HARDWARE OVERVIEW

  • 5 DoF, based on early Cog design,

avoid using custom boards

  • Visual, auditory, vestibular and control
  • 4 CCD cameras: ImagingSource DFM 5303P

and BT878 based PXC-200F framegrabbers

  • Microphones with low-noise pre-amplifier
  • 3 axis accelerometer/gyro & NuDAQ 9112 A/D
  • Motion control: Galil DMC-1850

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 3

slide-4
SLIDE 4

SOFTWARE OVERVIEW

  • QNX RTOS nodes: vision (SMP), binaural localisation, motor control. . .
  • Hard real time system essential for control tasks
  • Shared memory for intra-node communication
  • QNET Network transparency for inter-node communication
  • btlib, libmotors: BT848/878 and Galil QNX libraries developed in our Lab
  • Various other libraries: OpenCV, FFTW, dmcqnx, pxc200

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 4

slide-5
SLIDE 5

DRIVER ARCHITECTURE

Pure resource manager model Our hybrid resource manager

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 5

slide-6
SLIDE 6

DATA ACQUISITION SCHEME

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 6

slide-7
SLIDE 7

BINAURAL LOCALISATION

Geometrical problem overview

  • Duplex theory: simple ITD and IID physics
  • IDD = Dleft − Dright = IT D × Vsound

Ambiguity

  • Directional uncertainty: front-back & left-right
  • Path differences producing similar ITD values

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 7

slide-8
SLIDE 8

BINAURAL SYSTEM DESIGN

Data acquisition:

  • A/D conversion
  • window function
  • DFT processing (fftw library)

Spectral & correlation proc.:

  • tone extraction
  • ITD and IID calculation
  • IID enforced ITD calculation

Post-processing:

  • low-pass filtering
  • half-wave rectification
  • SCOT Transform

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 8

slide-9
SLIDE 9

MATCHING EYE MOVEMENTS

  • So far implemented: voluntary movements:

saccades, smooth pursuit, vergence.

  • All voluntary movements were based on

very simple normalised cross correlation

  • Partially implemented: VOR, OKR
  • Pre-attentive cues: motion, color, skin-tone
  • Other saliency maps: habituation, sound

source localisation (aids attention), no stereovision yet

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 9

slide-10
SLIDE 10

VOLUNTARY EYE MOVEMENTS

Saccade

Smooth pursuit

Vergence

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 10

slide-11
SLIDE 11

PREATTENTIVE CUES

Different saliency maps

  • motion segmentation,
  • habituation, sound sources,
  • (color, skin tone extraction)

Motion and sound

  • sound aids visual attention
  • bar width: probability
  • bar value: signal strength

Habituation and sound

  • activation: weighted sum
  • target selection: centroid of

the biggest region→saccade

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 11

slide-12
SLIDE 12

FUTURE WORK

  • Develop cognitive vision system that learns

from observation

  • Start working with log-polar representation,

implement new vision algorithms

  • New analytical model for auditory cue

templates (based on scattering theory)

  • Develop STM memory module

utilising real-time in-memory database system

  • Make PALADYN work with YARP

µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 12

slide-13
SLIDE 13

QUESTIONS

Thank You.

Binaural localisation: Lech B la˙ zejewski lechblaz@ai.pjwstk.edu.pl Motor control: Sebastian Pawlak trix@ai.pjwstk.edu.pl Vestibular system: Krzysztof Luks kluks@ai.pjwstk.edu.pl Visual behaviours: Piotr Ka´ zmierczak piotr@ai.pjwstk.edu.pl µ ¶ · ¸ the construction of humanoid robotic head: RobotCub Open Day – July 14, 2005 13