Augmented Reality: Fusing the Real and Synthetic Worlds Mauro - - PowerPoint PPT Presentation

▶
augmented reality
SMART_READER_LITE
LIVE PREVIEW

Augmented Reality: Fusing the Real and Synthetic Worlds Mauro - - PowerPoint PPT Presentation

Aug 18, 2022 201 likes •542 views

Augmented Reality: Fusing the Real and Synthetic Worlds Mauro Dalla Mura, Michele Zanin, Claudio Andreatta, Paul Chippendale, Fondazione Bruno Kessler, Trento, Italy Morphological and Attribute Profiles Spectral/spatial analysis What is

slide-1
SLIDE 1

Mauro Dalla Mura, Michele Zanin, Claudio Andreatta, Paul Chippendale, Fondazione Bruno Kessler, Trento, Italy

Augmented Reality:

Fusing the Real and Synthetic Worlds

slide-2
SLIDE 2

2

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Morphological and Attribute Profiles Spectral/spatial analysis IEEE IGARSS’12

What is Augmented Reality?

slide-3
SLIDE 3

3

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Morphological and Attribute Profiles Spectral/spatial analysis IEEE IGARSS’12

What is Augmented Reality?

Can it be useful for the GRS community?

slide-4
SLIDE 4

Outline

  • 1. Introduction to Augmented Reality
  • 2. AR and GRS: the proposed concept
  • 3. Augmenting the user’s view
  • 4. Augmenting geo-data
  • 5. Discussion and conclusion
  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-5
SLIDE 5

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-6
SLIDE 6

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

1950s 1966 Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-7
SLIDE 7

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

1950s 1966 2000 Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-8
SLIDE 8

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

1950s 1966 2000 2008 Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-9
SLIDE 9

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

1950s 1966 2000 2008 2014 Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-10
SLIDE 10

Augmented Reality

Augmented Reality (AR) blends virtual and real scenes in both time and space

1950s 1966 2000 2008 2014

. . .

Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-11
SLIDE 11

AR applications

 Gaming

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-12
SLIDE 12

AR applications

 Gaming  Entertainment

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-13
SLIDE 13

AR applications

 Gaming  Entertainment  Sport

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-14
SLIDE 14

AR applications

 Gaming  Entertainment  Sport  Commercial

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-15
SLIDE 15

AR applications

 Gaming  Entertainment  Sport  Commercial

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu Urban planning

slide-16
SLIDE 16

AR applications

 Gaming  Entertainment  Sport  Commercial

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu Urban planning  Tourism

slide-17
SLIDE 17

AR applications

 Gaming  Entertainment  Sport  Commercial

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu Urban planning  Tourism  Navigation

slide-18
SLIDE 18

AR applications

 Gaming  Entertainment  Sport  Commercial

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu Urban planning  Tourism  Navigation  . . .

. . .

slide-19
SLIDE 19

Augmented Reality for GRS

Augmented Reality exploiting geo-data  Availability of massive archives of geo-referenced data (e.g., remote sensing, geo-referenced social photo websites, GIS, geo-data)

A possible exploitation of geo-data

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

Augmentation of a 2D map superimposing the corresponding 3D model

 Effective integration of AR and geo-data is becoming feasible on consumer- level portable devices

slide-20
SLIDE 20

Augmented Reality for GRS

Available applications showing geo-data on smart-phones

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

Layar (since Jun 2009) Peak.AR (since Oct 2009) Wikitude (since May 2009) SwissPeaks (since Nov 2009) Peaks Mixare (since Mar 2010)

Device-world registration is usually not accurate (e.g., errors in horizontal

  • rientation of 30 deg if using the on-board compass)
slide-21
SLIDE 21

Photo-world registration

Accurate alignment of the device and the surrounding panorama is necessary

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

Captured Photograph DEM 2D Rendered DEM Aligned Photograph

Salient feature extraction Photo registration

 Camera position (ϕ0, λ0, h0) and pose (α, β, γ) should be estimated  Vision based approaches operates by finding the best match between visual features and synthetic ones (from a 3D model)  Hardware information (e.g., magnetometer, compass, GPS) can be used as initial clue of the pose  After photo-world registration, each pixel in the image can be associated to geographic coordinates

Panorama rendering

slide-22
SLIDE 22

AR for GRS: the proposed concept

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-23
SLIDE 23

Reality augmentation

Augmenting the user’s view

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-24
SLIDE 24

Augmenting the user’s view

 Once registered, geo-data can be delivered to the user

 Geographical toponyms  GIS layers  Topographic maps  3D building models  . . .

 New modality to query and access geo-data  Fast interactive modality (e.g., touchscreen)

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-25
SLIDE 25

Augmenting the geoscientist’s view

 Support tool for field campaigns:

 Real-time guidance for field surveys (e.g., ground truth collection)  In-situ validation of remote sensing data and products (e.g., land cover maps)

 Aid the interpretation of geo-referenced RS data

 data not in the visible range  time series

Forest regions from Corine land cover 2000 (25m resolution)

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

Topographic map

slide-26
SLIDE 26

Reality augmentation

Augmenting the geo-data archives

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-27
SLIDE 27

Augmenting the geo-data archives

Users’ portable devices become ground-based sensors

r c ϕ λ

Digital Elevation Model

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

position pose focal length

(r, c)  (ϕ, λ) f(F, ϕ0, λ0, h0, α, β, γ, DEM)

slide-28
SLIDE 28

Ground-based sensing

 Extract Information on the land cover

November snowline >1785 m December snowline >1212 m

Oct. 2007 Nov. 2007 Dec. 2007 Snow cover map

0.2 0.4 0.6 0.8 1 860 1060 1260 1460 1660 1860 2060 2260 2460 2660 2860 3060 Altitude (m) Snow (percentage) Sept Nov Jan

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-29
SLIDE 29

Ground-based sensors

 Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-30
SLIDE 30

Ground-based sensors

 Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution  Capture vertical surfaces  Massive databases of geo-tagged images (for free!)  Near-live acquisition (e.g., for damage assessment)

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-31
SLIDE 31

Ground-based sensors

 Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution  Capture vertical surfaces  Massive databases of geo-tagged images (for free!)  Near-live acquisition (e.g., for damage assessment)

Limited spatial coverage

Uneven resolution

Occlusions

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-32
SLIDE 32

Discussion and conclusion

The effectiveness, quality and accuracy of the AR result depends on many factors:  Geo-data projection and rendering are demanding operations and should be performed fast  Estimation of the camera pose and position for achieving a precise match for accurately overlay virtual information on the scene and for acquiring information on the landscape  High-speed wireless transfer of data (i.e., optimize network transfers) Conclusion  Hardware and software specification of consumer-level portable devices are constantly improving  The exploitation of portable devices for AR applications has just started  We proposed a concept that exploiting AR technology for GRS applications  Thanks to an accurate registration of the device to the world it is possible to perform  visualization of geo-data on the mobile device interactively  exploitation of the mobiles’ camera as a ground-based imaging sensors

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu

slide-33
SLIDE 33

Introduction

Discussion and conclusion

Morphological and Attribute Profiles Spectral/spatial analysis

from http://www.gls-tedesco-a-berlino.de/2065.html

Thanks for your attention!

https://venturi.fbk.eu/ http://tev.fbk.eu/marmota/

  • M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

IEEE IGARSS’12 dallamura@fbk.eu