Marco Cavallo Riverwalk: Incorporating Historical Photographs in - - PowerPoint PPT Presentation

▶
marco cavallo
SMART_READER_LITE
LIVE PREVIEW

Marco Cavallo Riverwalk: Incorporating Historical Photographs in - - PowerPoint PPT Presentation

Aug 30, 2023 11 likes •312 views

Marco Cavallo Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences POLITECNICO DI MILANO Marco Cavallo Research Presentation First of all some discussions about terminology used could be

slide-1
SLIDE 1

Marco Cavallo

Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-2
SLIDE 2

First of all…

… some discussions about terminology used could be done, but for now we will stick to the term «Augmented Reality» (Milgram, 1994)

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-3
SLIDE 3

Before…

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-4
SLIDE 4

…after

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-5
SLIDE 5

The Chicago 0,0 Project

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-6
SLIDE 6

The Chicago 0,0 Project

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Ongoing project

>> in collaboration with Geoffrey Alan Rhodes

  • Chicago History Museum’s massive photo archive
slide-7
SLIDE 7

The limits of markerless AR

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Flat images
  • Highly textured
  • Feature distribution
  • Avoid organic shapes
  • Avoid repetitive patterns
  • Good local contrast
slide-8
SLIDE 8

Guess what?

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Repetitive patterns (windows!)
  • Sometimes uniform in color
  • Lighting and weather conditions
  • Flat…?
slide-9
SLIDE 9

Additional complaints

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

What if…

  • No features to be tracked?
  • No flat images to track?
  • We wanted to create more complex behaviors?
  • We wanted to improve the design process?
slide-10
SLIDE 10

Our solution: geolocating content

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

… we live in a 3D world after all :)

Towards an absolute camera pose

Both overlays and fiducials are geolocated!

slide-11
SLIDE 11

The dual camera approach

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • ARCamera

>> pose estimated through pattern-based image tracking

  • SensorCamera

>> leveraging geomagnetic field, accelerometer and gyroscope

… an abstraction for Markerless & Location-based AR

slide-12
SLIDE 12

The ARCamera

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Temporary solution -> general purpose markerless AR and… oversampling :-/
  • Future solution -> specific algorithm for architectural features

… approach by itself algorithm-independent

slide-13
SLIDE 13

The ARCamera

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

But… what’s the novelty?

Absolute positioning!!

slide-14
SLIDE 14

The SensorCamera

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Absolute position: (A-)GPS

>> Corrected with step detectors, multi-sensors odometry, visual odometry

  • Absolute orientation: Inertial Measurement Unit (IMU)

>> Accelerometer, Gyroscope, Magnetic field

slide-15
SLIDE 15

Estimating the pose of the camera

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

4 cases to be handled:

  • Fiducial found
  • Fiducial lost
  • Multiple fiducials
  • No fiducials available
slide-16
SLIDE 16

General advantages

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Absolute Geolocation

>> Abstract location-based and marker-less AR as a single experience >> Load and unload content based on user location >> Display virtual content even if no tracking is available >> Know virtual content close to user or to other virtual content >> Prune the dataset of pattern images based on proximity >> Virtual environment where to edit and preview the application >> Dynamic resource management

slide-17
SLIDE 17

General advantages

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Absolute orientation:

>> Know how much user is moving away from a tracked object >> Display objects even if tracking has been lost >> Know how to orient user towards other virtual content >> Know how to filter virtual content if overlapping >> Signaling incoherent situations and false positives / negatives >> Intelligent camera smoothing (stabilization)

slide-18
SLIDE 18

Navigation & information browsing

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Color coding and showing available content
  • Showing target where the user has to aim
  • Indicating whete to move / rotate to next object
  • Indicate when device needs calibration
  • Allow transitions between adjacent / overlapping

content

  • Suggest the user where to position to see from a

particular perspective

  • Map + optional navigation
  • Allow user to move / modify / correct by himself?
slide-19
SLIDE 19

Just to be sure…

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Weather conditions
  • Absence of light
  • Device limitations

… always keep a classic user interface to allow users visualizing the content

slide-20
SLIDE 20

The designer perspective

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

… ever wondered?

slide-21
SLIDE 21

The designer perspective

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-22
SLIDE 22

Towards a first authoring tool

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Selecting view points
  • Placing overlays
  • Previewing them offline…

…simulating the user’s experience!!

slide-23
SLIDE 23

What’s next?

Ops… now we’ll go a bit off-topic :)

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

… or maybe not?

slide-24
SLIDE 24

Not that again!

Let’s broaden our term… let’s use <<Mixed Reality>> (Milgram, 1994)

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-25
SLIDE 25

Let’s think outside the box

What if we could…

  • Create a better way to design and edit MR experiences
  • Modify at runtime the MR experience
  • Study how users behave while using our application
  • Interact selectively with them

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

slide-26
SLIDE 26

Merging the two worlds

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

1:1 mapping to real-world locations

slide-27
SLIDE 27

An editor but also…

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

Not only a nice way to edit virtual content

slide-28
SLIDE 28

A portal between realities

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

Content modified real-time on clients

Users represented as avatars with 4+3 DOF

slide-29
SLIDE 29

Many new possibilities!!

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation

  • Real-time content customization in order to solve problems or satisfy

user needs

  • Possibily to analyze the behavior of users and to store significant data
  • Possibility to interact with users in MR

>> Create private audio and video channels >> See from their real-world perspective >> Debug the whole MR solution >> Add ourselves to their MR experience

slide-30
SLIDE 30

Thanks for watching ;)

POLITECNICO DI MILANO

Marco Cavallo – Research Presentation