Marco Cavallo Riverwalk: Incorporating Historical Photographs in Public Outdoor Augmented Reality Experiences POLITECNICO DI MILANO Marco Cavallo – Research Presentation
First of all … … some discussions about terminology used could be done, but for now we will stick to the term « Augmented Reality » (Milgram, 1994) POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Before … POLITECNICO DI MILANO Marco Cavallo – Research Presentation
… after POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The Chicago 0,0 Project POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The Chicago 0,0 Project • Ongoing project >> in collaboration with Geoffrey Alan Rhodes Chicago History Museum ’s massive photo archive • POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The limits of markerless AR • Flat images • Highly textured • Feature distribution • Avoid organic shapes • Avoid repetitive patterns • Good local contrast POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Guess what? • Repetitive patterns (windows!) • Sometimes uniform in color • Lighting and weather conditions • Flat … ? POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Additional complaints What if … • No features to be tracked? • No flat images to track? • We wanted to create more complex behaviors? • We wanted to improve the design process? POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Our solution: geolocating content Towards an absolute camera pose Both overlays and fiducials are geolocated! … we live in a 3D world after all :) POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The dual camera approach … an abstraction for Markerless & Location-based AR • ARCamera >> pose estimated through pattern-based image tracking • SensorCamera >> leveraging geomagnetic field, accelerometer and gyroscope POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The ARCamera • Temporary solution -> general purpose markerless AR and … oversampling :-/ • Future solution -> specific algorithm for architectural features … approach by itself algorithm-independent POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The ARCamera But … what’s the novelty? Absolute positioning!! POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The SensorCamera • Absolute position: (A-)GPS >> Corrected with step detectors, multi-sensors odometry, visual odometry • Absolute orientation: Inertial Measurement Unit (IMU) >> Accelerometer, Gyroscope, Magnetic field POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Estimating the pose of the camera 4 cases to be handled: • Fiducial found • Fiducial lost • Multiple fiducials • No fiducials available POLITECNICO DI MILANO Marco Cavallo – Research Presentation
General advantages • Absolute Geolocation >> Abstract location-based and marker-less AR as a single experience >> Load and unload content based on user location >> Display virtual content even if no tracking is available >> Know virtual content close to user or to other virtual content >> Prune the dataset of pattern images based on proximity >> Virtual environment where to edit and preview the application >> Dynamic resource management POLITECNICO DI MILANO Marco Cavallo – Research Presentation
General advantages • Absolute orientation: >> Know how much user is moving away from a tracked object >> Display objects even if tracking has been lost >> Know how to orient user towards other virtual content >> Know how to filter virtual content if overlapping >> Signaling incoherent situations and false positives / negatives >> Intelligent camera smoothing (stabilization) POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Navigation & information browsing • Color coding and showing available content • Showing target where the user has to aim • Indicating whete to move / rotate to next object • Indicate when device needs calibration • Allow transitions between adjacent / overlapping content • Suggest the user where to position to see from a particular perspective • Map + optional navigation • Allow user to move / modify / correct by himself ? POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Just to be sure … • Weather conditions • Absence of light • Device limitations … always keep a classic user interface to allow users visualizing the content POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The designer perspective … ever wondered? POLITECNICO DI MILANO Marco Cavallo – Research Presentation
The designer perspective POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Towards a first authoring tool • Selecting view points • Placing overlays • Previewing them offline … … simulating the user’s experience!! POLITECNICO DI MILANO Marco Cavallo – Research Presentation
What’s next? Ops … now we’ll go a bit off-topic :) … or maybe not? POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Not that again! Let’s broaden our term … let’s use << Mixed Reality >> (Milgram, 1994) POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Let’s think outside the box What if we could … ● Create a better way to design and edit MR experiences ● Modify at runtime the MR experience ● Study how users behave while using our application ● Interact selectively with them POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Merging the two worlds 1:1 mapping to real-world locations POLITECNICO DI MILANO Marco Cavallo – Research Presentation
An editor but also … Not only a nice way to edit virtual content POLITECNICO DI MILANO Marco Cavallo – Research Presentation
A portal between realities Content modified real-time on clients Users represented as avatars with 4+3 DOF POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Many new possibilities!! • Real-time content customization in order to solve problems or satisfy user needs • Possibily to analyze the behavior of users and to store significant data • Possibility to interact with users in MR >> Create private audio and video channels >> See from their real-world perspective >> Debug the whole MR solution >> Add ourselves to their MR experience POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Thanks for watching ;) POLITECNICO DI MILANO Marco Cavallo – Research Presentation
Recommend
More recommend
