Virtual and Augmented Reality 3D Scanning and Reconstruction (02) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz
How the lectures should look like #1 Ask questions, please!!! - Be communicative - www.slido.com #VAR01 - More active you are, the better for you! - 2
3D Model Representations 3
3D Object Representations How to represent a 3D object? 4
3D Object Representations Polygons… 5
3D Object Representations Voxels… 6
3D Object Representations Recreating artwork 7
3D Object Representations Mechanical objects 8
3D Object Representations Maps, Cities, Landscape 9
3D Object Representations Clouds, Smoke, Fog, Water 10
3D Object Representations Points Range Image, Point Cloud Surfaces Polygonal, Subdivision, Parametric, Implicit Solids Voxels, BSP Tree, CSG, Sweep, etc. Hierarchical Structures Scene graph, Application specific… 11
Why so many representation? Efficiency for different tasks Rendering Acquisition Manipulation Animation Analysis Data structures determine algorithms 12
Range Image Set of 3D points mapping to pixels of depth image Structured Point Cloud Acquired using a range scanner (eg. Kinect) 13
Point Cloud Unstructured set of 3D point samples Acquired from multiple range scans, vision, etc. 14
Polygonal Mesh Connected mesh of polygons (usually triangles) Most common representation, supported in OpenGL 15
Key Questions How to refine mesh ? Aim for properties like smoothness How to store mesh ? Aim for efficiency of implementing subdivision rules 16
Polygonal meshes V, E, F P, S 17
Polygonal Meshes Mesh Representations Independent faces Vertex and face tables Adjacency lists Winged-Edge 18
Independent Faces Each face lists vertex coordinates Redundant vertices No topology information 19
Vertex and Face Tables Each face lists vertex references Shared vertices Still no topology information 20
Adjacency Lists Store all vertex, edge and face adjacency Efficient topology traversal Extra storage 21
Partial Adjacency Lists Can we can store only some adjacency information and derive others? 22
Winged Edge Adjacency encoded in edges All adjacencies in O(1) time Little extra storage Arbitrary polygons 23
Winged Edge Example 24
Taxonomy of 3D representations 25
Computational Differences Efficiency Computational complexity ( O(n log n) ) Space/Time trade-off Numerical stability/accuracy Simplicity Hardware acceleration Ease of acquisition Software creation and maintenance Usability Designer vs. computational engine 26
Parametric vs. polygonal Parametric smooth, reparametrizable harder rendering precise rendering Polygonal discrete, hard to reparametrize faster rendering or rasterization approximative rendering 27
How the lectures should look like #2 Ask questions, please!!! - Be communicative - www.slido.com #VAR01 - More active you are, the better for you! - 28
3D Cameras 29
“Stereo Vision” > 2 cameras Photogrametry 30
“Time of Flight” Ligh pulses and its reflections captured by photodiodes 31
“Structured Light” Projection of patterns, capturing by camera 32
“Structured Light” Binary sequence – position 33
Scan Registration Iterative Closest Point (ICP) https://www.youtube.com/watch?v=k116t4cef-4 34
Scan Registration Iterative Closest Point (ICP) 35
Scan Registration Iterative Closest Point (ICP) 36
ICP Normal-space Sampling 37
ICP Point-to-Plane Error Metric Minimize the sum of the squared distance between a point and the tangent plane at its correspondence point [Chen & Medioni 91] 38
Filtering Multiview filtering Volumetric grid Sample radius ML-based using CNN 39
Mesh reconstruction Conversion into an implicit function Poisson reconstruction 40
Mesh reconstruction Isosurface extraction Marching cubes 41
Marching cubes Create cubes Classify vertices Build indices Lookup edge list 42
Marching cubes Create cubes Classify vertices Build indices Interpolate Triangle Vertices 43
Marching cubes Create cubes Classify vertices Build indices Interpolate Triangle Vertices Calculate normals 44
Marching cubes 45
Marching cubes 46
Marching cubes 47
Marching cubes 48
Marching cubes 49
PRAFOS Point cloud Rigid Alignment and Fusion of Scans 50
Rotable - Point cloud Rigid Alignment and Fusion of Scans 51
Rotable 52
TAROS 53
TAROS 54
TAROS 55
Inspection and Editing Inspection and Editing of Point Clouds in VR Bachelor thesis 2019 Bc. Luk áš Gajdošech 56
Inspection and Editing 57
Non-Rigid scanning and reconstruction Human Model Fusion 58
Ultimate Goal Real-time capturing of human performance and reconstruction rendering in virtual reality 59
VR Pipeline Skeleton and surface tracking Surface reconstruction fusion Compression & data streaming Surface reconstruction from textures and skeleton Rendering & applications 60
Skeleton Tracking 61
Point Cloud & Tracked Skeleton 62
Tracking Optical Flow 63
Skeletex Data Structure 64
Skeletex Skeletex: Skeleton-texture Co-representation for Topology-driven Real-time Interchange and Manipulation of Surface Regions PACIFIC GRAPHICS 2018 ( Martin Madaras, Adam Rie čický, Michal Mesároš, Martin Stuchlík, Michal Piovarči ) 65
Skeletex Pipeline Automatic conversion of standard mesh 66
Skeletex Pipeline Real time GPU reconstruction Pose-independent 67
Mesh Generation Regular quad lattice is generated for each segment 68
Skeletex Construction 69
Skeletons Automatically extracted Generated by artist Property neighboring surface segments are neighbors in the skeleton 70
Skeletex Layout 71
Separation Planes Space segmentation 72
Mesh Segmentation Flood-fill algorithm over vertices 73
Skeletex Segmentation 74
Hybrid Skeletex Parameterization 75
SPD Separation Planes Driven Deformed piecewise cylindrical parameterization Given directly by the structure 76
VSM Vertex Sampling Maximizing Balances the triangle area in texture space 77
Bone Tangent Space Pose-independent, normalized form of the surface 78
Animation support Roll quaternion 79
Animation support Yaw-pitch Quaternion 80
Quaternion Deformations Roll quaternion Yaw-pitch quaternion 81
Quaternion Deformations 82
Skeletex LoD 32x32 up to 1024x1024 83
Skeletex Reconstructions 84
Segment Interchange 85
Texture-space Morphing 86
Texture-space Morphing 87
Texture-space Morphing 88
Texture-space Morphing 89
Texture-space Morphing 90
Skeletex Fusion 91
Skeleton-based Segmentation 92
Segmentation Map 93
Height Map 94
Intensity Map 95
Fusion 96
State of the art dynamic fusion techniques 97
VolumeDeform 98
Fusion4D 99
BodyFusion 100
Recommend
More recommend
