Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz - - PowerPoint PPT Presentation

introduction 00
SMART_READER_LITE
LIVE PREVIEW

Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz - - PowerPoint PPT Presentation

Virtual and Augmented Reality Introduction (00) RNDr. Martin Madaras, PhD. madaras@skeletex.xyz Introduction Introduction Who am I? What do I do here? 2 About me Short research bio: - 2014 - finished PhD at FMFI UK - 2014 2016 -


slide-1
SLIDE 1

Virtual and Augmented Reality

Introduction (00)

  • RNDr. Martin Madaras, PhD.

madaras@skeletex.xyz

slide-2
SLIDE 2

2

Introduction Who am I? What do I do here?

Introduction

slide-3
SLIDE 3

3

Short research bio:

  • 2014 - finished PhD at FMFI UK
  • 2014 – 2016 - researcher & freelancer
  • 2016 – 2017 - PostDoc Researcher at TU Wien
  • 2017 – now - CEO, Research lead at Skeletex Research

Collaboration with universities:

  • 2015 – 2018 - research assistant FMFI UK
  • 2018 – now - assistant professor at FIIT STU
  • 2018 – now - assistant professor at FMFI UK

About me

slide-4
SLIDE 4

4

  • Explain basic principles of computer graphics and computer vision in

context of VR/AR applications

  • Tell a story about me, computer graphics and interesting projects
  • Motivate you, students, into CG and CV
  • Create some kind of collaboration between students and our company

What do I do here

slide-5
SLIDE 5

5

  • Freelancing research and development company
  • What we used to do:
  • motion capture, skeleton tracking, human body fusion, 3D cameras
  • Currently we are working on:
  • 3D scanners, scan registration, mesh reconstruction, point cloud segmentation
  • Cooperation with universities:
  • lectures, theses supervision, internships, research, publications
  • Cooperation with tech companies:
  • research and development

What does Skeletex Research do

slide-6
SLIDE 6

6

  • Openworm
  • Skeleton-based compression of particle simulation
  • Kinexact
  • Automatic extraction of skeleton
  • Hand scans
  • Optical-Inertial hybrid tracking of skeleton
  • Webcam based
  • Skeleton tracking and body fusion
  • Texture-space surface fusion
  • Skeletex data structure

Previous projects

slide-7
SLIDE 7

7

3D reconstructions and skeletons

slide-8
SLIDE 8

8

Openworm

slide-9
SLIDE 9

9

Kinexact Hand

slide-10
SLIDE 10

10

Kinexact Body

slide-11
SLIDE 11

11

Optinertial

slide-12
SLIDE 12

12

Optinertial

slide-13
SLIDE 13

13

Human skeleton tracking and fusion

slide-14
SLIDE 14

14

Skeletex data structure

slide-15
SLIDE 15

15

Skeletex data structure

slide-16
SLIDE 16

16

  • Capture human motion
  • Reconstruct in

VR

Our main goal

slide-17
SLIDE 17

17

  • 3D scan segmentation
  • Real-time (for 60fps camera)
  • CUDA implementation for GPU and TegraTX1, TX2 build in camera
  • Use hierarchical structure and flood fill approximation
  • 3D scan registration
  • Iterative Closes Point with fast camera space projections
  • Global optimization (use of scan graph)
  • Tracking (if real-time)
  • 3D model fusion and reconstruction
  • Multi-view filtering + Outlier removal
  • Dynamic and progressive triangulation
  • Rigid / Non-rigid? Real-time?

Current projects

slide-18
SLIDE 18

18

  • Real-time CUDA point cloud segmentation
  • a) calculate metrics based on curvature and distance
  • b) threshold the metrics
  • c) fill regions in parallel (accelerated by hierarchical structure)

Point cloud segmentation

slide-19
SLIDE 19

19

Point cloud Rigid Alignment and Fusion of Scans

PRAFOS

slide-20
SLIDE 20

20

  • Point cloud Rigid Alignment and Fusion of Scans

Rotable

slide-21
SLIDE 21

21

  • PhoXi 3D Meshing

P3DM

slide-22
SLIDE 22

22

MotionCam3D data

slide-23
SLIDE 23

23

PCVR

slide-24
SLIDE 24

24

RAVOS

slide-25
SLIDE 25

25

BinSim

 Synthetic data generation for ML

 Physically-based simulation & virtual scanning

slide-26
SLIDE 26

26

HIRO

slide-27
SLIDE 27

27

  • Computer Games Development
  • Common motivation
  • Ain’t no fun, very hard :/
  • Hard business
  • Financial problems / hard with capital investment
  • CG skills can be used in other fields as well:
  • Film industry
  • Medical applications
  • 3D printing
  • 3D scanning
  • Optical systems
  • Other software

Why CG? Common view…

slide-28
SLIDE 28

28

Slovakia, Bratislava T ech companies, Startups Jobs, University research

Situation

slide-29
SLIDE 29

29

  • CG and years 2010 vs 2014 vs 2019/2020
  • CG Companies in Bratislava
  • Photoneo
  • Capturing Reality
  • Vectary
  • etc.
  • CG Companies in Wien
  • VRVIS
  • Procedural Design
  • CG Companies in Czech
  • Corona

Companies

slide-30
SLIDE 30

30

“The Danube Valley”

Companies

slide-31
SLIDE 31

31

  • STU FIIT
  • Principles of Computer Graphics and Image Processing
  • Advanced Computer Graphics Methods
  • more computer vision courses
  • FMFI UK
  • Fundamentals of Computer Graphics and Image Processing
  • Advanced Computer Graphics
  • Virtual and Augmented Reality
  • Real-time Rendering
  • more computer vision courses
  • You can focus on CG along projects during the study
  • The most important – your B&D Theses!

Universities

slide-32
SLIDE 32

32

  • If you want to do graphics in you professional life after graduation
  • Try it in pro way, get used to such a cooperation during study
  • State of the art research
  • Cooperation of high-end startups and universities
  • Be guided by top experts in the field
  • Winners of student conferences
  • CESCG
  • ŠVOČ
  • PhD. Students internships
  • MIT
  • MPII

Why to collaborate with Skeletex

slide-33
SLIDE 33

33

Virtual and Augmented Reality

Lecture

slide-34
SLIDE 34

34

  • Introduction (0)

VAR

slide-35
SLIDE 35

35

  • Introduction (0)
  • Transformations (1)

VAR

slide-36
SLIDE 36

36

  • Introduction (0)
  • Transformations (1)
  • Animation, Character skeletal animation, Motion capture (2)

VAR

slide-37
SLIDE 37

37

  • Introduction (0)
  • Transformations (1)
  • Animation, Character skeletal animation, Motion capture (2)
  • 3D scanning and reconstruction, Human fusion (3)

VAR

slide-38
SLIDE 38

38

Acknowledgements

 Thanks to all the people, whose work is shown here and whose

slides were used as a material for creation of these slides:

Matej Novotný, GSVM lectures at FMFI UK Peter Drahoš, PPGSO lectures at FIIT STU Output of all the publications and great team work Very best data from 3D cameras

slide-39
SLIDE 39

39

www.skeletex.xyz madaras@skeletex.xyz

Questions ?!