Fundamentals of Optics and Color Instructors: Yasuhiro Mukaigawa, - - PowerPoint PPT Presentation

▶
fundamentals of optics and color
SMART_READER_LITE
LIVE PREVIEW

Fundamentals of Optics and Color Instructors: Yasuhiro Mukaigawa, - - PowerPoint PPT Presentation

Oct 13, 2023 154 likes •666 views

Visual Media Processing II 2020 No.1 Fundamentals of Optics and Color Instructors: Yasuhiro Mukaigawa, Takuya Funatomi, Kenichiro Tanaka Mini report 3-digit unique ID has been sent to you. If you havent gotten it, consult TA

slide-1
SLIDE 1

No.1 Fundamentals of Optics and Color

Instructors: Yasuhiro Mukaigawa, Takuya Funatomi, Kenichiro Tanaka

2020 Visual Media Processing II

slide-2
SLIDE 2

Visual Media Processing II (4098), 2020 Fall

Mini report

  • 3-digit unique ID has been sent to you.
  • If you haven’t gotten it, consult TA immediately.
  • Small questions are asked in the lecture.
  • Input your unique ID and answers to

Google Forms.

  • Today’s mini-report:

http://bit.ly/vmp2-2020-1

  • Also, link has been sent by email.

2

slide-3
SLIDE 3

Visual Media Processing II (4098), 2020 Fall

What is Computer Vision?

  • Computer vision is a method that extracts

visual information of real world from images.

  • It gives computers and robots the ability for

visually recognizing real world.

Real World Computer A red ball on the white table. It is soft. The picture was taken indoors.

Observation target Observation

slide-4
SLIDE 4

Visual Media Processing II (4098), 2020 Fall

3D reconstruction

  • Photometric stereo (#7)
  • Use multiple images taken with different

lighting directions

照度差ステレオ

slide-5
SLIDE 5

Visual Media Processing II (4098), 2020 Fall

3D reconstruction

  • Photometric stereo

http://openaccess.thecvf.com/content_iccv_2015/papers/Gotardo_Photogeometric_Scene_Flow_ICCV_2015_paper.pdf

slide-6
SLIDE 6

Visual Media Processing II (4098), 2020 Fall

Photometric analysis

  • Reflection, shade, shadow, scattering
  • Analysis by physics-based model

Attached shadow Cast shadow Specular reflection Undefined Diffuse reflection

陰 影 散乱 反射

slide-7
SLIDE 7

Visual Media Processing II (4098), 2020 Fall

Photometric consistency

  • Rendered object match to environment?
  • Color
  • Shadow
  • Shade

Etc.

  • Real environment should be measured.
  • Global lighting
  • Reflection model of sofa
  • Floor position

Etc.

7

[photo credit: Prof. Kiyokawa] Strange rendering Photorealistic

slide-8
SLIDE 8

Visual Media Processing II (4098), 2020 Fall

Computational photography

  • Breaking the limit of camera by computation.

Real-time vein visualization Computational bokeh control after capture

slide-9
SLIDE 9

Visual Media Processing II (4098), 2020 Fall

Non-Line-of-Sight Imaging

  • Imaging around corner.
  • Sophisticated light reflection analysis.

9

RGB camera Visible wall NLOS scene Obstacle Light paths

Scene Recovered

slide-10
SLIDE 10

Visual Media Processing II (4098), 2020 Fall

Mini-report – Access Google Forms and answer questions.

10

http://bit.ly/vmp2-2020-1

slide-11
SLIDE 11

Visual Media Processing II (4098), 2020 Fall

Photometric analysis pipeline

11

Image processing / image analysis (#3 - #8)

Optics (#1, Today) (#2) (#1, Today) (#1, #2) Sensor in-camera processing

slide-12
SLIDE 12

Visual Media Processing II (4098), 2020 Fall

Ray, Wave, Particle

  • Ray
  • Wave

12

  • Particle (Photon)

Explainable region of Light Photon (Quantum optics) Wave (Wave optics) Ray (Geometric optics)

slide-13
SLIDE 13

Visual Media Processing II (4098), 2020 Fall

Interaction between Light and Matter

13

Specular reflection (Highlight) Shadow Fresnel reflection (Water reflection) Diffuse reflection Refraction (Transmission) (Rayleigh) Scattering (Blue sky) (Mie) Scattering (Clouds) Interreflection (Indirect lighting)

slide-14
SLIDE 14

Visual Media Processing II (4098), 2020 Fall

Reflection

14

  • Specular reflection
  • Diffuse reflection
  • Dichroic reflection model (Refer #5)
  • Specular (highlight) is light source color.
  • Diffuse reflection is material color.

𝜄𝑗 = 𝜄𝑝 Perfectly smooth Microscopically rough （鏡面反射） （拡散反射）

  • Inherits incident light properties
  • Contains material’s properties

Mixture of reflections specular diffuse appearance

反射

slide-15
SLIDE 15

Visual Media Processing II (4098), 2020 Fall

Refraction

  • Snell’s law
  • Refractive index 𝜃: how slow compared to vacuum.
  • Fresnel transmission / reflection
  • Fresnel reflection is angular-dependent.
  • Dispersion
  • Refractive index is wavelength-dependent.

15 Reflection No reflection

Incident angle reflection transmission 0% 100% 90° 0°

𝜄𝑏 𝜄𝑐 sin 𝜄𝑏 sin 𝜄𝑐 = 𝑤𝑏 𝑤𝑐 = 𝜃𝑏𝑐

wavelength Refraction index

屈折率 スネルの法則 フレネル透過・反射 分散 屈折

slide-16
SLIDE 16

Visual Media Processing II (4098), 2020 Fall

Mini-report

16

http://bit.ly/vmp2-2020-1

slide-17
SLIDE 17

Visual Media Processing II (4098), 2020 Fall

Polarization

  • Linear polarization
  • Direction of oscillation is stable.
  • AoLP, DoLP
  • Polarization state can be observed by rotating

polarizer.

17

Rotation angle of polarizer Passed intensity of light A B 𝜚 Degree of Linear Polarization (DoLP) = A / (A+B) Angle of Linear Polarization (AoLP) = 𝜚 偏光 振動 偏光板

slide-18
SLIDE 18

Visual Media Processing II (4098), 2020 Fall

Polarization Filters

  • Cut polarized reflection from water surface.
  • Fresnel reflection again.
  • Fresnel reflection is polarized

18

kenko-tokina.co.jp

Normal photo With polarization filter

Incident angle Parallel component 0% 100% 90° 0° Perpendicular component

polarized

slide-19
SLIDE 19

Visual Media Processing II (4098), 2020 Fall

Liquid Crystal and LCD

  • Electrically rotate polarization angle.

19

tdk.com Liquid crystal Polarizers Groove Voltage

  • White pixel
  • Black pixel

Light By inducing voltage, polarized light is not rotated and blocked by 2nd polarizer. 液晶 液晶ディスプレイ

slide-20
SLIDE 20

Visual Media Processing II (4098), 2020 Fall

Circular Polarization

  • Polarization (oscillation direction) is rotating.
  • Clockwise, anticlockwise
  • Specular reflection changes the direction.

20

watson.jp Linear polarization Circular polarization CW Anti-CW 円偏光 時計回り・反時計回り Sony.jp 3D vision

slide-21
SLIDE 21

Visual Media Processing II (4098), 2020 Fall

Mini-report

21

http://bit.ly/vmp2-2020-1

slide-22
SLIDE 22

Controlling light rays

22

slide-23
SLIDE 23

Visual Media Processing II (4098), 2020 Fall

Lens

  • Converting point light to parallel light and vice versa.
  • Gathering lights

23

  • lympus-imaging.jp

A convex lens （凸レンズ） A concave lens （凹レンズ）

slide-24
SLIDE 24

Visual Media Processing II (4098), 2020 Fall

Mirrors

  • Plane
  • Mathematically curved

Paraboloid Ellipsoid Hyperboloid

24

Metal coating Everyday mirror Fresnel Main reflection Surface mirror Ideal reflection Flat at nano-level

放物面 楕円面 双曲面 表面鏡

slide-25
SLIDE 25

Visual Media Processing II (4098), 2020 Fall

Filters and Coatings

  • ND Filters
  • Color filters
  • Anti-reflection coatings
  • All filters and lens can have reflection.

25

Wavelength Transmission Blue filter Orange filter Destructive interference

反射防止膜

弱めあう ND filter

slide-26
SLIDE 26

Visual Media Processing II (4098), 2020 Fall

Beamsplitters

  • Split into transmission and reflection.
  • Dichroic mirrors
  • Wavelength dependent beamsplitters

26

X % (100-X) % Wavelength

reflection transmission

Wavelength Wavelength

reflection transmission

2色性 Half mirror (50%T-50%R)

slide-27
SLIDE 27

Visual Media Processing II (4098), 2020 Fall

Mini-report

27

http://bit.ly/vmp2-2020-1

slide-28
SLIDE 28

Visual Media Processing II (4098), 2020 Fall

Generating light rays

  • How to control light rays?
  • Projector as a ray generator

28

Projected image

slide-29
SLIDE 29

Visual Media Processing II (4098), 2020 Fall

3LCD Projector

  • Separate 3 color channel and combine

29

Nitto-optical.co.jp Dichroic mirrors to separate color channels Dichroic prism to merge color channels LCD panel for each color White lamp Lenses for gathering light Projection lens 投影レンズ

slide-30
SLIDE 30

Visual Media Processing II (4098), 2020 Fall

Time Division

  • Color is time divided
  • Quiz: How to identify time divided projectors?

30

Color wheel Projection lens Light source To screen

slide-31
SLIDE 31

Visual Media Processing II (4098), 2020 Fall

Mini-report

31

http://bit.ly/vmp2-2020-1

slide-32
SLIDE 32

What is Color?

Photometric conversion from light to color.

slide-33
SLIDE 33

Visual Media Processing II (4098), 2020 Fall

Appearance of object

  • A sphere is placed on a table.
  • The camera position is fixed.
  • But there are many appearances.
slide-34
SLIDE 34

Visual Media Processing II (4098), 2020 Fall

Color Representations in Computer

  • Color names
  • red, orange, yellow, green, blue, or purple.
  • Numerical parameters
  • How many parameters are necessary to define colors?
slide-35
SLIDE 35

Visual Media Processing II (4098), 2020 Fall

Linear color spaces: RGB

  • Invented for practical reasons.
  • Single wavelength primaries
  • R: 645.16nm
  • G: 526.32nm
  • B: 444.44nm
  • Represented as unit cube

色空間

slide-36
SLIDE 36

Visual Media Processing II (4098), 2020 Fall

Non-Linear color spaces: Hue, Saturation, and Value

  • Representing human intuitions about the topology of colors.
  • hues forms a circle from red through orange to yellow and

then green, cyan, blue, purple, and then red again.

  • HSV color space
  • Hue （色相）, Saturation（彩度）, Value（明度）
slide-37
SLIDE 37

Visual Media Processing II (4098), 2020 Fall

CMYK color space

  • Mixture of color inks.
  • Cyan, Magenta, Yellow, and Black

37

slide-38
SLIDE 38

Visual Media Processing II (4098), 2020 Fall

Limitation of common color spaces

  • Most color spaces are subset of perceptible

color region.

38

Vivid colors tend to outside color space.

slide-39
SLIDE 39

Visual Media Processing II (4098), 2020 Fall

Mini-report

39

http://bit.ly/vmp2-2020-1

slide-40
SLIDE 40

Visual Media Processing II (4098), 2020 Fall

Three Aspects of Color

(1) Spectral quantity (2) Recording using color filters (3) Human color perception

slide-41
SLIDE 41

Visual Media Processing II (4098), 2020 Fall

Physics of Color

  • Light is electromagnetic wave.
  • " The rays are not colored." by Sir Isaac Newton.

「光線には色はついていない」

  • Human can sense lights between 380nm to 780nm,

and feel different color for each wavelength.

Wikipedia 電磁波 波長

slide-42
SLIDE 42

Visual Media Processing II (4098), 2020 Fall

How to Record Colors?

  • CCD/CMOS has no function to sense color

without color filters.

  • Spectral imaging camera
  • Multi band color camera
  • 3CCD camera for capturing RGB images
  • 1CCD with Bayer pattern

16-band color camera 3CCD Bayer pattern Spectral camera

slide-43
SLIDE 43

Visual Media Processing II (4098), 2020 Fall

Model for Color Observation

  • Multiplication of all spectral effects.

Incident spectrum 𝐹(𝜇) Sensitivity of color sensor 𝑇𝑗(𝜇)

𝑊

𝑗 = න 𝜇=380 780

𝐹 𝜇 𝑆 𝜇 𝑇𝑗 𝜇 𝑒𝜇

𝑗 ∈ [𝑆, 𝐻, 𝐶]

Spectral reflectance 𝑆(𝜇)

R G B

slide-44
SLIDE 44

Visual Media Processing II (4098), 2020 Fall

Mini-report

44

http://bit.ly/vmp2-2020-1

slide-45
SLIDE 45

Visual Media Processing II (4098), 2020 Fall

Color is sometimes tricky

  • What colors are this dress?
slide-46
SLIDE 46

Visual Media Processing II (4098), 2020 Fall

How do the other side people see?

46

http://kazoo04.hatenablog.com/entry/2015/02/27/150642

slide-47
SLIDE 47

Visual Media Processing II (4098), 2020 Fall

Human color perception

  • Human perception of colors is a complex

function of context; illumination, memory,

  • bject identity, and emotion.
  • Color is a psychological effect.

https://engineering.purdue.edu/~bouman/ece637/notes/ColorConstancy/color/

知覚 心理学的

slide-48
SLIDE 48

Visual Media Processing II (4098), 2020 Fall

Color constancy （色恒常性）

  • Advanced visual ability of human that

perceive true color under varying illumination conditions.

slide-49
SLIDE 49

Visual Media Processing II (4098), 2020 Fall

Human eye vs camera

  • Color is physical and psychological effect.
  • Camera records the physical intensity.
  • Human perceive subjectively.

Human eye Camera

Computer reddish R:230, G:14, B:23

slide-50
SLIDE 50

Visual Media Processing II (4098), 2020 Fall

Mini-report

51

http://bit.ly/vmp2-2020-1