Adaptive Incident Radiance Field Sampling and Reconstruction Using - - PowerPoint PPT Presentation
Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning Yuchi Huo, Rui Wnag, Ruzhang Zheng, Hualin Xu, Hujun Bao, Sung-eui Yoon KAIST and CAD&CG Global Illumination Path Guiding Filtering
Yuchi Huo, Rui Wnag, Ruzhang Zheng, Hualin Xu, Hujun Bao, Sung-eui Yoon KAIST and CAD&CG
2
Filtering Path Guiding
3
Filtering Path Guiding
4
min KPCN Ours KPCN Ours min 1 2 8 30 60 120 Reference
5
6
QUALITY NETWORK Sample RECONSTRUCTION NETWORK Evaluate Reward high reward if Reconstruction à GT
7
Reward … Dataset Feature R-network Dataset Ground Truth Q-network Input Target Output Reconstructio n Output Input Target Reward Generation
8
ü To guide path tracing ü To generate preview ü Others v Extremely sparse samples v Memory/computation overhead v Hard to find ground-truth
φ θ
9
Q-network:
Adaptive samplin g and refining the radiance field, trai ned by DRL
φ θ
R-network:
Reconstruct 4D r adiance field in b
rection spaces
10
R-network
11
R-network
(a) Sampling in Direction Space
0.5π θ=0 φ=0 2π cos%& 0.5
12
R-network
(a) Sampling in Direction Space
0.5π θ=0 φ=0 2π cos%& 0.5
(b) Quality values of adap tive actions Q-network Prediction Refine
Resample
13
R-network
(a) Sampling in Direction Space
0.5π θ=0 φ=0 2π cos%& 0.5
(b) Quality values of adap tive actions Q-network Prediction Refine
Resample (c) Adaptive sampling and refining in Direction Space
14
15
𝑯+
CONVs (6 layers)
…
CONVs (3 layers)
θ φ
𝑺+
&
𝑺+
&-
𝑺+
&
𝑯+ 𝑮𝒆+
&
w h
𝑮𝒆+
&-
Rearrange
𝑮𝒆0
&
𝑮𝒆0
&-
𝒚2 3 𝑀25
6 (𝒚2)
𝑮𝒆0
6
…
…
…
Image Space Features: Normal, Radiance … Direction Space Feature Maps
16
Refinement
Resampling
17
Flatten
𝑮𝒆+
=
CONVs
(3 layers)
𝑯+
𝑺+
=
𝑺+
@(=)
𝑰+
=
FCs
(4 layers) Q-value of Refining Q-value of Resampling w h
𝑮𝒆2
=
Image Space Features: Normal, Radiance, Hierarchies … Direction Space Feature Maps
18
19
20
21
22
min KPCN Ours KPCN Ours min 1 2 8 30 60 120 Reference