image processing framework
play

Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 - PowerPoint PPT Presentation

Mar 29, 2024 β€’478 likes β€’1.33k views

FlexISP: A Flexible Camera Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 YUN-TA TSAI 1 NASA ROUF 1,2 DAWID PAJK 1 DIKPAL REDDY 1 ORAZIO GALLO 1 JING LIU 4 WOLFGANG HEIDRICH 5 KAREN EGIAZARIAN 1,6 JAN KAUTZ 1 KARI PULLI 1 1

  1. FlexISP: A Flexible Camera Image Processing Framework FELIX HEIDE 1,2 MARKUS STEINBERGER 3 YUN-TA TSAI 1 NASA ROUF 1,2 DAWID PAJΔ„K 1 DIKPAL REDDY 1 ORAZIO GALLO 1 JING LIU 4 WOLFGANG HEIDRICH 5 KAREN EGIAZARIAN 1,6 JAN KAUTZ 1 KARI PULLI 1 1 NVIDIA 2 UBC 3 TU GRAZ 4 UCSC 5 KAUST 6 TUT

  2. CAMERA IMAGE PROCESSING

  4. Image Signal Processor (ISP) 4

  5. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  6. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  7. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  8. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  9. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  10. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  11. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  12. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  13. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Demosaic Denoise Metering Correction Tone Image Enhancing Mapping

  14. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  15. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  16. IMAGE PROCESSING PIPELINE Exposure / Focus Adjustment Bad Pixel Black Level Correction Lens Denoise Demosaic Metering Correction Tone Image Enhancing Mapping

  17. NOVEL FEATURES Long Exposure Short Exposure Long Exposure Short Exposure

  18. NOVEL FEATURES Single Exposure Interleaved Exposure

  19. NOVEL FEATURES

  21. Depth Refocusing

  24. Kepler GPU

  25. OBJECTIVES NO PIPELINE FLEXIBILITY PERFORMANCE

  26. PROPOSED SOLUTION FLEXISP Bayer Burst iHDR Others …

  27. IMAGE FORMATION x B D z

  28. IMAGE FORMATION z = DBx x B D z

  29. IMAGE FORMATION z = A x B D

  30. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2

  31. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 errors w.r.t. observation A x

  32. LINEAR SYSTEM 2 π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 errors w.r.t. observation A x

  33. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 regularization

  34. TOTAL VARIATION (Rudin et al. β€˜92)

  35. CROSS-CHANNEL (Heide et al. β€˜13)

  36. COLLABORATIVE DENOISING

  37. COLLABORATIVE DENOISING

  38. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 + + Total variation Cross-channel Collaborative denoising

  39. IMPORTANCE OF PRIORS

  40. MORE ON PRIORS… TV [Rudin et al. β€˜92] EPLL [Zoran and Weiss β€˜11] Curvelets [ CandΓ¨s and Donoho β€˜99 ]

  41. MORE ON PRIORS…

  42. LINEAR SYSTEM 2 + Ξ»(𝑦) π‘π‘ π‘•π‘›π‘—π‘œ 𝑦 𝑨 βˆ’ 𝐡𝑦 2 Convex Optimization (Mostly)

  43. CONVERGENCE

  44. RESULTS

  45. CAMERA SYSTEMS Bayer Burst iHDR

  46. CAMERA SYSTEMS Bayer Burst iHDR

  47. CAMERA SYSTEMS Bayer Burst iHDR A

  54. CAMERA SYSTEMS Bayer Burst iHDR

  55. CAMERA SYSTEMS Bayer Burst iHDR

  56. CAMERA SYSTEMS Bayer Burst iHDR A

  64. CAMERA SYSTEMS Bayer Burst iHDR

  65. CAMERA SYSTEMS Bayer Burst iHDR

  66. CAMERA SYSTEMS Bayer Burst iHDR A

  73. PERFORMANCE GTX Titan Demosaic (5MP) 2.13 iHDR (13MP) 7.45 Image burst (0.4MPx16) 0.82 TIME IN SECONDS

  74. PERFORMANCE Tegra K1 Demosaic (5MP) 40.5 iHDR (13MP) 174.4 Image burst (0.4MPx16) 16.7 TIME IN SECONDS

  75. MORE ON PRIORS…

  76. PERFORMANCE (BM3D) Tegra K1 Demosaic (5MP) 40.5 iHDR (13MP) 174.4 Image burst (0.4MPx16) 16.7 TIME IN SECONDS

  77. PERFORMANCE (AVERAGING NLM) Tegra K1 Demosaic (5MP) 6.7 iHDR (13MP) 33.2 Image burst (0.4MPx16) 6.6 TIME IN SECONDS

  80. CONCLUSION

  81. Bayer Burst iHDR Color Camera Array + others such as: deconvolution, super-resolution, JPEG compression and de-blocking.

  82. Bayer Burst AOHDR Color Camera Array A FLEXISP

  83. THANK YOU!

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it

BBM 413 Today Fundamentals of What is image processing? Image Processing What does it mean, to see? Vision as a computational problem Sample image processing problems Erkut Erdem Dept. of Computer Engineering Hacettepe

399 views β€’ 13 slides
Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What

Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What

Introduction: What is Image Processing? CS 4640: Image Processing Basics January 10, 2012 What is Image Processing? Definition Image processing is the study of any algorithm that takes an image as input and returns an image as output.

530 views β€’ 16 slides
PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ??

PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ??

PLT Project SIP(Simplified Image Processing) A Language for image processing Why SIP ?? Effectively an image processing language. Concept can be extended for videos Features included to make operations on images short and effective

733 views β€’ 12 slides
Color image processing The use of color in image processing is primarily motivated by two Image

Color image processing The use of color in image processing is primarily motivated by two Image

Color image processing The use of color in image processing is primarily motivated by two Image Analysis factors: Humans can perceive thousands of colors but only about two Color processing dozen shades of gray. Color is a powerful descriptor

1.05k views β€’ 7 slides
BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE

BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE

CHUCK REEVES @MANCHUCK BACKGROUND JOB PROCESSING DO'S AND DON'TS BACKGROUND JOB PROCESSING - DO'S AND DON'TS IMAGE PROCESSOR UPLOAD IMAGE PROCESS IMAGE ENTERS QUEUE ZendCon 2016 BACKGROUND JOB PROCESSING - DO'S AND DON'TS THE PROBLEM

817 views β€’ 35 slides
Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012

Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012

Yayi A generic framework for morphological image processing IPOL Raffi Enficiaud June 2012 (Ex.) CMM - Mines Paris Forewords Yayi - A generic framework for morphological image processing http ://raffi.enficiaud.free.fr Recent library (1st

898 views β€’ 41 slides
. Inverse problems in signal and image processing and Bayesian inference framework: from basic to

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to advanced Bayesian computation Ali Mohammad-Djafari Laboratoire des Signaux et Syst` emes (L2S) UMR8506 CNRS-CentraleSup elec-UNIV PARIS SUD

990 views β€’ 77 slides
Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof

Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof

Digital Image Processing (CS/ECE 545) Lecture 1: Introduction to Image Processing and ImageJ Prof Emmanuel Agu Computer Science Dept. Worcester Polytechnic Institute (WPI) What is an Image? 2 dimensional matrix of Intensity (gray or color)

884 views β€’ 77 slides
Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image

Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image

Deep Learning in Image Processing Topics: Image Filtering 101 CNNs 101 Image Processing Pipelines Frank Dellaert CS 4476 Computer Vision Many slides from Stanfords CS231N by Fei-Fei Li, Justin Johnson, Serena Yeung, as well as

1.46k views β€’ 79 slides
Image Processing CS 110 Why Image Processing? Medical Images

Image Processing CS 110 Why Image Processing? Medical Images

Image Processing CS 110 Why Image Processing? Medical Images 2 Why Image Processing? Manufacturing Dig;al Image Processing, Spring 2006 3 What can you

717 views β€’ 24 slides
Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image

Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image

Lecture 1 Introduction Objectives Digital image processing, Why? Scope of digital image processing Exploration of application areas Components in a typical general-purpose image processing system Directions Digital image

511 views β€’ 30 slides
BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The

BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The

BLINK: A GPU-Enabled Image Processing Framework Mark Davey Lead HPC Engineer The Foundry The Foundry and HPC The Foundry Founded in 1996 We develop award-winning visual effects, computer graphics and design software used globally

560 views β€’ 21 slides
Image Processing Todays Class Image Representations: Matrices Image Representations: RGB,

Image Processing Todays Class Image Representations: Matrices Image Representations: RGB,

CS6501: Deep Learning for Visual Recognition Image Processing Todays Class Image Representations: Matrices Image Representations: RGB, HSV, etc Image Processing: Brightness Image Filtering: Mean Filter, Median Filter Image Filtering:

959 views β€’ 77 slides
Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 10, 2020 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

840 views β€’ 37 slides
Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) July 4, 2019 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

1.02k views β€’ 36 slides
Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama

Introduction to Computer Graphics Image Processing (1) June 8, 2017 Kenshi Takayama Todays topics Edge-aware image processing Gradient-domain image processing 2 Image smoothing using Gaussian Filter Smoothness parameter

1.25k views β€’ 38 slides
1 Gbps and 10 Gbps WAN Emulator IPNetSim Multi Stream IP WAN Emulator 818 West Diamond

1 Gbps and 10 Gbps WAN Emulator IPNetSim Multi Stream IP WAN Emulator 818 West Diamond

1 Gbps and 10 Gbps WAN Emulator IPNetSim Multi Stream IP WAN Emulator 818 West Diamond Avenue - Third Floor, Gaithersburg, MD 20878 Phone: (301) 670-4784 Fax: (301) 670-9187 Email: info@gl.com 1 1 Website: http://www.gl.com

806 views β€’ 27 slides
Observation of THz CSR Observation of THz CSR Burst at UVSOR- -II II Burst at UVSOR 1 Miho

Observation of THz CSR Observation of THz CSR Burst at UVSOR- -II II Burst at UVSOR 1 Miho

Observation of THz CSR Observation of THz CSR Burst at UVSOR- -II II Burst at UVSOR 1 Miho SHIMADA, 1,2 Masahiro KATOH, 1* Akira MOCHIHASHI, 1 Shin-ichi KIMURA, 2 Masahito HOSAKA, 2 Yoshifumi TAKASHIMA, 3 Toshiharu TAKAHASHI 1 UVSOR Institute

282 views β€’ 25 slides
Burst Buffer Simulation In Dragonfly Network Jian Peng, Michael Lang Illinois Institute of

Burst Buffer Simulation In Dragonfly Network Jian Peng, Michael Lang Illinois Institute of

Burst Buffer Simulation In Dragonfly Network Jian Peng, Michael Lang Illinois Institute of Technology, Los Alamos National Laboratory Purpose: Residing in the compute node network, and using solid state drives (SSD), burst buffers bring a

280 views β€’ 16 slides
Proposed MaxSec Merger www.fftsecurity.com Forward Looking Statements This presentation is given

Proposed MaxSec Merger www.fftsecurity.com Forward Looking Statements This presentation is given

Proposed MaxSec Merger www.fftsecurity.com Forward Looking Statements This presentation is given on behalf of Future Fibre Technologies Limited (FFT). Information in this presentation is for general information purposes only, and is not an offer

594 views β€’ 31 slides
Availability in Globally Distributed Storage Systems Robert Kozikowski Introduction Designing

Availability in Globally Distributed Storage Systems Robert Kozikowski Introduction Designing

Availability in Globally Distributed Storage Systems Robert Kozikowski Introduction Designing and optimizing the distributed systems for goals such as data availability relies on models of system behavior. This includes quantifying the impact

332 views β€’ 29 slides
Cloud-Integrated IP Design: Bursting EDA Workflows to the Public Cloud Jerome McFarland,

Cloud-Integrated IP Design: Bursting EDA Workflows to the Public Cloud Jerome McFarland,

Cloud-Integrated IP Design: Bursting EDA Workflows to the Public Cloud Jerome McFarland, Marketing Director, Elastifile Agenda Why Cloud? How Cloud? Real-World Example Why Cloud? IC Design Complexity is Steadily Increasing Process

604 views β€’ 31 slides
Introduction to dpdk-burst-replay tool By Jonathan Ribas 02/02/2019 Brussels TABLE OF CONTENT

Introduction to dpdk-burst-replay tool By Jonathan Ribas 02/02/2019 Brussels TABLE OF CONTENT

Introduction to dpdk-burst-replay tool By Jonathan Ribas 02/02/2019 Brussels TABLE OF CONTENT 01 04 TOOL OBJECTIVES INTERNAL DESIGN 02 05 WHAT CAN IT DO? FUTURE EVOLUTIONS 03 06 HOW TO USE IT QUESTIONS? Tool objectives 01 First,

249 views β€’ 23 slides
Architecture Decision Records in Action Michael Keeling Joe Runde IBM IBM @michaelkeeling

Architecture Decision Records in Action Michael Keeling Joe Runde IBM IBM @michaelkeeling

Architecture Decision Records in Action Michael Keeling Joe Runde IBM IBM @michaelkeeling @joerunde How do you share important design decisions? 2 3 Oral history is a great way to share design decisions while youre still exploring

818 views β€’ 54 slides

More recommend