shifting media app development into high gear
play

Shifting Media app development into high gear Using virtual drivers - PowerPoint PPT Presentation

Dec 11, 2022 •259 likes •756 views

Shifting Media app development into high gear Using virtual drivers to speed up development Helen Koike helen.koike@collabora.com Open First Creative Commons Attribution-ShareAlike 4.0 Summary Classic V4L2 API Vivid Driver Media

  1. Shifting Media app development into high gear Using virtual drivers to speed up development Helen Koike helen.koike@collabora.com Open First Creative Commons Attribution-ShareAlike 4.0

  2. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 2

  3. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 3

  4. ELC 2018 4

  5. Sepia Sensor Composer Scaler Greyscale ELC 2018 5

  6. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 6

  7. APP Read/Write/MMAP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Video stream Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 7

  8. APP Read/Write/MMAP User IOCTLs space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Standard Custom Video stream General confgs Controls Controls * Contrast * DRV DEF 1 * Img fmt Kernel * Brightness * DRV DEF 2 * Bufers space * Gamma * DRV DEF 3 * Video std . . * Frame rate . . . . . . ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 8

  9. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 9

  10. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 10

  11. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- The Virtual Video T est Driver (VIVID) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 11

  12. APP User space -------------------------------------------------------- /dev/video* -------------------------------------------------------- The Virtual Video T est Driver (VIVID) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 12

  13. Vivid driver ELC 2018 13

  14. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 14

  15. Media API Motivation ● Embedded systems: great variety of devices ● Hardware complexity ● Similar confguration for diferent entities ELC 2018 15

  16. contrast Sepia Sensor Composer Scaler Greyscale contrast ELC 2018 16

  17. APP Read/Write/MMAP User IOCTLs space -------------------------------------------------------- /dev/video* -------------------------------------------------------- Driver Standard Custom Video stream General confgs Controls Controls * Contrast * DRV DEF 1 * Img fmt Kernel * Brightness * DRV DEF 2 * Bufers space * Gamma * DRV DEF 3 * Video std . . * Frame rate . . . . . . ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 17

  18. Sepia Sensor Composer Scaler Greyscale ELC 2018 18

  19. Sepia Sensor Composer Scaler Greyscale ELC 2018 19

  20. /dev/video0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 20

  21. APP Subdev API contrast IOCTLs Sepia /dev/v4l- subdev0 APP APP R/W/MMAP IOCTLs Scaler IOCTLs /dev/v4l- /dev/video0 subdev2 APP contrast IOCTLs Greyscale /dev/v4l- subdev1 ELC 2018 21

  22. What is part of the device? What is the topology? APP ? ? ? ? ? ? ? ? ? ? ? ? /dev/v4l-subdev0 /dev/v4l-subdev98 /dev/v4l-subdev11 /dev/v4l-subdev14 /dev/video2 /dev/video0 /dev/v4l-subdev6 /dev/video42 /dev/v4l-subdev21 /dev/v4l-subdev31 ELC 2018 22

  23. /dev/video0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 23

  24. /dev/video0 out Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 24

  25. APP Media API contrast IOCTLs Sepia /dev/v4l- subdev0 APP APP APP R/W/MMAP IOCTLs IOCTLs Scaler IOCTLs /dev/media0 /dev/v4l- /dev/video0 subdev2 APP contrast IOCTLs Greyscale /dev/v4l- subdev1 ELC 2018 25

  26. APP /dev/video0 IOCTLs out /dev/media0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 26

  27. APP /dev/video0 IOCTLs out /dev/media0 Sepia /dev/v4l-subdev0 Sensor Composer Scaler /dev/v4l-subdev2 /dev/v4l-subdev1 Greyscale ELC 2018 27

  28. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 28

  29. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- Kernel Driver space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 29

  30. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- The Virtual Media Controller Driver (VIMC) Kernel space ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 30

  31. APP User space -------------------------------------------------------- /dev/media* -------------------------------------------------------- The Virtual Media Controller Driver (VIMC) Kernel space ? ---------------------------------------------------------------------------------------------------------------------------------- Physical device ELC 2018 31

  32. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 32

  33. APP Developer User space --------------- VIMC Confgfs API --------------------- /dev/media0 --------------------- /dev/media1 ----------- Kernel create space create ELC 2018 33

  34. Vimc: submodules Subdev1.ko Implementation of subdevice Subdev2.ko drivers are modularized and doesn’t need to alter Vimc’s core code. SubdevN.ko ELC 2018 34

  35. vimc-sensor.ko vimc-scaler.ko APP Developer Standard vimc-debayer.ko vimc-capture.ko ... Emulate diferent topologies to test the app in several scenarios custom-sd1.ko custom-sd2-ko Manufacturer custom-sd3.ko ... ELC 2018 35

  36. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 36

  37. Vimc Driver ● Proposed by Laurent Pinchart for Outreachy in 2015 ● Merged in Kernel 4.12 ● Moving slowly ELC 2018 37

  38. Vimc Driver: current state ● Basic standard submodules: – vimc-capture.ko – vimc-sensor.ko – vimc-debayer.ko – vimc-scaler.ko ● Hard-coded topology (re-compilation required) ELC 2018 38

  39. ELC 2018 39

  40. Vimc Driver ELC 2018 40

  41. Userspace tools ● v4l-utils media-ctl -d /dev/media0 --print-dot > /tmp/out.dot && xdot /tmp/out.dot media-ctl -d /dev/media0 -V " 'Sensor A':0 [ fmt: RGB888_1X24/600x600]" media-ctl -v -d /dev/media0 --links "'Debayer A':1 -> 'Scaler':0 [0] " ● Yavta (Yet Another V4L2 T est Application) yavta --format RGB24 --size 600x600 /dev/video0 ELC 2018 41

  42. Summary ● Classic V4L2 API → Vivid Driver ● Media API (extension) → Vimc Driver ● Vimc: ● Submodules ● Current state ● Confgfs API ● Future work ELC 2018 42

  43. Confgfs API: Topology ● Entities ‘Sensor A’ vimc-sensor.ko – Name 0 – Submodule ● Pads 0 ‘Debayer A’ vimc-debayer.ko – Source 1 – Sink ● Links 0 ‘Raw Capture 0’ vimc-capture.ko ELC 2018 43

  44. mkdir “MEDIA_NAME” ELC 2018 44

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

East Texas GEAR UP Advanced Placement, Concurrent Credit, and Dual Credit GEAR UPs parent

East Texas GEAR UP Advanced Placement, Concurrent Credit, and Dual Credit GEAR UPs parent

East Texas GEAR UP Advanced Placement, Concurrent Credit, and Dual Credit GEAR UPs parent messages for this quarter include: 1) earning college credit while still in high school and 2) financial literacy terms concerning postsecondary

510 views • 4 slides
EP TRAN Com m ittee: 1 2 April 2 0 1 8 GEAR 2 0 3 0 Gw enole Cozigou, Director DG GROW I

EP TRAN Com m ittee: 1 2 April 2 0 1 8 GEAR 2 0 3 0 Gw enole Cozigou, Director DG GROW I

EP TRAN Com m ittee: 1 2 April 2 0 1 8 GEAR 2 0 3 0 Gw enole Cozigou, Director DG GROW I ndustrial Transform ation & Advanced Value Chains Internal market, Industry, Entrepreneurship and SMEs GEAR 2 0 3 0 High Level Group

798 views • 14 slides
landscape Bangkok, 25 January 2016 Why look at media development? Potential of media to

landscape Bangkok, 25 January 2016 Why look at media development? Potential of media to

UNESCOs Media Development Indicators (MDIs) Tool for assessing national media landscape Bangkok, 25 January 2016 Why look at media development? Potential of media to foster good governance: transparency, accountability, dialogue

509 views • 11 slides
Getting Your Community In Gear Bike-Based Economic Development Insights for Non-Cyclists

Getting Your Community In Gear Bike-Based Economic Development Insights for Non-Cyclists

Getting Your Community In Gear Bike-Based Economic Development Insights for Non-Cyclists Samantha Pearson Walk It! Bike It! Lewisburg + Lewisburg Neighborhoods Corporation Getting Your Community In Gear Bike-Based Economic Development

934 views • 78 slides
Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, T emple University John

Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, T emple University John

Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, T emple University John Harris: GEAR UP Lead Coach, Benjamin Franklin High School Your Opinions and Feelings For Everyone: How do you feel when you need to talk to a

328 views • 14 slides
FAQs Wednesday (4/15) is the GEAR Session IV presentation Discussion will be available on

FAQs Wednesday (4/15) is the GEAR Session IV presentation Discussion will be available on

CS535 Big Data 4/13/2020 Week 12-A Sangmi Lee Pallickara CS535 Big Data | Computer Science | Colorado State University CS535 BIG DATA PART B. GEAR SESSIONS SESSION 4: LARGE SCALE RECOMMENDATION SYSTEMS AND SOCIAL MEDIA Sangmi Lee Pallickara

565 views • 32 slides
Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, Temple University John

Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, Temple University John

Student Leadership and Involvement Rachel Howe: GEAR UP Coordinator, Temple University John Harris: Temple University undergraduate, GEAR UP Lead Coach, Benjamin Franklin High School Your Opinions and Feelings How do you feel when apprentice

362 views • 15 slides
In-Line Non-Contact Gear Metrology Key Features 100% In-line Inspection No Fail Forward High

In-Line Non-Contact Gear Metrology Key Features 100% In-line Inspection No Fail Forward High

In-Line Non-Contact Gear Metrology Key Features 100% In-line Inspection No Fail Forward High Accuracy Metrology Full Surface Non-Contact Analysis Robust Quick Tooling Change 25 Second Tact Time Capable Typical Applications Pinion Gears

393 views • 8 slides
YOUR DIGITAL MARKETING KEY SOCIAL MEDIA | SEO | WEB DEVELOPMENT www.digitalchaabi.com Mob :

YOUR DIGITAL MARKETING KEY SOCIAL MEDIA | SEO | WEB DEVELOPMENT www.digitalchaabi.com Mob :

YOUR DIGITAL MARKETING KEY SOCIAL MEDIA | SEO | WEB DEVELOPMENT www.digitalchaabi.com Mob : 8012300095 ABOUT DIGITAL CHAABI NO FANCY PROMISES NO HIGH WORDS JUST REAL EFFORT 01 Our team consists of hand-picked members, who are dedicated to your

332 views • 14 slides
Development and Testing of Line Free Fishing Gear to Reduce Entanglement of North Atlantic Right

Development and Testing of Line Free Fishing Gear to Reduce Entanglement of North Atlantic Right

Development and Testing of Line Free Fishing Gear to Reduce Entanglement of North Atlantic Right Whales Richard P. Riels November 6, 2018 New Bedford Whaling Museum Outline Of Presentation Who we are What we have developed:

168 views • 12 slides
Piper Arrow IV Landing Gear System Opera8on Overview St.

Piper Arrow IV Landing Gear System Opera8on Overview St.

Piper Arrow IV Landing Gear System Opera8on Overview St. Louis Flying Club 2/9/16 Landing Gear System Switches Gear Up/Down WARN Switch and

3.4k views • 14 slides
Social Media Seminar for Development Educators Part 1: Social Media Basics How are these

Social Media Seminar for Development Educators Part 1: Social Media Basics How are these

Social Media Seminar for Development Educators Part 1: Social Media Basics How are these websites being used for social change? http://www.slideshare.net/mzkagan/what-the-fk-social-media On Road Media Is an award-winning social enterprise

1.07k views • 75 slides
Shifting Paradigms, Positive Behavioral Supports and Restorative Practices in an Urban High

Shifting Paradigms, Positive Behavioral Supports and Restorative Practices in an Urban High

Shifting Paradigms, Positive Behavioral Supports and Restorative Practices in an Urban High School in the Midwest Jan Petersen Restorative Practices Coordinator jpetersen@usd259.net Rob Simon Restorative Practices Advocate robsimonxi@aol.com

986 views • 32 slides
Shifting to a Standards- Based Mindset LMS Department Everett High School September 3, 2015

Shifting to a Standards- Based Mindset LMS Department Everett High School September 3, 2015

Shifting to a Standards- Based Mindset LMS Department Everett High School September 3, 2015 Todays Focus Build consistency and coherence as teams around formative and summative assessment The Standards Based Mindset Shifts: polices

513 views • 24 slides
An alternative approach to Synthetic Wind Turbine Gear Oil formulation. Phil Hutchinson. Evonik

An alternative approach to Synthetic Wind Turbine Gear Oil formulation. Phil Hutchinson. Evonik

An alternative approach to Synthetic Wind Turbine Gear Oil formulation. Phil Hutchinson. Evonik Oil Additives, Singapore. 1 Synthetic Industrial Gear-Oils / Wind Turbine Gear-oils (WTGO) Significant growth in the use of Synthetic Gear-

488 views • 13 slides
Chapter 06 A Shifting Mosaic: Rain Forest Development and Dynamics PLATE 6-1 Landslides such as

Chapter 06 A Shifting Mosaic: Rain Forest Development and Dynamics PLATE 6-1 Landslides such as

Chapter 06 A Shifting Mosaic: Rain Forest Development and Dynamics PLATE 6-1 Landslides such as this along a forested area on an Ecuadorian mountainside sometimes result in soil removal, and this results in slow primary succession. If soil

694 views • 32 slides
Introduction to Design and Photoshop Radhika Bhatt 4/19/2016 About the Instructor Front

Introduction to Design and Photoshop Radhika Bhatt 4/19/2016 About the Instructor Front

Introduction to Design and Photoshop Radhika Bhatt 4/19/2016 About the Instructor Front End Engineer, Commerce Data Service Skills include graphic design, user experience design, branding, prototyping and wireframing, front end

985 views • 52 slides
University of Illinois College of Medicine Peoria Class of 2020 Residency Match Results

University of Illinois College of Medicine Peoria Class of 2020 Residency Match Results

University of Illinois College of Medicine Peoria Class of 2020 Residency Match Results Zachariah A. Pooja A. David A. Oregon Health & Science Rush University Medical University of Kansas Center Illinois University School of

469 views • 15 slides
Concurrent Masters in Genomic Medicine for Medical Students 4 years to MD/MS Jeffery M.

Concurrent Masters in Genomic Medicine for Medical Students 4 years to MD/MS Jeffery M.

Concurrent Masters in Genomic Medicine for Medical Students 4 years to MD/MS Jeffery M. Vance, PhD, MD Professor, Dr. John T. Macdonald Foundation Department of Human Genetics Professor of Neurology John P . Hussman Institute for Human

390 views • 12 slides
Kevin ORourke Head of Low Carbon Technologies Sustainable Energy Authority of Ireland SEAI

Kevin ORourke Head of Low Carbon Technologies Sustainable Energy Authority of Ireland SEAI

Global Building Performance Network Webinar: Getting the Building Codes right: The importance of long-term energy targets and frequent revision cycles Wednesday 13 th November 2013 The Ireland experience drivers, experiences, achievements

727 views • 28 slides
Statistical Design Consideration Jessica Riley, Shells Artwork from Reflections Art in Health

Statistical Design Consideration Jessica Riley, Shells Artwork from Reflections Art in Health

Developing Similarity Criteria of Dissolution Profiles through the Weibull Model and a Statistical Design Consideration Jessica Riley, Shells Artwork from Reflections Art in Health Stan Altan, Hans Coppenolle Wim Van der Elst Manufacturing and

786 views • 41 slides
Mul$media Event Detec$on and Recoun$ng Task Reports Time ime Pres

Mul$media Event Detec$on and Recoun$ng Task Reports Time ime Pres

Mul$media Event Detec$on and Recoun$ng Task Reports Time ime Pres esent entation ion University of Amsterdam (MediaMill) 9:30 9:50 TNO (TNO) 9:50 10:10 Access to

615 views • 35 slides
Social media and news How social media is affecting the business and practice of journalism

Social media and news How social media is affecting the business and practice of journalism

Social media and news How social media is affecting the business and practice of journalism Drivers for change Journalistic examples Challenges and research questions @nicnewman April 2012 Journalist with radio, TV and online 10

749 views • 25 slides
Multi-Media Tools That Make Sharing Info with Students a Slam Dunk PRESENTERS: KALYN RYLL,

Multi-Media Tools That Make Sharing Info with Students a Slam Dunk PRESENTERS: KALYN RYLL,

Multi-Media Tools That Make Sharing Info with Students a Slam Dunk PRESENTERS: KALYN RYLL, KALYN.RYLL@SIMMONS.EDU CINDY FIRESTEIN, CINDY.FIRESTEIN@SIMMONS.EDU Goals for Our Presentation Best Practices for Sharing Information with

619 views • 33 slides

More recommend