7/26/2019 Data Ma Mana nagement for r Vide deo Ana nalyti tics Video data is everywhere. Brandon Haynes, Maureen Daum, Amrita Mazumdar, Magdalena Balazinsk ska, Luis Ceze, & Alvin Cheung 1 2 1 2 Width in pixels Height in pixels Existing systems treat video data like it’s the 20 th century 3 4 3 4 Many recent video applications require jointly querying multiple cameras, reasoning about position and orientation, or querying complex metadata. 5 6 5 6 1
7/26/2019 SELECT Metadata.Location FROM Cameras WHERE ‘James Bridle’ IN Metadata.Name 7 8 7 8 Modern Data Ma Mana nagement for r Vide deo Ana nalyti tics LightDB A Database System for Video Visual Road Virtual & Augmented Reality File System Metastore Video Applications A Database System for A Video Data Management Efficient querying of rich Optimized storage and Virtual & Augmented Reality Benchmark video content retrieval of video data Video Applications (Maureen Daum) 9 10 20 th century Imperative Code LightDB Query source = Scan(“kittens”) detection = source.Map( 𝑒𝑓𝑢𝑓𝑑𝑢 ) result = Union(source, detection) result.Save (“output.mp4”) How would you write this application? 12 11 12 2
7/26/2019 source = Scan(“kittens”) detection = source.Map( 𝑒𝑓𝑢𝑓𝑑𝑢 ) Encode result = Union(source, detection) Save HEVC result.Save (“output.mp4”) LightDB Query: source = Scan(“kittens”) Map Union (Grayscale) detection = source.Map( 𝑒𝑓𝑢𝑓𝑑𝑢 ) result = Union(source, detection) result.Save (“output.mp4”) Map Scan Scan Logical Plan 13 14 13 14 Object Overlay AR Application GPU GPU Overlay Overlay 45 GPUMap GPUMap GPUEncode GPUEncode 30 FPS GPU GPU GPUTee GPUTee 15 ToCPU ToCPU GPUQueued Homomorphic 0 Decode Union Timelapse Venice Coaster Cats SaveSingle ScanSingle (Light Field) File File Physical Plan LightDB Ffmpeg Scanner OpenCV 15 15 16 Modern Data Ma Mana nagement for r Vide deo Ana nalyti tics Key Features: • Data management system for VR & AR video applications • Unified data model for panoramic (360 ° ) and light field video • Declarative queries with automatic optimization Video Key Contributions: Visual Road File System Metastore • ∼ Τ 1 10 lines of code A Database System for A Video Data Management Efficient querying of rich Optimized storage and • Up to 4 × performance for real-world workloads Virtual & Augmented Reality Benchmark video content retrieval of video data • Reduced client bandwidth & power requirements Scan( “ LEGOS ” ) Video Applications (Maureen Daum) .Map( GRAYSCALE ) Brandon Haynes .Store( “ G RAYLEGOS ” ) bhaynes@cs.washington.edu lightdb.uwdb.io 17 18 3
7/26/2019 # Distinct videos performance teste stedby system Video System # Distinct Videos LightDB (2018) 4 Chameleon (2018) 5 The perfor ormance of video systems is evaluated like it’s still the 20 th century BlazeIt (2018) 6 NoScope (2017) 7 Focus (2018) 14 DeepLens (2019) ∼ 16 Scanner (2018) > 100 ( only 14 joined ) 19 20 19 20 Manually annotated Type Benchmarks Test Data with 1,210,000 hand- drawn boxes! Video (Visual Road) Ad Hoc Synthetic OLTP (TPC-H) Synthetic OLAP (SSB, DWEB) Synthetic Streaming (Linear Road, DTDW) Synthetic NoSQL (YCSB) Synthetic Graph (LDBC) Synthetic Privacy (SDV) Synthetic UA-DETRAC Dataset (2015) 10 hours, 24 Locations Video by Wen et al. 21 22 21 22 Video Data Management System Hyperparameters: scale, resolution, Execute Configure duration, seed Verify Synthetic Dataset Generate Video Benchmark Queries & Dataset Generator Verifier Q1 Q2 Q2b Q2c Q2d Q3 Q4 Q5 Q6a Q6b Q7 Q8 Q9 Q10 Pregenerated Datasets Commercial Gaming Autonomous Driving Engine Simulator 23 24 23 24 4
7/26/2019 Automatic ground truth generation 25 26 25 26 Q1 Q2(a) Q2(b) Query 7: Object Detection 2 2 12 1 1 6 0 0 0 Q2(c) Q2(d) Q3 72 16 16 ` 36 8 8 Traffic Camera 𝑊 𝑘 : Result for query instance 𝑗 : 0 0 0 Q4 Q5 Q6(a) 8 2 4 4 1 2 0 0 0 ⋮ 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 Scale Factor ( 4 × 𝑇𝑑𝑏𝑚𝑓 query instances) 27 28 27 28 Modern Data Ma Mana nagement for r Vide deo Ana nalyti tics visualroad.uwdb.io Key Features: • Video data management benchmark • Synthetic dataset generation • Unlimited scale, resolution, duration, and overlap Video • Extensible suite of computer vision, VR, and microbenchmarks Visual Road File System Metastore • Applicable to both general and specialized video DBMSs A Database System for A Video Data Management Efficient querying of rich Optimized storage and Virtual & Augmented Reality Benchmark video content retrieval of video data Video Applications (Maureen Daum) 29 29 30 5
7/26/2019 Modern Data Ma Mana nagement for r Vide deo Ana nalyti tics lightdb.uwdb.io visualroad.uwdb.io Key Features: Key Features: • Video data management benchmark • DBMS for VR & AR video applications • Synthetic dataset generation • Unified data model • Unlimited scale, resolution, duration • Declarative queries • Extensible suite of computer vision, VR, and Video • Automatic optimization Visual Road microbenchmark queries File System Metastore A Database System for A Video Data Management Efficient querying of rich Optimized storage and Virtual & Augmented Reality Benchmark video content retrieval of video data Video Applications (Maureen Daum) 31 31 32 Motivation • We want to enable rich, content-based queries over video data Towards ds Efficient t Querying of • Existing systems optimize running object detection over videos • As a result, they focus on simple queries only Rich h Vide deo Con ontent • We want to use this metadata to enable more complex queries Maureen Daum 33 34 SELECT pixels FROM video WHERE dog Metadata • Object labels • Weather conditions • Descriptions • License plate numbers • Aggregates 35 36 35 36 6
7/26/2019 Overlay the dog pixels from video 1 onto the background in video 2 Select sequences of frames that contain increasing numbers of cats 37 38 37 38 Metadata Model Executing Queries • Videos are stored in a compressed format • Encoding and decoding are expensive operators t Perc rcent nt of Que uery ry Time Spe pent in n Enc ncod ode or or Decode de Label Volume Draw boxes over people Dog 𝑢 0 ,𝑢 1 , 𝑦 0 , 𝑦 1 , [𝑧 0 ,𝑧 1 ] y Select frames with people Select frames with buses x Select frames with bicycles 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Encode and decode 39 40 39 40 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog This Photo by Unknown Author is licensed under CC BY-SA 41 This Photo by Unknown Author is licensed under CC BY-SA 42 41 42 7
7/26/2019 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog 43 44 This Photo by Unknown Author is licensed under CC BY-SA This Photo by Unknown Author is licensed under CC BY-SA 43 44 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog This Photo by Unknown Author is licensed under CC BY-SA 45 This Photo by Unknown Author is licensed under CC BY-SA 46 45 46 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog This Photo by Unknown Author is licensed under CC BY-SA 47 This Photo by Unknown Author is licensed under CC BY-SA 48 47 48 8
7/26/2019 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog 49 50 This Photo by Unknown Author is licensed under CC BY-SA This Photo by Unknown Author is licensed under CC BY-SA 49 50 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog Random access point Random access point Encoded as a delta Encoded as a delta This Photo by Unknown Author is licensed under CC BY-SA 51 This Photo by Unknown Author is licensed under CC BY-SA 52 51 52 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog Random access point Random access point Encoded as a delta Encoded as a delta This Photo by Unknown Author is licensed under CC BY-SA 53 This Photo by Unknown Author is licensed under CC BY-SA 54 53 54 9
7/26/2019 SELECT frames FROM video WHERE dog SELECT frames FROM video WHERE dog Random access point Random access point Encoded as a delta Encoded as a delta 55 56 This Photo by Unknown Author is licensed under CC BY-SA This Photo by Unknown Author is licensed under CC BY-SA 55 56 SELECT pixels FROM video WHERE dog SELECT pixels FROM video WHERE dog 57 58 57 58 SELECT pixels FROM video WHERE dog SELECT pixels FROM video WHERE dog 59 60 59 60 10
Recommend
More recommend
