Certifying Video Provenance Ashish Gehani I3P/SRI 1 INTRODUCTION - PowerPoint PPT Presentation

Certifying Video Provenance Ashish Gehani I3P/SRI 1

INTRODUCTION : Why certify? • Video cameras / editors ubiquitous • Data originates from multiple sources • Consumer trusts some producers • Must be able to validate provenance • Examples: – Surveillance camera, police matches, lawyer uses – Tank camera, classification edits, WMD analyst uses 2

INTRODUCTION : Why embed? • Auxiliary files: – Consistent copies – Synchronized access – Storage overhead – Loose coupling → Error prone • Embedding agnostic to: – Administrative domain – Transfer between hosts, operating systems • Video Embedding of Information for Lineage (VEIL) : – Inband encoding – Interoperable with legacy applications, libraries – Facilitate uptake 3

PROVENANCE : Desired properties • Sound – Signed elements • Necessary – Prune elements when possible • Complete – Don’t prune when output changes 4

PROVENANCE : Lineage tree Output Operation Input 1 Input n (a) Primitive operation (b) Compound operation tree 5

PROVENANCE : Granularity • ( Output, Executor, Input 1 , . . . , Input n ) • Level - Assembler, System call, File? • Fine → High overhead • Coarse → False positives • Can’t prune intra-process side-effects dynamically 6

PROVENANCE : Metadata format • Primitive operation format: Executor Signature Output Input Input n End 1 IP Address Inode Time • Executor: 32 bit IPv4 address, 32 bit user ID • Signature: 160 bits [ S = S IGN K E ( O, I 1 , . . . , I n ) ] • Input / Output File: – 32 bit IPv4 address – 32 bit inode – 32 bit time (Unix seconds count from 1 Jan, 1970) 7

PROVENANCE : Collection • Manually: > veil -o Output.mov -i Input 1.mov Input 2.mov Input 3.mov • Automatically: File 2 Read open() close() File 1 Read close() open() Process execution Time close() open() File 3 Write 8

RELATED WORK : • Provenance – Single host semantics • Steganography – Secrecy → Low capacity • Watermarking – Robustness → Low capacity 9

EMBEDDING : Interposition point Compression Residue Spatial to Entropy Frame Motion Quantization Calculation Signal Domain Coding Decomposition Estimation VEIL Interpolation VEIL Subpixel Analysis Decompression Entropy Signal to Block Frame Dequantization Decoding Spatial Domain Interpolation Reconstitution 10

EMBEDDING : Overview Primitive Operation Video Frame Metadata encoding operation �� �� �� �� �� �� Executor Signature Output Input Input n End 1 �� �� �� �� �� �� Executor Signature Output Input 1 Input n End �� �� �� �� �� �� Executor Signature Output Input 1 Input n End 11

EMBEDDING : Subpixel displacement • Human visual system tolerates subpixel shift: Outline of original frame Outline of block in interpolated frame Object in VEIL interpolated frame Object in original frame 12

EMBEDDING : Registration error compensation Camera Registration Grid Surface of object Points on surface mapped to first frame Points on surface mapped to second frame Points on surface mapped to third frame • v 1 , v 2 : Adjacent pixel intensities • v i : Interpolated pixel intensity • v i = v 1 + ( δx − δi )( v 2 − v 1 ) 13

EXTRACTION : Checking lineage Algorithm : C HECK L INEAGE ( D ) { E, S, O, I 1 , . . . , I n } ← G ET R OOT ( D ) O UTPUT ( E ) P E ← P KI L OOKUP ( E ) if I 1 , . . . , I n = {} then V ERIFY S IGNATURE ( P E , S, O ) 8 Result ← V ERIFY S IGNATURE ( P E , S, O | I 1 | . . . | I n ) > > > > if Result = T RUE > > > > < 8 else for i ← 1 to n < then > > do C HECK L INEAGE ( I i ) > : > > > > > else CheckFailed : 14

EXTRACTION : Interpolation estimation • Subpixel alignment minimizes visual distance: dx 15

EXTRACTION : Calculating subpixel displacement δx can be calculated in O ( β ) time using Theorem: O (log β ) space, where β is block size. Proof: x max y max MSE ( δx ) = 1 � � [ F ( x + δx, y ) − G ( x, y )] 2 β x = x min y = y min d d ( δx ) MSE ( δx ) = 0 δx = � x max � y max y = y min [ [ F ( x + 1 , y ) − F ( x, y )] [ F ( x, y ) − G ( x, y )] ] x = x min � x max � y max y = y min [ F ( x + 1 , y ) − F ( x, y )] 2 x = x min � 16

CONCLUSION : • – No auxiliary files to manage – No storage overhead – Legacy decoders work • – Fraction of data needed for lineage – All data needed to bind lineage to data – Legacy writes detectable – Real-time speed • – 21 writers, 4 inputs each, 8 x 8 blocks, 4 bits/block – Tree extraction: 0 . 06 seconds 17

More? 18

DESIGN : Why not watermark? • Security – Watermark protects producer – VEIL protects consumer • Robustness – Watermark is: ∗ Robust - survives aggressive distortion ∗ Fragile - localizes distortion – Consumer not adversary, constraints irrelevant • Visibility – Watermark introduces flickering, frame noise – VEIL avoids visual distraction 19

DESIGN : Why not watermark? [Continued] • Capacity – Video has heterogeneous signal, noise characteristics – Limited areas with: ∗ Low visual distortion ∗ High robustness ∗ High capacity – VEIL needs large capacity 20

PROVENANCE : Space requirement • Space requirement (in KB): Fan-in 1 2 3 4 Levels 2 0.09 0.14 0.19 0.24 3 0.14 0.34 0.65 1.05 4 0.19 0.75 2.02 4.30 5 0.24 1.56 6.13 17.30 21

EVALUATION : Extraction time • δx extraction in C, Mac OS 10.4, 2 GHz Intel • Time depends on block size: Block size Time to compute δx (in µ s) Overhead (in sec) for 1 sec of video ( 640 x 480 resolution, 30 fps) 4x4 1.3 0.723 8x8 5.2 0.746 16x16 18.2 0.648 32x32 72.2 0.649 64x64 292.1 0.657 22

EVALUATION : Embedding capacity • Tree - Fan-in: 4 , Levels: 4 , Storage: 35 , 264 bits • I frames: 5 % Bits encoded Block size Redundant copies ( ρ ) Video length needed (in sec) per block ( α ) ( β in bits) in 1 min video to reconstruct lineage tree 2 8x8 465 0.13 4 8x8 931 0.06 4 16x16 232 0.26 6 8x8 1396 0.04 23