FPGA Accelerated Seam Carving for Video A Design Overview B2: - - PowerPoint PPT Presentation

FPGA Accelerated Seam Carving for Video

A Design Overview

B2: Kimberly Lim, Eshani Mishra, Shruti Narayan

Application Area

Content-aware re-scaling intelligently targets parts of the frame to remove.

• Reduced video size (users often run out of space)
• Carve out unwanted pixels and save what’s important
• Draw attention to important aspects of video
• Highlight important aspects by removing unwanted seams
• Video processing is often slow
• FPGA for Acceleration

Application Area

Naive implementation of restitching of seam-carved images shown on left (spatial only). Static seam carving on right uses temporal and spatial so less distortion. Computational complexity becomes the bottleneck of the implementation of the algorithm. A hardware-oriented seam carving algorithm using FPGA is proposed to improve performance.

Overview of MVP

FPGA user input 5 seams at a time Camera input of 360x240 resolution video at 30 fps Monitor

Key

Block Diagram: Data Transfer through Hardware

Software

DE10-Standard FPGA

D8M-GPIO Camera Monitor

Hardware

ARM Processor FPGA Linux Cyclone V SDRAM SDRAM Hardware

M10K Embedded Memory Blocks Video Array Preprocessing and Video Array Formatting Script V i d e

• I

n p u t Video Array Seam Removal Script V i d e

• O

u t p u t Video Array Video Array Frame by Frame Load LE’s Seam to Remove Bought Borrowed Newly Designed Seam Carving Algorithm

Algorithm Overview

Stage 3 Stage 1 Stage 2

Memory Allocation in FPGA

Stage 3 Stage 1 Stage 2 5 5 7 M 1 K

Spatial Energy Map (75 blocks) Temporal Energy Map (150 blocks) Loading Frame (120 blocks) Processing Frame (120 blocks) 92 blocks left Energy Map (75 blocks) Accumulation Paths (75 blocks) 320 Accumulation Cell Copies (1 block each, 320 blocks) 87 blocks left 5 Accumulation Path Copies (75 blocks each, 375 total) 182 blocks left

Algorithm Implementation

Stage 3 Stage 1 Stage 2

Metrics and Validation

Risk Factors/Unknowns 03

• Remove less seams at a time (<5)
• Change blocking to optimize

parallelization

• Memory consumption + timing analysis

for test matrices using Quartus

Video Quality 02

• PSNR, Spatio-temporal SSIM
• User testing
• Goal: <10% error compared to results

from C++ implementation

Timing 01

• Compare against benchmark C++

implementation of seam carving

• Goal: 5x speedup

Benchmark Analysis (360x240 at 30 fps with 1.4 GHz Intel Core i5)

Time

• 10.646925

seconds

Seams

• 30 vertical

seams

• 132360837

cycles

Cycles

Work Distribution

High-level Algorithm Design

Quality Metrics, HPS to FPGA communication Software Benchmarking Hardware Implementation Design

Kimberly Eshani Shruti

Schedule

Ongoing and future tasks up until spring break. (Post spring break for slack time as well as adding planned extension steps)