OpenBR Open Source Biometric Recognition and Beyond Josh Klontz - - PowerPoint PPT Presentation

▶
openbr open source biometric recognition
SMART_READER_LITE
LIVE PREVIEW

OpenBR Open Source Biometric Recognition and Beyond Josh Klontz - - PowerPoint PPT Presentation

Apr 28, 2023 114 likes •637 views

OpenBR Open Source Biometric Recognition and Beyond Josh Klontz www.openbiometrics.org February 17, 2013 J. Klontz OpenBR February 17, 2013 1 / 18 Why Open Source? J. Klontz OpenBR February 17, 2013 2 / 18 Why Open Source?

slide-1
SLIDE 1

OpenBR – Open Source Biometric Recognition

and Beyond Josh Klontz

www.openbiometrics.org

February 17, 2013

  • J. Klontz

OpenBR February 17, 2013 1 / 18

slide-2
SLIDE 2

Why Open Source?

  • J. Klontz

OpenBR February 17, 2013 2 / 18

slide-3
SLIDE 3

Why Open Source?

Reproducible Research

Support a common set of file formats and tools for algorithm design, development, and evaluation.

  • J. Klontz

OpenBR February 17, 2013 2 / 18

slide-4
SLIDE 4

Why Open Source?

Reproducible Research

Support a common set of file formats and tools for algorithm design, development, and evaluation.

Decrease Time to Market

Provide a well-engineered and consistent framework for deploying new algorithms.

  • J. Klontz

OpenBR February 17, 2013 2 / 18

slide-5
SLIDE 5

Why Open Source?

Reproducible Research

Support a common set of file formats and tools for algorithm design, development, and evaluation.

Decrease Time to Market

Provide a well-engineered and consistent framework for deploying new algorithms.

Reduce Duplication

Supply state-of-the-art baseline components for algorithm design.

  • J. Klontz

OpenBR February 17, 2013 2 / 18

slide-6
SLIDE 6

Why Open Source?

Reproducible Research

Support a common set of file formats and tools for algorithm design, development, and evaluation.

Decrease Time to Market

Provide a well-engineered and consistent framework for deploying new algorithms.

Reduce Duplication

Supply state-of-the-art baseline components for algorithm design.

Improve Collaboration

Help foster a community where collaboration takes place at the source code level.

  • J. Klontz

OpenBR February 17, 2013 2 / 18

slide-7
SLIDE 7

What’s in it?

Off-the-shelf algorithms

Face Recognition Gender Classification Age Estimation Commercial Wrappers

  • J. Klontz

OpenBR February 17, 2013 3 / 18

slide-8
SLIDE 8

What’s in it?

Off-the-shelf algorithms

Face Recognition Gender Classification Age Estimation Commercial Wrappers

Tools for algorithm evaluation

Standardized set of file formats Automatic plot generation Command line interface supporting common biometrics tasks

  • J. Klontz

OpenBR February 17, 2013 3 / 18

slide-9
SLIDE 9

What’s in it?

Off-the-shelf algorithms

Face Recognition Gender Classification Age Estimation Commercial Wrappers

Tools for algorithm evaluation

Standardized set of file formats Automatic plot generation Command line interface supporting common biometrics tasks

Software framework for algorithm development

C++ plugin API for implementing new algorithms Grammar for image processing Automatic testing, packaging and deployment

  • J. Klontz

OpenBR February 17, 2013 3 / 18

slide-10
SLIDE 10

Software Architecture

Qt

Cross-platform application and UI framework

OpenCV

Image processing library

Eigen

Linear algebra library

CMake

Cross-platform build system

  • J. Klontz

OpenBR February 17, 2013 4 / 18

slide-11
SLIDE 11

Software Architecture

Qt

Cross-platform application and UI framework

OpenCV

Image processing library

Eigen

Linear algebra library

CMake

Cross-platform build system

br

Command line application for running algorithms and evaluating results.

C API

High-level interface for other programming languages.

C++ Plugin API

Core interface for using and developing algorithms.

  • J. Klontz

OpenBR February 17, 2013 4 / 18

slide-12
SLIDE 12

Supported Platforms

Now

  • J. Klontz

OpenBR February 17, 2013 5 / 18

slide-13
SLIDE 13

Supported Platforms

Now Soon

  • J. Klontz

OpenBR February 17, 2013 5 / 18

slide-14
SLIDE 14

Supported Platforms

Now Soon Future

  • J. Klontz

OpenBR February 17, 2013 5 / 18

slide-15
SLIDE 15

Algorithm Evaluation

40% 60% 80% 0.001 0.100 False Accept Rate True Accept Rate Algorithm COTS OpenBR COTS OpenBR 250 500 750 −1 1 2 3 Score Frequency / Algorithm Ground Truth Genuine Impostor

0.618 0.786 0.76 0.858

0.001 0.010 0.00 0.25 0.50 0.75 COTS OpenBR COTS OpenBR Algorithm / False Accept Rate True Accept Rate

Figure: OpenBR vs COTS face recognition on MEDS mugshot database.

  • J. Klontz

OpenBR February 17, 2013 6 / 18

slide-16
SLIDE 16

Algorithm Evaluation

40% 60% 80% 0.001 0.100 False Accept Rate True Accept Rate Algorithm COTS OpenBR COTS OpenBR 250 500 750 −1 1 2 3 Score Frequency / Algorithm Ground Truth Genuine Impostor

0.618 0.786 0.76 0.858

0.001 0.010 0.00 0.25 0.50 0.75 COTS OpenBR COTS OpenBR Algorithm / False Accept Rate True Accept Rate

Figure: OpenBR vs COTS face recognition on MEDS mugshot database.

OpenBR COTS-A COTS-B COTS-C COTS-D TAR @ FAR = 0.01 0.77 0.93 0.96 0.86 0.80 Template Size (kB) 0.75 2.8 5.0 36 74 Enrollment Speed 10 N/A N/A 1.3 1.2 Comparison Speed 3,800,000 N/A 110,000 19,000 2,000

  • J. Klontz

OpenBR February 17, 2013 6 / 18

slide-17
SLIDE 17

FRVT 2012 (OpenBR = ’K’)

  • J. Klontz

OpenBR February 17, 2013 7 / 18

slide-18
SLIDE 18

FRVT 2012 (OpenBR = ’K’)

  • J. Klontz

OpenBR February 17, 2013 8 / 18

slide-19
SLIDE 19

Algorithm Example: Face Recognition

$ br -algorithm FaceRecognition -compare me.jpg you.jpg

  • J. Klontz

OpenBR February 17, 2013 9 / 18

slide-20
SLIDE 20

Algorithm Example: Face Recognition

$ br -algorithm FaceRecognition -compare me.jpg you.jpg

FaceRecognition

FaceDetection!<FaceRegistration>!<FaceExtraction>+ <FaceEmbedding>+<FaceQuantization>:UCharL1

  • J. Klontz

OpenBR February 17, 2013 9 / 18

slide-21
SLIDE 21

Algorithm Example: Face Recognition

$ br -algorithm FaceRecognition -compare me.jpg you.jpg

FaceRecognition

FaceDetection!<FaceRegistration>!<FaceExtraction>+ <FaceEmbedding>+<FaceQuantization>:UCharL1

FaceDetection

Open+Cvt(Gray)+Cascade(FrontalFace)

  • J. Klontz

OpenBR February 17, 2013 9 / 18

slide-22
SLIDE 22

Algorithm Example: Face Recognition

$ br -algorithm FaceRecognition -compare me.jpg you.jpg

FaceRecognition

FaceDetection!<FaceRegistration>!<FaceExtraction>+ <FaceEmbedding>+<FaceQuantization>:UCharL1

FaceDetection

Open+Cvt(Gray)+Cascade(FrontalFace)

FaceRegistration

ASEFEyes+Affine(88,88,0.25,0.35)+FTE(DFFS)

  • J. Klontz

OpenBR February 17, 2013 9 / 18

slide-23
SLIDE 23

Algorithm Example: Face Recognition

$ br -algorithm FaceRecognition -compare me.jpg you.jpg

FaceRecognition

FaceDetection!<FaceRegistration>!<FaceExtraction>+ <FaceEmbedding>+<FaceQuantization>:UCharL1

FaceDetection

Open+Cvt(Gray)+Cascade(FrontalFace)

FaceRegistration

ASEFEyes+Affine(88,88,0.25,0.35)+FTE(DFFS) ...

FaceEmbedding

Dup(12)+RndSubspace(0.05,1)+LDA(0.98)+Cat+PCA(768)

  • J. Klontz

OpenBR February 17, 2013 9 / 18

slide-24
SLIDE 24

Live Coding

  • J. Klontz

OpenBR February 17, 2013 10 / 18

slide-25
SLIDE 25

Live Coding

Inventing on Principle

http://www.youtube.com/watch?v=PUv66718DII

  • J. Klontz

OpenBR February 17, 2013 10 / 18

slide-26
SLIDE 26

CPU Scaling

Figure: http://www.extremetech.com/computing/116561-the-death-of-cpu-scaling-from-one-

core-to-many-and-why-were-still-stuck

  • J. Klontz

OpenBR February 17, 2013 11 / 18

slide-27
SLIDE 27

Evolution of Hardware and Software

Figure: http://www.extremetech.com/computing/116561-the-death-of-cpu-scaling-from-one-

core-to-many-and-why-were-still-stuck

  • J. Klontz

OpenBR February 17, 2013 12 / 18

slide-28
SLIDE 28

Hardware Realities

Figure: i7 3930k Figure: GTX 680 Figure: Xeon Phi 5110p

  • J. Klontz

OpenBR February 17, 2013 13 / 18

slide-29
SLIDE 29

Hardware Realities

Figure: i7 3930k Figure: GTX 680 Figure: Xeon Phi 5110p

$570.00 $568.50 $2,649

  • J. Klontz

OpenBR February 17, 2013 13 / 18

slide-30
SLIDE 30

Hardware Realities

Figure: i7 3930k Figure: GTX 680 Figure: Xeon Phi 5110p

$570.00 $568.50 $2,649 76.8 GFLOPS 1665 GFLOPS 1011 GFLOPS

  • J. Klontz

OpenBR February 17, 2013 13 / 18

slide-31
SLIDE 31

Hardware Realities

Figure: i7 3930k Figure: GTX 680 Figure: Xeon Phi 5110p

$570.00 $568.50 $2,649 76.8 GFLOPS 1665 GFLOPS 1011 GFLOPS

Gotcha: Memory Bandwidth

12.8 GFLOPS 48.0 GFLOPS 80 GFLOPS

  • J. Klontz

OpenBR February 17, 2013 13 / 18

slide-32
SLIDE 32

Hardware Realities

Figure: i7 3930k Figure: GTX 680 Figure: Xeon Phi 5110p

$570.00 $568.50 $2,649 76.8 GFLOPS 1665 GFLOPS 1011 GFLOPS

Gotcha: Memory Bandwidth

12.8 GFLOPS 48.0 GFLOPS 80 GFLOPS

Gotcha: Code Duplication

Need a separate code base for optimized performance on each device!

  • J. Klontz

OpenBR February 17, 2013 13 / 18

slide-33
SLIDE 33

Requirements

What we want

Write once and run everywhere Automatically utilize all available hardware Run faster on future hardware

  • J. Klontz

OpenBR February 17, 2013 14 / 18

slide-34
SLIDE 34

Requirements

What we want

Write once and run everywhere Automatically utilize all available hardware Run faster on future hardware

What we need

Virtual machine or just-in-time compiler Express computations using induction variables or “kernels”: void example kernel(int *a, int *b, int i) { a[i] += b[i]; }

  • J. Klontz

OpenBR February 17, 2013 14 / 18

slide-35
SLIDE 35

Requirements

What we want

Write once and run everywhere Automatically utilize all available hardware Run faster on future hardware

What we need

Virtual machine or just-in-time compiler Express computations using induction variables or “kernels”: void example kernel(int *a, int *b, int i) { a[i] += b[i]; }

What we’re proposing

LLVM IR and JIT compiler Designing for OpenCL 2.0 standard

  • J. Klontz

OpenBR February 17, 2013 14 / 18

slide-36
SLIDE 36

Goals

Perfectly Composable Image Processing Primitives

A grammar for building algorithms from orthogonal primitive kernels with typeless semantics and optimized execution.

  • J. Klontz

OpenBR February 17, 2013 15 / 18

slide-37
SLIDE 37

Goals

Perfectly Composable Image Processing Primitives

A grammar for building algorithms from orthogonal primitive kernels with typeless semantics and optimized execution.

When we say...

Transform *lbpu2 = Transform::make("LBP(1)+U2");

  • J. Klontz

OpenBR February 17, 2013 15 / 18

slide-38
SLIDE 38

Goals

Perfectly Composable Image Processing Primitives

A grammar for building algorithms from orthogonal primitive kernels with typeless semantics and optimized execution.

When we say...

Transform *lbpu2 = Transform::make("LBP(1)+U2");

...we mean

Give me a pointer to a function that computes LBPu2

8,1 on an image,

minimizes main memory transactions by combining kernels, and is

  • ptimized for parallel execution on the hardware available.
  • J. Klontz

OpenBR February 17, 2013 15 / 18

slide-39
SLIDE 39

Goals

Perfectly Composable Image Processing Primitives

A grammar for building algorithms from orthogonal primitive kernels with typeless semantics and optimized execution.

When we say...

Transform *lbpu2 = Transform::make("LBP(1)+U2");

...we mean

Give me a pointer to a function that computes LBPu2

8,1 on an image,

minimizes main memory transactions by combining kernels, and is

  • ptimized for parallel execution on the hardware available.

Take-Home Message

Compilation = Source Code + Available Hardware + First Image

  • J. Klontz

OpenBR February 17, 2013 15 / 18

slide-40
SLIDE 40

The End

Slides

  • penbiometerics.org/slides.pdf

Source

github.com/biometrics/openbr

E-mail

  • penbr-dev@googlegroups.com

Thank You!

  • J. Klontz

OpenBR February 17, 2013 16 / 18

slide-41
SLIDE 41

Welcome to the Parallel Jungle!

Figure: http://www.drdobbs.com/parallel/welcome-to-the-parallel-jungle/232400273

  • J. Klontz

OpenBR February 17, 2013 17 / 18

slide-42
SLIDE 42

Plugin Example: Local Binary Patterns

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-43
SLIDE 43

Plugin Example: Local Binary Patterns

#include <openbr plugin.h>

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-44
SLIDE 44

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform {

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-45
SLIDE 45

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1)

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-46
SLIDE 46

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const {

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-47
SLIDE 47

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const { for (int r=radius; r<src.rows-radius; r++) for (int c=radius; c<src.cols-radius; c++) { float cval = p[r*src.cols+c]; dst(r, c) = (p[(r-radius)*src.cols+c-radius] >= cval ? 128 : 0) | (p[(r-radius)*src.cols+c] >= cval ? 64 : 0) | ...; }

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-48
SLIDE 48

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const { for (int r=radius; r<src.rows-radius; r++) for (int c=radius; c<src.cols-radius; c++) { float cval = p[r*src.cols+c]; dst(r, c) = (p[(r-radius)*src.cols+c-radius] >= cval ? 128 : 0) | (p[(r-radius)*src.cols+c] >= cval ? 64 : 0) | ...; } } };

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-49
SLIDE 49

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const { for (int r=radius; r<src.rows-radius; r++) for (int c=radius; c<src.cols-radius; c++) { float cval = p[r*src.cols+c]; dst(r, c) = (p[(r-radius)*src.cols+c-radius] >= cval ? 128 : 0) | (p[(r-radius)*src.cols+c] >= cval ? 64 : 0) | ...; } } }; BR REGISTER(Transform, LBP)

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-50
SLIDE 50

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const { for (int r=radius; r<src.rows-radius; r++) for (int c=radius; c<src.cols-radius; c++) { float cval = p[r*src.cols+c]; dst(r, c) = (p[(r-radius)*src.cols+c-radius] >= cval ? 128 : 0) | (p[(r-radius)*src.cols+c] >= cval ? 64 : 0) | ...; } } }; BR REGISTER(Transform, LBP) ... Transform *lbp = Transform::make(”LBP(1)”);

  • J. Klontz

OpenBR February 17, 2013 18 / 18

slide-51
SLIDE 51

Plugin Example: Local Binary Patterns

#include <openbr plugin.h> class LBP : public Transform { BR PROPERTY(int, radius, 1) void project(const Matrix &src, Matrix &dst) const { for (int r=radius; r<src.rows-radius; r++) for (int c=radius; c<src.cols-radius; c++) { float cval = p[r*src.cols+c]; dst(r, c) = (p[(r-radius)*src.cols+c-radius] >= cval ? 128 : 0) | (p[(r-radius)*src.cols+c] >= cval ? 64 : 0) | ...; } } }; BR REGISTER(Transform, LBP) ... Transform *lbp = Transform::make(”LBP(1)”); Transform *lbpu2 = Transform::make(”LBP(1)+U2”);

  • J. Klontz

OpenBR February 17, 2013 18 / 18