Towards Reversible De-Identification in Video Sequences Using 3D - - PowerPoint PPT Presentation

Towards Reversible De-Identification in Video Sequences Using 3D Avatars and Steganography

Karla Brkić

University of Zagreb Faculty of Electrical Engineering and Computing HR-10000 Zagreb, Croatia

Collaborators: Martin Blažević, Tomislav Hrkać

November 3, 2015 Karla Brkić 1

6th COST Action IC1206 Meeting, WG2

Outline

• Reversibly de-identifying people in videos

and images

– Challenges – Possible approaches

• A 3D avatar-based de-identification pipeline
• Preliminary results (using the Kinect sensor)
• Conclusions

November 3, 2015 Karla Brkić 2

Our goal: de-identifying people in videos and images

• concealing identities of people while maintaining informa

November 3, 2015 Karla Brkić 3

How to do proper de-identification?

Identity revealing features:

• biometric

– e.g. face

• soft biometric

– e.g. tattoos, scars, birthmarks

• non-biometric

– e.g. dressing style, hairstyle

We aim to de-identify soft and non-biometric features.

November 3, 2015 Karla Brkić 4

REVERSIBLE!

Additionally, we are interested in…

• concealing identities of people

while maintaining information

• n the action and its context
• reversible de-identification

November 3, 2015 Karla Brkić 5

Person detection Input: image or video sequence De-identifying transformation Identifier segmentation Output: de- identified content De-identifying transformation Identifier segmentation

Possible approaches (I)

• Find and de-identify individual soft and non-biometric identifiers

November 3, 2015 Karla Brkić 6

find hair, change its color find clothes, change their style find tattoos, blur them find birthmarks, blur them

Possible approaches (II)

• De-identification of the whole body

– replace the person with a virtual avatar

November 3, 2015 Karla Brkić 7

Our envisioned de-identification pipeline (I)

November 3, 2015 Karla Brkić 8

Our envisioned de-identification pipeline (II)

• focus of this work: the last two stages of the pipeline

– replacing the person in the video with a 3D model – steganographically encoding the replaced image in the new image

• prerequisite: pose estimation

– a hard problem – to investigate close-to-best scenario, we use Microsoft Kinect

November 3, 2015 Karla Brkić 9

Recovering the 3D skeleton from Kinect

• Kinect features:

– standard VGA color camera – multi-array microphone – depth sensor

• capabilities:

– motion capture – face gestures – voice detection

November 3, 2015 Karla Brkić 10

Hierarchy of joints defined by the skeletal tracking system

Rendering the 3D model

• the avatar consists of two parts:

– hierarchical set of interconnected bones – skeleton or rig – surface representation used to draw the character – skin or mesh

November 3, 2015 Karla Brkić 11

Concealing the original image (I)

November 3, 2015 Karla Brkić 12

• steganography: the science of hiding information
• 𝑔

𝐹: insertion

• 𝑔

𝐹 −1: extraction

• key: steganographic key
• stego: steganographic file
• carrier: data file into which the

hidden data is hidden

• message: secret message that

needs to be hidden

Sender Receiver

Concealing the original image (II)

• LSB insertion steganographic algorithm

November 3, 2015 Karla Brkić 13

Experiments – model rendering

November 3, 2015 Karla Brkić 14

Experiments – steganographic encoding (I)

November 3, 2015 Karla Brkić 15

(a) 1 hidden bit (b) 2 hidden bits

Experiments – steganographic encoding (II)

November 3, 2015 Karla Brkić 16

(c) 4 hidden bits (d) 7 hidden bits

Conclusions

• Our solution enables:

– detecting and tracking persons in videos – reversible hiding of their identity

• Limitations:

– dependent on the Kinect sensor – the 3D model cannot perform a broad range of movements – errors in tracking, pertaining to the Kinect device

• Future work:

– more natural human models – finer capture of joint dana and body movements

November 3, 2015 Karla Brkić 17

Thank you!

• Questions?

November 3, 2015 Karla Brkić 18