AI iThome CYBERSEC 2019 - - PowerPoint PPT Presentation

AI 的逆襲， 眼⾒見廓真的為憑嗎？

台灣資安⼤夨會 iThome CYBERSEC 2019
 Turkey & Melody 


台灣駭客協會 秘書長 HITCON GIRLS 共同創辦⼈亻 台灣駭客協會 專案經理痢 HITCON GIRLS 成員

ABOUT US

2

Turkey Melody

OUTLINE

3

• Deepfakes
• Lyrebirds
• 應⽤甩
• 反制技術

DEEPFAKES MAKES IT POSSIBLE TO CREATE FAKE VIDEOS OF ALMOST EVERYONE.

4

5

LET’S TALK ABOUT DEEPFAKES!

• 1. In December 2017, a user

named “DeepFakes” announce the tool on Reddit Community of developers.

• 2. DeepFakes is a tool that

utilizes deep learning to recognize and swap faces in pictures and videos.

5

6

HOW DEEPFAKES WORK?

HOW DEEPFAKES WORK?

https://github.com/iperov/DeepFaceLab

7

HOW DEEPFAKES WORK?

>DeepFakes Tools

• 1. Fakeapp
• 2. DeepFaceLab
• 3. Fakeswap
• 4. Openfaceswap
• 5. Myfakeapp

8

HOW DEEPFAKES WORK?

>DeepFakes Tools

• 1. Fakeapp

2.DeepFaceLab

• 3. Fakeswap
• 4. Openfaceswap
• 5. Myfakeapp

>

9

Source Target

10

HOW DEEPFAKES WORK?

>Steps for: DeepFaceLab

2 3 1

Extract image from source video. Extract faces from source and target image.

4

Manually remove error image.

5

Model Training

6

Extract image from target video. Debug & Convert image to mp4

11

HOW DEEPFAKES WORK?

>Extract image from source＆target video.

FFmpeg FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created.

ffmpeg -i clipname -vf fps=framerate -qscale:v 2 "imagename%04d.jpg" Command_

12

HOW DEEPFAKES WORK?

>Extract faces from source and target image

（Ref: Facial landmarks with dlib, OpenCV, and Python）

Divided a face into the following areas:

• 1. eyes(left/right)
• 2. eyebrows(left/right)
• 3. nose
• 4. chin

13

HOW DEEPFAKES WORK?

>Extract faces from source and target image

（Ref: Real-Time Face Pose Estimation ）

14

HOW DEEPFAKES WORK?

>Extract faces from source and target image

15

HOW DEEPFAKES WORK?

>Extract faces from source and target image

（Ref: One Millisecond Face Alignment with an Ensemble of Regression Trees）

16

HOW DEEPFAKES WORK?

>Manually remove error image

17

HOW DEEPFAKES WORK?

>Manually remove error image

18

HOW DEEPFAKES WORK?

>Manually remove error image

19

HOW DEEPFAKES WORK?

>Model Training

(Ref: https://deepfakes.com.cn/)

20

HOW DEEPFAKES WORK?

>Model Training

(Ref: https://deepfakes.com.cn/)

21

HOW DEEPFAKES WORK?

>Model Training

(Ref: https://deepfakes.com.cn/)

22

HOW DEEPFAKES WORK?

>Model Training

Models Types:

• 1. H64
• 2. H128
• 3. DF
• 4. LIAEF
• 5. LIAEF128YAW
• 6. MIAEF128
• 7. AVATAR
• 8. SAE

H128

23

HOW DEEPFAKES WORK?

>Model Training

Models Types:

• 1. H64
• 2. H128
• 3. DF
• 4. LIAEF
• 5. LIAEF128YAW
• 6. MIAEF128
• 7. AVATAR
• 8. SAE

H128

Train 越久效果越好！

24

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

25

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

26

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

27

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

28

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

29

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

30

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

Debug Mode

Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default

31

HOW DEEPFAKES WORK?

>Debug & Convert image to mp4

ffmpeg -f image2 -i imagename%04d.jpg -vcodec libx264 -crf 15 -pix_fmt yuv420p Command_

32

(Downlink: https://github.com/iperov/DeepFaceLab)

HOW DEEPFAKES WORK?

>Download DeepFaceLab

33

34

LET’S DEMO !

HOW DEEPFAKES WORK?

>⼩尐插曲

實測跑評：RTX 2080 ⼤夨約需跑 6 個⼩尐時以上會有不錯的成果

作者跪求 RTX 2080 Ti 中 XD

35

HOW DEEPFAKES WORK?

>不要讓貧窮限制你的 AI

＃讓 AI 聰明前 先飽滿你的⼝古袋

36

Source Target

37

38

（Link:https://youtu.be/yfsdZ5gLuOo）

CAN AI MIMIC YOUR VOICE?

39

CAN YOU TELL A FAKE VOICE FROM A REAL ONE?

40

（Ref: https://lyrebird.ai/）

• create a digitized version of your own voice.
• Uses deep learning frameworks on AWS to develop and

train its AI models

What is Lyrebird?

Record & Upload voice samples Train by DL Framework

• n AWS EC2 P3

Digital Voice Generated Optput audio of requested dialog （Ref: https://lyrebird.ai/）

41

LYREBIRD LIVE DEMO

This is my honored to talk to everyone here. However, can you trust your eyes and ears to perceive reality? Machine learning can artificially mimic natural sounds to create digital voice. Just like me!

42

Ｗhy my voice sounds awful?

43

（Ref: https://lyrebird.ai/）

Voice Cloning Market Generation Is Coming

44

CHALLENGES FOR OUR FUTURE

45

Why do you believe what I am telling you now?

NEGATIVE EFFECTS

46

POSITIVE EFFECTS

Create images and voices from the past.

47

（Ref:apple） （Ref:Fast & Furious 6）

（Ref:https://www.youtube.com/watch?v=5iZuffHPDAw）

FIGHT AGAINST FAKE VIDEOS!

49

FAKE VIDEO DETECT SOLUTION

>Detect malicious alterations with AI

• abnormal compression signatures
• lip sync analysis
• Video metadata analysis
• noise patterns analysis

50

FAKE VIDEO DETECT SOLUTION

> Fingerprint source videos & track provenance with blockchain smart contracts

Create Hashing Blackchan record submitt

51

REFERENCE

• https://deepfakes.com.cn/
• https://www.youtube.com/watch?v=K98nTNjXkq8
• ttps://github.com/Fabsqrt/BitTigerLab
• https://github.com/iperov/DeepFaceLab
• https://www.youtube.com/watch?v=CR5Jr6Z1KI
• https://lyrebird.ai/
• https://www.darpa.mil/
• https://medium.com/amber-video

52

THANKS!

53