AI iThome CYBERSEC 2019 - PowerPoint PPT Presentation

眼⾒見廓真的為憑嗎？ AI 的逆襲， 台灣資安⼤夨會 iThome CYBERSEC 2019 
 Turkey & Melody 


ABOUT US Turkey Melody 台灣駭客協會 秘書長 台灣駭客協會 專案經理痢 HITCON GIRLS 共同創辦⼈亻 HITCON GIRLS 成員 2

OUTLINE • Deepfakes • Lyrebirds • 應⽤甩 • 反制技術 3

DEEPFAKES MAKES IT POSSIBLE TO CREATE FAKE VIDEOS OF ALMOST EVERYONE. 4

LET’S TALK ABOUT DEEPFAKES! 1. In December 2017, a user named “DeepFakes” announce the tool on Reddit Community of developers. 2. DeepFakes is a tool that utilizes deep learning to recognize and swap faces in pictures and videos. 5 5

HOW DEEPFAKES WORK? 6

HOW DEEPFAKES WORK? https://github.com/iperov/DeepFaceLab 7

HOW DEEPFAKES WORK? >DeepFakes Tools 1. Fakeapp 2. DeepFaceLab 3. Fakeswap 4. Openfaceswap 5. Myfakeapp 8

HOW DEEPFAKES WORK? >DeepFakes Tools 1. Fakeapp > 2.DeepFaceLab 3. Fakeswap 4. Openfaceswap 5. Myfakeapp 9

Source Target 10

4 2 3 1 5 6 HOW DEEPFAKES WORK? >Steps for: DeepFaceLab Extract faces Extract image Extract image from source from source from target and target video. video. image. Manually Debug & Convert Model remove image to mp4 Training error image. 11

HOW DEEPFAKES WORK? >Extract image from source ＆ target video. FFmpeg FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. ffmpeg -i clipname -vf fps=framerate -qscale:v 2 "imagename%04d.jpg" Command_ 12

HOW DEEPFAKES WORK? >Extract faces from source and target image Divided a face into the following areas: 1. eyes(left/right) 2. eyebrows(left/right) 3. nose 4. chin （ Ref: Facial landmarks with dlib, OpenCV, and Python ） 13

HOW DEEPFAKES WORK? >Extract faces from source and target image （ Ref: Real-Time Face Pose Estimation ） 14

HOW DEEPFAKES WORK? >Extract faces from source and target image 15

HOW DEEPFAKES WORK? >Extract faces from source and target image （ Ref: One Millisecond Face Alignment with an Ensemble of Regression Trees ） 16

HOW DEEPFAKES WORK? >Manually remove error image 17

HOW DEEPFAKES WORK? >Manually remove error image 18

HOW DEEPFAKES WORK? >Manually remove error image 19

HOW DEEPFAKES WORK? >Model Training (Ref: https://deepfakes.com.cn/) 20

HOW DEEPFAKES WORK? >Model Training (Ref: https://deepfakes.com.cn/) 21

HOW DEEPFAKES WORK? >Model Training (Ref: https://deepfakes.com.cn/) 22

HOW DEEPFAKES WORK? >Model Training Models Types: 1. H64 2. H128 3. DF 4. LIAEF 5. LIAEF128YAW 6. MIAEF128 7. AVATAR 8. SAE H128 23

Train 越久效果越好！ HOW DEEPFAKES WORK? >Model Training Models Types: 1. H64 2. H128 3. DF 4. LIAEF 5. LIAEF128YAW 6. MIAEF128 7. AVATAR 8. SAE H128 24

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 25

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 26

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 27

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 28

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 29

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 30

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 Debug Mode Use predicted mask? 1 Erosion (-100 to +100): (default = 0) 1 Seamless Erosion (0 to 40): (default = 0) 20 Blur (-200 to +200): (default = 0) 40 Hist-match threshold (0 to 255): (default = 255) default Face Scale (-50 to +50): (default = 0) default Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT Degrade Color Power of Final Image: (default = 0) default 31

HOW DEEPFAKES WORK? >Debug & Convert image to mp4 ffmpeg -f image2 -i imagename%04d.jpg -vcodec libx264 -crf 15 -pix_fmt yuv420p Command_ 32

HOW DEEPFAKES WORK? >Download DeepFaceLab (Downlink: https://github.com/iperov/DeepFaceLab) 33

LET’S DEMO ! 34

HOW DEEPFAKES WORK? > ⼩尐插曲 作者跪求 RTX 2080 Ti 中 XD 實測跑評： RTX 2080 ⼤夨約需跑 6 個⼩尐時以上會有不錯的成果 35

HOW DEEPFAKES WORK? > 不要讓貧窮限制你的 AI ＃讓 AI 聰明前 先飽滿你的⼝古袋 36

Source Target 37

（ Link:https://youtu.be/yfsdZ5gLuOo ） 38

CAN AI MIMIC YOUR VOICE? 39

CAN YOU TELL A FAKE VOICE FROM A REAL ONE? （ Ref: https://lyrebird.ai/ ） 40

What is Lyrebird? • create a digitized version of your own voice. • Uses deep learning frameworks on AWS to develop and train its AI models Record & Upload Train by DL Framework Digital Voice Optput audio of voice samples on AWS EC2 P3 Generated requested dialog （ Ref: https://lyrebird.ai/ ） 41

LYREBIRD LIVE DEMO This is my honored to talk to everyone here. However, can you trust your eyes and ears to perceive reality? Machine learning can artificially mimic natural sounds to create digital voice. Just like me! 42

Ｗ hy my voice sounds awful? （ Ref: https://lyrebird.ai/ ） 43

Voice Cloning Market Generation Is Coming 44

CHALLENGES FOR OUR FUTURE 45

NEGATIVE EFFECTS Why do you believe what I am telling you now? 46

（ Ref:Fast & Furious 6 ） POSITIVE EFFECTS （ Ref:apple ） Create images and voices from the past. 47

（ Ref:https://www.youtube.com/watch?v=5iZuffHPDAw ）

FIGHT AGAINST FAKE VIDEOS! 49

FAKE VIDEO DETECT SOLUTION >Detect malicious alterations with AI •abnormal compression signatures •lip sync analysis •Video metadata analysis •noise patterns analysis 50

FAKE VIDEO DETECT SOLUTION > Fingerprint source videos & track provenance with blockchain smart contracts Create Hashing Blackchan record submitt 51

REFERENCE • https://deepfakes.com.cn/ • https://www.youtube.com/watch?v=K98nTNjXkq8 • ttps://github.com/Fabsqrt/BitTigerLab • https://github.com/iperov/DeepFaceLab • https://www.youtube.com/watch?v=CR5Jr6Z1KI • https://lyrebird.ai/ • https://www.darpa.mil/ • https://medium.com/amber-video 52

THANKS! 53