DEXCALIBUR AUTOMATE YOUR ANDROID APP REVERSE Or hooking for dummies - - PowerPoint PPT Presentation
DEXCALIBUR AUTOMATE YOUR ANDROID APP REVERSE Or hooking for dummies https://github.com/FrenchYeti/dexcalibur.git WHO AM I ? GEORGES-B. MICHEL @FrenchYeti yeti@0xff.ninja Aka @FrenchYeti Software Security Evaluator at Thales Day
AUTOMATE YOUR ANDROID APP REVERSE
Or hooking for dummies
https://github.com/FrenchYeti/dexcalibur.git
WHO AM I ?
GEORGES-B. MICHEL
▸ @FrenchYeti ▸ yeti@0xff.ninja ▸ Software Security Evaluator at Thales ▸ Day : Reverse engineering (Android + TEE) apps
▸
HCE Payment applications, Trusted Applications, ARM binaries
▸ Night : Develop reverse / pentest / appsec tools
▸
Frida addict
Aka @FrenchYeti
MOTIVATION
LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file & JNI libs Ciphered secondary .dex file
DECIPHER & LOAD NATIVE FUNCTIONS
MOTIVATION
LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file & JNI libs Ciphered secondary .dex file
DECIPHER & LOAD
INVOKE BY REFLECTION
NATIVE FUNCTIONS
MOTIVATION
LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file & JNI libs Ciphered secondary .dex file
DECIPHER & LOAD
INVOKE BY REFLECTION Ciphered JNI lib
DECIPHER & LOAD NATIVE FUNCTIONS WHITE BOX CRYPTO
MOTIVATION
LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file & JNI libs Ciphered secondary .dex file
DECIPHER & LOAD
INVOKE BY REFLECTION Ciphered JNI lib
DECIPHER & LOAD
Class loaded from the network (NetworkClassLoader)
DOWNLOAD, DECIPHER & LOAD JNI FUNCTIONS NATIVE FUNCTIONS WHITE BOX CRYPTO
MOTIVATION
LET’S IMAGINE AN OBFUSCATED MULTI-DEX APPLICATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file & JNI libs Ciphered secondary .dex file
DECIPHER & LOAD
INVOKE BY REFLECTION Ciphered JNI lib
DECIPHER & LOAD
Class loaded from the network (NetworkClassLoader)
DOWNLOAD, DECIPHER & LOAD JNI FUNCTIONS NATIVE FUNCTIONS WHITE BOX CRYPTO
MOTIVATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Ciphered secondary .dex file
DECIPHER
INVOKE BY REFLECTION Ciphered JNI lib
DECIPHER & LOAD
Class loaded from the network (NetworkClassLoader)
DOWNLOAD, DECIPHER & LOAD JNI FUNCTIONS
YOU CAN HOOK ONLY WHAT YOU SEE
WHAT CAN I HOOK ?
Clear .dex file & JNI libs
NATIVE FUNCTIONS
MOTIVATION
PACKER CLASS LOADER DEX LOADER APP CLASSES & METHODS
Clear .dex file Ciphered secondary .dex file
DECIPHER
INVOKE BY REFLECTION Ciphered JNI lib
DECIPHER & LOAD
Class loaded from the network (NetworkClassLoader)
DOWNLOAD, DECIPHER & LOAD JNI FUNCTIONS
IT REQUIRES SEVERAL HOOKING SESSIONS
Clear .dex file & JNI libs
NATIVE FUNCTIONS
WHAT IS INTERESTING TO HOOK ?
THE IDEA
▸ Deobfuscate waste of time
MOTIVATION
THE IDEA
▸ Deobfuscate waste of time ▸ Manage hooks not so easy
MOTIVATION
THE IDEA
▸ Deobfuscate waste of time ▸ Manage hooks not so easy ▸ Manual tasks can be automated (start App, …)
MOTIVATION
THE IDEA
▸ Deobfuscate waste of time ▸ Manage hooks not so easy ▸ Manual tasks can be automated (start App, …) ▸ Several devices hooked simultaneously
MOTIVATION
THE IDEA
▸ Deobfuscate waste of time ▸ Manage hooks not so easy ▸ Manual tasks can be automated (start App, …) ▸ Several devices hooked simultaneously ▸ Application size explore bytecode/libs is boring
MOTIVATION
THE IDEA
▸ Show functions invoked dynamically as « xrefs » ▸ Discover automatically classes & bytecode loaded dynamically
(DexFile ..)
▸ Generate hook with a single click on the function ▸ Debug a single hook while others are active ▸ Enable/disable hook without lose
CHRISTMAS WISH LIST 1/2 :
THE IDEA
▸ Multi-user : share the same instrumentation with my friends ▸ Instrumente several devices and merge hook logs
(Workflow / IoT)
▸ Be able to run with rooted & non-rooted devices ▸ Offer user-friendly GUI and API, ▸ Free & open-source ! ( license APACHE 2 )
CHRISTMAS WISH LIST 2/2 :
NOT JUST A TOOLBOX
DEX DISASSEMBLER
Baksmali
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
FILE IDENTIFIERS & PARSERS DEX DISASSEMBLER
Baksmali
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS DEX DISASSEMBLER
Baksmali
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS MODULAR HEURISTIC & SEARCH ENGINE DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS MODULAR HEURISTIC & SEARCH ENGINE DEVICE MANAGER & FRIDA UTILS DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS MODULAR HEURISTIC & SEARCH ENGINE WEB SERVER & UI DEVICE MANAGER & FRIDA UTILS CONTROLS & CUSTOMIZE DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS MODULAR HEURISTIC & SEARCH ENGINE WEB SERVER & UI DEVICE MANAGER & FRIDA UTILS IMPROVES AT RUNTIME CONTROLS & CUSTOMIZE DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
NOT JUST A TOOLBOX
STATIC BYTECODE ANALYZER DYNAMIC BYTECODE ANALYZER FILE IDENTIFIERS & PARSERS MODULAR HEURISTIC & SEARCH ENGINE WEB SERVER & UI DEVICE MANAGER & FRIDA UTILS IMPROVES AT RUNTIME CONTROLS & CUSTOMIZE
DEXCALIBUR
DEX DISASSEMBLER
Baksmali
INSTRUMENTATION TOOL
WHAT IS DEXCALIBUR ?
WHAT IS DEXCALIBUR ?
POWERED BY …
ANDROID SDK APKTOOL + BAKSMALI
Today
NATIVE HOOK CANNOT BE GENERATED NO BYTECODE SYMBOLIC EXEC
Functions contained into JNI/native libs can be hooked, but decompilers/analyzers dont support it. So, native hook cannot be generated.
NICE TOOLS :-)
WHAT IS DEXCALIBUR ?
POWERED BY …
ANDROID SDK APKTOOL + BAKSMALI
Today
NATIVE HOOK CANNOT BE GENERATED NO BYTECODE SYMBOLIC EXEC
Functions contained into JNI/native libs can be hooked, but decompilers/analyzers dont support it. So, native hook cannot be generated.
ADD NATIVE LIBRARIES SUPPORT SMALI SYMBOLIC EXEC
NICE TOOLS :-)
LIEF R2 LIEF
Tomorrow
RetDec
SMALI VM Z3 SOLVER
AND MORE !
HOW IT WORKS ?
1) START PHASE - FILE ANALYSIS
UNCOMPRESS APK
APK FILE DEVICE
FILE ANALYZER
Files identified & categorized: key stores, libs, properties, xml, shared pref, cache, … Pull Application data /data/data/xxx … Undetected / high entropy files are tagged notify
1 3 4
Parse APK content
2
HOW IT WORKS ?
1) START PHASE - ANDROID API ANALYSIS
UNCOMPRESS APK
APK FILE
ANDROID API/STUB
Application Graph Statically built
DEX DISASSEMBLER
SAST
FILE ANALYZER
3 1 2
Create app graph
DEVICE
HOW IT WORKS ?
1) START PHASE - APPLICATION BYTE CODE ANALYSIS
UNCOMPRESS APK
APK FILE
ANDROID API/STUB
DEX DISASSEMBLER
notify
SAST
Application Graph Statically built
DEX DISASSEMBLER
SAST
FILE ANALYZER
1 2 4 3
Update app graph
DEVICE
HOW IT WORKS ?
2) INSTRUMENTATION PHASE - BEFORE RUN
notify Categorized Files Application + Android API Graph Statically built
DYNAMIC LOADER BYTE ARRAY CLASSIFIER FILE ACCESS KEY STORES
…
1 MODULAR HEURISTIC ENGINE
HOW IT WORKS ?
notify Categorized Files Application + Android API Graph Statically built
DYNAMIC LOADER NATIVE LIB / JNI FILE ACCESS DESCRIPTORS STREAMS
Search pattern & method Correlate static files Bind a file to a method
KEY STORE
…
1 2 2’ MODULAR HEURISTIC ENGINE
2) INSTRUMENTATION PHASE - BEFORE RUN
HOW IT WORKS ?
Categorized Files Application + Android API Graph Statically built
DYNAMIC LOADER NATIVE LIB / JNI FILE ACCESS DESCRIPTORS STREAMS KEY STORE
…
HOOK MANAGER
Get method signature
ASK FOR INSTRUMENTATION
Generate frida code
HOOKS
3 4 5 MODULAR HEURISTIC ENGINE
2) INSTRUMENTATION PHASE - BEFORE RUN
HOW IT WORKS ?
2) INSTRUMENTATION PHASE - RUNTIME
Application + Android API Graph Statically built
DYNAMIC LOADER NATIVE LIB / JNI FILE ACCESS DESCRIPTORS STREAMS KEY STORE
…
HOOK MANAGER DEVICE
HOOKS
Starts app & deploys Hook data : args, return, this, …
6 7
Correlate graph & intercepted data
8 MODULAR HEURISTIC ENGINE
HOW IT WORKS ?
Application + Android API Graph Statically built
DYNAMIC LOADER NATIVE LIB / JNI FILE ACCESS DESCRIPTORS STREAMS KEY STORE
…
HOOK MANAGER DEVICE
HOOKS
Starts app & deploys Hook data : args, return, this, …
6 7
Correlate intercepted data
8
Push discovered elements & tag node
9 MODULAR HEURISTIC ENGINE
2) INSTRUMENTATION PHASE - RUNTIME
DRAW A COMPLETE PICTURE OF THE APPLICATION
MIX * ANALYSIS WITH INSTRUMENTATION RESULTS
GRAPHS
STATIC ANALYSIS
ANDROID INTERNALS CALLS STATIC VALUES
DRAW A COMPLETE PICTURE OF THE APPLICATION
GRAPHS SYMBOLIC VALUES
STATIC ANALYSIS DYNAMIC ANALYSIS
ANDROID INTERNALS CALLS STATIC VALUES SOLVE CONSTRAINT …
MIX * ANALYSIS WITH INSTRUMENTATION RESULTS
DRAW A COMPLETE PICTURE OF THE APPLICATION
GRAPHS SYMBOLIC VALUES
STATIC ANALYSIS DYNAMIC ANALYSIS
ANDROID INTERNALS CALLS STATIC VALUES
FILE ANALYSIS
KEYSTORES PROPERTIES LIBS & DEX … STRUCTURES SOLVE CONSTRAINT …
MIX * ANALYSIS WITH INSTRUMENTATION RESULTS
DRAW A COMPLETE PICTURE OF THE APPLICATION
GRAPHS SYMBOLIC VALUES PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
KEYSTORES PROPERTIES LIBS & DEX … STRUCTURES SOLVE CONSTRAINT …
MIX * ANALYSIS WITH INSTRUMENTATION RESULTS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
METHOD INVOKED DYNAMICALLY
From a static point-of-view only two methods are called :
Smali code
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED START APP
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED START APP HOOK TRIGGED
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED START APP HOOK TRIGGED HOOK GATHERS METHOD INFO
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED START APP HOOK TRIGGED HOOK SHOWS STACK TRACE HOOK GATHERS METHOD INFO
DYNAMIC UPDATE OF « XREF FROM »WITH INVOKED METHODS
DYNAMIC UPDATE OF XREF WITH INVOKED METHODS
GRAPHS PARAMS & RETURNS VALUES
STATIC ANALYSIS DYNAMIC INSTRUMENTATION
ANDROID INTERNALS CALLS STATIC VALUES DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
REFLECTION API INSTRUMENTED START APP HOOK TRIGGED HOOK SHOWS STACK TRACE HOOK GATHERS METHOD INFO HEURISTIC ENGINE UPDATE DB
BEFORE RUNTIME
DYNAMIC UPDATE OF THE CALL GRAPH
METHOD INVOKED DYNAMICALLY
BEFORE RUNTIME AFTER RUNTIME
METHOD INVOKED DYNAMICALLY
DYNAMIC UPDATE OF THE CALL GRAPH
UPDATE OF THE CALL GRAPH
Green nodes are internal Android or Java methods Pink node are invoked dynamically and not discovered statically Gray nodes have been discovered statically DYNAMIC UPDATE OF THE CALL GRAPH
DEMO #2
DYNAMIC UPDATE OF XREFS WITH INVOKED METHODS
DYNAMIC UPDATE OF THE CALL GRAPH
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
DEX LOADING API INSTRUMENTED START APP
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
DEX LOADING API INSTRUMENTED DEXFILE CONSTRUCTORS TRIGGED START APP
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
DEX LOADING API INSTRUMENTED DEXFILE CONSTRUCTORS TRIGGED START APP HOOKS ASK IF DEX FILES ARE ALREADY KNOWN
Dex File already analyzed ?
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
DEX LOADING API INSTRUMENTED DEXFILE CONSTRUCTORS TRIGGED START APP HOOKS ASK IF DEX FILES ARE ALREADY KNOWN
Dex File already analyzed ?
COPY OR GET DEX FILE
ANALYZE DEX FILE LOADED DYNAMICALLY
ANALYZE DEX FILE LOADED DYNAMICALLY
PARAMS & RETURNS VALUES
DYNAMIC INSTRUMENTATION
DATA READ/WRITE SECONDARY DEX & LIBS STACK TRACE RUNTIME CONTEXT
FILE ANALYSIS
LIBS & DEX CLASS GRAPH
STATIC ANALYSIS
ANDROID INTERNALS CALLS
DEX LOADING API INSTRUMENTED DEXFILE CONSTRUCTORS TRIGGED START APP HOOKS ASK IF DEX FILES ARE ALREADY KNOWN
Dex File already analyzed ?
COPY OR GET DEX FILE DECOMPILE DEX & UPDATE DB
ANALYZE DEX FILE LOADED DYNAMICALLY
BYTECODE CLEANER
BYTECODE CLEANER
BYTE CODE CLEANER : REMOVE NOP
BEFORE
BYTECODE CLEANER
BYTE CODE CLEANER : REMOVE NOP
BEFORE AFTER
REMOVE USELESS GOTO
BEFORE
BYTECODE CLEANER
REMOVE USELESS GOTO
BEFORE AFTER
BYTECODE CLEANER
DEXCALIBUR - NEXT STEPS
IMPROVEMENTS
DEXCALIBUR
Thanks
HOW TO INSTALL ?
HOW TO INSTALL ?
git clone https://github.com/FrenchYeti/dexcalibur.git cd dexcalibur npm install docker pull frenchyeti/dexcalibur docker run -it \
frenchyeti/dexcalibur
DEXCALIBUR - WHAT IS IT ?
SEARCH BYTE ARRAY