IVRE Network Recon Framework Network cartography using passive - - PowerPoint PPT Presentation

▶
ivre network recon framework
SMART_READER_LITE
LIVE PREVIEW

IVRE Network Recon Framework Network cartography using passive - - PowerPoint PPT Presentation

Jan 25, 2023 120 likes •387 views

IVRE Network Recon Framework Network cartography using passive traffic analysis VENUTI Vivien CEA ( Alternative Energies and Atomic Energy Commission ) 11 avril 2019 CEA ( Alternative Energies and Atomic Energy Commission ) | 11 avril 2019 | PAGE

slide-1
SLIDE 1

IVRE Network Recon Framework

Network cartography using passive traffic analysis

VENUTI Vivien CEA (Alternative Energies and Atomic Energy Commission) 11 avril 2019

CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 1/26

slide-2
SLIDE 2

Contents

1

IVRE

2

Context

3

Active Scan

4

Passive Monitoring

5

View

6

What’s next

CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 2/26

slide-3
SLIDE 3

IVRE

Network Recon Framework

Started by Pierre Lalet in 2011, GPL licensed Written in python, support from 2.6 to 3.7 Build test using Travis Quality check using flake8, codacy, ... CLI, GUI, Python API DB backends : MongoDB, Postgresql, SQLite, Neo4j

Links

Repository : https://github.com/cea-sec/ivre/ WebSite : https://ivre.rocks/ Twitter : @IvreRocks

IVRE CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 3/26

slide-4
SLIDE 4

IVRE Architecture

IVRE CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 4/26

slide-5
SLIDE 5

Use cases Use cases

Everyday Monitoring

Zeek + passiverecon / p0f (ipinfo) Zeek / Argus / NetFlow (flow)

Audit, Pentesting

Nmap / Masscan (scancli) Zeek / ... (ipinfo/flow)

Context CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 5/26

slide-6
SLIDE 6

Addressed Issue Issue :

Various use cases Heterogeneous tools Common goals

Solution :

Keep independant databases Unify analysis tools under UI

Context CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 6/26

slide-7
SLIDE 7

Active tools

Active Scan CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 7/26

slide-8
SLIDE 8

SYN Scan

SYN SYN|ACK / (RST) ACK

SYN Scan

Determine open TCP ports Fast and small footprint Most common way of scanning

Active Scan CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 8/26

slide-9
SLIDE 9

XML output format

1 <?xml version=”1.0”?> 2 <!-- masscan v1.0 scan --> 3 <?xml-stylesheet href=”” type=”text/xsl”?> 4 <nmaprun scanner=”masscan” start=”1524396448” version=”1.0-BETA” xmloutputversion=”1.03”> 5 <scaninfo type=”syn” protocol=”tcp” /> 6 <host endtime=”1524396450”><address addr=”130.236.252.73” addrtype=”ipv4”/>< ports><port protocol=”tcp” portid=”102”><state state=”open” reason=” response” reason_ttl=”114” /><service name=”s7comm” banner=”\x03\x00\x00\ x0b\x06\x80\x00”></service></port></ports></host> 7 <runstats> 8 9 <finished time=”1524397449” timestr=”2018-04-22␣11:44:09” elapsed=”1003” /> 10 <hosts up=”24720” down=”0” total=”24720” /> 11 </runstats> 12 </nmaprun>

Active Scan CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 9/26

slide-10
SLIDE 10

ftp-anon

Nmap Anonymous FTP Listing

1 PORT STATE SERVICE 2 21/tcp open ftp 3 | ftp-anon: Anonymous FTP login allowed (FTP code 230) 4 | -rw-r--r-- 1 1170 924 31 Mar 28 2001 .banner 5 | d--x--x--x 2 root root 1024 Jan 14 2002 bin 6 | d--x--x--x 2 root root 1024 Aug 10 1999 etc 7 | drwxr-srwt 2 1170 924 2048 Jul 19 18:48 incoming [NSE: writeable] 8 | d--x--x--x 2 root root 1024 Jan 14 2002 lib 9 | drwxr-sr-x 2 1170 924 1024 Aug 5 2004 pub

Active probing scripts Can establish connection with encrypted services Can check configuration errors (eg: root without password) Can check for vulnerabilities

Active Scan CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 10/26

slide-11
SLIDE 11

CANNOT scan on every system!

Research Reactor OSIRIS Fusion Research Installation Tore-Supra

source : <www.cea.fr/multimedia/>

Active Scan CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 11/26

slide-12
SLIDE 12

Passive tools

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 12/26

slide-13
SLIDE 13

Passiverecon Use

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 13/26

slide-14
SLIDE 14

Passiverecon Script

Record format

Custom, fixed set of fields Event type identified by recon_type Meaning of the fields depends on event

1 type Info : record { 2 ts : time &log ; 3 uid : string &log ; 4 host : addr &log &optional ; 5 srvport : port &log &optional ; 6 recon_type : Type &log &default =UNKNOWN; 7 source : string &log &optional ; 8 value : string &log ; 9 targetval : string &log &optional ; 10 } ;

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 14/26

slide-15
SLIDE 15

Banner extraction

Raw log

1 #fields ts uid host srvport recon_type source value targetval 2 #types time string addr port enum string string string 3 1213964540.267709 C5PFZj40WKttMOTD36 10.0.0.2 22 PassiveRecon::SSH_SERVER

  • SSH-2.0-Cisco-1.25
  • 4 1213964540.267709

C5PFZj40WKttMOTD36 10.0.0.1

  • PassiveRecon::SSH_CLIENT
  • SSH-1.99-Cisco-1.25
  • 5 #close

2019-04-04-21-51-27

Formatted data

1 10.0.0.2 22 SSH_SERVER SSH-2.0-Cisco-1.25 (1 time) 2008-06-20 14:22:20

  • 2008-06-20 14:22:20

2 service_name: ssh 3 service_extrainfo: protocol 2.0 4 service_ostype: IOS 5 service_product: Cisco SSH 6 service_version: 1.25

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 15/26

slide-16
SLIDE 16

Nmap fingerprints

1 match ssh m|^SSH-(\d[\d.]+)-Cisco-(\d[\d.]+)\r?\n$| p/Cisco SSH/ v/$2/ i/ protocol $1/ o/IOS/ cpe:/a:cisco:ssh:$2/ cpe:/o:cisco:ios/a

Regexp based Extraction of variable fields Easy to use and store in JSON Generic rules to match partially

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 16/26

slide-17
SLIDE 17

Passiverecon Format

1 { 2 ”addr”: ”10.0.0.2”, 3 ”sensor”: ””, 4 ”count”: 1, 5 ”firstseen”: ”2008-06-20 14:22:20.267709”, 6 ”lastseen”: ”2008-06-20 14:22:20.267709”, 7 ”port”: 22, 8 ”recontype”: ”SSH_SERVER”, 9 ”source”: ””, 10 ”targetval”: ””, 11 ”value”: ”SSH-2.0-Cisco-1.25”, 12 ”infos”: { 13 ”service_name”: ”ssh”, 14 ”service_ostype”: ”IOS”, 15 ”service_product”: ”Cisco SSH”, 16 ”service_version”: ”1.25”, 17 ”service_extrainfo”: ”protocol 2.0” 18 } 19 }

Passiverecon data as stored in IVRE

Passive Monitoring CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 17/26

slide-18
SLIDE 18

Combine active and passive

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 18/26

slide-19
SLIDE 19

Active vs Passive

Active scan will detect all hosts and services if up Passive monitoring will detect all streams that occur A rogue client won’t be detected by a scan Proper traffic sniff will not cause perturbation Time period of monitoring, or moment of scan has great impact!

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 19/26

slide-20
SLIDE 20

db2view

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 20/26

slide-21
SLIDE 21

Passive to View Conversion

Passive to View Generic conversion process 19 different passive recontype

  • eg: SSH_SERVER, HTTP_SERVER, P0F, TCP_BANNER, ...

8 extractors currently implemented

  • Value is parsed again (eg : nmap fingerprints)
  • Easily extensible

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 21/26

slide-22
SLIDE 22

Conversion with Extraction

Passive record

1 { ’ _id ’ : 4 , 2 ’ addr ’ : ’ 1 7 2 . 2 8 . 2 . 3 ’ , 3 ’ count ’ : 1 , 4 ’ firstseen ’ : datetime . datetime (2003 , 6 , 11 , 2 , 6 , 18 , 756932) , 5 ’ infos ’ : { ’ algo ’ : ’ ssh

−

rsa ’ , 6 ’ bits ’ : 1024 , 7 ’ exponent ’ : ’35 ’ , 8 ’md5 ’ : ’207 ce596b04ecea4dbe4aa29e8909807 ’ , 9 ’modulus ’ : ’1170991138...0455603093695918136982250777549537817 ’ , 10 ’ sha1 ’ : ’2 a1c5cb270cb1cce82879a42f37affb95556cfcb ’ , 11 ’ sha256 ’ : ’ b8162aaee40c4c13fda7af52aaaf73e7016f3011bf33dfaebb921e9e4e3bb296 ’ } , 12 ’ lastseen ’ : datetime . datetime (2003 , 6 , 11 , 2 , 6 , 18 , 756932) , 13 ’ port ’ : 22 , 14 ’ recontype ’ : ’SSH_SERVER_HOSTKEY ’ , 15 ’ sensor ’ : ’ ’ , 16 ’ source ’ : ’ SSHv2 ’ , 17 ’ targetval ’ : ’ ’ , 18 ’ value ’ : ’\\ x00\\x00\\x00\\x07ssh

−

rsa \\x00\\x00\\x00\\x01 . . . \ \ x11 \\x02\\x19 ’ } View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 22/26

slide-23
SLIDE 23

Conversion with Extraction

Active record

1 { ’ addr ’ : ’ 1 7 2 . 2 8 . 2 . 3 ’ , 2 ’ endtime ’ : datetime . datetime (2003 , 6 , 11 , 2 , 6 , 18 , 756932) , 3 ’ openports ’ : { ’ count ’ : 1 , ’ tcp ’ : { ’ count ’ : 1 , ’ ports ’ : [ 2 2 ] } } , 4 ’ ports ’ : [ { ’ port ’ : 22 , 5 ’ protocol ’ : ’ tcp ’ , 6 ’ scripts ’ : [ { ’ id ’ : ’ ssh

−

hostkey ’ , 7 ’ key ’ : { ’ bits ’ : 1024 , 8 ’ fingerprint ’ : ’207 ce596b04ecea4dbe4aa29e8909807 ’ , 9 ’ key ’ : ’ AAAAB3NzaC1yc2EAAAABIwAAA . . . = ’ , 10 ’ type ’ : ’ ssh

−

rsa ’ } , 11 ’ output ’ : ’\n ’ 12 ’ 1024 ’ 13 ’20:7 c : e5 : 9 6 : b0 :4 e : ce : a4 : db : e4 : aa : 2 9 : e8 : 9 0 : 9 8 : 0 7 ’ 14 ’ ( RSA ) \n ’ 15 ’ ssh

−

rsa ’ 16 ’ AAAAB3NzaC1yc2EAAAABIwAAA . . . = ’ , 17 ’ ssh

−

hostkey ’ : [ { ’ bits ’ : 1024 , 18 ’ fingerprint ’ : ’207 ce596b04ecea4dbe4aa29e8909807 ’ , 19 ’ key ’ : ’ AAAAB3NzaC1yc2EAAAABIwAAA . . . = ’ , 20 ’ type ’ : ’ ssh

−

rsa ’ } ] } ] , 21 ’ service_name ’ : ’ ssh ’ , 22 ’ state_reason ’ : ’ passive ’ , 23 ’ state_state ’ : ’open ’ } ] , 24 ’ schema_version ’ : 11 , 25 ’ starttime ’ : datetime . datetime (2003 , 6 , 11 , 2 , 6 , 18 , 756932) , 26 ’ state ’ : ’up ’ , 27 ’ state_reason ’ : ’ passive ’ } View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 23/26

slide-24
SLIDE 24

Conversion Only

Passive record

1 {’_id’: 8, 2 ’addr’: ’66.59.111.190’, 3 ’count’: 1, 4 ’firstseen’: datetime.datetime(2003, 6, 11, 2, 6, 18, 756932), 5 ’infos’: {’distance’: 0, ’version’: ’2.4-2.6’}, 6 ’lastseen’: datetime.datetime(2003, 6, 11, 2, 6, 18, 756932), 7 ’port’: 0, 8 ’recontype’: ’P0F’, 9 ’sensor’: ’’, 10 ’source’: ’’, 11 ’targetval’: ’’, 12 ’value’: ’Linux’}

Active record

1 {’addr’: ’66.59.111.190’, 2 ’endtime’: datetime.datetime(2003, 6, 11, 2, 6, 18, 756932), 3 ’openports’: {’count’: 0}, 4 ’schema_version’: 11, 5 ’starttime’: datetime.datetime(2003, 6, 11, 2, 6, 18, 756932), 6 ’state’: ’up’, 7 ’state_reason’: ’passive’}

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 24/26

slide-25
SLIDE 25

Merge

Parse records recursively Many fields of variable size and content Tedious work, a lot of special cases

  • overwritable fields
  • updatable fields
  • difficult cases -> dropped

View CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 25/26

slide-26
SLIDE 26

Enhancement

Actively working on the project (2-3 people) WIP on new backends: SQLite and Elasticsearch Flow being rewritten Scan Diff (Machine learning incoming!) Looking for discussion and contribution Links Repository : https://github.com/cea-sec/ivre/ WebSite : https://ivre.rocks/ Twitter : @IvreRocks

What’s next CEA (Alternative Energies and Atomic Energy Commission) | 11 avril 2019 | PAGE 26/26