Hardware Acceleration: An Essential Part of Cyber Security in - - PowerPoint PPT Presentation

Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks

Jiří Novotný Pavel Čeleda Radek Krejčí

novotny@ics.muni.cz celeda@ics.muni.cz krejci@liberouter.org

DeepSec – In-Depth Security Conference 2010 – November 25, 2010, Vienna, Austria

Part I Motivation

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 2 / 41

World Is Changing Quickly

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 3 / 41

World Is Changing Quickly

Cyber security become to be very important. Income from cyber crime is higher than from drugs.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 3 / 41

World Is Changing Quickly

Cyber security become to be very important. Income from cyber crime is higher than from drugs. SPAM, phishing, social engineering, stealing of confidential information and many others. Botnet business (e.g. Chuck Norris botnet).

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 3 / 41

World Is Changing Quickly

Cyber security become to be very important. Income from cyber crime is higher than from drugs. SPAM, phishing, social engineering, stealing of confidential information and many others. Botnet business (e.g. Chuck Norris botnet). DDoS attacks against Estonia and Georgia.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 3 / 41

World Is Changing Quickly

Cyber security become to be very important. Income from cyber crime is higher than from drugs. SPAM, phishing, social engineering, stealing of confidential information and many others. Botnet business (e.g. Chuck Norris botnet). DDoS attacks against Estonia and Georgia.

Internet is battlefield

• f today.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 3 / 41

How Well Do You Know Your Network?

Do you know what is happening on your network? Are you sure that your network is secure? Are you able to detect and prove network incidents? Or does your network looks like Pandora’s box?

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 4 / 41

Network Monitoring in Time

Originally Basic functionality

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 5 / 41

Network Monitoring in Time

Originally Basic functionality Then Incident handling Network forensics

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 5 / 41

Network Monitoring in Time

Originally Basic functionality Then Incident handling Network forensics Now Intrusion detection

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 5 / 41

Present Computer Security

Main Issues Huge amount of data passing through network. Huge amount of monitoring data. Software-only monitoring solutions are not fast enough. Many of security tools are too complex for configuration. Hardware appliances are not flexible enough. Data from network devices have no sufficient quality.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 6 / 41

Our Vision of the Network Security Monitoring System

10Gbps Core Network Access Network HAMOC Access Network HAMOC 10Gbps Security Operations Center - SOC

NetFlow Analysis Lawful Interception ... Deep Packet Inspection Firewall Rules CSIRT Bad Guys Users Victims

1 2 1 1

HAMOC

High-speed acceleration – COMBOv2 hardware accelerator. Flexibility – Server PC box with monitoring software.

2

Security Operations Center.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 7 / 41

Part II Hardware Accelerated Monitoring Center (HAMOC)

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 8 / 41

HAMOC Goals

Makes use of hardware acceleration more user-friendly. Set of third-party tools tunned to work with COMBOv2. Use-cases and best practices how to work with COMBOv2.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 9 / 41

HAMOC Hardware – COMBOv2 Family

COMBOI-1G4 – 4x1 Gb/s COMBOI-10G2 – 2x10 Gb/s COMBOI-10G4TXT – 4x10 Gb/s

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 10 / 41

Firmware

NetCOPE – SDK for the COMBO Hardware Accelerator

TX RX ibuf

• buf

COMBOv2 card SZE

RAM

NETWORK

RX TX Port0 Interface sze0:0 Interface

PCI express BUS

BURST/SINGLE

16 KiB 16 KiB

packet transfer

64 MiB 64 MiB RX DMA TX DMA 16 KiB 16 KiB Acceleration core

RAM ... COMBOv2 with HANIC FW

ibuf

• buf

10 Gb/s Interface 1 ibuf

• buf

10 Gb/s Interface 0

16KiB

Hash-based Packet Distribution

16KiB 16KiB 16KiB

RX DMA 1 RX DMA 2 RX DMA 8 TX DMA 1 TX DMA 2 Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 11 / 41

Firmware

NetCOPE – SDK for the COMBO Hardware Accelerator

TX RX ibuf

• buf

COMBOv2 card SZE

RAM

NETWORK

RX TX Port0 Interface sze0:0 Interface

PCI express BUS

BURST/SINGLE

16 KiB 16 KiB

packet transfer

64 MiB 64 MiB RX DMA TX DMA 16 KiB 16 KiB Acceleration core

Hardware Accelerated NIC (HANIC) Firmware

RAM ... COMBOv2 with HANIC FW

ibuf

• buf

10 Gb/s Interface 1 ibuf

• buf

10 Gb/s Interface 0

16KiB

Hash-based Packet Distribution

16KiB 16KiB 16KiB

RX DMA 1 RX DMA 2 RX DMA 8 TX DMA 1 TX DMA 2 Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 11 / 41

Software Architecture

RX TX RX TX

sze0:1

libsze2 libpcap-sze sze0:0 sze0:1 built in modules szedata2 szedata2-cv2

sze0:0

ceth1 ceth0 SZE applications PCAP applications RAM

native ethernet interface

standard applications ceth1 ceth0 sze0:0 sze0:1 Linux TCP/IP stack control modules: combov2, combo6core libpcap FPGA

HANIC firmware

COMBO card

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 12 / 41

Remote Configuration

NETCONF Agent Local Configuration Datastore HAMOC Box #1 HAMOC Configuration Daemon NETCONF Manager Security Operations Center

SSH connection configuration data status information, notifications

NETCONF Agent Local Configuration Datastore HAMOC Box #N HAMOC Configuration Daemon

NETCONF Protocol Secured data transport over SSH (Secure Shell) version 2. XML data format. Event notifications capability. Separated configuration datastores:

startup, running, candidate.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 13 / 41

Connection to Network

pass 0 on 1 pass 1 on 0

HAMOC

Port 0 Port 1

HAMOC

Port 0 Port 1 TAP pass 2 on 1 pass 3 on 0 Forward to SW

In-line Mode 10 Gb/s TAP Mode 10 Gb/s

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 14 / 41

HAMOC – Test Results

2 4 6 8 10 12 64 128 256 512 800 1024 1280 1518 20 40 60 80 100 120

Throughput [Gb/s] CPU Load [%] Packet Length [B] HANIC Single Port Throughput Results at 10 Gb/s Ethernet

Measured Throughput Measured CPU Load Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 15 / 41

HAMOC Summary

Based on COMBOv2 hardware accelerators. Uses NetCOPE platform for rapid firmware development. Changing filtering rules without packet loss. Several API for applications (standard stack, PCAP, SZE2). Uses third party well known applications (e.g. Wireshark). Simple development of new applications. Remote configuration via NETCONF.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 16 / 41

Part III Use Cases – Deep Packet Inspection

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 17 / 41

Nanosecond Timestamps – I

Motivation COMBOv2 hardware supports nanosecond timestamps.

RAM NIC CPU Host Bridge NTP server

(tik.cesnet.cz)

GPS receiver

(GARMIN 18 LVC)

COMBO-LXT COMBOL-GPS

PCIe BUS PPS CLK+PPS

correction algorithm

tsuctl C L K

Problem libpcap library supports microsecond timestamps only. Wireshark supports nanoseconds PCAP file format.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 18 / 41

Nanosecond Timestamps – II

Solution sze2pcap tool – writes network traffic to wireshark nanosecond PCAP format with nanosecond precision.

RAM sze0:1 libsze2 libpcap-sze tcpdump, tshark, wireshark sze2pcap Port 0 Port 1 TAP sze0:0

timestamp ss.nsec timestamp ss.usec

SZE 10 Gb/s Ethernet

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 19 / 41

Nanosecond Timestamps – III

Usage $ sze2pcap -c 1000 -i 0 -w /tmp/dump.pcap $ wireshark /tmp/dump.pcap µs timestamps ← −

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 20 / 41

Nanosecond Timestamps – III

Usage $ sze2pcap -c 1000 -i 0 -w /tmp/dump.pcap $ wireshark /tmp/dump.pcap µs timestamps ← − ns timestamps ← −

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 20 / 41

Remote Packet Capture

client server

tcpdump wireshark

ADVANCED REMOTE CAPTURE FUNCTIONS LIBPCAP RPCAP API

rpcapd

LIBPCAP RAW Socket SZE2 ...

control data

COMBO - LXT Standard NIC

data packets control connection dynamically allocated data connection

COMMAND PORT (2002) DATA PORT (dynamically set) UDP/TCP

network

IP 10.0.0.5 IP 10.0.0.10

# tcpdump

• i rpcap://10.0.0.5/eth0

# rpcapd -n -b 10.0.0.5

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 21 / 41

Use Case – VoIP Analyzer

Captures control protocols (e.g. SIP, H.323 and H.248) and transport protocols (e.g. RTP, RTCP and SRTP). Uses Wireshark packet analyzer to analyze VoIP traffic. 10 Gb/s Ethernet Line HAMOC with SIP+RTP Filter Wireshark

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 22 / 41

Use Case – Snort over HAMOC

Sniffer – displaying network traffic. Packet Logger – saving display traffic to file. Network Intrusion Detection System – IDS. Inline Mode – Intrusion Prevention System – IPS. 8 parallel instances of Snort → performace increase.

RAM

RX DMA 1 RX DMA 2 RX DMA 8 TX DMA 1 TX DMA 2

...

Snort 2

...

Snort 1 Snort 8

CPU Cores Applications COMBOv2 with HANIC FW

ibuf

• buf

10 Gb/s Interface 1 ibuf

• buf

10 Gb/s Interface 0

16KiB

Hash-based Packet Distribution

NETWORK

16KiB 16KiB 16KiB

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 23 / 41

Part IV Use Cases – Advanced Flow Analyses

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 24 / 41

Flow Based Monitoring

Provides information about who communicates with whom, for how long, which protocol, how much data and so on. Based on CISCO NetFlow v5/v9 technology and IETF IPFIX. Enables you to watch your network traffic in real-time. GÉANT2 Security Toolset = FlowMon probe + NfSen. Detailed network view with NetFlow data.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 25 / 41

Use Case – NetFlow Principles

Useing FlowMon probe to generate NetFlow or IPFIX data.

Src and Dst Port Protocol Number Sum of Bytes Src and Dst IP Addr Number of Packets Timestamps TCP Flags, ... FROM 172.16.96.48:15094 TO 209.85.135.147:80 HTTP Request Browser Web HTTP Response FROM 209.85.135.147:80 TO 172.16.96.48:15094 Web Server

Flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Packets Bytes 09:41:21.763 0.101 TCP 172.16.96.48:15094

• > 209.85.135.147:80

.AP.SF 4 715 09:41:21.893 0.031 TCP 209.85.135.147:80

• > 172.16.96.48:15094

.AP.SF 4 1594 Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 26 / 41

Use Case – Web Access Analyzer

Only specific part of traffic is analyzed. Uses TAP or mirror port to get traffic to analyze. Uses httpry utility to analyze HTTP traffic.

User Web Server 172.16.30.15 www.evil.net www.angel.net www.evil.net virtual host www.angel.net virtual host

Timestamp Source-IP Dest-IP Method Host 2010-03-18 20:35:09 172.16.30.2 172.16.30.15 > GET www.angel.net 2010-03-18 20:35:24 172.16.30.2 172.16.30.15 > GET www.evil.net

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 27 / 41

Use Case – Tunneled IPv6 Traffic Monitoring

IPv6 is hidden inside IPv4 tunnel – possible security risk. Support for common IPv6 transition mechanisms (Teredo, 6to4, ISATAP). Exporting statistics of envelope IPv4 as well as of tunneled IPv6 traffic using modified NetFlow protocol.

IPv6 datagram IPv4 datagram

IPv4 network tunnel

IPv4 datagram IPv6 datagram IPv6 datagram IPv6 datagram

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 28 / 41

Network Behavioral Analysis

Full manual analysis of flow data is manually intensive. Naive, high speed attacks are easy to detect. Automated solution needed to detect sophisticated attacks. Incident analysis and reporting. Available approaches:

thresholds, trend analysis, attack-specific patterns, anomaly detection.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 29 / 41

Part V Use Cases – Network Defence

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 30 / 41

Network Defense

With acquired information you are able to do filtering and firewalling, network traffic splitting, packet sniffing, load balancing. That all at full linerate and without packet loss.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 31 / 41

Use Case – Network Protector

Auto-disconnects infected or enemy users from network. Transparent for good guys, leakproof for bad guys. Deployed as last device before network gateway.

Protected Network port 1 port 0 ceth2

HAMOC

network gateway eth1 ceth3 eth0 Internet iptables NAT web server

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 32 / 41

Use Case – Traffic Limiter

Limits specific traffic filtered by HAMOC firmware. Uses iptables traffic shaping features.

Network 1 port 1 port 0

HAMOC

Network 2 iptables traffic to limit non-limited traffic ceth2 ceth3

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 33 / 41

Part VI Even Chuck Norris Can’t Resist the Hardware Acceleration

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 34 / 41

Chuck Norris Botnet

Linux malware – IRC bots with central C&C servers. Attacks poorly-configured Linux MIPSEL devices. Vulnerable devices – ADSL modems and routers. Uses TELNET brute force attack as infection vector. Users are not aware about the malicious activities. Missing anti-malware solution to detect it. Discovered by hardware accelerated FlowMon probes at Masaryk University on 2 December 2009.

The botnet got the Chuck Norris moniker from a comment in source code: [R]anger Killato : in nome di Chuck Norris !

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 35 / 41

Botnet Size and Evaluation

Size estimation based on NetFlow data from Masaryk University. 33000 unique attackers (infected devices) from 10/2009 – 02/2010.

Most Infected ISPs Telefonica del Peru Global Village Telecom (Brazil) Turk Telecom Pakistan Telecommunication Company China Unicom Hebei Province Network

100000 200000 300000 400000 500000 Oct 1 Nov 1 Dec 1 Jan 1 Feb 1 Mar 1 Apr 1 500 1000 1500 2000 2500

Telnet Scans Against Masaryk University Network Unique Attackers

Telnet Scans Against Masaryk University Network

Unique attackers targeting the MU network Month Min Max Avr Mdn October 854 502 621 November 41 628 241 136 December 69 1321 366 325 January 9 1467 312 137 February 180 2004 670 560 Total 2004 414 354

Botnet stopped activity

• n 23 February 2010.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 36 / 41

Botnet Size and Evaluation

Size estimation based on NetFlow data from Masaryk University. 33000 unique attackers (infected devices) from 10/2009 – 02/2010.

Most Infected ISPs Telefonica del Peru Global Village Telecom (Brazil) Turk Telecom Pakistan Telecommunication Company China Unicom Hebei Province Network

100000 200000 300000 400000 500000 Oct 1 Nov 1 Dec 1 Jan 1 Feb 1 Mar 1 Apr 1 500 1000 1500 2000 2500

Telnet Scans Against Masaryk University Network Unique Attackers

Telnet Scans Against Masaryk University Network Unique Attackers

Unique attackers targeting the MU network Month Min Max Avr Mdn October 854 502 621 November 41 628 241 136 December 69 1321 366 325 January 9 1467 312 137 February 180 2004 670 560 Total 2004 414 354

Botnet stopped activity

• n 23 February 2010.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 36 / 41

Botnet Size and Evaluation

Size estimation based on NetFlow data from Masaryk University. 33000 unique attackers (infected devices) from 10/2009 – 02/2010.

Most Infected ISPs Telefonica del Peru Global Village Telecom (Brazil) Turk Telecom Pakistan Telecommunication Company China Unicom Hebei Province Network

100000 200000 300000 400000 500000 Oct 1 Nov 1 Dec 1 Jan 1 Feb 1 Mar 1 Apr 1 500 1000 1500 2000 2500

Telnet Scans Against Masaryk University Network Unique Attackers botnet discovery 2.12.2009 botnet shutdown 23.2.2010

Telnet Scans Against Masaryk University Network Unique Attackers

Unique attackers targeting the MU network Month Min Max Avr Mdn October 854 502 621 November 41 628 241 136 December 69 1321 366 325 January 9 1467 312 137 February 180 2004 670 560 Total 2004 414 354

Botnet stopped activity

• n 23 February 2010.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 36 / 41

Part VII Conclusion

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 37 / 41

Conclusion

Hardware acceleration enables reliable wirespeed traffic processing even in worst case scenarios – DoS/DDoS. NetCOPE platform allows rapid firmware development. Simple development of new applications due to PCAP API. Works even on encrypted and tunneled traffic. HAMOC is being deployed at CESNET network. The results of the research activities were transferred into spin-off company.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 38 / 41

Future Work

Extend portfolio of HAMOC applications. Adopt 40/100G Ethernet. Deploy HAMOC to more partners.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 39 / 41

Future Work

Extend portfolio of HAMOC applications. Adopt 40/100G Ethernet. Deploy HAMOC to more partners.

We are looking for new R&D partners.

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 39 / 41

Research and Development Background

R&D is held by CESNET (Czech NREN) in a frame of Optical National Research Network and its New Applications together with: Masaryk University Brno University of Technology Team has about 60 members (most of them are students) Hardware Software Testing Support

Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 40 / 41

Thank You For Your Attention

Jiří Novotný

novotny@ics.muni.cz

Pavel Čeleda

celeda@ics.muni.cz

Radek Krejčí

krejci@liberouter.org

Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks

This material is based upon work supported by grants from the Czech Ministry of Education, EU Funds, European Research Office of the US Army, Ministry of Defece and Armed Forces of the Czech Republic. Jiří Novotný et al. Hardware Acceleration: An Essential Part of Cyber Security in High-Speed Networks 41 / 41