SIDN Domain name registry for .nl ccTLD > 5,6 million domain - - PowerPoint PPT Presentation

▶
sidn
SMART_READER_LITE
LIVE PREVIEW

SIDN Domain name registry for .nl ccTLD > 5,6 million domain - - PowerPoint PPT Presentation

Aug 20, 2023 299 likes •521 views

Klik om de s+jl te bewerken Klik om de models+jlen te bewerken Tweede niveau TLD Data Analysis Derde niveau Vierde niveau Vijf jfde niveau ICANN Tech Day, Dublin October 19th 2015 Maarten Wullink, SIDN Wie zijn wij? | Mijlpalen

slide-1
SLIDE 1

Klik om de s+jl te bewerken

Klik om de models+jlen te bewerken § Tweede niveau § Derde niveau § Vierde niveau Vijf jfde niveau Wie zijn wij? | Mijlpalen | Organisa@e | Het huidige internet | Missie - Visie | Diensten | Referen@es | SamenvaJng 1

ICANN Tech Day, Dublin

October 19th 2015 Maarten Wullink, SIDN

TLD Data Analysis

slide-2
SLIDE 2

SIDN

  • Domain name registry for .nl ccTLD
  • > 5,6 million domain names
  • 2,46 million domain names secured with DNSSEC
  • SIDN Labs is the R&D team of SIDN
slide-3
SLIDE 3

DNS Data @SIDN

  • > 3.1 million dis@nct resolvers
  • > 1.3 billion query's daily
  • > 300 GB of PCAP data daily
slide-4
SLIDE 4

ENTRADA

  • Goal: data-driven improved security & stability of .nl and the Internet at large
  • Problem: Exis@ng solu@ons for analyzing network data do not work well with large datasets and have

limited analy@cal capabili@es.

  • Main requirement: high-performance, near real-@me data warehouse
  • Approach: avoid expensive pcap analysis:
  • Convert pcap data to a performance-op@mized format (key)
  • Perform analysis with tools/engines that leverage that

ENhanced Top-Level Domain Resilience through Advanced Data Analysis

slide-5
SLIDE 5

Focussed on increasing the security and stability of .nl

Use Cases

  • Visualize DNS pagerns (visualize traffic pagerns for phishing domain names)
  • Detect botnet infec@ons
  • Real-@me Phishing detec@on
  • Sta@s@cs (stats.sidnlabs.nl)
  • Scien@fic research (collabora@on with Dutch Universi@es)
  • Opera@onal support for DNS operators
slide-6
SLIDE 6

Example Applica@ons

  • DNS security scoreboard
  • Resolver reputa@on
slide-7
SLIDE 7

DNS Security Scoreboard

Goal: Visualize DNS pagerns for malicious ac@vity How: Combine external phishing feeds with DNS data

slide-8
SLIDE 8

Web UI Security feed I Security feed II Event Analyzer Hadoop PostgreSQL

new event new event save enriched event retrieve event data

REST API

Architecture

slide-9
SLIDE 9

Traffic Visualiza@on

slide-10
SLIDE 10

Resolver Reputa@on (RESREP)

Goal: Try to detect malicious ac@vity by assigning reputa@on scores to resolvers How: “fingerprin@ng” resolver behaviour

slide-11
SLIDE 11

RESREP Concept

Malicious ac@vity:

  • Spam-runs
  • Botnets like Cutwail
  • DNS-amplifica@on agacks

ISP Resolvers DNS ques@ons and responses Authorita@ve DNS .nl .nl Registry

slide-12
SLIDE 12

RESREP Architecture

Resolvers Root operator Child operator (example.nl)

 Œ   Ž

ISP network User www.example.nl

‘ ’ “ ” HTTP

ENTRADA Plaqorm AbuseHUB Abusedesk RESREP Privacy Policy .nl Privacy Board RESREP service



slide-13
SLIDE 13

ENTRADA Architecture

  • ‘DNS big data’ system
  • Goal: develop applica@ons and services that

further enhance the security and stability of .nl, the DNS, and the Internet at large

  • ENTRADA main components
  • Applica@ons and services
  • Plaqorm and data sources
  • Privacy framework
  • Plaqorm + privacy framework = ENTRADA plumbing
slide-14
SLIDE 14

ENTRADA Privacy Framework

  • Part of the “ENTRADA plumbing”
  • Key concepts
  • Applica@on-specific privacy policy
  • Privacy Board
  • Enforcement Points
  • Policy elements include
  • Purpose
  • Data used
  • Filters
  • Reten@on period
  • Type of applica@on (R&D vs. produc@on)

PEP#G% PEP#A% PEP#O% PEP#V% PEP#G% PEP#A% PEP#O% PEP#V% PEP#G% PEP#A% PEP#O% PEP#V% .nl%nameservers% DNS%query’s%en%responses% resolvers% Privacy% Board% Auteur% (Ontwikkelaar% toepassing%T1)% Aanpassingen% Concept% policy% voor%T1% Policy% voor%T1% Template% R&D% licenJe% ENTRADA%data%plaKorm%(technisch)% Juridisch%en%organisatorisch% ENTRADA%privacyraamwerk% toepassingssilos%

Database%queries% DNS%packets%(PCAP)%

Security%en%stability%% services%en%dashboards% Data#analyse%% algoritmes% Opslag% Verzameling%

T1% T2% TN%

slide-15
SLIDE 15

ENTRADA Technical Architecture

ENTRADA-specific components Open source Hadoop (generic components) HDFS IMPALA Support DNS Library PCAP Conversion Workflow Services Parquet ENTRADA plaqorm

slide-16
SLIDE 16

Workflow

Query data available for analysis within 10 minutes name server PCAP staging PCAP decode Import

Hadoop

Parquet Impala

Analyst

Enrich Join Filter Monitoring Metrics Applica@on Y Applica@on X

slide-17
SLIDE 17

Performance

1 Year of data is 2.2TB Parquet ~ 52TB of PCAP

select concat_ws(’-’,day,month,year), count(1) from dns.queries where ipv=4 group by concat_ws(’-’,day,month,year) Example query, count # ipv4 queries per day. Query response @mes

slide-18
SLIDE 18

Name server feeds 2 Queries per day ~320M Daily PCAP volume(gzipped) ~70GB Daily Parquet volume ~14GB Months opera@onal 18 Total # queries stored > 74B Total Parquet volume > 3TB HDFS (3x replica@on) > 9TB Cluster capacity ~150B-200B tuples

ENTRADA Status

slide-19
SLIDE 19

Conclusions

Technical:

  • Hadoop HDFS + Parquet + Impala is a winning combina@on!

Contribu@ons:

  • Research by SIDN Labs and universi@es
  • Iden@fied malicious domain names and botnets
  • External data feed to the Abuse Informa@on Exchange
  • Insight into DNS query data
slide-20
SLIDE 20

Future Work

  • Combine data from .nl authorita@ve name server with scans of the

complete .nl zone and ISP data.

  • Get data from more name servers and resolvers
  • Expand Open Data program
slide-21
SLIDE 21

Ques@ons and Feedback

Maarten Wullink Senior Research Engineer maarten.wullink@sidn.nl @wulliak www.sidnlabs.nl

hgps://stats.sidnlabs.nl