melting the snow detecting snowshoe spam domains using
play

Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS - PowerPoint PPT Presentation

Dec 22, 2023 •34 likes •784 views

Olivier van der Toorn <o.i.vandertoorn@utwente.nl> November 13, 2018 University of Twente, Design and Analysis of Communication Systems NOMS 2018 Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS Measurements

  1. Olivier van der Toorn <o.i.vandertoorn@utwente.nl> November 13, 2018 University of Twente, Design and Analysis of Communication Systems NOMS 2018 Melting the Snow: Detecting Snowshoe Spam Domains Using Active DNS Measurements

  2. Introduction

  3. 1

  4. 1

  5. 1

  6. 2 Snowshoe Spam Internet

  7. 2 • few hosts Snowshoe Spam Internet Spam

  8. 2 • few hosts Snowshoe Spam Internet Spam • many messages per host

  9. 2 • few hosts Snowshoe Spam Internet Spam Snowshoe Spam • many hosts • many messages per host

  10. 2 • few hosts Snowshoe Spam Internet Spam Snowshoe Spam • many hosts • many messages per host • few messages per host

  11. 3 Assumption

  12. 4 Assumption

  13. ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’

  14. Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 ip4:217.72.207.0/27 -all Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’

  15. Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all

  16. Typical usage of SPF v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139;

  17. v=spf1 ip4:213.165.64.0/23 ip4:74.208.5.64/26 ip4:74.208.122.0/26 ip4:212.227.126.128/25 ip4:212.227.15.0/24 ip4:212.227.17.0/27 ip4:74.208.4.192/26 ip4:82.165.159.0/24 Received-SPF: fail (google.com: domain of paypoint_sanchez@consultant.com does not v=spf1 a mx ip4:167.160.22.0/24 -all 5 Assumption: Background SPF record from ‘consultant.com’ ip4:217.72.207.0/27 -all Email from ‘paypoint_sanchez@consultant.com’ designate 103.10.4.139 as permitted sender) client-ip=103.10.4.139; Typical usage of SPF

  18. 6 While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Hypothesis

  19. Snowshoe spam + SPF While snowshoe spammers are hard to detect, but still leave a trace in the DNS. 6 Hypothesis

  20. Snowshoe spam + SPF While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Many hosts + a DNS record for each host or a long SPF record 6 Hypothesis

  21. While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Snowshoe spam + SPF Many hosts + a DNS record for each host or a long SPF record 6 Hypothesis Domain with many records or long SPF records

  22. While snowshoe spammers are hard to detect, but still leave a trace in the DNS. Snowshoe spam + SPF Many hosts + a DNS record for each host or a long SPF record Domain with many records or long SPF records Active DNS measurements are a good way to detect snowshoe spam domains. 6 Hypothesis

  23. Methodology

  24. OpenINTEL 7 (DNS data source) Overview

  25. 8 OpenINTEL (DNS data source) Machine Learning (processing) Overview

  26. 8 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) Overview

  27. 8 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Overview

  28. • Queries more than 60% of registered domain names (in total more than 206 million) • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background

  29. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  30. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  31. • Active DNS measurement platform • A • AAAA • MX • NS • … • Every 24 hours a measurement is started 9 OpenINTEL: Background • Queries more than 60% of registered domain names (in total more than 206 million)

  32. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  33. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  34. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  35. 37 features • Simple: number of MX addresses • Complex: number of IP addresses inside an SPF record These features are not computed for every domain in OpenINTEL. 10 OpenINTEL: Datasets & Features

  36. 11 OpenINTEL: Long Tail Analysis

  37. 12 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Machine Learning

  38. The performance of each classifier is compared based on the precision metric. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. Precision True Positives True Positives False Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms

  39. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. True Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms The performance of each classifier is compared based on the precision metric. Precision = True Positives + False Positives

  40. We have trained and evaluated 12 Machine Learning algorithms. • Training dataset from domains on the long tail which appear in known blacklists. True Positives Selected the ‘AdaBoost’ classifier as our classifier of choice, since it had the highest precision (98% with a FPR of 1%). 13 Machine Learning: 12 algorithms The performance of each classifier is compared based on the precision metric. Precision = True Positives + False Positives

  41. 14 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) Realtime Blackhole List (RBL)

  42. 15 OpenINTEL (DNS data source) Machine Learning (processing) Realtime Blackhole List (storage) SURFnet (validation) SURFnet

  43. We have selected the AdaBoost classifier as our classifier of choice, since it had the highest precision metric. The results of our daily detections are stored in an RBL. Active DNS measurements of more than 60% of registered domain names forms the source of our data. We filter out large domains via the Long Tail Analysis. We have evaluated the RBL in SURFmailfilter. 16 Methodology: Recap

  44. The results of our daily detections are stored in an RBL. Active DNS measurements of more than 60% of registered domain names forms the source of our data. We filter out large domains via the Long Tail Analysis. We have evaluated the RBL in SURFmailfilter. 16 Methodology: Recap We have selected the AdaBoost classifier as our classifier of choice, since it had the highest precision metric.

Download Presentation
Download Policy: The content available on the website is offered to you 'AS IS' for your personal information and use only. It cannot be commercialized, licensed, or distributed on other websites without prior consent from the author. To download a presentation, simply click this link. If you encounter any difficulties during the download process, it's possible that the publisher has removed the file from their server.

Recommend

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der Toorn February 1, 2018 University of Twente, Design and Analysis of Communication Systems Introduction Olivier

1.46k views • 90 slides
Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der

Melting the Snow Using Active DNS Measurements to Detect Snowshoe Spam Domains Olivier van der Toorn November 13, 2018 University of Twente, Design and Analysis of Communication Systems Introduction Olivier

825 views • 71 slides
Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt;

Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt;

Combating Snowshoe Spam with Fire Olivier van der Toorn &lt;o.i.vandertoorn@utwente.nl&gt; November 13, 2018 University of Twente, Design and Analysis of Communication Systems ICT OPEN 2018 Overview Introduction Methodology Results

1.05k views • 55 slides
Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why

Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why

Detecting Product Review Spammers using Rating Behaviors Itay Dressler What is Spam? Why should you care? How to detect Spam? 2 What is Spam? What is Spam? All forms of malicious manipulation of user generated data so as to

943 views • 56 slides
Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a

Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a

Spam, Spam, Spam Why is spam interesting? Everyone can observe spam. Spam / Anti-spam is a highly evolved form of information warfare. Fascinating socioeconomic study with many players - users, ISPs, spammers, technologists, legal

322 views • 17 slides
WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy,

WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy,

WITCH A new algorithm for detecting Web spam using page features and hyperlinks Jacob Abernethy, UC Berkeley (Thanks to Yahoo! Research for two internships and a fellowship!) Joint work with Olivier Chapelle and Carlos Castillo (Chato) from

514 views • 28 slides
Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 ,

Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 ,

Query-log mining for detecting spam queries Carlos Castillo 1 , Claudio Corsi 2 , Debora Donato 1 , Paolo Feraggina 2 , Aristides Gionis 1 1 Yahoo! Research Labs, Barcelona, Spain 2 University of Pisa, Italy motivation Query logs provide valuable

547 views • 16 slides
Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam

Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam

Spam Fighting at CERN 28 April 2004 Emmanuel Ormancey 1 What is Spam ? What is Spam ? Spam is the friendly name given to unsolicited mail everyone receives in the mailbox. Comes from a Monty Python sketch, where in a caf everything

505 views • 13 slides
Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang

Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang

Detecting Spammers with SNARE: Spatio-temporal Network-level Automatic Reputation Engine Shuang Hao, Nadeem Ahmed Syed, Nick Feamster, Alexander G. Gray, Sven Krasser Motivation Spam: More than Just a Nuisance Spam: Ham: unsolicited bulk

523 views • 28 slides
Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All

Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All

Opinion Spam and Analysis NITIN JINDAL &amp; BING LIU, WSDM 08 UIUC Opinion/Review Spam All spam is spam but some spam is more spam than others Opinion spam similar to web spam or email spam in intent, but different in form / content Web

912 views • 38 slides
The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation

The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation

The CAN-SPAM Act of 2003 D E C E M B E R 2 0 0 3 THE CAN-SPAM ACT OF 2003 Status of Legislation When Congress returns from recess on December 8, 2003, it will finish its work on the CAN-SPAM Act of 2003 (Controlling the Assault of

323 views • 4 slides
Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to

Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to

Link Spam Alliances Zoltn Gyngyi Hector Garcia-Molina Class List Spam 101 Intro to web spam Spam 221 Spamming PageRank Spam farm model Optimal farm structure Alliances of two farms Larger alliances Spam

876 views • 30 slides
Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and

Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and

Web Dynamics Web Spam Web Spam Dr. Marc Spaniol Dr. Marc Spaniol Saarbrcken, June 24, 2010 Databases and Information Systems Prof. Dr. G. Weikum MPII-Sp-0710-1/49 Agenda Web spam - Why and what? Web Spam - Spam taxonomy

669 views • 49 slides
Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information

Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information

Web Dynamics Web Spam Web Spam Marc Spaniol Marc Spaniol Saarbrcken, July 23, 2009 Databases and Information Systems Prof. Dr. G. Weikum MPII-Sp-0709-1/49 Agenda Web spam - Why and what? Web Spam - Spam taxonomy Overview

806 views • 49 slides
Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid.

Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid.

Snow Ice crystals come together in a cloud to make a snowflake . Snow is a solid. The most is in the North Pole. Sleet Rain that is partly frozen or mixed with snow is sleet. Sleet falls faster then snow.

486 views • 4 slides
Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency?

Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency?

Snow and Ice Control Update October 22, 2018 Comments from last winter Why no snow emergency? Address priority areas sooner? Start earlier? Clear snow from driveways? 2018-19 Updates Added 8 miles of snow emergency route

103 views • 7 slides
Introduction to Machine Learning CMU-10701 3. Bayes classification Barnabs Pczos &amp; Aarti

Introduction to Machine Learning CMU-10701 3. Bayes classification Barnabs Pczos &amp; Aarti

Introduction to Machine Learning CMU-10701 3. Bayes classification Barnabs Pczos &amp; Aarti Singh 2014 Spring What about prior knowledge ? (MAP Estimation) 2 What about prior knowledge, Domain knowledge, expert knowledge We know the

746 views • 39 slides
CS615 - Aspects of System Administration HTTPS, TLS, SMTP Department of Computer Science Stevens

CS615 - Aspects of System Administration HTTPS, TLS, SMTP Department of Computer Science Stevens

CS615 - Aspects of System Administration Slide 1 CS615 - Aspects of System Administration HTTPS, TLS, SMTP Department of Computer Science Stevens Institute of Technology Jan Schaumann jschauma@stevens.edu https://stevens.netmeister.org/615/

902 views • 72 slides
A Forecasting Competition: First Results Michael Binder 1 , Mtys Farkas 2 , Zexi Sun 1 , John

A Forecasting Competition: First Results Michael Binder 1 , Mtys Farkas 2 , Zexi Sun 1 , John

3rd Conference of the Macroeconomic Modelling and Model Comparison Network June 13, 2019 A Forecasting Competition: First Results Michael Binder 1 , Mtys Farkas 2 , Zexi Sun 1 , John Taylor 3 , Volker Wieland 1 , Maik Wolters 4 1 Goethe

343 views • 19 slides
Sufi Mohamed Agile Tour Zrich 2019 22.11.2019 Retrospecting the retrospective A

Sufi Mohamed Agile Tour Zrich 2019 22.11.2019 Retrospecting the retrospective A

Prospect &amp; Adapt Sufi Mohamed Agile Tour Zrich 2019 22.11.2019 Retrospecting the retrospective A problem-focused scrum event Photo: @lionrevolt via Twenty20 21.11.19 www.spf-consulting.ch 2 Mini. Regardless of what we discover, we

287 views • 16 slides
OpenINTEL an infrastructure for long-term, large-scale and high-performance active DNS

OpenINTEL an infrastructure for long-term, large-scale and high-performance active DNS

OpenINTEL an infrastructure for long-term, large-scale and high-performance active DNS measurements DACS DACS Design and Analysis of Communication Systems Why measure DNS? (Almost) every networked service relies on DNS DNS

773 views • 15 slides
Constructing Strictly Positive Types A Joint venture of the Nottingham Container Consortium and

Constructing Strictly Positive Types A Joint venture of the Nottingham Container Consortium and

Constructing Strictly Positive Types A Joint venture of the Nottingham Container Consortium and the Epigram Team Peter Morris w/ Thorsten Altenkirch pwm@cs.nott.ac.uk University of Nottingham Constructing Strictly Positive Types p. 1/10

588 views • 27 slides
Uninterpreted Functions: Their use in Code Transformation Catherine Olschanowsky Mary Hall

Uninterpreted Functions: Their use in Code Transformation Catherine Olschanowsky Mary Hall

Uninterpreted Functions: Their use in Code Transformation Catherine Olschanowsky Mary Hall Michelle Strout CHiLL-I/E Team, Collaborators, and Funding Mary Hall (University of Utah) Michelle Strout (University of Arizona) Catherine

324 views • 10 slides
When to take the option When to exercise discretion When to get an opinion When to get something

When to take the option When to exercise discretion When to get an opinion When to get something

10/18/2019 UFLC 2019 Ashley Voss, CPA Ratliff &amp; Associates Options &amp; Opinions: The Art of Accounting 1 Session Summary When to take the option When to exercise discretion When to get an opinion When to get something else 2 1

248 views • 9 slides

More recommend