Digital Intelligence Gathering Using The Powers Of OSINT For Both - PowerPoint PPT Presentation

Digital Intelligence Gathering Using The Powers Of OSINT For Both Blue And Red Teams BSidesSF February 2016 1 / 71

Ethan Dodge  DFIR @ Nuna Health.  DFIR professional and perpetual learner.  @__eth0  dodgesec.com 2 / 71

Nuna Health  We work with the government and self-insured employers to understand and improve how people use healthcare. 3 / 71

Nuna Health  We work with the government and self-insured employers to understand and improve how people use healthcare.  Security is the foundation of our culture and products. 4 / 71

Nuna Health  We work with the government and self-insured employers to understand and improve how people use healthcare.  Security is the foundation of our culture and products.  We're accepting resumes! 5 / 71

 OSINT 6 / 71

What is OSINT? Using information available to everyone to gather intelligence 7 / 71

What is OSINT? Using information available to everyone to gather intelligence  Social Networks 8 / 71

What is OSINT? Using information available to everyone to gather intelligence  Social Networks  Public Data Records 9 / 71

What is OSINT? Using information available to everyone to gather intelligence  Social Networks  Public Data Records  Leaked Customer Data 10 / 71

11 / 71

12 / 71

Why OSINT?  Private Investigators/Detectives  Investigative Journalism  Criminal Activity/Law Enforcement  Threat Intelligence 13 / 71

14 / 71

15 / 71

DISCLAIMER 16 / 71

Basic Work � ow Identify Source  Identify possible sources of intel  Validate  Automate 17 / 71

Basic Work � ow Identify Source  Does it apply to our target? Analyze  Determine probability  Apply con � dence  Generate new potential sources 18 / 71

Basic Work � ow Identify Source  Add context to target Analyze Enrich  Add probability, con � dence level to details  Develop narrative 19 / 71

 Maltego 20 / 71

Mal...what? Link Analysis Visualization Tool  Enrich entity with other sources of information automatically  Identify relationships between entities  Visualize relationships 21 / 71

Common Terms  Entities 22 / 71

Common Terms  Entities  Transforms 23 / 71

Common Terms  Entities  Transforms  Machine 24 / 71

Transform Example 25 / 71

Transform Development Primer from MaltegoTransform import * 26 / 71

Transform Development Primer me = MaltegoTransform() me.parseArguments(sys.argv) location = sys.argv[1] 27 / 71

Transform Development Primer ent = me.addEntity("maltego.Location","DNA Lounge") me.returnOutput() 28 / 71

 Gavel  Custom maltego transform we developed. 29 / 71

 Gavel  Custom maltego transform we developed.  Digs up court case records from individual states. 30 / 71

 Gavel  Custom maltego transform we developed.  Digs up court case records from individual states.  Tons of sensitive information. 31 / 71

 Gavel  Custom maltego transform we developed.  Digs up court case records from individual states.  Tons of sensitive information.  https://github.com/brianwarehime/gavel 32 / 71

Gavel Example 33 / 71

 Story Time 34 / 71

Ever seen this? 35 / 71

Or this? 36 / 71

37 / 71

Twitter Data  Start with best source of data - Twitter 38 / 71

Twitter Data  Start with best source of data - Twitter  We needed a way to parse through all the data 39 / 71

Twitter Data  Start with best source of data - Twitter  We needed a way to parse through all the data  We identi � ed it, validated it, now we analyze... 40 / 71

Get the tweets def download_tweets (screen_name,number_of_tweets,max_id=None): api_url = "%s/statuses/user_timeline.json?" % base_twitter_url api_url += "screen_name=%s&" % screen_name api_url += "count=%d" % number_of_tweets if max_id is not None : api_url += "&max_id=%d" % max_id # send request to Twitter response = requests.get(api_url,auth=oauth) if response.status_code == 200: tweets = json.loads(response.content) return tweets 41 / 71

All the tweets! def download_all_tweets (username): full_tweet_list = [] max_id = 0 tweet_list = download_tweets(username,200) oldest_tweet = tweet_list[::-1][0] while max_id != oldest_tweet['id']: full_tweet_list.extend(tweet_list) max_id = oldest_tweet['id'] time.sleep(3) tweet_list = download_tweets(username,200,max_id-1) if len(tweet_list): oldest_tweet = tweet_list[-1] full_tweet_list.extend(tweet_list) return full_tweet_list 42 / 71

43 / 71

Where the tweets at? for tweet in tweets: templist = [] if tweet.has_key("geo") and tweet['geo']: latitude,longitude = tweet['geo'].get("coordinates") r = requests.get("http://maps.googleapis.com/maps/api/geocode/json? latlng="+str(latitude)+","+str(longitude)+"&sensor=true") res = json.loads(r.text) try : for i in res['results'][0]['address_components']: if "neighborhood" in i['types'] or "administrative_area_level_2" in i['types'] or "postal_code_suffix" in i['types'] or "country" in i['types'] or "postal_code" in i['types']: pass else : templist.append(i['long_name']) except : pass 44 / 71

Enriching Maltego last = Counter(newlist).most_common(5) x = 5 for address in last: ent = me.addEntity("maltego.Location",address[0]) ent.addAdditionalFields('link#maltego.link.thickness','','',x) x = x - 1 45 / 71

Transform in Action 46 / 71

Then we found this... 47 / 71

28702eb195e8b292ceb8c5eb4d8b94d05b6c2a62 SHA1 : 3nd3rwiggin 48 / 71

49 / 71

Unique Username = Win 50 / 71

51 / 71

52 / 71

What we found: Personal  Home address (Twitter & Etsy)  Class locations (Twitter)  Password (Have I Been Pwnd?)  Close Friends (Twitter & Instagram)  Job History (LinkedIn & Facebook)  Home IP Address (Reddit Login History)  Birthdate (Etsy)  Barber (Twitter) 53 / 71

What we found: Personal  Addresses (Whitepages & Property Records) Family  Members (Google+)  Names (Maltego) 54 / 71

 Use Cases Red Team 55 / 71

56 / 71

57 / 71

 Use Cases Blue Team 58 / 71

WALK THE LINE 59 / 71

Blue Team Twitter  See if public activity is malicious  Following with competitors?  Talking with competitors?  Talking about your brand? 60 / 71

Blue Team Twitter  Work badges Instagram  Passwords  Network Diagrams 61 / 71

Blue Team Twitter  Committed sensitive � les Instagram  Committed proprietary code Github  Committed company info 62 / 71

Blue Team Twitter  See if public activity is malicious Instagram  Friends with competitors? Github  Talking about your brand? Facebook 63 / 71

Blue Team Twitter  Scumblr by Net � ix Instagram  Monitor Forum Chatter Github  Monitor Your Name Facebook Brand Monitoring 64 / 71

Blue Team Twitter  Most is going to be accidental Instagram  Who's your most active employee? Github  Monitor them closer Facebook Brand Monitoring Rate Employees 65 / 71

Blue Team Twitter  Alert Instagram  Correlate Github Facebook Brand Monitoring Rate Employees SEIM 66 / 71

Interrogator  Web Application  Continuous OSINT Monitoring of Workforce  Visualize relationships with a Graph Database  Coming mid 2016! 67 / 71

68 / 71

Reccomendations  Justin Seitz - @jms_dot_py  The Grugq - @thegrugq  automatingosint.com  bellingcat.com 69 / 71

 Q&A 70 / 71

Ethan Dodge  @__eth0  ethan@nuna.com  dodgesec.com 71 / 71