SLIDE 1
Search + Discovery Peter Bourgon Evolution of search Relevance - - PowerPoint PPT Presentation
Search + Discovery Peter Bourgon Evolution of search Relevance - - PowerPoint PPT Presentation
Search + Discovery Peter Bourgon Evolution of search Relevance ranking A bit about SOA EVOLUTION OF SEARCH KATAMARI DAMACY, NAMCO HOW TO FIX? HOW TO FIX? ElasticSearch HOW TO FIX? ElasticSearch Greenfield development HOW TO FIX?
SLIDE 2
SLIDE 3
EVOLUTION OF SEARCH
SLIDE 4
SLIDE 5
SLIDE 6
SLIDE 7
SLIDE 8
KATAMARI DAMACY, NAMCO
SLIDE 9
SLIDE 10
HOW TO FIX?
SLIDE 11
HOW TO FIX?
ElasticSearch
SLIDE 12
HOW TO FIX?
ElasticSearch Greenfield development
SLIDE 13
HOW TO FIX?
ElasticSearch Greenfield development Parallel dark launch
SLIDE 14
HOW TO FIX?
ElasticSearch Greenfield development Parallel dark launch Migrate traffic slowly
SLIDE 15
HOW TO FIX?
ElasticSearch Greenfield development Parallel dark launch Migrate traffic slowly Scale out
SLIDE 16
SLIDE 17
WHY ELASTICSEARCH
SLIDE 18
WHY ELASTICSEARCH
Good API
SLIDE 19
WHY ELASTICSEARCH
Good API Clear path for growth
SLIDE 20
WHY ELASTICSEARCH
Good API Clear path for growth Batteries included
SLIDE 21
WHY ELASTICSEARCH
Good API Clear path for growth Batteries included Works like you expect it to
SLIDE 22
COMPONENTS
SLIDE 23
RELEVANCE RANKING
SLIDE 24
PageRank
SLIDE 25
DiscoRank
SLIDE 26
SLIDE 27
DISCORANK DETAILS
SLIDE 28
DISCORANK DETAILS
Calculations done offline
SLIDE 29
DISCORANK DETAILS
Calculations done offline Serialize to compact array
SLIDE 30
DISCORANK DETAILS
Calculations done offline Serialize to compact array Load in ES custom scorer
SLIDE 31
DISCORANK DETAILS
Calculations done offline Serialize to compact array Load in ES custom scorer “Hybrid” 2-phase queries
SLIDE 32
DISCORANK DETAILS
Calculations done offline Serialize to compact array Load in ES custom scorer “Hybrid” 2-phase queries Recalculate/reload every N
SLIDE 33
SLIDE 34
DISCORANK++
SLIDE 35
DISCORANK++
+ Time component → “Hotness”
SLIDE 36
DISCORANK++
+ Time component → “Hotness” + User behavior → “Trust”
SLIDE 37
THE BIGGER PICTURE
SLIDE 38
SLIDE 39
SLIDE 40
INDEXING
Single-purpose application(s) Stateless and idempotent Build, iterate, iterate, iterate Full catalog ~= 1 hour
SLIDE 41
SLIDE 42
SEARCHING
Single-purpose application Stateless and idempotent SC/ES DSL translation Open-source ES library
SLIDE 43
SLIDE 44
EXPLORING
Single-purpose application “Stateless” and idempotent Special, complex searches Cached on regular interval
SLIDE 45
SERVICE ORIENTED ARCHITECTURE
SLIDE 46
SLIDE 47
SOA AT SOUNDCLOUD
SLIDE 48
SOA AT SOUNDCLOUD
Bazooka platform
SLIDE 49
SOA AT SOUNDCLOUD
Bazooka platform API as firewall
SLIDE 50
SOA AT SOUNDCLOUD
Bazooka platform API as firewall 12 Factor applications
SLIDE 51
SLIDE 52
12 FACTOR APPS
SLIDE 53
12 FACTOR APPS
Single codebase (repository)
SLIDE 54
12 FACTOR APPS
Single codebase (repository) Config stored in environment
SLIDE 55
12 FACTOR APPS
Single codebase (repository) Config stored in environment Backing services as resources
SLIDE 56
12 FACTOR APPS
Single codebase (repository) Config stored in environment Backing services as resources Stateless
SLIDE 57
12 FACTOR APPS
Single codebase (repository) Config stored in environment Backing services as resources Stateless Horizontal scaling with processes
SLIDE 58
SLIDE 59
WHY SOA?
SLIDE 60
WHY SOA?
Velocity
SLIDE 61
WHY SOA?
Velocity Developer happiness
SLIDE 62
WHY SOA?
Velocity Developer happiness Distributed systems are complex
SLIDE 63
ElasticSearch core + SOA infrastructure
SLIDE 64