Neo Technology, Inc Confidential
Neo4j Introduction No Sql Search Roadshow
Dirk Möller dirk.moeller@neotechnology.com
Donnerstag, 19. September 13
Neo4j Introduction No Sql Search Roadshow Dirk Mller - - PowerPoint PPT Presentation
Neo4j Introduction No Sql Search Roadshow Dirk Mller dirk.moeller@neotechnology.com Neo Technology, Inc Confidential Donnerstag, 19. September 13 Graphs are everywhere 2 Donnerstag, 19. September 13 The Graph Trend Neo Technology, Inc
Neo Technology, Inc Confidential
Dirk Möller dirk.moeller@neotechnology.com
Donnerstag, 19. September 13
2
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Evolution of Web Search
WWW Indexing
Discrete Data
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Evolution of Web Search
WWW Indexing
Discrete Data
Google Invents PageRank
Connected Data (Simple)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Evolution of Web Search
WWW Indexing
Discrete Data
Google Invents PageRank
Connected Data (Simple)
Google Knowledge Graph, Facebook Graph Search
Connected Data (Rich)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Evolution of Online Job Search
Resume Searching & Scoring
Discrete Data
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Evolution of Online Job Search
Resume Searching & Scoring
Discrete Data
Social Job Search
Connected Data
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
(Actual Neo4j Graphs)
CMDB (Network Inventory)
Social Networks Customer & Employee
Organizational Hierarchy Product Subscriptions
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
(Actual Neo4j Graphs)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management
(Actual Neo4j Graphs)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management
(Actual Neo4j Graphs)
Insurance Risk Analysis
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management Geo Routing
(Public Transport)
(Actual Neo4j Graphs)
Insurance Risk Analysis
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management
Network Cell Analysis
Geo Routing
(Public Transport)
(Actual Neo4j Graphs)
Insurance Risk Analysis
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management Network Asset Management
Network Cell Analysis
Geo Routing
(Public Transport)
(Actual Neo4j Graphs)
Insurance Risk Analysis
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Entitlements & Identity Management Network Asset Management
Network Cell Analysis
Geo Routing
(Public Transport)
BioInformatics
(Actual Neo4j Graphs)
Insurance Risk Analysis
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
The Company
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
The Company
The Product
Donnerstag, 19. September 13
13
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
performance for complex, connected data
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
performance for complex, connected data
cycles
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
performance for complex, connected data
cycles
model
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
performance for complex, connected data
cycles
model
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Load Balancer Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Specialized Hardware Load Balancer Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Specialized Hardware Load Balancer Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
System Monitoring & Management Specialized Hardware Load Balancer Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
EAI
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
EAI
ETL
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
EAI
Indexing Providers
ETL
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
EAI
Indexing Providers
ETL
Bulk Analytic Infrastructure
(e.g. Graph Compute Engine)
ETL
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Application
Cache
System Monitoring & Management Specialized Hardware Load Balancer
Message Queue
Neo4j Cluster End User
Graph Visualization
Other Databases
EAI
Indexing Providers
ETL
*(Runtime)
Bulk Analytic Infrastructure
(e.g. Graph Compute Engine)
ETL
Reporting
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Core Industries & Use Cases: Web / ISV Finance & Insurance Datacom / Telecom Network & Data Center Management MDM Social Geo
(What we expected to happen - view from several years ago)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
*Community Users Not Included
Core Industries & Use Cases: Web / ISV Financial Services Telecomm- unications Network & Data Center Management MDM Social Geo
Select Commercial Customers* Across Anticipated Segments
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Core Industries & Use Cases: Web / ISV Finance & Insurance Telecomm- unications Network & Data Center Management MDM Social Geo
Select Commercial Customers* Across Anticipated Segments
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Core Industries & Use Cases: Web / ISV Finance & Insurance Telecomm- unications Network & Data Center Management MDM Social Geo
Select Commercial Customers* Across Anticipated Segments
Core Industries & Use Cases:
Software Financial Services
Telecomm unications
Health Care & Life Sciences
Web Social, HR & Recruiting
Media & Publishing
Energy, Services, Automotive, Gov’t, Logistics, Education, Gaming, Other
Network & Data Center Management MDM / System of Record
Social Geo Recommend- ations Identity & Access Mgmt Content Management
BI, CRM, Impact Analysis, Fraud Detection, Resource Optimization, etc.
Accenture
Finance Energy Aerospace
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
(e.g. Network Dependency Analysis, Network Inventory, etc.)
(mobile apps, social recommendations, collaboration)
(creating inferred social graph, churn reduction, etc.)
(org & product hierarchy, data governance, IAM)
(enterprise collaboration)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
(e.g. Fraud Detection, Credit Risk Analysis, Chargebacks...)
(org drillthru, product recommendations, mobile payments, etc.)
(identity & access management, authorization)
(portfolio analytics, risk analysis, trading, compliance)
(enterprise collaboration, corporate hierarchy, data governance)
Finance
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
(e.g. referrals, patient management, research)
(support communities, doctor recommendations, clinical trials)
(drug research, genetic screening, plant engineering, etc.)
(biological master data, evolutionary taxonomy, etc.)
(collaborative medicine, clinical trials, etc.)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Online Job Search Use case: Social / Recommendations
anonymized inside information to job seekers
Neo Technology Confidential
Background
Sausalito, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Online Job Search Use case: Social / Recommendations
anonymized inside information to job seekers
Business problem
found through personal & professional connections
network data. Facebook was the ideal choice.
competitive market
Person Company
KNOWS
Person Person
K N O W S
Company
KNOWS WORKS_AT WORKS_AT
Neo Technology Confidential
Background
Sausalito, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Online Job Search Use case: Social / Recommendations
anonymized inside information to job seekers
Business problem
found through personal & professional connections
network data. Facebook was the ideal choice.
competitive market
Solution & Benefits
through their network of Facebook friends
social graph
being brought online as graph size and load have increased
Person Company
KNOWS
Person Person
K N O W S
Company
KNOWS WORKS_AT WORKS_AT
Neo Technology Confidential
Background
Sausalito, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Industry: Logistics Use case: Parcel Routing
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Business problem
scalability
any point, to any point
Industry: Logistics Use case: Parcel Routing
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Business problem
scalability
any point, to any point
Solution & Benefits
clustering
complex routing
variance much better than relational
Industry: Logistics Use case: Parcel Routing
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
for managing employee permissions to research resources (documents and application services) Industry: Financial Services Use case: Entitlements/IAM
London
Large Investment Bank
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
for managing employee permissions to research resources (documents and application services)
Business problem
permissions were manually provisioned via a complex manual process. Traders lost an average of 5 days of trading, waiting for the permissions to be granted
the onboarding process. Needed to respond quickly.
trader needed access to hundreds of resources. Industry: Financial Services Use case: Entitlements/IAM
London
Large Investment Bank
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
for managing employee permissions to research resources (documents and application services)
Business problem
permissions were manually provisioned via a complex manual process. Traders lost an average of 5 days of trading, waiting for the permissions to be granted
the onboarding process. Needed to respond quickly.
trader needed access to hundreds of resources.
Solution & Benefits
in Neo4j
the productivity advantage of a domain fit
provision permissions themselves
relational data store Industry: Financial Services Use case: Entitlements/IAM
London
Large Investment Bank
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Network Management
Background
Vivendi Group, partnering with Vodafone
Paris, France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Network Management
Background
Vivendi Group, partnering with Vodafone
Business problem
plan, because of the need to model network impacts
ensure resilience during unplanned network outages
need for additional redundancy
with daily changes to network infrastructure
Router Service
DEPENDS_ON
Switch Switch Router Fiber Link Fiber Link Fiber Link
Oceanfloor Cable
DEPENDS_ON D E P E N D S _ O N D E P E N D S _ O N DEPENDS_ON D E P E N D S _ O N DEPENDS_ON DEPENDS_ON DEPENDS_ON DEPENDS_ON L I N K E D LINKED LINKED DEPENDS_ON
Paris, France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Network Management
Background
Vivendi Group, partnering with Vodafone
Business problem
plan, because of the need to model network impacts
ensure resilience during unplanned network outages
need for additional redundancy
with daily changes to network infrastructure
Solution & Benefits
support modeling, aggregation & troubleshooting
network
allows new applications to access network data
1:1 mapping between the real world and the graph
requirements
Router Service
DEPENDS_ON
Switch Switch Router Fiber Link Fiber Link Fiber Link
Oceanfloor Cable
DEPENDS_ON D E P E N D S _ O N D E P E N D S _ O N DEPENDS_ON D E P E N D S _ O N DEPENDS_ON DEPENDS_ON DEPENDS_ON DEPENDS_ON L I N K E D LINKED LINKED DEPENDS_ON
Paris, France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Social gaming
Background
consumers and businesses, as well as internet services, television, and other services
Frankfurt, Germany
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Interactive Television Programming
Industry: Communications Use case: Social gaming
Background
consumers and businesses, as well as internet services, television, and other services
Frankfurt, Germany
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Interactive Television Programming
Industry: Communications Use case: Social gaming
Background
consumers and businesses, as well as internet services, television, and other services
Frankfurt, Germany
Business problem
interactive experience while watching sports
changing data
spikes of activity during popular games
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Interactive Television Programming
Industry: Communications Use case: Social gaming
Background
consumers and businesses, as well as internet services, television, and other services
Solution & Benefits
experience the game more closely
with information, promotions, and ads
Frankfurt, Germany
Business problem
interactive experience while watching sports
changing data
spikes of activity during popular games
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
& services
key application inside HP’s OSS Assurance portfolio
determination, root cause & service impact analysis
complex and fast changing networks Industry: Web/ISV, Communications Use case: Network Management
Global (U.S., France)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
& services
key application inside HP’s OSS Assurance portfolio
determination, root cause & service impact analysis
complex and fast changing networks
Business problem
problems causes on the network
Management System alarms by event correlation Industry: Web/ISV, Communications Use case: Network Management
Global (U.S., France)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
& services
key application inside HP’s OSS Assurance portfolio
determination, root cause & service impact analysis
complex and fast changing networks
Business problem
problems causes on the network
Management System alarms by event correlation
Solution & Benefits
Industry: Web/ISV, Communications Use case: Network Management
Global (U.S., France)
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Resource Authorization & Access Control
Background
the Nordics
admins manage employee subscriptions and plans
responsiveness is critical to customer satisfaction
Oslo, Norway
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Resource Authorization & Access Control
Background
the Nordics
admins manage employee subscriptions and plans
responsiveness is critical to customer satisfaction
Business problem
minutes while system retrieved access rights
Highly interconnected data set w/massive joins
problem, but meant data was no longer current
workaround projected to reach 9 hours by 2014: longer than the nightly batch window
Subscription Account Customer Customer
SUBSCRIBED_BY CONTROLLED_BY PART_OF
User
USER_ACCESS
Oslo, Norway
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Resource Authorization & Access Control
Background
the Nordics
admins manage employee subscriptions and plans
responsiveness is critical to customer satisfaction
Business problem
minutes while system retrieved access rights
Highly interconnected data set w/massive joins
problem, but meant data was no longer current
workaround projected to reach 9 hours by 2014: longer than the nightly batch window
Solution & Benefits
straightforward, as the domain is inherently a graph
responses: measured in milliseconds
Subscription Account Customer Customer
SUBSCRIBED_BY CONTROLLED_BY PART_OF
User
USER_ACCESS
Oslo, Norway
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Recommendations
customers with Support Services
encourage use of online knowledge base
internal master data management solution.
recommendations
Neo Technology Confidential
Background
San Jose, CA
Cisco.com
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Recommendations
customers with Support Services
encourage use of online knowledge base
internal master data management solution.
recommendations
Neo Technology Confidential
Background Business problem
improving the efficacy of online self service
service cases, solutions, articles, forums, etc.
costs, needed to be lowered
Support Case Support Case Knowledge Base Article Solution Knowledge Base Article Knowledge Base Article Message
San Jose, CA
Cisco.com
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Industry: Communications Use case: Recommendations
customers with Support Services
encourage use of online knowledge base
internal master data management solution.
recommendations
Solution & Benefits
for cross-reference links, and represented in Neo4j
decreased reliance on customer support
Neo Technology Confidential
Background Business problem
improving the efficacy of online self service
service cases, solutions, articles, forums, etc.
costs, needed to be lowered
Support Case Support Case Knowledge Base Article Solution Knowledge Base Article Knowledge Base Article Message
San Jose, CA
Cisco.com
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
(after LinkedIn)
Industry: Professional Social Network Use case: Social, Recommendations
Silicon Valley & France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
(after LinkedIn)
Business problem
to attract new users and retain existing ones
connected to any other member
MySQL cluster. Batch precompute meant stale data.
Industry: Professional Social Network Use case: Social, Recommendations
Silicon Valley & France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
(after LinkedIn)
Business problem
to attract new users and retain existing ones
connected to any other member
MySQL cluster. Batch precompute meant stale data.
Solution & Benefits
time programmers
responsiveness with up-to-date data.
relationships.
Viadeo’s professional network, connecting 50M+ professionals Industry: Professional Social Network Use case: Social, Recommendations
Silicon Valley & France
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Creative Suite users to collaborate via the Cloud Industry: Web/ISV Use case: Content Management, Social, Access Control
San Jose, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Creative Suite users to collaborate via the Cloud Industry: Web/ISV Use case: Content Management, Social, Access Control
Business problem
distributed global system, supporting collaboration for users of its highest revenue product line
managing access rights for (eventually) millions of users, groups, collections, and pieces of content
connected to whom, and who could see or edit what, proved a significant technical challenge
User-Content-Access Graph
San Jose, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Background
Creative Suite users to collaborate via the Cloud
Solution & Benefits
performance, were the two major selection factors.
performance requirements. 24x7 with no downtime.
complex queries, even as the system grows
across three Amazon EC2 regions: U.S., Europe, Asia Industry: Web/ISV Use case: Content Management, Social, Access Control
Business problem
distributed global system, supporting collaboration for users of its highest revenue product line
managing access rights for (eventually) millions of users, groups, collections, and pieces of content
connected to whom, and who could see or edit what, proved a significant technical challenge
User-Content-Access Graph
San Jose, CA
Donnerstag, 19. September 13
Neo Technology, Inc Confidential
Donnerstag, 19. September 13