Applications of Machine Learning to Performance Evaluation Daniel - - PowerPoint PPT Presentation
Random walks on graphs Hidden Markov Models and prediction Applications of Machine Learning to Performance Evaluation Daniel Sadoc Menasche 1 Edmundo de Souza e Silva 2 Federal University of Rio de Janeiro 1 Computer Science Department,
1/82
Random walks on graphs Hidden Markov Models and prediction
Daniel Sadoc Menasche1 Edmundo de Souza e Silva2
Federal University of Rio de Janeiro
1Computer Science Department, Institute of Mathematics 2Systems Engineering and Computer Science Department, COPPE
2012
2/82
Random walks on graphs Hidden Markov Models and prediction
Large amount of data produced and consumed everyday social networks
microblogging genome information How to obtain insights from data?
data insight ?
3/82
Random walks on graphs Hidden Markov Models and prediction
We would like to recommend for search engine users, what is the most important webpage? for clients of an online store, which book to suggest based on browsing history? when distributing vaccines, which person to immunize first? when watching a movie, which resolution to use?
4/82
Random walks on graphs Hidden Markov Models and prediction
Machine learning = automatic pattern recognition Performance evaluation = model building and analysis In this talk: how machine learning tools can help to solve performance evaluation problems (and vice versa)
5/82
Random walks on graphs Hidden Markov Models and prediction
Traces provide structural (relational) information about entity relationships
structure encoded in graphs (networks) graph set of vertices and edges (links)
propositional information about individual entities
user preferences (e.g., rate given by a user to a movie) user characteristics (e.g., sex of a user) path properties (e.g., loss probability and delay in a network path)
Structural and propositional information may or may not vary over time
6/82
Random walks on graphs Hidden Markov Models and prediction
School dating network Sexual contact network
Data: connections between people Recommendation: who is the most influential person?
7/82
Random walks on graphs Hidden Markov Models and prediction
black=clinical disease pink=dormant virus green=exposed to virus gray=unknown
Data: connections between people/virus exposure Recommendation: whom to imunize first?
8/82
Random walks on graphs Hidden Markov Models and prediction
Data: connections between researchers Recommendation: who is the most relevant researcher in field A from the perspective of field B?
9/82
Random walks on graphs Hidden Markov Models and prediction
Data: connections between interacting proteins Recommendation: which proteins to use for diagnosis?
10/82
Random walks on graphs Hidden Markov Models and prediction
Link between books A and B, if users that bought A also bought B red=republican blue=democrat magenta=neutral
Data: items bought together Recommendation: which item to recommend next?
11/82
Random walks on graphs Hidden Markov Models and prediction
Data: connections between routers Recommendation: which routers should be more protected?
12/82
Random walks on graphs Hidden Markov Models and prediction
Data: connections between webpages Recommendation: which webpage to suggest to a web user?
13/82
Random walks on graphs Hidden Markov Models and prediction
Newman’s website
http://www-personal.umich.edu/~mejn/networks/
Kreb’s website
http://www.orgnet.com/
14/82
Random walks on graphs Hidden Markov Models and prediction
Provide performance evaluation perspective on machine learning problems Focus on problems rather than details about tools and techniques
15/82
Random walks on graphs Hidden Markov Models and prediction
How to find the most influential node in a network? How to rank nodes? How to devise better recommendation mechanisms? What are good generative models for traffic load? How to predict behavior?
16/82
Random walks on graphs Hidden Markov Models and prediction
Markov chain based models traces with structural and propositional information e.g., 1) links between nodes, 2) user preferences, 3) loss probabilities recommendations inference
17/82
Random walks on graphs Hidden Markov Models and prediction
1
Random walks on graphs
2
Hidden Markov Models and prediction
18/82
Random walks on graphs Hidden Markov Models and prediction
Given a trace, social network movie correlation matrix web pages links questions are how to efficiently infer most central (influential) node? how to rank nodes? given item A and items B and C, which item to recommend after A? But how to define metrics that allow us to answer the questions above?
19/82
Random walks on graphs Hidden Markov Models and prediction
Classic centrality metrics Degree: number of links incident upon node Betweenness: fraction of shortest paths between pairs that pass through node Random walk based centrality metrics Random walk betweenness PageRank
20/82
Random walks on graphs Hidden Markov Models and prediction
What is a random walk? Experiment by a random surfer (e.g., user navigating through web pages)
Start from a random node Choose any link on that node at random Follow that link, and choose any link on that node at random, and so on...
Record how often each node was visited
21/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p3 p4 p1 # of Visits Fraction of Visits Probability of Visit 1 2 3 4 Random Walker Power Method 1 1 1 0 0 0 0 0 0 0 0 0 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1
22/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4 # of Visits Fraction of Visits Probability of Visit 1 2 3 4 Random Walker Power Method 1 0.5 0 0 0 0.33 0 0 0.33 1 0.5 0.33 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0 1 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0
23/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4 # of Visits Fraction of Visits Probability of Visit 1 2 3 4 Random Walker Power Method 1 0.33 0.1667 0 0 0 1 0.33 0.3333 1 0.33 0.5 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0 2 0.16 0 0.33 0.5 0 0 1 0 0 0.16 0.66 0.16 0.5 0 0 0.5
24/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4 # of Visits Fraction of Visits Probability of Visit 1 2 3 4 Random Walker Power Method 1 0.25 0.1667 0 0 0.0556 1 0.25 0.5556 2 0.5 0.2222 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0 3 0.16 0.05 0.55 0.22 0.5 0 0 0.5 0.33 0 0.16 0.5 0 0.16 0.66 0.16
25/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4 # of Visits Fraction of Visits Probability of Visit 1 2 3 4 Random Walker Power Method 200 0.20 0.20 67 0.06 0.06 400 0.40 0.40 333 0.33 0.33 0 0.33 0.33 0.33 0 0 0 1 0.5 0 0 0.5 0 0 1 0 1000 0.2 0.06 0.4 0.33 0.2 0.06 0.4 0.33 0.2 0.06 0.4 0.33 0.2 0.06 0.4 0.33
26/82
Random walks on graphs Hidden Markov Models and prediction
Important vertex connected to many vertices or few important vertices A = adjacency matrix Aij = 1, if i connected to j Aij = 0, otherwise di = out degree of vertex i πi = centrality of vertex i, πi =
Aij dj πj
27/82
Random walks on graphs Hidden Markov Models and prediction
28/82
Random walks on graphs Hidden Markov Models and prediction
πi = centrality of vertex i, πi =
Aij dj πj source = node that has in degree zero sink = node that has out degree zero (dangling node) What is the centrality of sources and sinks?
29/82
Random walks on graphs Hidden Markov Models and prediction
Recall that citation networks are usually acyclic web page networks may have sources and sinks (e.g., pdf files)
1/3 1/3 1/3 1 1/2 1/2
p2 p1 p3 p4
The random surfer is stuck
30/82
Random walks on graphs Hidden Markov Models and prediction
After entering a dangling node ⇒ hyperlink to any page (equal probability) There is a set of trusted web pages.
with probability 1 − d the random surfer does not follow hyperlinks. instead, jump to a trusted page (with equal probability).
(1-d)/2+d/3 d/3 (1-d)/2+d/3 d d/2 d/2
p2 p1 p3 p4
(1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 d
31/82
Random walks on graphs Hidden Markov Models and prediction
Let π be the vector of page ranks for the entire web T set of trusted pages nT = |T | cardinality of set of trusted pages T be a vector which has all zero’s except for non-zero values in positions corresponding to the trusted pages. In those locations the value is 1/nT.
(1-d)/2+d/3 d/3 (1-d)/2+d/3 d d/2 d/2
p2 p1 p3 p4
(1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 (1-d)/2 d
32/82
Random walks on graphs Hidden Markov Models and prediction
Google’s equation πi = d
πj 1 cj + (1 − d)T[i] ∀ i Everyday, new applications of random surfer abstraction
Rank web pages [Brin and Page, 1998] Recommend movies [Bogers et al., CARS 2010] Control and planning [Mahadevan, AAAI 2010] Cure cancer [Winter et al., PLOS 2012]
Heinrich Hertz (regarding Maxwell’s equation)
One cannot escape the feeling that these mathematical formulas have an independent existence and an intelligence
their discoverers, that we get more out of them than was
32/82
Random walks on graphs Hidden Markov Models and prediction
Google’s equation πi = d
πj 1 cj + (1 − d)T[i] ∀ i Everyday, new applications of random surfer abstraction
Rank web pages [Brin and Page, 1998] Recommend movies [Bogers et al., CARS 2010] Control and planning [Mahadevan, AAAI 2010] Cure cancer [Winter et al., PLOS 2012]
Heinrich Hertz (regarding Maxwell’s equation)
One cannot escape the feeling that these mathematical formulas have an independent existence and an intelligence
their discoverers, that we get more out of them than was
33/82
Random walks on graphs Hidden Markov Models and prediction
Until now: steady state analysis of Markov chains Applications of transient analysis
Movement models for mobile computing Obtaining relevance score between two nodes (one the fundamental problems in data mining) Spectral partitioning of a network into clusters Given a node A, how closely related are B and C to A?
33/82
Random walks on graphs Hidden Markov Models and prediction
Until now: steady state analysis of Markov chains Applications of transient analysis
Movement models for mobile computing Obtaining relevance score between two nodes (one the fundamental problems in data mining) Spectral partitioning of a network into clusters Given a node A, how closely related are B and C to A?
34/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4
1/3 1/3 1/3 1 1
p2 p1 p3 p4
1 1/3 1/3 1/3 1
p2 p1 p3 p4
1/2 1/2 1
mean number of transitions to reach p3 = 1/π3 mean number of transitions to reach p2 = 1/π2
Initial page: p1
35/82
Random walks on graphs Hidden Markov Models and prediction
1/3 1/3 1/3 1 1 1/2 1/2
p2 p1 p3 p4
1/3 1/3 1/3 1 1
p2 p1 p3 p4
1 1/3 1/3 1/3 1
p2 p1 p3 p4
1/2 1/2 1
mean number of transitions to reach p3 = 1/π3 = mean number of transitions to reach p2 = 1/π2 =
Initial page: p1
3 11
36/82
Random walks on graphs Hidden Markov Models and prediction
Abstract model: graph model, probabilities associated with edges Useful theory
reversible MCs theory aggregation/disaggregation theory transient analysis Markov chains
applied to new problems
37/82
Random walks on graphs Hidden Markov Models and prediction
How to account for time component?
Graph is discovered online over time Graph changes over time
38/82
Random walks on graphs Hidden Markov Models and prediction
In real networks Network might be too large to compute exact centralities Network structure might be hidden from public view Question: how to efficiently identify top k most central nodes without complete access to entire network? Local versus global computations Degree centrality can be locally obtained Other centrality metrics require global knowledge
38/82
Random walks on graphs Hidden Markov Models and prediction
In real networks Network might be too large to compute exact centralities Network structure might be hidden from public view Question: how to efficiently identify top k most central nodes without complete access to entire network? Local versus global computations Degree centrality can be locally obtained Other centrality metrics require global knowledge
39/82
Random walks on graphs Hidden Markov Models and prediction
Original Network Sampling Identification Sampled Network Sampled Network
40/82
Random walks on graphs Hidden Markov Models and prediction
Original Network Sampling Identification Sampled Network Sampled Network Node in the k most central nodes might not be sampled by sampling algorithm Estimated the k most central nodes can be in- correct top k in actual network
41/82
Random walks on graphs Hidden Markov Models and prediction
Rank correlation between degree and other centralities
Set Type # of nodes # of edges Description AS-Snapshot Device 22,963 48,436 Snapshot of Internet at level of AS ca-CondMat Collaboration 23,133 186,936 ArXiv Condense Matter ca-HepPh Collaboration 12,008 237,010 ArXiv High Energy Physics email-Enron Social 36,692 367,662 Email network from Enron
42/82
Random walks on graphs Hidden Markov Models and prediction
Sampling phase Random-walk sampling Identification phase Recalculation in sampled network Degree as alias to other centralities
43/82
Random walks on graphs Hidden Markov Models and prediction
Start from randomly selected node Visit next node uniformly at random Assume random-walker can query degrees of visited nodes
2 1 3
Node Queried degree node1 5 node2 6 node3 4 . . .
44/82
Random walks on graphs Hidden Markov Models and prediction
Random-walker quickly includes desired top k nodes into sampled set Fraction of top k nodes in sampled set
ca-CondMat RW efficiently col- lects most central nodes with small sampled fraction
45/82
Random walks on graphs Hidden Markov Models and prediction
Sampling phase Random-walk sampling Identification phase Recalculation in sampled network Degree as alias to other centralities
46/82
Random walks on graphs Hidden Markov Models and prediction
Centralities recalculated on sampled network Recalculated centralities are approximation of original centralities
Centrality calcu- lation on sampled net- work
47/82
Random walks on graphs Hidden Markov Models and prediction
Visited nodes sorted according to queried degrees Top k nodes in sorted list taken as top k highest other centralities
Use side in- formation for identifying most central nodes
48/82
Random walks on graphs Hidden Markov Models and prediction
How accurately identify desired top k nodes in sampled network? Overlap ration between identified top k nodes and original top k nodes
ca-CondMat Deg-alias Re- calc. Facebook: more than 500 million users
49/82
Random walks on graphs Hidden Markov Models and prediction
Online Estimating the k Central Nodes of a Network, Y. Lim, D. Menasche, B. Ribeiro, D. Towsley, P . Basu, IEEE Network Science Workshop, Westpoint, 2011
50/82
Random walks on graphs Hidden Markov Models and prediction
How to infer most influential nodes in a changing world? Break for advertisement from authors Related work (main conference)
Characterizing Continuous Time Random Walks on Time Varying Graphs, Daniel Figueiredo (Federal University of Rio de Janeiro - UFRJ), Philippe Nain (INRIA), Bruno Ribeiro (UMass Amherst), Edmundo de Souza e Silva (UFRJ) and Don Towsley (UMass Amherst)
50/82
Random walks on graphs Hidden Markov Models and prediction
How to infer most influential nodes in a changing world? Break for advertisement from authors Related work (main conference)
Characterizing Continuous Time Random Walks on Time Varying Graphs, Daniel Figueiredo (Federal University of Rio de Janeiro - UFRJ), Philippe Nain (INRIA), Bruno Ribeiro (UMass Amherst), Edmundo de Souza e Silva (UFRJ) and Don Towsley (UMass Amherst)
50/82
Random walks on graphs Hidden Markov Models and prediction
How to infer most influential nodes in a changing world? Break for advertisement from authors Related work (main conference)
Characterizing Continuous Time Random Walks on Time Varying Graphs, Daniel Figueiredo (Federal University of Rio de Janeiro - UFRJ), Philippe Nain (INRIA), Bruno Ribeiro (UMass Amherst), Edmundo de Souza e Silva (UFRJ) and Don Towsley (UMass Amherst)
51/82
Random walks on graphs Hidden Markov Models and prediction
1
Random walks on graphs
2
Hidden Markov Models and prediction
52/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
speech recognition signal processing artificial intelligence computational biology image processing finance medical diagnosis
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
53/82
Random walks on graphs Hidden Markov Models and prediction
Use of HMMs
Given a time series, how to parameterize model to predict future values?
inferring customer behavior modeling network channel losses modeling traffic generating workload
Note: we have traces of time series of one or more variables. Is there a structure behind the data?
54/82
Random walks on graphs Hidden Markov Models and prediction
Performance/Availability
Analyst understands how the system works Markovian models:
Important issue: choice of state variables Models parameterized from some prior knowledge of the system behavior
Key point: the system state is directly observable
54/82
Random walks on graphs Hidden Markov Models and prediction
Performance/Availability
Analyst understands how the system works Markovian models:
Important issue: choice of state variables Models parameterized from some prior knowledge of the system behavior
Key point: the system state is directly observable
54/82
Random walks on graphs Hidden Markov Models and prediction
Performance/Availability
Analyst understands how the system works Markovian models:
Important issue: choice of state variables Models parameterized from some prior knowledge of the system behavior
Key point: the system state is directly observable
54/82
Random walks on graphs Hidden Markov Models and prediction
Performance/Availability
Analyst understands how the system works Markovian models:
Important issue: choice of state variables Models parameterized from some prior knowledge of the system behavior
Key point: the system state is directly observable
54/82
Random walks on graphs Hidden Markov Models and prediction
Performance/Availability
Analyst understands how the system works Markovian models:
Important issue: choice of state variables Models parameterized from some prior knowledge of the system behavior
Key point: the system state is directly observable
55/82
Random walks on graphs Hidden Markov Models and prediction
Hidden Markov Models
System state is assumed to not be directly observable. But... can observe values that are a probabilistic function
55/82
Random walks on graphs Hidden Markov Models and prediction
Hidden Markov Models
System state is assumed to not be directly observable. But... can observe values that are a probabilistic function
56/82
Random walks on graphs Hidden Markov Models and prediction
Example
Customer browsing through the web pages of an online bookstore Problem: determine the probability that a specific customer is ready to order an item based on their past behavior Assumed that we have access to data that includes the intention of a customer
56/82
Random walks on graphs Hidden Markov Models and prediction
Example
Customer browsing through the web pages of an online bookstore Problem: determine the probability that a specific customer is ready to order an item based on their past behavior Assumed that we have access to data that includes the intention of a customer
56/82
Random walks on graphs Hidden Markov Models and prediction
Example
Customer browsing through the web pages of an online bookstore Problem: determine the probability that a specific customer is ready to order an item based on their past behavior Assumed that we have access to data that includes the intention of a customer
57/82
Random walks on graphs Hidden Markov Models and prediction
Example
JB IP RO
0.02 0.2 0.3 0.6 0.18 0.3 0.2
LV
0.5 0.4 1.0 0.3 0.1
JB, just browsing IP, interested in product RO, ready to order LV, leaving
58/82
Random walks on graphs Hidden Markov Models and prediction
Example
But... user’s state of intent is not directly observable Suppose only the types of pages a customer visits are
O product overview D product details C set of products within a category S shopping cart P purchase E exit
Trace: O C D C D C D O C D C D S O C D S P E
58/82
Random walks on graphs Hidden Markov Models and prediction
Example
But... user’s state of intent is not directly observable Suppose only the types of pages a customer visits are
O product overview D product details C set of products within a category S shopping cart P purchase E exit
Trace: O C D C D C D O C D C D S O C D S P E
58/82
Random walks on graphs Hidden Markov Models and prediction
Example
But... user’s state of intent is not directly observable Suppose only the types of pages a customer visits are
O product overview D product details C set of products within a category S shopping cart P purchase E exit
Trace: O C D C D C D O C D C D S O C D S P E
58/82
Random walks on graphs Hidden Markov Models and prediction
Example
But... user’s state of intent is not directly observable Suppose only the types of pages a customer visits are
O product overview D product details C set of products within a category S shopping cart P purchase E exit
Trace: O C D C D C D O C D C D S O C D S P E
59/82
Random walks on graphs Hidden Markov Models and prediction
Example
States are correlated with the sequence of observations
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
JB, just browsing IP, interested in product RO, ready to order LV, leaving 6 observable symbols (page types)
60/82
Random walks on graphs Hidden Markov Models and prediction
Example
Note: In most problems we do not know how many states the hidden chain contains. the interpretation of the hidden states (e.g. the “states of intent”)
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
hidden states
(page types)
JB, just browsing IP, interested in product RO, ready to order LV, leaving
60/82
Random walks on graphs Hidden Markov Models and prediction
Example
Note: In most problems we do not know how many states the hidden chain contains. the interpretation of the hidden states (e.g. the “states of intent”)
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
hidden states
(page types)
JB, just browsing IP, interested in product RO, ready to order LV, leaving
60/82
Random walks on graphs Hidden Markov Models and prediction
Example
Note: In most problems we do not know how many states the hidden chain contains. the interpretation of the hidden states (e.g. the “states of intent”)
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
hidden states
(page types)
JB, just browsing IP, interested in product RO, ready to order LV, leaving
61/82
Random walks on graphs Hidden Markov Models and prediction
HMM Elements
The HMM elements are a set of hidden states a set of symbols the state transition probability matrix the probabilities of emitting each symbol at each state
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
JB, just browsing IP, interested in product RO, ready to order LV, leaving
62/82
Random walks on graphs Hidden Markov Models and prediction
What is the most probable hidden state (the customer intent) given the observed sequence of pages visited? What are the model parameters that maximize the probability that the observed sequence is generated by the model? Assume 2 types of customers: young and mature.
Given a user session and the sequence of page clicks (but not the customer type), what is the probability that the customer is young versus mature?
62/82
Random walks on graphs Hidden Markov Models and prediction
What is the most probable hidden state (the customer intent) given the observed sequence of pages visited? What are the model parameters that maximize the probability that the observed sequence is generated by the model? Assume 2 types of customers: young and mature.
Given a user session and the sequence of page clicks (but not the customer type), what is the probability that the customer is young versus mature?
62/82
Random walks on graphs Hidden Markov Models and prediction
What is the most probable hidden state (the customer intent) given the observed sequence of pages visited? What are the model parameters that maximize the probability that the observed sequence is generated by the model? Assume 2 types of customers: young and mature.
Given a user session and the sequence of page clicks (but not the customer type), what is the probability that the customer is young versus mature?
62/82
Random walks on graphs Hidden Markov Models and prediction
What is the most probable hidden state (the customer intent) given the observed sequence of pages visited? What are the model parameters that maximize the probability that the observed sequence is generated by the model? Assume 2 types of customers: young and mature.
Given a user session and the sequence of page clicks (but not the customer type), what is the probability that the customer is young versus mature?
63/82
Random walks on graphs Hidden Markov Models and prediction
Problem 1
Given the observation sequence OT = O1O2 . . . OT and a model M, compute the probability of observing the output sequence OT given the underlying model M, i.e., P[OT|M].
O O O D C S P E P E what is probability? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
0.85
probability
64/82
Random walks on graphs Hidden Markov Models and prediction
Problem 2
Given the observation sequence OT = O1O2 . . . OT and a model M, how to determine a best state sequence QT = q1, . . . , qT which best explains the output sequence OT.
O O O D C S P E P E what is best sequence of states? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
JB JB IP RO LV JB
sequence of states
65/82
Random walks on graphs Hidden Markov Models and prediction
Two versions of problem 2
Given the observation sequence OT = O1O2 . . . OT
1
For each time t, what is the most probable state of the underlying MC?
2
What is the most probable sequence of states?
O O O D C S P E P E for each time t, what is the most probable state? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
JB JB IP LV IP LV IP
sequence of states
If pIP,LV =0, * in (1), we could have s t=IP and s t+1=LV * in (2), the sequence
could never occur.
66/82
Random walks on graphs Hidden Markov Models and prediction
Two versions of problem 2
Given the observation sequence OT = O1O2 . . . OT
1
For each time t, what is the most probable state of the underlying MC?
2
What is the most probable sequence of states?
O O O D C S P E P E what is most probable sequence of states? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
JB JB IP LV IP LV JB
sequence of states
67/82
Random walks on graphs Hidden Markov Models and prediction
Problem 3
Given the observation sequence OT = O1O2 . . . OT, construct an underlying model M, such that P[OT|M] is maximized.
what are best model parameters? O O O D C S P P E P P E trace JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
68/82
Random walks on graphs Hidden Markov Models and prediction
Problem 4
Given several possible models Mi 1 ≤ i ≤ K and a sequence
model, i.e. P[Mj|O].
O O O D C S P P E P P E
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
what is most likely model?
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
69/82
Random walks on graphs Hidden Markov Models and prediction
Problem 1 There is an iterative algorithm for computing P[OT|M] which has complexity O(N2T).
O O O D C S P E P E what is probability? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
0.85
probability
70/82
Random walks on graphs Hidden Markov Models and prediction
Problem 2 The forward-backward algorithm solves a specific version of problem 2 computes P[qT = si|M, OT] and has complexity O(N2T)
O O O D C S P E P E at time T, what is the most probable state? trace
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model
JB
sequence of states
71/82
Random walks on graphs Hidden Markov Models and prediction
Problem 3 Procedure for adjusting the model parameters based on: the maximum likelihood estimation (MLE) method a technique derived from the Expectation-Maximization (EM) algorithm known as Baum-Welch iterative procedure to obtain a local maximum for the likelihood function.
what are best model parameters? O O O D C S P P E P P E trace JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
Hidden Markov Model ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
72/82
Random walks on graphs Hidden Markov Models and prediction
Problem 4 Basically this is a classification problem and one simply applies Bayes Theorem.
O O O D C S P P E P P E
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
what is most likely model?
JB IP RO LV
O D C S P E O D C S P E O D C S P E O D C S P E
73/82
Random walks on graphs Hidden Markov Models and prediction
Traffic Modeling
Build accurate traffic models for the packet flow generated by different applications such as SMTP , HTTP , network games, instant messaging.
Observable data: packet inter-arrival time and packet size. Application: traffic generators, capacity planning.
Traffic models for aggregate traffic.
Application: traffic generators, capacity planning.
Traffic classification.
Application: Identifying distinct applications by observing traffic. This is an example of Problem 4.
73/82
Random walks on graphs Hidden Markov Models and prediction
Traffic Modeling
Build accurate traffic models for the packet flow generated by different applications such as SMTP , HTTP , network games, instant messaging.
Observable data: packet inter-arrival time and packet size. Application: traffic generators, capacity planning.
Traffic models for aggregate traffic.
Application: traffic generators, capacity planning.
Traffic classification.
Application: Identifying distinct applications by observing traffic. This is an example of Problem 4.
73/82
Random walks on graphs Hidden Markov Models and prediction
Traffic Modeling
Build accurate traffic models for the packet flow generated by different applications such as SMTP , HTTP , network games, instant messaging.
Observable data: packet inter-arrival time and packet size. Application: traffic generators, capacity planning.
Traffic models for aggregate traffic.
Application: traffic generators, capacity planning.
Traffic classification.
Application: Identifying distinct applications by observing traffic. This is an example of Problem 4.
74/82
Random walks on graphs Hidden Markov Models and prediction
Loss process in a channel
Objective: model the loss process of an communication channel. Application: include model in simulation studies. Hierarchical HMMs have also been used.
74/82
Random walks on graphs Hidden Markov Models and prediction
Loss process in a channel
Objective: model the loss process of an communication channel. Application: include model in simulation studies. Hierarchical HMMs have also been used.
74/82
Random walks on graphs Hidden Markov Models and prediction
Loss process in a channel
Objective: model the loss process of an communication channel. Application: include model in simulation studies. Hierarchical HMMs have also been used.
75/82
Random walks on graphs Hidden Markov Models and prediction
Predictors of events
Use HMMs as predictors of events in the future. Applications:
Select a path from two or more distinct paths to communicate with the application peer. Estimate future channel loss statistics to better adapt their encoding algorithms.
75/82
Random walks on graphs Hidden Markov Models and prediction
Predictors of events
Use HMMs as predictors of events in the future. Applications:
Select a path from two or more distinct paths to communicate with the application peer. Estimate future channel loss statistics to better adapt their encoding algorithms.
75/82
Random walks on graphs Hidden Markov Models and prediction
Predictors of events
Use HMMs as predictors of events in the future. Applications:
Select a path from two or more distinct paths to communicate with the application peer. Estimate future channel loss statistics to better adapt their encoding algorithms.
76/82
Random walks on graphs Hidden Markov Models and prediction
Predictors of events H F T τ Measure prediction Model training
Prediction instant History sample Prediction window Training instant Training sample Training intervals
τ τ ψ ψ ψ ψ ψ
Prediction intervals
77/82
Random walks on graphs Hidden Markov Models and prediction
Workload Generation
Objective: generate synthetic workload at the application level, instead of the packet level. For anomaly detection, model is trained using a sequence of requests made by normal web users. Workload at the application level may involve different time scales.
Example: ON state, browser sends requests for a page and its in-line objects; OFF state, user reads the page contents Hierarchical HMMs are useful tools to represent different time scales.
77/82
Random walks on graphs Hidden Markov Models and prediction
Workload Generation
Objective: generate synthetic workload at the application level, instead of the packet level. For anomaly detection, model is trained using a sequence of requests made by normal web users. Workload at the application level may involve different time scales.
Example: ON state, browser sends requests for a page and its in-line objects; OFF state, user reads the page contents Hierarchical HMMs are useful tools to represent different time scales.
77/82
Random walks on graphs Hidden Markov Models and prediction
Workload Generation
Objective: generate synthetic workload at the application level, instead of the packet level. For anomaly detection, model is trained using a sequence of requests made by normal web users. Workload at the application level may involve different time scales.
Example: ON state, browser sends requests for a page and its in-line objects; OFF state, user reads the page contents Hierarchical HMMs are useful tools to represent different time scales.
78/82
Random walks on graphs Hidden Markov Models and prediction
Workload Generation
Student behavioral model accessing a video-lecture. Observed symbols are play, jump forward, rewind, pause, end of session, next slide. Goal: generate a synthetic workload.
78/82
Random walks on graphs Hidden Markov Models and prediction
Workload Generation
Student behavioral model accessing a video-lecture. Observed symbols are play, jump forward, rewind, pause, end of session, next slide. Goal: generate a synthetic workload.
79/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
Break for advertisement from authors Tangram-II has a specific module that allows users to work with regular or hierarchical HMMs Supports three different types of HHMM From data trace, user can, for instance, train the model, or calculate the log likelihood of the observation sample Tool implements a forecast algorithm
79/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
Break for advertisement from authors Tangram-II has a specific module that allows users to work with regular or hierarchical HMMs Supports three different types of HHMM From data trace, user can, for instance, train the model, or calculate the log likelihood of the observation sample Tool implements a forecast algorithm
79/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
Break for advertisement from authors Tangram-II has a specific module that allows users to work with regular or hierarchical HMMs Supports three different types of HHMM From data trace, user can, for instance, train the model, or calculate the log likelihood of the observation sample Tool implements a forecast algorithm
79/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
Break for advertisement from authors Tangram-II has a specific module that allows users to work with regular or hierarchical HMMs Supports three different types of HHMM From data trace, user can, for instance, train the model, or calculate the log likelihood of the observation sample Tool implements a forecast algorithm
79/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
Break for advertisement from authors Tangram-II has a specific module that allows users to work with regular or hierarchical HMMs Supports three different types of HHMM From data trace, user can, for instance, train the model, or calculate the log likelihood of the observation sample Tool implements a forecast algorithm
80/82
Random walks on graphs Hidden Markov Models and prediction
Tangram-II
81/82
Random walks on graphs Hidden Markov Models and prediction
Markov chain based models traces random surfer model behavior information Random walk model Hidden Markov model recommendations, insights, ... + + inference content centrality user state
82/82
Random walks on graphs Hidden Markov Models and prediction
http://www.land.ufrj.br/~sadoc/tutorial/