SLIDE 1
How I Spent My Sabbatical
My location My host: Mau Porfiri My commute
Transfer entropy for network reconstruction in a simple dynamical - - PowerPoint PPT Presentation
Transfer entropy for network reconstruction in a simple dynamical - - PowerPoint PPT Presentation
Transfer entropy for network reconstruction in a simple dynamical model Roy Goodman NJIT Dept. of Mathematical Sciences How I Spent My Sabbatical My location My commute My host: Mau Por fi ri The Por fi ri Lab A big group working in a lot of
SLIDE 2
SLIDE 3
The Porfiri Lab
A big group working in a lot of areas, both theoretical and through laboratory experiments:
- Fluid mechanics: fluid structure interactions during water
impact
- Artificial Muscles and Soft Robotics
- Telerehabilitation
- Network-based modeling of infectious diseases
- Fish schooling
- Using robotics and zebrafish to study substance-abuse
disorders
- Information-theoretic analysis of social science datasets
- more…
A unifying theme in this work is using methods from information theory for modeling and analysis
SLIDE 4
What is this talk about?
A lot of words to define here:
Transfer entropy for network reconstruction in a simple dynamical model
- Network: a graph composed of vertices and edges, the subject
at the heart of graph theory
- Transfer entropy: a quantity describing the transfer of
information from one evolving variable to another, from information theory
- The dynamical model: to be described, a simple probabilistic
dynamics.
- Reconstruction: figure out properties of the graph based on the
dynamics
18 months ago I knew 𝞋 about any of this
SLIDE 5
Graph Theory
A graph is defined by a set V of vertices, connected by a set E of edges. The graph at right is both directed and weighted.
An important notion for us will be the weighted incoming degree .
δi = P
j Wij
<latexit sha1_base64="3oRIr54WmvCXtvbZVTLQLfwvi30=">ADCnicbVJNi9QwGM7Ur7V+zepRD8Ey4EG6bCgF3FgL5klZmdhbaUNH3byW6SliRdGUpBPsXvHt1b7I38Tco/gd/hOnMKNdXyg8eZ73SZ8kb1Jyps1o9KvnXLl67fqNnZvurdt37t7r794/1EWlKMxowQt1lBANnEmYGWY4HJUKiEg4zJOT/Vafn4LSrJBTsywhEiSXLGOUGEvF/UdhCtyQuGYNfoFDXYm4Pm7w3BLHTdz3RsPRqvBl4G+AN3n57dP3+fvD+Ld3o8wLWglQBrKidaBPypNVBNlGOXQuGloST0hOQWCiJAB3Vq2M0eGCZFGeFsp80eMW6g23L1I/qVgNJm6isxQyUnH7s5xzirdbaih4qBORTfC9n54gF8DpJsEU3tXPt4vJFVgYNtk067JBruhHe0EILItA5JopvAbhyEzI7a0bzw8VyxcmVO2q6fYzKUE1wfifg8icg+c/9cZ/XSumE7kmQulSGxcQcxCX9Ra8n9aUJnseVQzWVab01otqzg2BW7nAqdMATV8aQGhitn3wnRBFKHGTo9rJ8G/+O6XweF46O8N9743uQtWtcOeogeoyfIR8/QBL1CB2iGKPqAPqMv6Mz56Jw5X53zdavT23geoE453/8AvsMArQ=</latexit>Graph theory is central to the mathematics of Computer Science, describing the connections between interacting agents.
W = 0.3 0.55 0.3 0.4 0.6 1 0.1 0.4 1 0.9 0.45
<latexit sha1_base64="cSFswFU9E1zR8dRebv0rXnmP4vU=">ADtnicbVLbtQwFE0aHiW8prBkYxENYoGiZJhCWVRU6oYVKmimU2mSVo5zk7HqOJHtFEZRvoaf4Quo+BuczEPJlCtd6+YcH/vE90YFo1J53l9z7p3/8HD/Uf24ydPnz0fHLw4l3kpCExJznJxEWEJjHKYKqoYXBQCcBYxmEXpw0/uwEhac4nalAmOGU04QSrDR0Nfg9Q8coiClvIoyrAT9WdseoM8932zbtI9PGzLILC9Lr7NDdGX9alxu37okH5Luv72mnELdpXup90D3XHrxQ6Ax1vTVwPHc7020N3CXxfO51v7uPh1a59dHZh/gjgnZQZcEYalnPteocIKC0UJg9oOSgkFJtc4hbkuOc5AhlX75DUaiRGS50coVa1B52JRM/rBoOKn7jExiSHDJ9GUpZYyWsr+hgpKBuMn6FrnoSH6ChCvHUx0X310mnMiQEFXpN2uwBrZAYcfJM8yrB8twJGs5/rAgEGiAqYnRDl+IGi6UIFovur+fso5iHo+2iowTxk4/jtntFG1SM9yhTMpl1mk7eomLeQu14D/4+alSo7CivKiXP+t5pKSIZWjZoZRTAUQxZa6wERQ3S9EFlhgovSk23oS/N2+3y3OR64/dsfPOfku7GKfeOV8dp4a/jGR+PE+GKcGVODmK45MUPz0jqyLi2w0tXWPXOteWn0wir+AZh9GeU=</latexit>The weight matrix has entries Wij defined as follows
- If an edge exists from node j to node i the
entry is the associated weight
- If no edge exists, the entry zero.
δE = 1 + 0.4 + 0.1 = 1.5
<latexit sha1_base64="LwXKVndP6bQLhYuhEREnmP3bS6A=">AD3icbVJNixMxGE7Hr3X82K4evUSHiqCUSamsF6GwCJ5kV9rdhU4pmcw7bdgkMySZXcowZ8978S948urexKv+AEH8M2baKp1dXwg8zvk3mSvHEuLFh+KvlXbt+4+atrdv+nbv37m+3dx4cmqzQDEYsE5k+jqkBwRWMLcCjnMNVMYCjuKTvVo/OgVteKaGdpHDRNKZ4iln1Dpq2n4cJSAsnZaRluWbqsKvMXkedvtukRp3X07bQdgNl4WvArIGweDpwe8fH8/7U93Wj+jJGOFBGWZoMaMSZjbSUm15UxA5UeFgZyEzqDsYOKSjCTcnmWCnck+A024pi5es39m0DMmkrDVQrGoqJk0gpYVwP5txIXhmg0lFAL0qWxG2NwPd/A7gGSdYOgujOC9TDENFjZNLu2KrLAfKThjmZRUJWVEY1ON3YaRgNRGwl29DUik+WxuI1/Vc1+rhToatz756BqJiAgL4LeX9eSaUQuqTRmIWMXV1I7N5e1mvyfNi5s+mpScpUX69M6LS0EthmuhwMnXAOzYuEAZq798JsTjVl1o2Q7yaBXH73q+Cw1yX9bv+ABIP3aFVb6BF6gp4hgnbRAL1F+2iEGPqAPqMv6MI79y68r963VavXWnseokZ53/8At2H+0w=</latexit>In this example
SLIDE 6
Information Theory
- Initiated by Claude Shannon’s 1948 “A Mathematical Theory of
Communication”
- Originally used to study the transmission of signals down noisy
channels and to develop optimal strategies for encoding information
- Recently become popular tool for analyzing dynamical systems
Fundamental quantity:
Consider a discrete random variable X drawn from a sample space X The information associated with the event X=x measures how “surprising” it is that X=x
I(x) = − log (Pr(X = x))
<latexit sha1_base64="+LxPrbxaQrwLNIb/AbHZefKoL54=">ADGXicbVJNj9MwEHXD1I+tgtHLhZRoRVQNdVKcFmp0l7gpqd1eq8pxJqm1thPZzrJVlB/BmR/ClT0gEFckJP4NTltQs8tIkcbvzXOeZybMBDe23/d8K5dv3Hz1s7t5p279+7vtvYeHJk01wmLBWpPgmpAcEVTCy3Ak4yDVSGAo7D08OKPz4DbXiqxnaZwUzSRPGYM2odNG89e9M57+ID/IKINCmIgNh2iKR2oWUx0mXn5OC8SzRPFrZbzlt+v9dfBb6aBJvEHw7I029fn+yO5nuNXyRKWS5BWSaoMdOgn9lZQbXlTEDZJLmBjLJTmsDUpYpKMLNi9aoStx0S4TjV7lMWr9Bme1syDmZFxYFiZ0xcQxzYX7WcKF4LmpFxSQC9Bnsm5h+z7cxm8Bo2DsWtdgA9TxTRY2BY5t2uwxE2i4ANLpaQqKgNTl1F656SoQbgvWDdSuJrk5lvZ4rBbqcDv4pqEoE+MFzf/BXtUJqlgsqjVnK0NmthmYucxX4P26a2/jVrOAqyzevdVycC2xTXK0JjrgGZsXSJZRp7uaF2YJqyqxbpqbhODy3K8mR4NesN/bfxf4w/doHTvoEXqMOihAL9EQvUYjNEMfUSf0Rd04X3yLrzv3o91qdfYaB6iWng/wC7sQOI</latexit>H(X) = E[I(X)] = − P
x∈X Pr(X = x) log Pr(X = x)
<latexit sha1_base64="h0CN0sprwUAmbI4BC5SLR0I3FA=">ADUnicdVJdixMxFE1bP9a6ul19CU4FrqgpVMWdB8KhUVYX6RKu1uYlJLJ3JmGzWSGJLNuGeaX+BP8Hz74LIJ/wGdf3SczbZXOrl4I3JxzT3KSe/1UcG16ve+1euPW7Tt3d+417+8+eLjX2n90qpNMZiwRCRq6lMNgkuYG4ETFMFNPYFnPnxyV/dgFK80SOzTKFWUwjyUPOqLHQvBWcdKYHeIBJTM1CxfnrwntjkdngBdFZPM8vMeFyzTIq8mlRYJKkoKhJlKQx5CNVdKaDywNMRBL9h5u3nF63twp8M3E3iTM8+nz049PHZ6P5fu0bCRKWxSANE1Rrz+2lZpZTZTgTUDRJpiGl7JxG4Nm0vE3P8tV3FLhtkQCHibJLGrxCm+1tydid5SUHkhVRocBhDQT9rKIC8EzXS3IROgLuKqhe3zcBu/BQg2Dsb2z18nEimwMC2yLpdgwVuEgkfWBLHVAY5ob4uPHsgERAaImz3jOMSxaOFIarcFdV6LiWowuv/VAZCXDc507/j2qFVCznNZ6GfvWbtlbfZ0rwX9xXmbCV7OcyzTbvNZyYSawSXA5XzjgCpgRS5tQprjtF2YLqigzdgqbdhLc632/mZz2u+5h9/Cd6wzfo3XsoCfoKeogF71EQ3SCRmiCGPqCfqJf6Kr+tX7VqDUa69J6baN5jCrR2P0NZI8Y0w=</latexit>The Shannon entropy of the random variable is the expectation value fo the information
SLIDE 7
Basic example: biased coin toss
Consider a biased coin that gives heads with probability p and tails with probability 1-p
- When p≈0 or p≈1, entropy small since surprising outcomes rarely
- ccur
- When p≈0.5, entropy large since both outcomes equally likely
SLIDE 8
Transfer entropy Schreiber 2000
Consider two random variables X and Y. Define the joint entropy and the conditional entropy:
H(X, Y) = − X
x∈X,y∈Y
Pr(X = x, Y = y) log Pr(X = x, Y = y) H(X|Y) = − X
x∈X,y∈Y
Pr(X = x, Y = y) log Pr(X = x|Y = y)
<latexit sha1_base64="FwI3fry1QsFcWeCEHvCkZP1kRU=">AD1nicvVNa9tAEJWifqTqR5z2MtS4eKAayxjaC+GQC45hbTYiYzXmNVqJC9ZrcTuKo1RlFvptb+u0H/Qn9GV4xYrKT12QDB6b2bnLW82zDlTut/Ye84Dx4+erz7xH367PmLvdb+yzOVFZLChGY8k0FIFHAmYKZ5hDkEkgacjgPL45q/vwSpGKZGOtVDvOUJILFjBJtoEXrJzruBN3pAXo7Qu+wKtJFeYUwEwinRC8p4WVQdGqCU2rCuEsB0l0JgVJoTyVScYXWno9UBwjxL/sljt56/b+mXtf8ouX1e/1oPuJv0k8axOni37O4yWqQgNOVEqZnfz/W8JFIzyqFycaEgJ/SCJDAzaT1Rzcu1JRVqGyRCcSbNJzRao257u2Xsz8uaA0GrJqPiCGJScDMsYZyzQjULSig4yMu0KWH7PNRGJwDRsHY+O6jo0xQCRq2m4zaW7BCLhbwmWZpSkRUYhKqamYOxBxijbnZIO35WLJkqbGs/6pmPRMCZDUb/OkgIuHg+V1v8LtrjTQklyRVapWGRm7tsbrL1eDfuFmh4w/zkom82NzWcHBkc5QveMoYhKo5iuTECqZ8QvRJZGEavMSXLMJ/l3f7ydng54/7A0/Dr3DT5ud2LVeW2+sjuVb761D69g6tSYWtU9sbVf2jRM4N84X5+t6Y696XlNcL59guXUEHJ</latexit>Transfer entropy measures the reduction in the uncertainty of predicting X(t + 1) from both X(t) and Y(t) relative to predicting it from X(t) alone. Entropy can be defined analogously for stationary stochastic processes Transfer entropy from Y to X is the difference between the entropy of X(t+1) conditioned on X(t) and that conditioned on both X(t) and Y(t)
TEY→X = H(X(t + 1)|X(t)) − H(X(t + 1)|X(t), Y(t)) = X
x+∈X x∈X y∈Y
- Pr [X(t + 1) = x+, X(t) = x, Y(t) = y] × log Pr [X(t + 1) = x+|X(t) = x, Y(t) = y]
Pr [X(t + 1) = x+|X(t) = x]
- <latexit sha1_base64="yAy5hPIWO/AbZjOjDkKU1KXu8vw=">AETXicjVPfT9swE4pG6z7AWyPezmtYqKCoQYhbS9ISGgSTxOTCrSqS+W4TrFwnMp2GJXxn7fnvU3a37DX8TZNuzQZIsCmnZTk83d3n8+5czSRwth2+1trj7/4OHC4qPG4ydPny0trzw/MmGT9kqUx1N6KGS6H4oRVW8u5Ec5pEkh9HZ3u5/icayNS1bHTCR8kdKxELBi1SA1XaieARhJqT3XiOu/9iesRm0LXw2vYgf217pdD1uX+Gm13lTXG72cBEIauUYeTkyWDB2+I2MpO3MXQ7fugQhV7MCodChMCFzcR06rZM97pCWPLXHkQBewD7gzrEPY2oFCfSNn8hWi3gyhjhbjUzsAYkXCDWamYyCxpgz+pXT5dyX/n4nX4QUgfrjcbG+2ZwZ3QViCZlDaAXbkKxmlLEu4skxSY/phe2IHjmormOS+QTLDJ/hz6Zj3ESqKJxy42SR4WEVmBHGq8VEWZmxj9WZKJxy43McV81WPiUc8pnEzcZCSpGZaoDjmeT6PKmWcFMPVuED56Oyg6OWwh7qWKaW34zCastSA8NovgnliYJVSNHaGR8HwWLrkscXNsMy1+p85WvxguluPb9resMqsaSN8ON5tafrBlTKdnRxJhpEmG5+aiZ276cvM/Xz2z8buCEmTladEXZxLwuRXC0ZCc2blFAFlWmC/gJ1SnDqLF7CBkxDe7vtdcLS1GW5vbn/cbu7uljOxGLwMXgVrQRi8DXaD/eAgOAxY7XPte+1H7ar+pX5V/1n/VYTO1cqcF0HF5hd+Aw1+YS0=</latexit>
SLIDE 9
Contrived transfer entropy example
Source: A Tutorial for Information Theory in Neuroscience Nicholas M. Timme and Christopher Lapish 2018
- Series 1: X & Y uncorrelated: TE≈0
- Series 2 & 3: X tends to fire before Y: TE large
- Series 4: X(t+1) determined entirely from X(t), no improvement
from knowing Y(t): TE=0
SLIDE 10
Transfer entropy used to infer climate network
Hlinka et al. 2013
- Divide the earth into N
patches using principal component analysis
- Look at time series of
surface-area temperature deviations
- Compute transfer entropies,
estimating PDFs using specially-tuned kernel density estimators
- Threshold to find links with
non-negligible transfer entropy
SLIDE 11
Transfer Entropy used to analyze connections between corporations
Sandoval 2014
Analysis based on time series of stock prices. Transfer entropy predicts that the price of a stock has an influence
- n other stocks in its sector and geographic area
SLIDE 12
Advantages and disadvantages of Transfer Entropy
- Model free
- Nonlinear
- Relatively simple to compute
- Requires the estimation of underlying probability
distribution, e.g. by binning or kernel density estimation
- Lots of hidden parameters to fiddle with that might effect
the computation
- Inherently dyadic: quantifies the interaction between two
agents while ignoring the effects of others. Unclear how much this matters. j i Wij TEj→i
SLIDE 13
Random Boolean Network (RBN) Model
Porfiri & Ruiz Marín 2018
A system of nodes Xi(t), i=1…n, each of which can take values in {0,1} Conceived as a model “Policy Diffusion”—How the passage of laws in
- ne jurisdiction influences the passage of laws in other jurisdictions.
The terms Wij≥0 represent the “network of influence.” At time step t+1, the state of node i depends on the state of the system at time step t, according to
Pr ⇥ Xi(t + 1) = 1|X1(t) = x1, . . . , XN(t) = xN⇤ = ✏ 2 41 +
N
X
j=1
Wijxj 3 5
<latexit sha1_base64="iOzG1l8i5mc1OqhZEy4QmEPQKxo=">ADm3icbVLdahNBFN5t/KnxL7WXIhwMgYolZEJBeyEc1NESoSkCWRjnMyeTSadnVlmZqth3UfyCXwPwUvfxNlNLEnrgWG/+b7zt2fOLBHc2Fbrt79XuXP3v39B9WHjx4/eVo7eHZhVKoZDpgSo9m1KDgEgeW4GjRCONZwKHs8tuoQ+vUBuZN+uEpzEdC5xBm1jprWfgQqQU2t0pLGmPV0DoHAyI5h9DnjORxZeA3kFbwDAt8LjhRcf9W4mPnHiprHDi+ZbocKD5fGEnAFB1x9HlJ8DEcKEkbCoRVyEwaTzNlkWZvAwdTjMOyxyKTMvrTNavdVslQa3AdmAeufsz/DLaeNnb3rg/wpCxdIYpWCGjMmrcROMqotZwLzapAaTCi7pHMcO1gMwUycq45NBwTQqS0O9JCyVYb2yF9MskKDSXLdxUThRjRVLhicy4ET82uQ4apQH0V7awnQ8acI4Ybjrou8cj0FWSabS4HeS6XZM5VAOJX5mKYyrDLKAzk49dwnLMgXBrYOtkPclAF7d8159LiToft68jqJwLrJPjevtfVMnstJzR2JhVPHPtxtQuzE2tIP+njVMbvZ1kXCbp5m+dFqUCrIJiUSHkGpkVKwco09y9F7AF1ZRZt85Vtwnk5rvfBhftJjlpnxyK9Hx1rbvPfdekce8d54He/M63kDj/mH/qn/3u9WXlS6lQ+Vj2vXPX8Tc+jtWGXwF3gbJx8=</latexit>Sample time series
SLIDE 14
Porfiri & Ruiz Marín’s result (2018)
TEj→i = ✏2G(2)(Wij) + O(✏3)
<latexit sha1_base64="TzJLhlFMlL1as2418Nct38Ps7Po=">ADQXicbVJdixMxFE3Hr7V+dfVRkLCl0KIsnbqgL0JhEX3SFdvtQqctaeZOm91MiSZlRLy5C/xZ/jsfxB8tV9E9/EFzNtXTq7Xhg4c849mZO5d5pxpk27/a0SXLl67fqNrZvVW7fv3L1X275/qGWuKPSp5FIdTYkGzgT0DTMcjIFJ1yGExP9gt9cApKMyl6ZpHBKCUzwRJGifHUpPY+khkoYqQSJAXbe+nG9jgyEjOHX+AIMs24FOMOfjW2zU7LNQcTy45dCz/GUrMnBJu37qmPe986lqTWr29214WvgzCNah3dz59/P0olgeT7crXKJY0T0EYyonWw7CdmZElyjDKwVWjXENG6AmZwdDIqoe2eXtHW54JsaJVP4RBi/ZamPT0gtHtBAUFdWdBJDQnLuPzZjnLNclxs5BzUaVqOsHkebuA3APE6Qc/4hDvS0EVGNg0+bQr0uFqJOADlWlKRGwjMtVu6A+MOCQm4n5Yph5Gis3mJlLFmyv3MyFAuWHn3EHEjEM9fFLv/HMtmVJkS1KtF+nUxy2mpi9qBfk/bZib5PnIMpHl69t6Lck59htSrBOmQJq+MIDQhXz8J0ThShxi9d1W9CeHul8FhZzfc291751ei1a1hR6iHdREIXqGug1OkB9RNFn9B39QGfBl+As+Bn8WrUGlbXnASpV8OcvCj0UrQ=</latexit>G(2)(x) = −x + (1 + x) log (1 + x) ⇡ 1 2x2 for x ⌧ 1
<latexit sha1_base64="NmshCSNFobHpaEg/uYMTYq6IlSU=">ADPHicbVLPb9MwGHXDr1F+dXAEIYuqUqtB1UQTcEGq1AOcpiG126SmK47zJbXmOJHtjFRW/ivO/BlIcGWcEFfOuGlBzcZ3env+znfC/IOFN6MPjacK5dv3Hz1s7t5p279+4/aO0+PFJpLilMaMpTeRIQBZwJmGimOZxkEkgScDgOzkYr/fgcpGKpGOtlBrOExIJFjBJtqXnr4O2p6Xq9slv08Bv8osB7uOvuFT2fp7GpUIl9kmUyLbAfSUKNWxqvLE497GsotMFRKnGJrco5duet9qA/qApfBe4GtIcvn4x+eE8/HM53G1/8MKV5AkJTpSauoNMzwyRmlEOZdPFWSEnpEYphYKkoCamerhJe5YJqwcRKnQuGKbne2RsTszKw0ELeuKikKISM7tZTHjnOWq3mAg5yDPk7qF7fNwBx8AhBsHY/t3XTxKBZWgYXvIul2TJW76Aj7SNEmICI1PAlVO7YE+h0j73O5Jt1fsnihfbn6Kuv9TAiQ5dT7N0FEzKHtPm97f6cqpmbZkESpZRJYuwnRC3VZW5H/06a5jl7PDBNZvnmt1aKcY53iVZJwyCRQzZcWECqZ3RemC2Ijom3emjYJ7uW9XwVHXt/d7+/t5EYonXtoMfoGeoiF71CQ/QOHaIJougT+oa+owvns3Ph/HR+rVudxmbmEaqV8/sPTroOQg=</latexit>where
Takeaway: To leading order transfer entropy from j to i depends
- n the strength of the weight from j to i
Obvious next question: By calculating next term in the expansion, can we quantify the effect the global topology of the network has on the computed transfer entropy? Setting 𝜗≪1 allows the use of perturbation methods to approximately calculate Transfer Entropy in terms of the weights Inverting gives an approximate formula for Wij in terms of transfer entropy
SLIDE 15
An example to show that next-order terms matter
The transpose 𝚫⊤ of a directed graph 𝚫 has the same vertices, oppositely- directed edges, and a weight matrix W⊤ A 50-vertex, 282-edge directed Barabási-Albert network with weight matrix W, random weights
Two networks 𝚫 and 𝚫⊤ that behave differently
Procedure:
- Compute dynamics for 105 steps
- Repeat 100 times
- Compute TE from time series
- Estimate Wij from formula
SLIDE 16
Result: Distribution of error much wider for 𝚫 than 𝚫⊤
Nonzero computed weights vs exact weights Error in computed weights
SLIDE 17
Strategy for analyzing the dynamics
- Recast the system as a Markov chain, dependent on a
small parameter 𝛝
- Calculate the stationary vector of the Markov chain
via perturbation theory in 𝛝
- Use the stationary probability vector and the
transition law to derive a formula for transfer entropy in terms of the weights Wij
- Invert to get approximation for the weights in terms
- f the pairwise transfer entropy
- Apply to numerically-generated time series of the
model
SLIDE 18
To analyze: recast as a Markov Chain So, a quick review
Consider a discrete-time finite-state Markov chain Z(t), i.e.
Z(t), t ∈ N, takes values z1, . . . , zM in a space Z of cardinality M ν(t) ∈ RM
+ is the probability vector νi(t) = Pr [Z(t) = zi]
Transition matrix P with entries Pij = Pr[Z(t + 1) = zj|Z(t) = zi] Then ν evolves according to ν(t + 1) = ν(t)P The long term behavior is ν(t) − − − →
t→∞
π, where the stationary vector π satisfies π = πP under mild assumptions on P
<latexit sha1_base64="X5bjYzMKI/bhXlSAwtGYkJ7/es=">AFI3icbVTNbtGEKZVtU3Vv7g9FgEGFQ0kiGCIRov2UsBALrkUAs7CaxVieVyKG283CV2l3Jkhre+Si+5Jm+RW9BLD32EvkNnKdkwk+6BHM7PN/MNPzKrlHR+Ov17b/DR8ONPr312ejzL786uvb+98ca2Ak+FUcY+y7hDJTWeukVPqs8jJT+DQ7fxDiT9donT6xG8qXJR8qWUhBfkSvcHd1iGS6kb6bGUl9iOWLAgPrvr703A5MaWMn9Ksuax+0kBs/P0cGaq5pu8WaTJjKjXeTy/RDJTNwVcIMRdmeCqOWtjMAUIbnOpuZJ+A/Gj+KoT0zX16jX6rU3v/x7QHPgVQmVNxjPZFa5ReGO7qlSGul+AzWwzP9val6lctFfQJ5ZrJwNRIGArX0A8i+FC+hWgpmd0oziGWdrI5+0WJ8DcT+4FnOfwEgLoFhPia9AV6q59DLg2ak1b4EKYQG0J3gTEjhHBMwtuVmvHpQJuWhLyHDF15IEdXrTbywcrny3FpzMfMG1pN4TeLpgVWyXgCFyu02G3G+e49cntjMZQyAkd+V+wYkiuMQtdujlrnaKGUKgfuXF1WAcIBbYnWM2Ko82s1pLfH08Npd+BDI9kZ42h3Zun+3r8sN6IuacFCEfw8mVZ+0XDrpVBXrVDksc5X+KcTM1LdIumE3ILB+TJoSAahdEeOu/o4GbJSbJoQgy1aPsRV+RY8FpRs6VUStaun9BgrdCuy94ITRCcN0a5/mQ328ABPEbMd4Od0EeUwAOjhUWPSyxc7YwYhovhClLTrtkPHPtnACZwsIzUoz14R175jZ8NT286XWaNv50XUF10uF42QyPrq6jx9Jrx0blNmNG7g5N6PBef/xea1L35eNFJX9Y4txYpakZQh/DAgl5aUpTZkcGHpaxIgVtxyQfp1IxJI8r4cPjSeHB0mPxz+OvR+Ph4J5Vb0XfR9HdKIl+io6jh9EsOo3E4I/Bq8HrwZvhn8O3w3fDv7apg71dzbdR7wz/+Q9LorC5</latexit> SLIDE 19
Recasting as a Markov chain: Main Idea
The states are binary vectors The state vector is a 2N-dimensional probability vector 𝜉 with components
Z1 = . . . , Z2 = 1 . . . , Z3 = 1 . . . , Z4 = 1 1 . . . , . . . , Z2N = 1 1 1 . . . 1
<latexit sha1_base64="N8RZ1u6SaJ2uE4+oUFuEm3blfcw=">AEa3iclVPNbtNAELbAMX8tKU34LAisQhirKhC6gSL1wqoqUtBVxiNbrsbPqetfaXQciy8/QV4I7L8BD9MKVHlgnpor7g8pItmbnm/l2ZvfbIOVMm273p7u23rhz97Gfe/Bw0ePN7e2nxqmSkKQyq5VMcB0cCZgKFhsNxqoAkAYej4GSvxI9moDSTYmDmKYwTEgsWMUqMDU2XfA5RMbPvU8TjN4hP4CYiTxNiFHsa4G6yPdXfv4slEZXSxDhRWIb+W1L0buGAv8fxaubusC3pti9qYvbUfAFVBLlvc/7xT/JcJ0M18k8X7F4avxistXsdroLQ1cdXDnN/vtT/e37+emBvZgzP5Q0S0AYyonWI9xNzTgnyjDKwTJnGlJCT0gMI+sKkoAe5wtBFKhlIyGKpLKfMGgR9VqrJQM8zksMBC3qiI5CiEjG7WYx45xlup6Q8ZBzZJaC7kdeGqk5Lre2eo2qIX2AcKqsYEVI0Z7UlAFBmpctAoWyPMFfKEySYg9VJ8EuhZwqViuZW1aeLqhFW5Kur5TAhQxah3UFEzKGJ283e36pFpD4JSbSeJ4Ft5xJX8bK4HXYKDPR23HORJpV01osyjgyEpUPD4VMATV8bh1CFbPXiOiUKEKNfZ6eFQi+LIerzmGvg3c7rz9apfSdpW04z5wXzksHO2+cvPBOXCGDnV/uL/c3+75+ljp/G08XyZuZWNTtOzRqtP2r0c94=</latexit>νi(t) = Pr[X(t) = Zi]
<latexit sha1_base64="okzMPUF6nEODqgOuD9qGJ1ySgJ4=">ADH3icbVLitswFXcx6TpK9Pu2o2oCUyhBDt0aDeFwGy6KhlIZkJtY2T52hEjS0aSpwRj6K90237F92VbucHSjf9h8pJWsYzvSA4Oude6V7pJCVn2njeRc+5cfPW7b3+ncHde/cfPBzuPzrRslIUFlRyqZYJ0cCZgIVhsOyVECKhMNpcnbU6qfnoDSTYm7WJUQFyQXLGCXGUvHwSiqmB2Y5/gNDmcqWG7h+5hF8dD1xt4m8HXg74A7xce/fvb3Dmfxfu93mEpaFSAM5UTrwPdKE9VEGUY5NIOw0lASekZyCwUpAd1ZshGjyTIozqewSBm/YwehydyP6lYDQZuorMUMlJxe1nOGeV7ibUHFQ50WnhbogZmWk5Lrb2eVr8Ai/A0h3jc3tA/r4SAqwEDnLojGzwIBXygsiISOuQJLoJ7IEh8yE3H6Fcf1QsXxlQtXum4+EwJUE0z+VRCRc3D9F+7kb9WG6U5Cq3XRWLbWfSV7W/J8WVCZ7HdVMlNVuWqtlFcdG4tYsOGUKqOFrCwhVzH4jpiuiCDXWUgNrEP+qHa6Dk8nYfzk+PLZOmaJt9NFT9AwdIB+9QlP0Fs3QAlH0EX1GX9BX5Pzfnu/NimOr1dzWPUCefiDwpJBW8=</latexit>The transition law is determined from the dynamics
SLIDE 20
Recasting as a Markov Chain: Ugly Details
Our RBN Model:
Pr ⇥ Xi(t + 1) = 1|X1(t) = x1, . . . , XN(t) = xN⇤ = ✏ 2 41 +
N
X
j=1
Wijxj 3 5
<latexit sha1_base64="iOzG1l8i5mc1OqhZEy4QmEPQKxo=">ADm3icbVLdahNBFN5t/KnxL7WXIhwMgYolZEJBeyEc1NESoSkCWRjnMyeTSadnVlmZqth3UfyCXwPwUvfxNlNLEnrgWG/+b7zt2fOLBHc2Fbrt79XuXP3v39B9WHjx4/eVo7eHZhVKoZDpgSo9m1KDgEgeW4GjRCONZwKHs8tuoQ+vUBuZN+uEpzEdC5xBm1jprWfgQqQU2t0pLGmPV0DoHAyI5h9DnjORxZeA3kFbwDAt8LjhRcf9W4mPnHiprHDi+ZbocKD5fGEnAFB1x9HlJ8DEcKEkbCoRVyEwaTzNlkWZvAwdTjMOyxyKTMvrTNavdVslQa3AdmAeufsz/DLaeNnb3rg/wpCxdIYpWCGjMmrcROMqotZwLzapAaTCi7pHMcO1gMwUycq45NBwTQqS0O9JCyVYb2yF9MskKDSXLdxUThRjRVLhicy4ET82uQ4apQH0V7awnQ8acI4Ybjrou8cj0FWSabS4HeS6XZM5VAOJX5mKYyrDLKAzk49dwnLMgXBrYOtkPclAF7d8159LiToft68jqJwLrJPjevtfVMnstJzR2JhVPHPtxtQuzE2tIP+njVMbvZ1kXCbp5m+dFqUCrIJiUSHkGpkVKwco09y9F7AF1ZRZt85Vtwnk5rvfBhftJjlpnxyK9Hx1rbvPfdekce8d54He/M63kDj/mH/qn/3u9WXlS6lQ+Vj2vXPX8Tc+jtWGXwF3gbJx8=</latexit>Pr ⇥ Xi(t + 1) = 1|X(t) = z ⇤ = ✏ ⇥ 1 + e>
i Wz
⇤
<latexit sha1_base64="bL8fGhqMXSuCa7jnZD/i8Oes3GM=">ADWnicbVJNa9tAEF3b/UiUfjhtb70MNS4pLcYygfZSMOTSU3HAjg2SYlbrkbNktRK7qxRH1al/Kr+jh0KhP6THrmS1WEkHFr19M292NDNhKrg2w+HPVrtz7/6Dh3v7zsGjx0+edg+fnekUwxnLBGJWoRUo+ASZ4YbgYtUIY1DgfPw8qT0z69QaZ7IqdmkGMR0LXnEGTWnaFP1HgC4yMB4vznBdwZOAtuG/gI7jwFRb2XuJr8BVfX5gABx7LFV9fEw1F4mEOolrxbjk57mvYvBNkhYwh+tau+z2hoNhZXAXuDXojV/f8G+ni5vJ8rD1w18lLItRGiao1p47TE2QU2U4E1g4fqYxpeySrtGzUNIYdZBXbSmgb5kVRImyRxqoWKe/K5m6QV76ULKi6dHRCiOaCfvYmgvBM90MyDETqK7iZgm7+aAPnxFXdQVT23sXThLJFBrcFdlqt2QBji/xC0vimMpV7tNQF5NWDXWF3aKpuduO+mr8lY047mUqApv9E9B5Vpgz3XG/1VUyj5JzGWm/i0JYbU3Ohb/tK8n8+LzPRhyDnMs3qv7W+KBNgEij3DFZcITNiYwFlit5AbugijJjt9Gxm+DenvtdcDYauMeD41O7EmOytT3ykrwiR8Ql78mYfCITMiOMfCe/W61Wu/2r0+7sdw62oe1WrXlOGtZ58QfI7BFg</latexit>Letting z=[x1,…,xN] this becomes Taking a product of such terms yields the Markov transition matrix
Pij(t) = Pr [Z(t + 1) = zj|Z(t) = zi] =
N
Y
k=1
- 1 − e>
k zj
- + ✏
- 2e>
k zj − 1
⇥ 1 + e>
k Wzi
⇤ = P(0) + ✏P(1) + ✏2P(2) + O(✏3)
<latexit sha1_base64="eVBqhjbT1cFYeWvgEyXV/EeWdac=">AEI3icbVPLbtQwFE06PEp4tbBkc8WoaEbQ0XgYCTaVKnXDqgxSX2IyM/I4ztSt40S2U1SMv4Q1H8KW7hBCYsG/4DxaTVosRbk+95zrk9zrecaZ0v3+H3+ldev2nbur94L7Dx4+ery2/uRApbkdJ+kPJVHc6woZ4Lua6Y5Pcokxcmc08P56U6RPzyjUrFU7OnzjE4SvBAsZgRrB83W/eFoZhic2I7uQrAF4UhCyGmsx/Cxo+EloO4WfJ6ZEwtfCqTaMQuhZItjPQEIXoCTZTKNZubUhchOza6tioSmfHUAwSZQl7fTUKdZXbAq0XWHhDRTjKcCoOYPbrI3XZFLRe0QOekS7/CateodOiAsTY6mptPv2uXzCg1oOmgBAcVmGB9TDA3723nivC6G8zW2v1ev1xwM0B10PbqNXL/+XcYpSRPqNCEY6XGqJ/picFSM8KpDcJc0QyTU7ygYxcKnFA1MWV/LWw4JI4le4RGko02FiW7KGJKXJUENvMqDiMc65O2zBOGe5ahIMzTmVZ0nTwnI92IBdSqPawZ4bIgQ7qSCSaroscm4r0EIQCvqJpEmCRWRCPFd27ApWI8HdOo2qpsji51t8pkQVNrx4EqBxYLTNnrVHlyqSqRh2eBEqfNk7uwWTVPXcwX4v9w41/HbiWEiy+uvdbk456BTKC4MRExSovm5CzCRzPULyDGWmGh3rYpJQNf7fjM4GPTQsDf8MGxvb9czseo9857HQ95b7xt75038vY94n/1v/s/IvWt9ZF62frV0Vd8WvNU6+xWn/AfJrVnk=</latexit>Allowing xi+ ∊{0,1}
Pr ⇥ Xi(t + 1) = xi
+
|{z}
2{0,1}
|Z(t) = z ⇤ =
- 1 − xi
+
- |
{z }
=
- 1,
xi + = 0 0, xi + = 1
+✏
- 2xi
+ − 1
- |
{z }
=
- −1,
xi + = 0 1, xi + = 1
⇥ 1 + e>
i Wz
⇤
<latexit sha1_base64="kKj8AjQld5eCXdJZFronj+JLBIQ=">AEaHiclVNbxMxEN2koS3hoykcEOIyalSUKk21G0BwiVSpF06oH6JOI283klq1etd2d5CMP6bSNw48ytwNqFK2nJgpJVmZ+a9eR6P41xwbcLwZ6W6Uru3urZ+v/7g4aPHG43NJyc6KxTDY5aJTJ3FVKPgEo8NwLPcoU0jQWexpcH0/zpFSrNM3lkJjkOUjqWfMQZNT403KxMyKECEvNxH87OectAG6Id6AEpZIqVpSh/Tq0bXfO3dACKVtahYmzhEtiw92IOe+w+eW2enBt5JrAFDv1RcpiMCRaUEHZizAVF8fGF2POsSqeFyYnskxjGXlvmjaQfRLrz0uLZH9SAEQiC8jvQigjKZV3qrt4FgrnIJMAdErXAjpezn+I6NxWEf1TBZTd+r5DG3BouTsnJsvhdDqgsuWgPmw0w72wNLjtRHOnuf9p5RU9IZ1Df1k/SJKxIkVpmKBa96MwNwNLleFMoPMD15hTdknH2PeupCnqgS3P5WDbRxIYZcp/0kAZrW8vQo6igZ3mUDK3nNGjBEe0EL7ZmAvBC71cYLEQqK7SZQmLfLANHxCTuYIjv4kRHGSKTS4CPJqZ0EHdSLxC8vSlPrBEhpr1/eE5VyJ8DtmtFslERN/9xyPZcSlet3rxFUjgU2o91m9y+qjCxJtjTVepLGXm5KzYW+mZsG78r1CzN6N7Bc5sX8tD43KgSYDKavDhKukBkx8Q5liv7AnZB/Voa/zanmxDdvPfbzkl3L3q9+ajX4n9YGbrwYtgK2gFUfA2A/eB4fBcAqv6qr1Y1qY+V3rVF7Vns+K61W5pinwZLVtv4A/iJw5g=</latexit> SLIDE 21
Setting up perturbation calculation
Expand 𝛒=𝛒(0)+𝛝𝛒(0)+𝛝2𝛒(2)+… Separating by orders yields Actually solving this was really hard…
O(1) : ⇡(0) ⇣ I − P(0)⌘ = 0
N
X
j=1
⇡(0)
j
= 1, O(✏) : ⇡(1) ⇣ I − P(0)⌘ = ⇡(0)P(1),
N
X
j=1
⇡(1)
j
= 0 O(✏2) : ⇡(2) ⇣ I − P(0)⌘ = ⇡(0)P(2) + ⇡(1)P(1)
N
X
j=1
⇡(2)
j
= 0
<latexit sha1_base64="QgNGutko3ZqNh0hz8RdfXE5EuQ=">AEaniclZPfaxNBEMcvadR4/mrik/RlMbakGMNtChCodAXfakR+guyadjszSXb7u2du3uVcty/Kfjqs3+Eez+sSVsrDhzMzsx353PM7CwWXBvP+16rzXu3X/QfOg+evzk6bP1VvtIR4licMgiEamTGdUguIRDw42Ak1gBDWcCjmfne3n+AKU5pE8MJcxTEI6lzgjBobmrZqKQmpWTAq0k9ZF2+/R+RLQn1EYn6adr3tDBEBgel+RG/QqIwQxecLs420A7yXGRtCxGdhNP0bAdnp/u5dnpWqbd2cM8lxF3uQiDWXETyqlxSdkR/6vjH7BRWd67CwEXCN7fCE4HtzEM/o8hL3+9hF9x3YU1KLGm6x2v7xWGbjq4cjpOZSM7rm/Ej1gSgjRMUK3H2IvNJKXKcCYgc0miIabsnM5hbF1JQ9CTtFiTDG3aiI+CSNlPGlRE3c1lyQGepHkOJMtWMzrwIaCJsM3mXAie6NWCFBIB6iJcRVi+D2ifQC/Ijiwu4jRXiSZAgPLIktbBjPkEglfWRSGVPopoTOdje2FxWiIsFtOrgcC1H5KVut51KCysaDKwWVcwEd3OsMfquKyApySkOtL8OZxc3RV/P5cHbcuPEBO8mKZdxUv2tzQWJQCZC+btDPlfAjLi0DmWK23khtqCKMmNfp2s3AV+f+03naNDHw/7w87Czu1vtRNPZcF46XQc7b51d54Mzcg4dVvtRb9Zb9fbaz0a78aKxUZbWa5XmubNijVe/ALYrX8E=</latexit> SLIDE 22
Solving the perturbation series I
The matrices P(j) are 2N×2N so we can only write them down explicitly for small N, need to work “in the abstract,” hybrid pencil&paper/Mathematica workflow
P(0)
ij
=
N
Y
k=1
⇥ 1 − e>
k zj
⇤ =
- 1,
kzjk = 0, 0, kzjk > 0.
<latexit sha1_base64="BKu1mBxrBvV8k+6RFJm7nuqLqc=">ADyXicbVLNbtNAEHYaoG34S+HIZUlEohsqMKuACRygGBVAUpStlU2u9HjvbrNdmd1ILZ94m74Fj8E78BCs4DihJGsHX/fLOzM+MlnClt279qe/U7d+/tHxw27j94+Ohx8+jJRMWpDCmMY/lhUcUcCZgrJnmcJFIJH4dxbnBb8+TVIxWIx0sEZhEJBQsYJdpAbvN26GbsKr/MOvZxjt4inMjYd7OFcR2DnuUIcwj0tDw6yEvEbiLS6zjBN24VwhLFs718fqcGV0DGcMehExk1JSm8hJyugFwiKWUWaExWV2F2G84uwt7h2ye2UeEP46i9ts2T17ZWjXcdZOa/Dh9+3Pw0/doXu0t4/9mKYRCE05UWrq2ImeZURqRjnkDZwqSAhdkBCmxhUkAjXLVk3NUdsgPgpiaT6h0QptDclI2eWFRwImlcZFfgQkJSby0LGOUtVNSCDlIO8jqolbOZDbXQG4K8rGJnJOeg0FlSChk2RqbYETZOxgG80jiJiWoaJp/KpSbiaG+ZmB3TLKYeEZfGXV+OL3m8KJhXBZFfAhACZT/v/FESEHFpOt9X/q1ohlTdmJFJqGXnmfRHRc7XNFeD/uGmqgzezjIkXbfHcEHKkY5RsdbIZxKo5kvjECqZGTCicyIJ1Wb5G2Z1nO1F2XUm/Z5z0jv5YnZoYJV2YD2znlsdy7FeWwProzW0xhatNWuvau9rg/rn+tf69/pNGbpXW2ueWhWr/gDh7k4pQ=</latexit>where kZk = PN
k=1 xk
<latexit sha1_base64="kTYnG+NZyJuV2b+48y9QrqCLq24=">ADQnicbVJNb9QwEHXDR0v4aAtHLhbpShzQKlVgkulXrhVBVpd1uRhJXjTLW2k5kOy2rKD+AX8MV7vwJ/gI3xJUDzjagTZeRL28N8+ZGU9ScqaN73/fce7cvXd/d+B+/DR4yf7B4dPZ7qoFIUpLXihLhOigTMJU8Mh8tSAREJh4tkedrqF1egNCvkxKxKiAXJcsYJcZS8wPvegEK8FEkCyXq9w0+wZGuxLxengTNhzP8cb48sln+0F8H3gZBzUxfn80NmN0oJWAqShnGgdBn5p4powyiHxo0qDSWhS5JDaKEkAnRcr7tp8MAyKc4KZY80eM26g03LJIjrVgNJm76isxQyUnH7s5xzirdT6ih4qCuRL+EzfvwAJ8BpF0FEzuyAJ8WkiowsGmy1d6QDXYjCde0EILItI5IopvQXhxyEzE7fCNF0SK5QsTqfar6e3g980zHqG2baBSQmqCUf/HETmHLzglTf61ozvR5rIrReicT2J4hZ6NtaS/5PCyuTvYlrJsuqG4/VsopjU+B2n3DKFDVxYQqph9YEwXRBFq7Na5dnWC24uyDWajYXA8PH438sbjbon20HP0Ar1EAXqNxugtOkdTRNEn9Bl9QV+db84P56fz6ybV2ek8z1AvnN9/AH6mEP0=</latexit>P(2)
ij
=
N
X
r,s=1 r>s
8 > < > :
- 2e>
r zj − 1
⇥ 1 + e>
r Wzi
⇤ 2e>
s zj − 1
⇥ 1 + e>
s Wzi
⇤
N
Y
k=1 k6=r,s
- 1 − e>
k zj
- 9
> = > ; = 8 > > > > > < > > > > > : PN
r,s=1 r>s
⇥ 1 + e>
r Wzi
⇤ ⇥ 1 + e>
s Wzi
⇤ , kzjk = 0, − ⇥ 1 + z>
j Wzi
⇤ ⇥ N − 1 +
- 1>
N − z> j
- Wzi
⇤ , kzjk = 1, ⌦ 1 + z>
j Wzi +
h e>
I1(zj)Wzi
i h e>
I2(zj)Wzi
i↵ , kzjk = 2. 0, kzjk > 2,
<latexit sha1_base64="ygQwWNR9ce7LpAFKILDLYMY1Y=">AGhXicnVRNbxoxEN2ElqT0K2mPvYyKEhEFEItQ20vTSLm0lyiVEhKJStjDHw7lLbmyq1/A/7B/o3em0PnV0WELaRLUEHmbejN8D+6NBVe60fixslp48LC4tv6o9PjJ02fPNzZftFUS8pOaCQiedYjigkeshPNtWBnY8lI0BPstDc6SOKnV0wqHoXH+nrMugEZhnzAKdHo8jcLDHAd+YZf2nNTae5YgG14D56KA9/gd09pQkdGQhU+l3wPJCwB8oi/tCJ9hAeybdKtAE5stzT0dj+OZfQi3BSz680DsTYAcdu3PMKaIMtxmC7kqRtl71JmhliqNZdT3zWhCmSQEXgh+wJpJznuFQTUsNRoftzsqHT3EOiVEplQmHT3emzIQ0NRd2VTD/yfYPeR4+5WM5ZVyKhsY6ORDAy2YpFBYxZAJplVg3xBP6TwGF6AbszvXzsY5JQm+dORbtRo7rIBwWpwlRPmIpxK4nd6fHYuxcQfUGJMJ+sb1xbSW5p52963J7VnN137orvlPSGls36XMvMaCSgPGoPmtXJmLCwnw2Jv1Fu1BvpgmXDzYyk60jf3N1zetHNA5YqKkgSnXcxlh3DZGaU8FsyYsVG+OgkSHroBmSgKmuSR8EC1vo6cMgkvgJNaTe0lY+5djtmiTGQmoXI2rQZwMSCzxsyIXgsVoEGBYLJq+CRQr5erAFh4z1MwbH+Oq4cBCFVDLN8knIduK0UMK/5lcaBQFByTzSU7aDBSfTIfD90mU3uxOZ/LKL+ET7fEJ7IaG9nMDkEnbac4ySDgUrOxWy81pVupZ6NGQKnroIf9JXOlbsYS52xTqwH7qGh+M4kwdjg1iAjiB5kqHPJaNaXKNBqOR4wUAviCRU48NdwtFxbw7KstFu1t1WvfW5Vd7fz4Zo3XnlvHYqju8dfadj86Rc+LQwvfCz8Kvwu/iWrFWbBXfTKCrK1nOS2dhFT/8AfLoGhc=</latexit> SLIDE 23
Solving the Perturbation Series II
Solve for the stationary vector order by order
SLIDE 24
Calculating Transfer Entropy
from node 2 to node 1
TE2→1 = X
x1
+,x1,x2
Pr h X1
(t+1) = x1 +, X1 t = x1, X2 t = x2i
log Pr h X1
(t+1) = x1 +|X1 t = x1, X2 t = x2i
Pr h X1
(t+1) = x1 +|X1 t = x1
i
<latexit sha1_base64="S1KvAe2y8n9wLriRS5qpG1LnLg=">AEXicjVPaxNBFN501db4K9Wjl8EQqKSE7FKwl0ihCHqRCkayKRhMnmbDJ2dWdma8u4XvXuX+N9Ch4139C8C9wdhNLtpXig1m+/b73vZ15+2aScKZNu/2jsuZfu35jfeNm9dbtO3fv1Tbv97VMFYUelVyqwYRo4ExAzDYZAoIPGEw+HkeD/XD09AaSZF15wlMIrJTLCIUWIcNa79Ri6wTEARI5UgMdjus+zIhtjIOtgncZje3oUjJvb7ulWmFULy4GymENkhgMn2i3TDB5ndNx0yXljOmcFigsUIgVm83NaOFmMuZxZEi1F5RB71FV1RalPpf/7kLZWhcq7db7SLQZRAsQX3vxfdfX1/3D0Yb6t46mkaQzCUE60HgbtxIwsUYZRDlkVpxoSQo/JDIYO5l3UI1v8nAw1HDNFkVRuCYMKtpYtXSDkc01EDQrKzqaQkRS7j42Y5yzVJcTLKQc1Elc3sJqPdRALwGmyx103QEaF8KqsDAqsntdkFmqIoFvKEyjomYWkwmOhu6gkWTMXezZOrLdmKVv2XlfCFVvGrolwz9ywYmBKhsGJ47iJhxqAfb9fCvq2BKZ7Qk1vosnrjzxcTM9UtJ/+lDVMT7Y4sE0m6bI/TopQjI1F+PdCUKaCGnzlAqGLuByM6J25SjbtEVTc6wcVBuQz6YSvYae28cjP01FvEhvfQe+RteYH3xNvznsHXs+jlUHlXeV95YP/0f/kf/a/LFLXKkvPA68U/rc/6K1Y4A=</latexit>- Use the transition rule and the asymptotic expansions
to approximate the various probabilities and conditional probabilities
- Use some tricks to avoid dividing by small numbers
- Get terms like
SLIDE 25
The next-order correction
G(2)(x) = −x + (1 + x) log (1 + x)
<latexit sha1_base64="bIP1EncozD+4yDhQRciq4nAPJB0=">ADEXicbVJNb9MwGHYzYCV8bePIxSKq1GpQJdUQXJAq7QAntEntNqkJk+O8Sa05TmQ7o1WUH8CZH8KV3RBX7kj8AcTPwEkLajZeKdLj53kf57H9hjlnSrvuz461dev2ne3uXfve/QcPH+3s7p2orJAUpjTjmTwLiQLOBEw10xzOcgkDTmcheHtX56CVKxTEz0MocgJYlgMaNEG+p8x3nzvuyPBlV/McCv8fMF3sd9b38x8HmWlA2qsOlyh25T+Cbw1sAZ4+Pfv7rbL47Odzs/CijRQpCU06UmnluroOSM0oh8r2CwU5oRckgZmBgqSgrI5TYV7holwnEnzCY0b1u5tWiZeUNYaCFq1FRVHEJOCm58ljHNWqHZDCQUHeZm2I2zuh3v4HUC0TjAxV+bhw0xQCRo2TSbtiqyw7Qv4QLM0JSIqfRKqamY29DnE2ufm8rXj+ZIlc+3LelW1+5kQIKvZ6J+DiISD4z1zRn9dDdOKXJUqWUamrgp0XN1XavJ/2mzQsevgpKJvFif1mhxwbHOcD0eOGISqOZLAwiVzLwXpnMiCdVmiGwzCd71d78JTkZD72B4cGxGYoxW1UVP0FPURx56icboLTpCU0TR/QZfUFX1ifryvpqfVu1Wp215zFqlfX9D7To/Xg=</latexit>G(3)
ij (W) = Wij (Wij − di − dj) + log (1 + Wij) (di − Wij + (1 + Wij)dj)
<latexit sha1_base64="DipDZuoXv/VqJpN6cLqajy+lnqE=">ADbXicbVLNb9MwFHdaPkbGRwfsBEKGqlrLoGrKJLgVdoBTmhI7TqpKZXjvKTuHCeynaEqivgrQeLCbWf+BZw0oGbjSZaefx8vL37PSzhTejD4aTWaN27eur1zx969e+/+g9bew1MVp5LChMY8lmceUcCZgIlmsNZIoFEHoepd35c8NMLkIrFYqzXCcwjEgoWMEq0gRatbx8WGVvlX7Lum17enfbwe4ynBsKr3OUQ6G51e0vmDkrjLErWbjUPRsXcYhdHodZ1zmshL0cb4yFocKMakuAyzJVlUWrPegPysDXE6dK2qOXv74fXD6fniz2rB+uH9M0AqEpJ0rNnEGi5xmRmlEOue2mChJCz0kIM5MKEoGaZ+VL5bhjEB8HsTRHaFyidmfbMnbmWcGBoHmdUYEPAUm5+VjIOGepqgsySDnIi6jewnY93MGfAPyqg7EZh4OPY0ElaNg2mW43YI5tV8BXGkcREX7mEk/lM1OwfGXm8HqtrN5SVcWt7yuZ0KAzGfDfw4iQg5t51V7+NdVIrWMxIptY48025E9FJd5Qrwf9ws1cG7ecZEklZ/a7g5VjHuFg97DMJVPO1SQiVzMwL0yWRhGqzoLbZBOfq3K8np8O+c9Q/+mxWYoQ2sYOeoBeoixz0Fo3QR3SCJoiS2vXemztN34395tPm820oZVeR6hWjQP/gDLHBlQ</latexit>dj =
N
X
k=1
Wjk.
<latexit sha1_base64="Y+tyzCpga5/wxcAXTbSE3Tf+8=">ADCHicbVLatwFNW4r8R9TdpduxE1A12EwR4C7SYwkC5KFyGFmUzAnhpZvYoI8lGklMGY+i6H9JFN82udNl+RKH/0I+o5tEyTnrBcHTOPfKRdJOSM218/1fHuXHz1u07O7vu3Xv3Hzs7j061UWlKIxpwQt1lhANnEkYG2Y4nJUKiEg4TJL50VKfXIDSrJAjsyhKkguWcYoMZaKu0/S+Bwf4khXIq7nh0Hz7hP4vp83vTjruf3/VXh6yDYAG/46vfn7tv9k/ivc7PKC1oJUAayonWYeCXZloTZRjl0LhRpaEkdE5yC2URICe1qtDNLhnmRnhbKfNHjFur1tyiY1ksNJG3ais5SyEjF7c9yxjmrdLuhoqDuhDtCNv74R4+Bkg3CUb2pgJ8VEiqwMC2yaZdkw12IwnvaSEkWkdkUQ3od0w4pCZiNs7N14QKZbPTKSWq6bdz6QE1YSDfw4icw5esO8N/rpWTCtyTYTWC5HYuIKYmb6qLcn/aWFlspfTmsmy2pzWalnFsSnwcipwyhRQwxcWEKqYfS9MZ0QRauzsuHYSgqvfh2cDvrBQf/grR2JIVrXDnqKnqHnKEAv0BC9RidojCj6gD6hL+jS+ehcOl+db+tWp7PxPEatcn78AXyo/Uc=</latexit>TEj→i = ✏2G(2)(Wij) + ✏3G(3)
ij (W) + O(✏4)
<latexit sha1_base64="M6DAKcjaG9PyS+IhAlEyuCwLHk=">ADV3icbVLNjxIxFC+gK+IX6NFLlRBZNYRhSfRiQrIx60nXBJZNGCl8wbKdjqTtrOGNP0bPWmyF2/+CV5dO4Ars+tLmrz5fbx57XuzhDOl2+2LQrF06/benfLdyr37Dx4+qtYen6g4lRQGNOaxPJ0RBZwJGimOZwmEkg04zCcnR1m/PAcpGKx6OtVAuOIzAULGSXaQdMq8yOiFzIy/fd2Ypa+jGz+B32IVGMx2LSwUcT0+zs2+Zwahe2n386h97gI8cunTW5kEmWZNZRUq4+WSbV8Lu/rRab7fa68A3E2+b1Hsvf3x78fPZ8HhaK3z3g5imEQhNOVFq5LUTPTZEakY52IqfKkgIPSNzGLlUkAjU2KzfxOKGQwIcxtIdofEarTR2LX1vbDIOBLV5RoUBhCTl7mdzxjlLV5gIOUgz6N8C7v1cAN/BAi2HfTdw3v4MBZUgoZdk+t2A1pc8QV8oXEUEREYn8yUHbmCPodQ+9yNUNc9X7L5Qvsy+7J5PRMCpB1rhxEzDnUvdf1zl/XGsm1bEik1CqauXazmanrXAb+jxulOnw7Nkwk6fa2jgtTjt32ZEuGAyaBar5yCaGSuXlhuiCSUO1WseI2wbs+95vJSafldVvdz24lemgTZfQUPUdN5KE3qIc+oGM0QBR9Rb/Qb3RZvChelvZK5Y20WNh6nqBclGp/AGqjGMc=</latexit>Finally, we arrive at where and dj is the weighted in-degree of node j
SLIDE 26
Solving for the weights: one more perturbation expansion
Tij ≡ TEj→i ✏2 = G(2)(Wij) + ✏G(3)
ij (W) + O(✏2)
<latexit sha1_base64="gnH70lN3LM2jYRPJgtK6gOsYMo=">ADbHicbVJdjxIxFC3gx4pfrLtvq6aRYCAawuBGfTEhIUafdE1g2YQClnIHynY6Y9vBkKb+TBPf9dXfYAdYhV1vMsntOfcue0940RwbRqNH7l84dr1Gzf3bhVv37l735p/8GpjlPFoMtiEauzMdUguISu4UbAWaKARmMBvfF5O+N7C1Cax7JjlgkMIjqVPOSMGg+NSt86I8vnDhP4kvIFJqGizJKImpmKbOetG9o5MTHmzlkCieYilsOmw2/wu6GtNmu2vN6PHc1/AxfFKy4FzW3Zq9FZe1ZFTYj676r1FtVCo36o1V4KtJsEnKrZcP27+ajz+fjPZz38kZmkE0jBte4HjcQMLFWGMwGuSFINCWXndAp9n0oagR7Y1UM5XPHIBIex8p80eIUWK9uSTjCwGQeSuV1GhxMIaSr8z6ZcCJ7q3QILqQC1iHZH2O6HK/gDwGQzQcdvI8DtWDIFBrZFfto16HCRSPjK4icmIJHWvX9w2JgNAQ4fdqygFRfDozRGUnt1vPpQTl+s2/CiqnAsrB83LzQrVCdka2NJ6GY39uNnO9GUuA/H9VMTvh5YLpN0c1vPhanA3j6Z8/CEK2BGLH1CmeJ+X5jNqPeb8f4seicEl/d+NTlt1oPj+vEnb4kWsceOkJPUBUF6BVqofoBHURQz9zxdxB7jD/u3BYOCo8WpfmcxvNAdqJwtM/t1Uc/w=</latexit>W(0)
ij
= h G(2)i−1 (Tij) W(1)
ij
= − G(3)
ij (W(0)) d dwG(2) (w)
- w=W(0)
ij
<latexit sha1_base64="ampS8YBzgF/jYtpViwIoxO+iNM=">ADonicbVLbhMxEN1tuJTl0hQe+2IRghKJRrshErxECupDUZFQK5KmUjZEjnc2MfV6I9vbKL8WXwKD/wN3kshaRlp5fE5c2ZmxzNfMSqV7/9292oPHj56vP/Ee/rs+YuD+uHLS5lmgsCIpCwV3MsgVEOI0UVg6uVAJzMGYzn1yc5P74BIWnKh2qzgmCF5zGlGBloVn953imKfphvuW3zZv+zpkEKvJqb132yYUdLFU8seB6Y1LEPbYegha/+UgVUi1EfHYSw0aeWKPD3bdMal5nbxtOoyN1BZVRkdLQ2t4VyprUu67VRedrDtq5met2vshSJDTKzesPv+IWh+05QOQ2nsvPZofsrjFKSJcAVYVjKSeCv1FRjoShYLwk7DC5BovYGJdjhOQU13M16CmRSIUp8J+XKEC9ZrbkmEw1TkHnJhdRsYRxDhjtiCMkYzuRugIWMgbpLdFrbzoSb6ChBVHQztIwboJOVEgIJtke2BA3yQg5rkiYJ5pEO8VyaiU1YDlk+UwbQTnicsJmN5yDsJMun8VmC8YNIJ3je6tqkB2WtY4kXKTzG27CVZLeZfLwf9xk0zFH6ea8lVW/a3l4owhlaJ8YVFEBRDFNtbBRFD7Xogsd0gZdfaLkJw9nvO5fdTtDr9C56jcGgWol958h57bScwPngDJzPzrkzcoh75H5yz9wvtTe1s9pF7VsZudWmlfOjtXCP+bhLAY=</latexit>Letting and we arrive at desired formula
SLIDE 27
Interpreting the correction term
G(3)
ij (W) ∼
W2
ij
2 (dj − di + Wij) + O(||W||3)
<latexit sha1_base64="q1Tyv1IH9M5eL4ZloHq7VyBqeY4=">ADSHicbVJNbxMxEHW2fJTwlcIRhCyiSolKo+y2Ao6RcoATFClpKmXTxeudTdx6vZHtLYoc/ziu3DnwAxDXckLcLIBJS0jWXr73jzveGbiKWdKt9vfKt7WjZu3bm/fqd69d/Bw9rOo2OVF5JCn+Y8lycxUcCZgL5msPJVALJYg6D+Ly70AcXIBXLRU/PpjDKyFiwlFGiHRXVhm8iw87sqWkcNG1j0MShYhkOU0moGZRSYE1gQw6pbiTR2X4Ssb1SCSUbT3QT7+EwI3pCTfvbcPM54P5/PTANqNavd1qLwNfB/4K1Dsvn3a/B8+HkU7la9hktMiA6EpJ0oN/fZUjwyRmlEOthoWCqaEnpMxDB0UJAM1MsuWLzrmASnuXRHaLxkq7vrlp4/MgsNBLWbikoTSEnB3c/GjHNWqM0EAwUHeZFtlrB+H97F7wCSVQU912ofd3NBJWhYN7lqS9LiaijgE82zjIjEhCRWduguXHY65G5ou6XLQ7l4stu5jMhQNph8M9BxJhD3X9RD/6lsxGyYZkSs2y2JW7GJm6qi3I/2nDQqevR4aJabF6rdPSgmOd48Va4YRJoJrPHCBUMjcvTCfEbZF2y+cWwb869uvgOGj5h63D24jOqiMbfQEPUcN5KNXqIPeoiPURxR9Rj/QJfrpfEuvV/e7zLVq6w8j9FGbHl/ADoqFek=</latexit>If Wij ⌧ 1 then the correction to the computed transfer entropy is
<latexit sha1_base64="tGW/Sf2tr08RpsHAmntbvyMneFY=">ADNnicbVJLb9NAEN6YVwmPpnDksKNxAFdlQJjpF6oRdUpKSpFvRej12lu7D2l0XRZ/E1f+SA9cyglx5Sewdg2KW0ba1efvmxnP7ExScGZsEHwfePfuP3j4aO/x8MnTZ8/3RwcvzowqNYUFVzp84QY4EzCwjL4bzQETCYZlcHDf68hK0YUrO7baAWJBcsoxRYh21Hp2cZPhwua7Y5xpHnOPwENsNyObCVGkNtPHDVnWMKEoLKbaSJOBxiCtVsUWM7Me+cEkaA3fBWEHfNTZ6fpgcBWlipbC5aCcGLMKg8LGFdGWUQ71MCoNFIRekBxWDkoiwMRV23ONx45Jca0O9Lilh2Od0PmYVw1Gkha9xWTpZCRkruf5YxzVpq+QwUlB30p+iXs5sNj/BEg7SqYu4cN8bGSVIOF3SBX7Q1Z42Ek4Yt7QEFkWkUkMfXKJYw4ZDbibkTWDyPN8o2NdPNV9/2ZlKDr1fRfBJE5Bz9860/RrVMr+SKCGO2InHlCmI35rbWkP/TVqXN3scVk82w26dlpW8XQPXK05Zsxh86wChmrl5YbohmlDrVs0tQnh7HfB2XQSHk2OPk392axbiT30Cr1Gb1CI3qEZ+oBO0QJR9BVdoWv0w/vmXs/vV83rt6gi3mJeub9/gOl6w0Y</latexit>W(1)
ij
∼ W(0)
ij
2 ⇣ d(0)
j
− d(0)
i
+ W(0)
ij
⌘ + O ✓
- W(0)
- 2◆
This yields a correction to the computed weight
The difference in weighted in-degree of the two nodes
j i Wij TEj→i
SLIDE 28
Return to numerical example
The network 𝚫 was constructed so that its in-degrees vary more widely than its out-degrees This leads to a larger variance in the computed weights for 𝚫 than for 𝚫⊤
SLIDE 29
Putting in the correction
Γ >
<latexit sha1_base64="ZaRT9gDq/u5Y38NFAEiXyPM1dE=">AC+HicbVLbtNAFJ2YVwmPtsCOzQgrEgsU2VEl2BGpi7JCRUraShlTjcfXzqjzMDPjotTyd7AtK8SGBd/AL7BgyZ8wTgKW65k6fice8bHd25aCm5dFP3qBTdu3rp9Z+tu/979Bw+3d3YfHVldGQZTpoU2Jym1ILiCqeNOwElpgMpUwHF6t/qx+dgLNdq4hYlJIWiuecUephBxQKen7mjhdNqc7YTSMloWvg3gNwtc/Ln4fHtSH57u9n6STLNKgnJMUGtncVS6pKbGcSag6ZPKQknZGS1g5qGiEmxSL1M3eOCZDOfa+Ec5vGT7g03LJE7qVgPFmq5i8wxyWgn/sYILwSvbaihEmDOZTfC5nl4gN8CZOsEz+aGO9rxQw42DT5tCuywX2i4CPTfmAqwlNbTPzBxIBuSPCD9mFMTG8mDti2rem28+VAtPMRv8cVBUCwvhFOPrWjKdyDWV1i5k6uNK6ub2qtaS/9NmlctfJTVXZbX+W6/lcBO43YNcMYNMCcWHlBmuL8vzObUOb8svhFiK9e+3VwNBrGe8O9d1E4HqNVbaGn6Bl6jmL0Eo3RG3SIpoihD+gTukSfg4vgMvgSfF21Br215zHqVPD9D2lj+Ug=</latexit>Γ
<latexit sha1_base64="wkxhtICFEtu9KQb9eVAMKy+ByE=">AC8XicbVLbtNAFJ2YVwmPtsCOzQgrEgsU2VEl2BGpi7JCRUrSrZVXY+vk6EzY2tmXBQs/wPbskJskfgKfoEFS/6EcRJQ3HIlS8fn3DM+vnfSUnBjg+BXz7tx89btOzt3+/fuP3i4u7f/aGaKSjOcskIU+jQFg4IrnFpuBZ6WGkGmAk/S8NWP7lAbXihJnZYiJhrnjOGVhHzeIjkBLO9vxgGKyKXgfhBvivf3z8fT9SX18t/7GWcFqyQqywQYE4VBaZMatOVMYNOPK4MlsHOY+SgAokmqVdxGzpwTEbzQrtHWbpi+4NtyRM6lZDxZquYvIMc6iE+9icC8Er02osRKoL2Q3wvZ5dEDfImabBM3k5AeFoptLhtcmnXZEP7scIPrHCTUlkdQ2qayB0YC8xtLNx0rR/Gms8XNtbtW9Pt50qhbqLRPweouUA/fOGP/rpWTCdyDdKYpUxdXAl2Ya5qLfk/Laps/iqpuSqrzd86La8EtQVt908zrpFZsXQAmOZuX5QtQAOz7pa4ixBeXft1MBsNw4PhwbvAH4/JunbIU/KMPCcheUnG5A05JlPCyHvyiVySz57xLr0v3td1q9fbeB6Tnf/gCh/vX9</latexit> SLIDE 30
Question: Is this a general phenomenon?
Does this long and painful calculation actually tell us anything about connection between the network structure and the accuracy of the computation? Let’s look at another model. The tent map is a simple chaotic system
xt+1 = F(xt) ≡
- 2xt,
0 ≤ xt < 1
2
2(1 − xt)
1 2 ≤ xt ≤ 1
<latexit sha1_base64="tZO+8W+hHCovYLtLH2jnsX3oOts=">ADanicbVJda9swFWSfbTeV5o+jY4hlqWkrAtWKNseNgExp5GB0lbiEImy9eJqCx7kpw1GP/MPex57G0/YspHR9xOYDg656r63tvkEphrO/rFRrd+7eu7+z6z14+Ojxk/pe48wkmeYw5IlM9EXADEihYGiFlXCRamBxIOE8uOwv9fM5aCMSNbCLFMYxmyoRCc6soyb14mqS21ekwB/wx/bVxB5hCt8yMfdoAFOhcu5ym8LrYqcd40PsYyphecHvMY04zkp8m6BKXUxbfJ6leKwLG0MK0A8Ciq8TjupN/2Ovzr4NiAb0Oy9edb/3X3+9XSyV/lBw4RnMSjLJTNmRPzUjnOmreASCo9mBlLGL9kURg4qFoMZ56s+FbjlmBHiXafsnjFeq1ty4CM86UGihdlxUQhRCyT7rGpkFJkphyQyZBz+NyCdv5cAt/Bg3FQzcMAjuJ4prsLBtctWuyQJ7VMF3nsQxcz2jLDFyCV0fYwslW6stkmoFtOZpXp5K8rxQinQxaj7z8HUVEKTHDe7164VUyo5Z7Exizhw5cbMzsxNbUn+TxtlNno3zoVKs83fOi3KJLYJXi4eDoUGbuXCAca1cPCfMbcmli3nm4RyM2x3wZn3Q456Zx8cRvRQ+uzgw7QC9RGBL1FPfQJnaIh4uhXZbfSqOxX/9Qatae1g3VotbLx7KPSqb38CwRNGDQ=</latexit>xi
t+1 = F
- xi
t
- + ✏
N
X
j=1
Wij ⇣ F ⇣ xj
t
⌘ − F
- xi
t
⌘ + ni(t)
<latexit sha1_base64="S3vtnuSwqsDvmrVb0HV4D51aYjY=">ADjHicbVLBbtNAELVrCiVASEGcuKyIqUKRHEUBKFKlSxYWqSElTKU6t9XrsbLpeW7vrQmT5W/gRfoMDZ36DA+s4NHbkawdvzfzdnZmvIRqXq93+ae9WD/4aODx7UnT5/VnzcOX5zLOBUEJiRmsbjwsARGOUwUVQwuEgE48hMvavjgp9eg5A05mO1SmAe4ZDTgBKsNOQ2fny/zGjuZqpj58MTh0Gg2iWkHEHDhTpCNVRax4FEUhZz5Mg0crPl0M4vT2/oqZvRZV4qbIW6F398pvju0lkoYRrwIaqsjt9HsdXtrQ3cde+M0Ryc/v/z986p+5h6avxw/JmkEXBGpZzZvUTNMywUJQzympNKSDC5wiHMtMtxBHKerTuZo5ZGfBTEQn9coTVa+2mjO15VnDASV5lZOBDgFOmLwspYzSV1YAMUgbiOqWsKuHWugUwN9UMNbjstFxzIkABbtJutoSzFHN4fCNxFGEuZ852JP5TAuG+0wPXjVtMsO6L4y6vxlHMQ+ax/k4F5yKBpv232/2etkUrJGY6kXEWeLjfCaiFvcwV4HzdLVfBxnlGepJvXai5IGVIxKlYT+VQAUWylHUwE1fNCZIEFJkovsF4E+/bY7zrn/a496A6+6o0YGaUdGK+N0bsI0Pxsj4bJwZE4OY+2bHJjvrbo1sD5ZwzJ0z9zkvDQqZp38A0kdJgc=</latexit>We consider a system of coupled tent maps with noise
SLIDE 31
The coupled tent map example
Simulate for two 30-node networks 𝚫 and 𝚫⊤ and observe oscillators going in and out of synchronization The networks constructed such that their distribution of weighted in- degrees very different Variance in computed transfer entropy increases with variance in weighted in-degree
Γ
<latexit sha1_base64="wkxhtICFEtu9KQb9eVAMKy+ByE=">AC8XicbVLbtNAFJ2YVwmPtsCOzQgrEgsU2VEl2BGpi7JCRUrSrZVXY+vk6EzY2tmXBQs/wPbskJskfgKfoEFS/6EcRJQ3HIlS8fn3DM+vnfSUnBjg+BXz7tx89btOzt3+/fuP3i4u7f/aGaKSjOcskIU+jQFg4IrnFpuBZ6WGkGmAk/S8NWP7lAbXihJnZYiJhrnjOGVhHzeIjkBLO9vxgGKyKXgfhBvivf3z8fT9SX18t/7GWcFqyQqywQYE4VBaZMatOVMYNOPK4MlsHOY+SgAokmqVdxGzpwTEbzQrtHWbpi+4NtyRM6lZDxZquYvIMc6iE+9icC8Er02osRKoL2Q3wvZ5dEDfImabBM3k5AeFoptLhtcmnXZEP7scIPrHCTUlkdQ2qayB0YC8xtLNx0rR/Gms8XNtbtW9Pt50qhbqLRPweouUA/fOGP/rpWTCdyDdKYpUxdXAl2Ya5qLfk/Laps/iqpuSqrzd86La8EtQVt908zrpFZsXQAmOZuX5QtQAOz7pa4ixBeXft1MBsNw4PhwbvAH4/JunbIU/KMPCcheUnG5A05JlPCyHvyiVySz57xLr0v3td1q9fbeB6Tnf/gCh/vX9</latexit>Γ >
<latexit sha1_base64="ZaRT9gDq/u5Y38NFAEiXyPM1dE=">AC+HicbVLbtNAFJ2YVwmPtsCOzQgrEgsU2VEl2BGpi7JCRUraShlTjcfXzqjzMDPjotTyd7AtK8SGBd/AL7BgyZ8wTgKW65k6fice8bHd25aCm5dFP3qBTdu3rp9Z+tu/979Bw+3d3YfHVldGQZTpoU2Jym1ILiCqeNOwElpgMpUwHF6t/qx+dgLNdq4hYlJIWiuecUephBxQKen7mjhdNqc7YTSMloWvg3gNwtc/Ln4fHtSH57u9n6STLNKgnJMUGtncVS6pKbGcSag6ZPKQknZGS1g5qGiEmxSL1M3eOCZDOfa+Ec5vGT7g03LJE7qVgPFmq5i8wxyWgn/sYILwSvbaihEmDOZTfC5nl4gN8CZOsEz+aGO9rxQw42DT5tCuywX2i4CPTfmAqwlNbTPzBxIBuSPCD9mFMTG8mDti2rem28+VAtPMRv8cVBUCwvhFOPrWjKdyDWV1i5k6uNK6ub2qtaS/9NmlctfJTVXZbX+W6/lcBO43YNcMYNMCcWHlBmuL8vzObUOb8svhFiK9e+3VwNBrGe8O9d1E4HqNVbaGn6Bl6jmL0Eo3RG3SIpoihD+gTukSfg4vgMvgSfF21Br215zHqVPD9D2lj+Ug=</latexit> SLIDE 32
Extending the model: a multi-layered graph
There are many generalizations
- f the concept of a graph, e.g.
multigraph in which there exist different types of edges.
- rganized in layers
Maurizio’s interest: Fish communicate over multiple channels Stimulus fish Responding fish Fluid flow, lateral line, delayed Light, eyes, instaneous
SLIDE 33
Multilayered delay RBN model
Pr (xi(t) = 1) = ✏ 1 +
M
X
m=1 N
X
j=1
W(m)
ij
xj(t − m)
<latexit sha1_base64="/Pr1IunCsSXQ/zbIrhMUbesYog=">ADXicbVLBbtNAEN0kpJRQSgoHDlxWRJESAVEcVQIOkSL10gtVkJKmUpya9WacbLq7tnbXpZHlL+M7QOKE+A8OrO2A6paRLM28N292PDN+xJk2/f6PSrX2oL73cP9R4/HBk8OnzaNn5zqMFYUpDXmoLnyigTMJU8Mh4tIARE+h5l/dZLxs2tQmoVyYrYRLARZSRYwSoyFvGaE3bFKXA6B6eAbj3VMd+hgV7HV2nRTPMQNjF2INOhxEWaRZzXro6Fl4ihk15+xEWwyYIzPMStkvk47opjfepmPeim5epajpNVv9Xj83fN9xdk5rdPpr9vlD+vYO6p8c5chjQVIQznReu70I7NIiDKMckgbqwhIvSKrGBuXUkE6EWSjybFbYscRAq+0mDc7TRvi2ZOIsk40DStMzoYAkBibl9bMU4Z7EuJyQc1DXotzC7Xq4jc8AlrsOJnb+Dj4JVg4LbIdluAKW64Er7QUAgil4lLfJ3ObcF8i63mzQtp5ikq7IoLeczKUGl8E/BZErDi3nTWvwV5UjpZYTIrTeCt+2K4hZ67tcBv6Pm8cmeL9ImIzi3d9aLog5NiHObg0vmQJq+NY6hCpm94XpmihCjb1IewjO3bXfd84HPe4d/zJXsQIFbaPXqJXqIMc9A6N0Ckaoymi6Dv6XalX9qo/a/XaQe2wSK1WdprnqGS1F38AMAEV6A=</latexit>Rewrite as a suspended system with coordinate
y(t) = x(t) x(t − 1) . . . x(t + 1 − M) = y(1)(t) y(2)(t) . . . y(M)(t) ∈ {0, 1}MN
<latexit sha1_base64="ekU6V6RaMFEKZd+tjOLZwptwlS0=">ADr3icdVJNb9NAELVrPkoKJYUjlxVRpES0UTaqBekoF4QUqIipU2kbBrW63Gy6npt7a5DI8v/izP/gQP/hnViStyWkSzPvDdvPJ4ZPxFcm273t7vnPXr85On+s9rB8xeHL+tHry51nCoGFywWsZr4VIPgEi4MNwImiQIa+QLG/vVZwY9XoDSP5cisE5hFdCF5yBk1FprXf65bpo0+ohpCxIcFl1kSUaP4TV5ANwVJSPE+wRuPrILY6BJ7h08G7UIJMqjo/ltwPc9auJ2XZYuodxv9K13gw3+UHCJSJZ9xiT/CobDPN5vdHtdDeG7ju4dBr9zo8vh58O2Pn8yP1FgpilEUjDBNV6iruJmWVUGc4E5DWSakgou6YLmFpX0gj0LNsMO0dNiwQojJV9pEbtNbclYzwLCs4kCyvMjoMIKSpsB9bcCF4qsJGaQC1CqtrBbDzXRECAoOxjZjWJ0FkumwMCuyHa7Be3IiITvLI4iagdJqK/zqS1IBISGCHsbpoGJ4oulIaqI8mo+lxJUPu3dKqhcCGjg40bvr2qDVFrOaKT1OvJtu3ZxS32XK8CHuGlqwg+zjMskLf/WcmEqkIlRcb0o4AqYEWvrUKa43RdiS6oM/bG7SHgu2u/71z2Ovi0c/rVXkTf2dq+8Z567Qc7Lx3+s5n59y5cJjbcofu2J142Bt7V963beqeW2peOxXz+B+n7SxE</latexit>The first N coordinates are updated probabilistically
Pr (yi(t) = 1) = ✏ @1 +
MN
X
j=1
Wijyj(t − 1) 1 A , where W = h W(1) W(2) · · · W(M)i
<latexit sha1_base64="M1ehtGYSMg1Cvt96Rvq3kDVjA=">ADlnicbVLNbhMxEN5t+CnhpylckLhYRJESUaI4KgIOKZUqRC+tgtQ2leI0eL2ziVuvd2V7WyLD8Qz8BwcOPImOJsUNS0jWfr8fTPj8cxEueDadDq/w7XKvfsPHq4/qj5+8vTZRm3z+YnOCsXgmGUiU6cR1SC4hGPDjYDTXAFNIwGD6GJvrg8uQWmeySMzy2GU0onkCWfUeGpc+0H6yhIBiWnOxrxpWj1MFJ9MTctVEer5QyDXGQSLbwQfoOILtKxPe9hd2YPDt1gbPm5QzNPuaZ5i1tomWILVYmB78aiqykoQK46QL1FniEanNkmbjlEtkrYXUDC4szoa/JgTpa5Rmhcq3fandLQXYCXoL67/2fw7WPjZ3+8Gf4icaKFKRhgmo9xJ3cjCxVhjMBrkoKDTlF3QCQw8lTUGPbNlThxqeiVGSKX+kQSVbdwMOcIjO9dAMreq6CSGhBbCPzbhQvBCrzpYKASoy3S1hJv5UAMdAsTLCo784DayRTYOBmkK92QTrfaglXLEtTKmNLaKTd0Ccsm02EXwFTX06WqPnNrfpzKUG5YfdfBJUTAXW8Ve9eR5XMSsmWplrP0siXm1Iz1be1Ofk/bViY5MPIcpkXy96LSkEMhmaLymKuQJmxMwDyhT380JsShVlxq+yXwR8e+x3wUm3jbfb21/9RuwGC1sPXgWvg2aAg/fBbrAf9IPjgIUb4btwJ/xUeVnZqXyufFm4roXLmBfBilX6fwE4Jid4</latexit>and the remaining (N-1)M deterministically
y(m)(t) = y(m−1)(t − 1), for m = 2, . . . , M
<latexit sha1_base64="82JpwygGLYlpqxiI1crSTf692yQ=">ADH3icbVJNa9tAEF2rX6n75bTHXpbKBgdcY4lAewkYcumlJQU7CVjGrFYje8l+iN1VihH6H732j/SankovPeTfdKWoxUo6BzHz3rzdt5qJM86MnUyuO969+w8ePtp73H3y9NnzF739l6dG5ZrCnCqu9HlMDHAmYW6Z5XCeaSAi5nAWXxX/NklaMOUnNltBktB1pKljBLroFUv7G9XxVAclEN7gI9wXbwNqtJ9+yOMU6VxXxyFo4gnyprRx/6q50/Gkzrw3SRoEh81cbLa7/yOEkVzAdJSToxZBJPMLguiLaMcym6UG8gIvSBrWLhUEgFmWdSPK/HAIUltI1XS4hrtDnYls2BZVBxIWrYZkyaQkpy7y9aMc5abdkMBOQd9KdoWds/DA/wJIGkczNwfDPCxklSDhV2Rc3sDlrgbSfhClRBEJkVEYlMu3IERh9RG3M3C+kGk2XpjI1VZbufSQm6XIT/FESuOfjByA/qmqkZbkgwpitiJ1dQezG3OYq8H/cIrfp+2XBZJY3r3VcmnNsFa62BSdMA7V86xJCNXPzwnRDNKHW7ZRbhOD2O8mp+E4OBwfg796bRZiT30Gr1BQxSgd2iKPqATNEcUfUXf0RX64X3zryf3q+bVq/TaF6hVnjXfwBpVAb</latexit> SLIDE 34
The result of the calculation
- Repeat our procedure
- Reformulate as a Markov chain on a 2MN dimensional
state space
- Calculate stationary vector
- Use the transition law and the stationary vector to
compute transfer entropy in terms of weights Wij
- This is the worst calculation I had to do in my entire
life
- The result is entirely analogous to what I obtained in
the first problem
SLIDE 35
One last numerical experiment
SLIDE 36