SLIDE 1 On Construction of Probabilistic Boolean Networks Wai-Ki CHING Advanced Modeling and Applied Computing Laboratory Department of Mathematics, The University of Hong Kong Abstract
Modeling genetic regulatory networks is an important problem in genomic re-
- search. Boolean Networks (BNs) and their extensions Probabilistic Boolean Net-
works (PBNs) have been proposed for modeling genetic regulatory interactions. We are interested in system synthesis which requires the construction of such a
- network. It is a challenging inverse problem, because there may be many networks
- r no network having the given properties, and the size of the problem is huge.
The construction of PBNs from a given transition-probability matrix and a given set of BNs is an inverse problem of huge size. In this talk, we shall propose some construction methods. In particular, we propose a maximum entropy approach for the captured problem. Newton’s method in conjunction with the Conjugate Gradient (CG) method is then applied to solving the inverse problem. We investi- gate the convergence rate of the proposed method. Numerical examples are also given to demonstrate the effectiveness of our proposed method.
1
SLIDE 2
The Outline (0) Motivations and Objectives. (1) Boolean Networks (BNs) and Probabilistic Boolean Net- works (PBNs). (2) The Inverse Problem. (3) The Maximum Entropy Approach. (4) Numerical Experiments.
2
SLIDE 3
- 0. Motivations and Objectives.
- An important issue in systems biology is to model and under-
stand the mechanism in which the cells execute and control a large number of operations for their normal functions and also the way in which they fail in diseases such as cancer (25000 genes in human Genome). Eventually to design some control strategy (drugs) to avoid the undesirable state/situation (cancer).
- Mathematical models (A review by De Jong 2002):
- Boolean networks (BNs) (Kaufman 1969)
- Differential equations (Keller 1994)
- Probabilistic Boolean networks (PBNs) (Shmulevich et al.
2002)
- Multivariate Markov chain model (Ching et al. 2005)
- Petri nets (Steggles et al. 2007) etc.
- Since genes exhibit “switching behavior”, BNs and PBNs mod-
els have received much attention.
3
SLIDE 4
- 1. Boolean Networks and Probabilistic Boolean Networks.
1.1 Boolean Networks
- In a BN, each gene is regarded as a vertex of the network and is
then quantized into two levels only (expressed: 1 or unexpressed: 0) though the idea can be extended to the case of more than two levels.
- The target gene is predicted by several genes called its input
genes via a Boolean function.
- If the input genes and the corresponding Boolean functions are
given, a BN is said to be defined and it can be considered as a deterministic dynamical system.
- The only randomness involved in the network is the initial system
state.
4
SLIDE 5
1.1.1 An Example of a BN of Three Genes vi(t + 1) = f(i)(v1(t), v2(t), v3(t)), i = 1, 2, 3. Network State v1(t) v2(t) v3(t) f(1) f(2) f(3) 1 1 1 2 1 1 1 3 1 1 1 4 1 1 1 1 5 1 1 6 1 1 1 7 1 1 1 1 8 1 1 1 1 1 Table 1 Attractor Cycle : (0, 1, 1) ↔ (0, 1, 1), (0, 0, 0) → (0, 1, 1), Attractor Cycle : (1, 0, 1) → (1, 0, 0) → (0, 1, 0) → (1, 1, 0) → (1, 0, 1), (0, 0, 1) → (1, 0, 1), (1, 1, 1) → (1, 1, 0).
5
SLIDE 6
- The transition probability matrix of the 3-gene BN is then given
by 1 2 3 4 5 6 7 8 A3 =
1 1 1 1 1 1 1 1
.
- We note that each column has only one non-zero element and
the column sum is one (a column stochastic matrix). We call it a Boolean Network (BN) matrix.
6
SLIDE 7 1.2 A Summary of BNs
- A BN G(V, F) actually consists of a set of vertices
V = {v1, v2, . . . , vn}. We define vi(t) to be the state (0 or 1) of the vertex vi at time t.
- There is also a list of Boolean functions (fi : {0, 1}n → {0, 1}):
F = {f1, f2, . . . , fn} to represent the rules of the regulatory interactions among the genes: vi(t + 1) = fi(v(t)), i = 1, 2, . . . , n where
v(t) = (v1(t), v2(t), . . . , vn(t))T
is called the Gene Activity Profile (GAP).
7
SLIDE 8
- The GAP can take any possible form (state) from the set
S = {(v1, v2, . . . , vn)T : vi ∈ {0, 1}} (1) and thus totally there are 2n possible states.
- Since BN is a deterministic model, to overcome this deterministic
rigidity, extension to a probabilistic setting is natural.
- Reasons for a stochastic model:
- The biological system has its stochastic nature;
- The microarray data sets used to infer the network structure are
usually not accurate because of the experimental noise in the complex measurement process. ...
8
SLIDE 9 1.3 Probabilistic Boolean Networks (PBNs)
- For each vertex vi in a PBN, instead of having only one Boolean
function as in the case of BN, there are a number of Boolean func- tions (predictor functions) f(i)
j
(j = 1, 2, . . . , l(i)) to be chosen for determining the state of Gene vi.
- The probability of choosing f(i)
j
as the predictor function is c(i)
j , 0 ≤ c(i) j
≤ 1 and
l(i)
∑
j=1
c(i)
j
= 1 for i = 1, 2, . . . , n.
9
SLIDE 10
- We let fj be the jth possible realization,
fj = (f(1)
j1 , f(2) j2 , . . . , f(n) jn ),
1 ≤ ji ≤ l(i), i = 1, 2, . . . , n. If the selection of the Boolean function for each gene is independent (an independent PBN), the probability of choosing the j-th BN pj is given by pj =
n
∏
i=1
c(i)
ji ,
1, 2, . . . , N. (2)
N =
n
∏
i=1
l(i) (3) different possible realizations of BNs.
10
SLIDE 11
- We note that the transition process among the states in the set
S in (1) is a Markov chain process. Let a and b be any two column vectors (binary unit vector) in S. Then the transition probability Prob {v(t + 1) = a | v(t) = b} =
N
∑
j=1
Prob {v(t + 1) = a | v(t) = b, the jth network is selected } · pj.
- By letting a and b to take all the possible states in S, one can
get the transition probability matrix for the process. In fact, the transition matrix is given by A = p1A1 + p2A2 + · · · + pNAN. Here Aj is the transition probability matrix (a BN matrix) of the j-th BN and pj is the corresponding selection probability.
- There are at most N2n nonzero entries for the transition proba-
bility matrix A.
11
SLIDE 12
Ilya Shmulevich and Edward R. Dougherty, Probabilistic Boolean Networks: The Modeling and Control of Gene Regulatory Networks, Philadelphia: SIAM, 2010.
Trairatphisan et al., Recent Development and Biomedical Applica- tions of Probabilistic Boolean Networks, Cell Communication and Signaling, 2013, 11:46. (http://www.biosignaling.com/content/11/1/46).
12
SLIDE 13 Recent Works in Construction and Control of BNs and PBNs
- T. Akutsu, M. Hayashida, W. Ching, M. Ng, Control of Boolean
networks: Hardness results and algorithms for tree structured net- works, Journal of Theoretical Biology, 244 (2007) 670–679.
- W Ching, S. Zhang, Y. Jiao, T. Akutsu, N..
Tsing, A. Wong, Optimal control policy for probabilistic Boolean networks with hard constraints, IET System Biology, 3 (2008) 90-99.
- D. Cheng and H. Qi, Controllability and Observability of Boolean
Control Networks, Automatica, 45 (2009) 1659-1667.
- W. Ching, X. Chen and N. Tsing, Generating Probabilistic Boolean
Networks from a Prescribed Transition Probability Matrix, IET Sys- tems biology, 3 (2009) 453-464.
13
SLIDE 14
- D. Cheng, Z. Li and H. Qi, Realization of Boolean Control Net-
works, Automatica, 46 (2010) 62-69.
- S. Zhang, W. Ching, X. Chen and N. Tsing, Generating Proba-
bilistic Boolean Networks from a Prescribed Stationary Distribution, Information Sciences, 180 (2010) 2560-2570.
- F. Li and J. Sun, Controllability of Boolean Control Networks with
Time Delays in States, Automatica, 47 (2011) 603-607.
- D. Cheng, Disturbance Decoupling of Boolean Control Networks,
IEEE Tran. Auto. Cont. 56 (2011) 1-10.
- F. Li and J. Sun, Controllability of Probabilistic Boolean Control
Networks, Automatica, 47 (2011) 2765-2771.
14
SLIDE 15
- X. Chen, H. Jiang, Y. Qiu and W. Ching (2012), On Optimal Con-
trol Policy for Probabilistic Boolean Network : A State Reduction Approach, BMC Systems Biology, 6 (Suppl 1):S8.
- H. Li and Y. Wang, On Reachability and Controllability of Switched
Boolean Control Networks, Automatica, 48 (2012) 2917-2922.
- X. Chen, T. Akutsu, T. Tamura and W. Ching, Finding Optimal
Control Policy in Probabilistic Boolean Networks with Hard Con- straints by Using Integer Programming and Dynamic Programming, International Journal of Data Mining and Bioinformatics, 7 (2013) 322-343.
- D. Laschov, M. Margaliot and G. Even, Observability of Boolean
Networks: A Graph-theoretic Approach, Automatica, 49 (2013) 2351-2362. · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·
15
SLIDE 16
2.1 The Motivation
- We study the problem of constructing a PBN from a given steady-
state distribution.
- Such problems are very useful for network inference from steady-
state data, as most microarray data sets are assumed to be obtained from sampling the steady-state.
- This is an inverse problem of huge problem size.
The inverse problem is ill-posed, meaning that there will be many networks or no network having the desirable properties.
- Ching et al. (2008), a modified Conjugate Gradient (CG) method
has been proposed to give some possible solutions of PBNs. How- ever, there are infinitely many possible PBNs and the algorithm ends up with different PBNs with different initial guesses.
16
SLIDE 17
- The problem can be decomposed into two parts.
- (I) Construct a transition probability matrix from a given steady-
state probability distribution.
- A mathematical formulation based on entropy rate theory has been
proposed for (I) Ching and Cong (2009).
- (II) Construct a PBN based on a given transition probability
matrix and a given set of BNs.
- We will focus on this problem here.
17
SLIDE 18 2.2 The Formulation
- Suppose that the possible BNs constituting the PBN are known
and their BN matrices are denoted by {A1, A2, . . . , AN}.
- Transition probability matrix is observed and they are related as
follows: A =
N
∑
i=1
qiAi. (4)
- We are interested in getting the parameters qi, i = 1, 2, . . . , N when
A is given.
18
SLIDE 19
- Since the problem size is huge and A is usually sparse. Here we
assume that each column of A has at most m non-zero entries. In this case, we have N = m2n and we can order A1, A2, · · · , Am2n systematically.
- We note that qi and Ai are non-negative and there are only m · 2n
non-zero entries in A. Thus we have at most m · 2n equations for m2n unknowns.
- To reconstruct the PBN, one possible way to get qi is to consider
the following minimization problem: min
q
m2n
∑
i=1
qiAi
F
(5) subject to 0 ≤ qi ≤ 1 and
m2n
∑
i=1
qi = 1.
19
SLIDE 20
- Here ∥ · ∥F is the Frobenius norm of a matrix. Let us define a
mapping F from the set of l × l square matrices to the set of l2 × 1 vectors by F
a11 · · · a1l . . . . . . . . . . . . . . . . . . al1 · · · all
= (a11, . . . , al1, a12, . . . , al2, . . . , . . . , a1l, . . . all)T.
(6)
U = [F(A1), F(A2), . . . , F(Am2n)] and
p = F(A)
(7) then Eq. (5) becomes min
q
∥Uq − p∥2
2
(8) subject to 0 ≤ qi ≤ 1 and
m2n
∑
i=1
qi = 1.
20
SLIDE 21
||Uq − p||2
2 = (Uq − p)T(Uq − p)
(9) and (Uq − p)T(Uq − p) = qTUTUq − 2qTUTp + pTp. (10)
- Thus the minimization problem (10) without constraints is equiv-
alent to min
q {qTUTUq − 2qTUTp}.
(11)
- The matrix UTU is a symmetric positive semi-definite ma-
- trix. The minimization problem without constraints is equivalent to
solving UTUq = UTp (12) with the Conjugate Gradient (CG) method.
21
SLIDE 22
- We note that if there is q satisfying the equation Uq = p with
1Tq = 1 and 0 ≤ q ≤ 1. Then the CG method can yield a solution.
- To ensure that 1Tq = 1, we add a row of (1, 1, . . . , 1) to the
bottom of the matrix U and form a new matrix ¯ U.
- At the same time, we add an entry 1 at the end of the vector p
to get a new vector ¯
- p. Thus we consider the revised equation:
¯ UT ¯ Uq = ¯ UT ¯
p.
(13)
- This method can give a solution of the inverse problem.
But usually there are too many solutions. Extra constraint or criterion has to be introduced in order to narrow down the set of solutions
- r even a unique solution.
22
SLIDE 23
- 3. The Maximum Entropy Approach
- One possible and reasonable approach is to consider the solution
which gives the largest entropy as q itself can be considered as a probability distribution.
- This means we are to find q such that it maximizes
−
m2n
∑
i=1
qi log(qi). (14)
- Similar method has been used by Wilson (1970) in traffic demand
estimation in a transportation network and it has become more popular (Ching et al. 2004).
23
SLIDE 24
- We recall that for the inverse problem, we have m · 2n equations
for m2n unknowns. Thus one may have infinitely many solutions.
- Since q can be viewed as a probability distribution, one possible
way to get a better choice of qi is to consider maximizing the entropy
- f q subject to the given constraints, i.e., the following maximization
problem: max
q
m2n
∑
i=1
(−qi log qi)
(15) subject to ¯ Uq = ¯
p
and 0 ≤ qi i = 1, 2, . . . , m2n. (16)
- We remark that the constraints that qi ≤ 1 can be discarded as
we required that
m2n
∑
i=1
qi = 1 and 0 ≤ qi i = 1, 2, . . . , m2n.
24
SLIDE 25
- The dual problem of (15) is therefore of the type
min
y
max
q
L(q, y) (17) where y is the multiplier and L(·, ·) is the Lagrangian function L(q, y) =
m2n
∑
i=1
(−qi log qi) + yT(¯
p − ¯
Uq). (18)
- The optimal solution q∗(y) of the inner maximization problem of
(17) solves the equations ∇qiL(q, y) = − log qi − 1 − yT ¯ U·i = 0, i = 1, 2, . . . , m2n and is thus of the form: q∗
i (y) = e−1−yT ¯ U·i,
i = 1, 2, . . . , m2n (19) where ¯ U·i is the ith column of the matrix ¯ U.
25
SLIDE 26
- After substituting q∗(y) back into (18) the dual problem (17)
can be simplified to min
y
m2n
∑
i=1
e−1−yT ¯
U·i + yT ¯
p
.
(20)
- The solution of the primal problem (17) is obtained from the
solution of the dual problem (19) through (20).
- Thus we have transformed a constrained maximization problem
with m2n variables into an unconstrained minimization problem of m · 2n + 1 variables.
- We will then apply Newton’s method in conjunction with Con-
jugate Gradient (CG) method to solving the dual problem.
26
SLIDE 27
4.1 Newton’s Method
- In the following, we will explain how Newton’s method in conjunc-
tion with the conjugate gradient method can be used. To this end we denote by f(y) =
m2n
∑
i=1
e−1−yT ¯
U·i + yT ¯
p
(21) the function to be minimized.
- The gradient and the Hessian of f are respectively of the forms:
∇f(y) = −¯ Uq∗(y) + ¯
p
(22) and ∇2f(y) = ¯ U · diag(q∗(y)) · ¯ UT (23) where q∗(y) is as defined in (19) and diag(q∗(y)) is the diagonal matrix with diagonal entries (q∗(y)).
27
SLIDE 28 Newton’s Method Choose starting point y0 ∈ Im(¯ U) k = 1; while ||∇f(yk)||2 > tolerance find pk with ∇2f(yk−1)pk = −∇f(yk−1); set yk = yk−1 + pk; k = k + 1; end.
(23), we observe that f is strictly convex on the subspace Im(¯ U).
- Newton’s method will produce a sequence of points yk according
to the iteration yk = yk−1 + pk, where the Newton step pk is the solution of the Hessian matrix system: ∇2f(yk−1)pk = −∇f(yk−1). (24)
28
SLIDE 29
- We note that ∇2f(yk−1) is a one-to-one mapping of the con-
cerned subspace onto itself.
- Moreover, from Eq. (22) ∇f(y) ∈ Im(¯
U) as we have ¯
p ∈ Im(¯
U) (from Eq. (16)). Hence, Eq. (24) has an unique solution and therefore Newton’s method for minimizing f is well defined.
- If we start with y0 ∈ Im(¯
U) the Newton sequence will remain in the subspace. Moreover, it will converge locally at a quadratic rate.
- To enforce global convergence one may wish to resort to line
search or trust region techniques. However, we did not find this necessary in our computational experiments.
29
SLIDE 30 4.2 Conjugate Gradient Method
- In each iteration of the Newton’s method, one has to solve the
linear system of the form in Eq. (24). We propose to solve the linear system (24) by Conjugate Gradient (CG) method.
- The convergence rate of CG method depends on the effective
condition number λ1(∇2f(y)) λs(∇2f(y)) (25)
- f ∇2f(y). Since ∇2f(y) is singular we have to consider the second
smallest eigenvalue λs(∇2f(y)). Theorem : For the Hessian matrix ∇2f(y), we have 2n · e−2(m·2n+1)·∥y∥∞ ≤ λ1(∇2f(y)) λs(∇2f(y)) ≤
(√
2n + √m
)2 · e2(m·2n+1)·∥y∥∞.
30
SLIDE 31 4.3 Some PBN Examples
- For Newton’s method, we set the tolerance to be 10−7 while the
tolerance of CG method is 10−10. Example 1. In the first example, we consider the case n = 2 and m = 2 and we suppose that the observed/estimated transition probability matrix of the PBN is given as follows: A2,2 =
0.1 0.3 0.5 0.6 0.0 0.7 0.0 0.0 0.0 0.0 0.5 0.0 0.9 0.0 0.0 0.4
.
(26)
31
SLIDE 32
- Then there are 16 possible BNs for constituting the PBN and they
are listed below:
A1 =
( 1
1 1 1
)
A2 =
( 1
1 1 1
)
A3 =
( 1
1 1 1
)
A4 =
( 1
1 1 1
)
A5 =
( 1
1 1 1
)
A6 =
( 1
1 1 1
)
A7 =
( 1
1 1 1
)
A8 =
( 1
1 1 1
)
A9 =
( 0
1 1 1 1
)
A10 =
( 0
1 1 1 1
)
A11 =
( 0
1 1 1 1
)
A12 =
( 0
1 1 1 1
)
A13 =
( 0
1 1 1 1
)
A14 =
( 0
1 1 1 1
)
A15 =
( 0
1 1 1 1
)
A16 =
( 0
1 1 1 1
)
. 32
SLIDE 33
A =
16
∑
i=1
qiAi and the followings are the 8 equations governing qi (cf. (7)):
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16
=
0.1 0.0 0.0 0.9 0.3 0.7 0.0 0.0 0.5 0.0 0.5 0.0 0.6 0.0 0.0 0.4
.
33
SLIDE 34 2 4 6 8 10 12 14 16 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
- Fig. 1. The Probability Distribution q for the case of A2,2.
34
SLIDE 35
State v1(t) v2(t) f (1) f (2) 1 1 1 2 1 1 3 1 4 1 1 Table 2: The Truth Table for A13. State v1(t) v2(t) f (1) f (2) 1 1 1 2 1 1 3 1 4 1 1 1 1 Table 3: The Truth Table for A14. State v1(t) v2(t) f (1) f (2) 1 1 1 2 1 1 3 1 1 4 1 1 Table 4 : The Truth Table for A15. State v1(t) v2(t) f (1) f (2) 1 1 1 2 1 1 3 1 1 4 1 1 1 1 Table 5 : The Truth Table for A16. 35
SLIDE 36 Example 2. We then consider the case n = 3 and m = 2 and we suppose that the observed transition matrix of the PBN is given as follows: A3,2 =
0.1 0.3 0.5 0.6 0.2 0.1 0.6 0.8 0.0 0.7 0.0 0.0 0.8 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.9 0.0 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0
.
- There are 256 possible BNs for constituting the PBN. The solution
is shown in Figure 2. We note that the PBN is dominated (over 60%) by 25 BNs.
36
SLIDE 37 50 100 150 200 250 300 0.01 0.02 0.03 0.04 0.05 0.06 0.07
- Fig. 2. The Probability Distribution q for the case of A3,2.
37
SLIDE 38
Example 3. We then consider a popular PBN (Shmulevich et al. (2002)): Network State f(1)
1
f(1)
2
f(2)
1
f(3)
1
f(3)
2
000 001 1 1 1 010 1 1 1 011 1 1 100 1 101 1 1 1 1 110 1 1 1 111 1 1 1 1 1 c(i)
j
0.6 0.4 1 0.5 0.5 Table 6: Truth Table (Taken from Shmulevich et al. (2002)). BN1 1 7 7 6 3 8 6 8 BN2 1 7 7 5 3 7 5 8 BN3 1 7 7 2 3 8 6 8 BN4 1 7 7 1 3 7 5 8 Table 7: The Four BNs.
38
SLIDE 39
- We consider adding some perturbations to the first two rows and
the non-zeros entries of the transition probability A4,4 as follows:
1.0 − δ δ δ 0.2 + δ δ δ δ δ δ δ δ 0.2 + δ δ δ δ δ 0.0 0.0 0.0 0.0 1.0 − 2δ 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.3 − δ 0.0 0.0 0.5 − δ 0.0 0.0 0.0 0.0 0.3 − δ 0.0 0.0 0.5 − δ 0.0 0.0 1.0 − 2δ 1.0 − 2δ 0.0 0.0 0.5 − δ 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.5 − δ 0.0 1.0 − 2δ
.
- For δ = 0.01, 0.02, 0.03 and 0.04, we apply our algorithm and
- btain 16 major BNs (out of 10368 BNs) (Table 8) and these BNs
actually contribute, respectively, 94%, 90%, 84% and 79% of the network.
- We note that the 1st, 8th, 9th and the last major BNs match with
the four BNs (BN1, BN2, BN3, BN4) in Table 7.
39
SLIDE 40
BNs qi(δ = 0.01) qi(δ = 0.02) qi(δ = 0.03) qi(δ = 0.04) 1* 1 7 7 1 3 7 5 8 0.047 0.045 0.042 0.040 2 1 7 7 1 3 7 6 8 0.047 0.045 0.042 0.040 3 1 7 7 1 3 8 5 8 0.047 0.045 0.042 0.040 4 1 7 7 1 3 8 6 8 0.047 0.045 0.042 0.040 5 1 7 7 2 3 7 5 8 0.047 0.045 0.042 0.040 6 1 7 7 2 3 7 6 8 0.047 0.045 0.042 0.040 7 1 7 7 2 3 8 5 8 0.047 0.045 0.042 0.040 8* 1 7 7 2 3 8 6 8 0.047 0.045 0.042 0.040 9* 1 7 7 5 3 7 5 8 0.071 0.067 0.063 0.059 10 1 7 7 5 3 7 6 8 0.071 0.067 0.063 0.059 11 1 7 7 5 3 8 5 8 0.071 0.067 0.063 0.059 12 1 7 7 5 3 8 6 8 0.071 0.067 0.063 0.059 13 1 7 7 6 3 7 5 8 0.071 0.067 0.063 0.059 14 1 7 7 6 3 7 6 8 0.071 0.067 0.063 0.059 15 1 7 7 6 3 8 5 8 0.071 0.067 0.063 0.059 16* 1 7 7 6 3 8 6 8 0.071 0.067 0.063 0.059 Table 8: The 16 Major BNs. 40
SLIDE 41 4.4 The performance of Newton Method and CG Method We also present the number of Newton’s iterations required for con- vergence and the average number of CG iterations in each Newton’s iteration in the following table. n m Number of BNs Newton’s Iterations Average Number
2 2 16 9 9 2 3 81 7 9 3 2 256 7 7 3 3 6561 11 13 Table 9 : Number of Iterations.
41
SLIDE 42 4.5 The Projection-based Gradient Descent Method
- We developed a projection-based gradient descent method
(Wen et. al. (2013)) for solving the following problem: min
q∈Ω φ(q) ≡ 1
2∥Uq − p∥2
2
(27) where Ω =
q : qi ≥ 0, ∑
i
qi = 1
.
- The challenge comes from the fact that the matrix U is huge in
practice such that it is not desirable to store the matrix. A matrix free method is therefore desirable for computational purpose.
- We prove its convergence and apply it to the PBN problem. The
solutions obtained are more sparse.
42
SLIDE 43
5 10 15 0.05 0.1 0.15 0.2
q Value
5 10 15 0.05 0.1 0.15 0.2
q Value
The probability distribution q : Entropy Method (Left) and Projection-based Method (Right)
5 10 15 20 0.02 0.04 0.06 0.08
Iterations Obj Function
The convergence curve of our method. 43
SLIDE 44
20 40 60 80 0.02 0.04 0.06 0.08 0.1
q Value
20 40 60 80 0.01 0.02 0.03 0.04 0.05 0.06
q Value
The probability distribution q: Entropy Method (Left) and Projection-based method (Right)
5 10 15 20 25 0.02 0.04 0.06 0.08 0.1 0.12
Iterations Obj Function
The convergence curve of our method. 44
SLIDE 45 References
- Akutsu, T. et al. (2000) Inferring qualitative relations in genetic
networks and metabolic arrays, Bioinformatics, 16, 727-734.
(2007) Control of Boolean networks: hard- ness results and algorithms for tree structured networks, Journal of Theoretical Biology, 244, 670-79.
- Chen, X., Jiang, H. and Ching, W. (2012), On Construction
- f Sparse Probabilistic Boolean Networks, East Asian Journal of
Applied Mathematics, 2, 1-18.
- Chen, X., Jiang, H., Qiu, Y. and Ching, W. (2012), On Op-
timal Control Policy for Probabilistic Boolean Network : A State Reduction Approach, BMC Systems Biology, 6 (Suppl 1):S8.
45
SLIDE 46
- Chen, X., Akutsu, T., Tamura, T. and Ching, W. (2013), Finding
Optimal Control Policy in Probabilistic Boolean Networks with Hard Constraints by Using Integer Programming and Dynamic Program- ming, International Journal of Data Mining and Bioinformatics, 7, 322-343.
- Ching, W., Scholtes, S. and Zhang, S. (2004), Numerical Algo-
rithms for Estimating Traffic Between Zones in a Network, Engi- neering Optimisation, 36, 379-400.
(2005) On construction of stochastic genetic networks based on gene expression sequences, International Journal
- f Neural Systems, 15, 297-310.
- Ching, W. and Cong, Y. (2009) A New Optimization Model for the
Construction of Markov Chains, Proceedings of CSO2009, Hainan, IEEE Computer Society Proceedings, Sanya, Hainan, China, April 24-26, 551-555, 2009.
46
SLIDE 47
- Ching, W. and Ng, M. (2006) Markov Chains : models, algorithms
and applications. International Series on Operations Research and Management Science, Springer, New York.
(2007) An approximation method for solving the steady-state probability distribution of probabilistic Boolean net- works, Bioinformatics, 23 1511-1518.
- Ching, W. et al. (2007) Optimal finite-horizon control for proba-
bilistic Boolean networks with hard constraints, Proceedings of the International Symposium on Optimization and Systems Biology.
- Ching, W., Zhang, S., Jiao, Y., Akutsu, T., Tsing, N. and Wong,
- A. (2009), Optimal Control Policy for Probabilistic Boolean Net-
works with Hard Constraints, IET on Systems Biology, 3 90-99.
47
SLIDE 48
- Ching, W., Chen, X. and Tsing, N. (2009) Generating Probabilistic
Boolean Networks from a Prescribed Transition Probability Matrix, IET on Systems Biology, 6 453-464.
(2003) External control in Markovian genetic regulatory networks, Machine Learning, 52, 169-91.
- De Jong, H. (2002) Modeling and simulation of genetic regulatory
systems: A literature review, J. Comp. Biol., 9, 67-103.
(2000) Coefficient of determination in nonlinear signal processing. Signal Processing, 80, 2219-35.
(2008) On approximate stochastic control in genetic regulatory networks, IET Systems Biology, 6, 361-368.
- Huang, S. (1999) Gene expression profiling, genetic networks, and
cellular states: An integrating concept for tumorigenesis and drug discovery, J. Mol. Med., 77, 469-480.
48
SLIDE 49
- Jiang, H., Chen, X., Qiu, Y. and Ching, W. (2012), On Gener-
ating Optimal Probabilistic Boolean Networks from a Set of Sparse Matrices, East Asian Journal of Applied Mathematics, 2, 353-372.
- Kauffman, S. A. (1969) Metabolic stability and epigenesis in ran-
domly constructed genetic nets. Theoretical Biology, 22, 437-467.
- Kauffman, S. A. The origins of order: Self-organization and se-
lection in evolution. New York: Oxford Univ. Press, 1993.
- Keller, A.D. (1994) Specifying Epigenetic States with Autoregu-
latory Transcription Factors. Journal of Theoretical Biology, 153, 181-194.
- Ng, M. and Ching W. et al. (2006) A control model for Marko-
vian genetic regulatory network. Transactions on Computational Systems Biology, Springer, 4070, 36-48.
- Pal, R. et al. (2005) Intervention in context-sensitive probabilistic
Boolean networks, Bioinformatics, 21 (7), 1211-1218.
49
SLIDE 50
- Pal, R. et al. (2006) Optimal Infinite-Horizon Control for Prob-
abilistic Boolean networks. IEEE Tran. Signal Processing, 54 (6), 2375-2387.
- Rabiner, L. (1989). A tutorial on hidden Markov models and se-
lected applications in speech recognition, Proceedings of the IEEE, 77, 257-286.
(2002) Probabilistic Boolean Networks: A rule-based uncertainty model for gene regulatory networks, Bioin- formatics, 18, 261-274.
- Shmulevich, I. et al. (2002) From Boolean to probabilistic Boolean
networks as models of genetic regulatory networks, Proceedings of the IEEE, 90, 1778-1792.
- Shmulevich, I. et al. (2002) Gene perturbation and intervention
in probabilistic Boolean networks, Bioinformatics, 18, 1319-1331.
50
SLIDE 51
(2002) Control of stationary behavior in probabilistic Boolean networks by means of structural intervention, Biological Systems, 10, 431-46.
- Shmulevich, I. et al (2003) Steady-state analysis of genetic reg-
ulatory networks modeled by probabilistic Boolean networks, Com- parative and Functional Genomics, 4, 601-608.
- Smolen, P. et al. (2000) Mathematical modeling of gene network,
Neuron, 26, 567-580.
- Steggles, L. J. et al. (2007) Qualitatively modelling and analysing
genetic regulatory networks: a Petri net approach, Bioinformatics, 23, 336-343.
(1998) From specific gene regulation to ge- nomic networks: A global analysis of transcriptional regulation in Escherichia coli, BioEssays, 20(5), 433-40.
51
SLIDE 52
- Wilson, A. (1970), Entropy in Urban and Regional Modelling,
Pion, London.
- Xu, W., Ching, W., Zhang, S., Li, W. and Chen X. (2011), A
Matrix Perturbation Method for Computing the Steady-state Prob- ability Distributions of Probabilistic Boolean Networks with Gene Perturbations, Journal of Computational and Applied Mathematics, 25, 2242-2251.
- Zhang, S. and Ching, W. et al. (2007) Simulation study in prob-
abilistic Boolean network models for genetic regulatory networks, Journal of Data Mining and Bioinformatics, 1, 217-40.
- Zhang, S. and Ching, W. et al. (2007) Algorithms for finding small
attractors in Boolean networks, EURASIP Journal on Bioinformatics and Systems Biology, Article ID 20180.
- Zhang, S., Ching, W., Tsing, N., Leung, H. and Guo, D. (2010) A
New Multiple Regression Approach for the Construction of Genetic Regulatory Networks, Journal of Artificial Intelligence in Medicine, 48, 153-160.
52
