L ECTURE 21: S WARM I NTELLIGENCE 7 / A NT C OLONY O PTIMIZATION 3 I - - PowerPoint PPT Presentation

LECTURE 21: SWARM INTELLIGENCE 7 / ANT COLONY OPTIMIZATION3

INSTRUCTOR: GIANNI A. DI CARO

15-382 COLLECTIVE INTELLIGENCE – S18

2

TYPES OF PROBLEM SCENARIOS

Offline: Full problem description and all data are accessible Online: Problem description / data become accessible sequentially, individually or in a bulk, while dealing with the

• problem. A model of data arrival can be available or not

Stationary: Problem parameters / structure do not change

• ver time (fixed), or keep being the same, on average

Dynamic: Problem parameters / structure do change over time and do not follow any globally stationary pattern. A model of changes can be available or not

3

COMPUTING / CONTROL ARCHITECTURES

Distributed problem § Data / code reside in physically different computing units § No shared memory: potentially need for communication channels § Delays in information / code propagation § Updated global system state is not instantaneously available Centralized problem § All data / code reside in the same computing unit / system § Shared memory § Shared time clock § Global system state available

4

CONTROL ARCHITECTURES IN MULTI-AGENT SYSTEMS

Centralized MA control § A central unit gathers data and perform decision-making for the agents sending controls back Distributed MA control § Control elements (controllers) are distributed throughout the system § Final decision-making can be done in a centralized or decentralized manner, or in- between Decentralized MA control § Each peer unit / agent makes local autonomous decisions towards its individual (global) goals § Peers might interact with each other and share information or provide service to

• ther peers

§ Emerging computation / behaviors § Local hierarchies can be present

15781 Fall 2016: Lecture 18

MORE ON CONTROL ARCHITECTURES

5

Distributed Control System

15781 Fall 2016: Lecture 18

PROBLEM TYPES: SO FAR AND NEXT ONES

6

Ø The optimization problem we have considered so far for ACO are: § Centralized § Can be solved according to any centralized or distributed control scheme of choice § Parallelization / Multi-Threading is quite immediate § Stationary Ø ? What about problems that are: § Distributed § Dynamic § They seem “closer” to ant colonies, by the way Ø Is the pheromone model (definition of stigmeric variables) the “same” for all types of centralized / stationary optimization problem? Ø Ant colonies perform automatic division of labor (diverging stigmergy) à Task Allocation Problems

15781 Fall 2016: Lecture 18

DYNAMIC SCENARIO: DATA ROUTING IN NETWORKS

7

Ø Routing problems: move an item (object, vehicle, data, agent) from one start location to one final destination or to a set of other locations using some interconnection infrastructure (a network) Ø A number of constraints may need to be considered regarding: capacity

• f the vehicle and/or the infrastructure, time/length budget, ordering for

visiting the locations, etc. Ø Goal: optimize the route followed to deliver/move the item § TSP: routing of an agent over a set of predefined locations, minimize traveled length § VRP (Vehicle Routing Problem): routing of a vehicle over predefined locations (to serve customers) subject to capacity and time constraints § Network routing: routing of data across a telecommunication network infrastructure (wired or wireless)

• Distributed
• Online
• Constraints defined by the network technology

15781 Fall 2016: Lecture 18

DATA ROUTING IN NETWORKS

8

§ Decision system to forward data packets from origin to destination nodes § Unicast, multicast, broadcast, anycast § Optimization of network-wide metrics § Best-effort / QoS: Throughput, end-to-end delay, losses, energy, … § Constraints imposed by the used transmission technology § Wired (point-to-point, bus, optical), wireless (omni, LoS, DCF), ... § Constraints imposed by the hardware used for packet forwarding § Specialized routers, workstations, laptops, mobiles . . .

15781 Fall 2016: Lecture 18

NETWORK ROUTING

9

§ Routing is the core component of the network layer in the OSI stack § The routing protocol specifies how routers communicate with each

• ther to disseminate information useful for routing

§ The routing algorithm uses this information to build a routing table (routing information database) and implements route selection

10

NETWORK ROUTING: META-PROTOCOL

• 1. Constant acquisition and organization of information concerning the

local state: information on the local traffic flows and on the status of the locally available resources (.e.g., neighbors, connection capacity)

• 2. Build a view of the global network state, possibly by exchanging local

state information with the other nodes

• 3. Use of the global view to set up the values of the local routing table

and define the local routing policy aiming to optimize some measure

• f network performance
• 4. Forward user traffic according to the defined routing policy
• 5. Asynchronously and concurrently with the other nodes repeat the

previous activities over time

At each network node / router:

11

LINK-STATE VS. DISTANCE-VECTOR ALGORITHMS

§ Each node maintains a complete description of the network (hierarchical, dynamic) § Flooding of link-state information § Each node computes shortest paths (e.g., Dijkstra-like algorithm) using its local network description § Each node 𝑜 maintains a vector 𝐸𝑗𝑡𝑢(𝑗,𝑒) of distance estimates for reaching each network destination 𝑒 through each of it links 𝑗 § Distributed (asynchronous) DP for computing

• ptimal routes

§ Node 𝑜 = state of the DP algorithm, value(𝑜) = distance to each possible destination

12

ROUTING IN MOBILE AD HOC NETWORKS (MANETS)

§ Infrastructure-less network of mobile devices connected wirelessly § Because of individual node mobility, wireless connection links can keep changing (breaking and creating) à Continual self-reconfiguring § Data traffic is forwarded in a multi-hop way, with each node acting as a source and a router at the same time § Wireless shared medium: collisions, interference, … § Heartbeat: each node must keep advertising its presence, in order to let neighbor nodes aware of its presence (and possible use as a router)

13

ANTHOCNET: ACO FOR ROUTING IN MANETS

§ Goal: Adaptively, build routing tables for data forwarding

§ Routing tables have an entry for each known destination 𝑒 and each known current neighbor 𝑜 § “Known”: things keep changing! § 𝑅𝑜𝑒 is a pheromone variable, it represents the estimated goodness/quality of routing decision 𝑜 for reaching destination 𝑒 (i.e., forwarding to neighbor 𝑜 a data packet with destination 𝑒) § Data packets are forwarded probabilistically, proportionally to the value of pheromone variables § Basic ACO idea: Ant agents are generated at each node to sample (solution) paths to known destinations, pheromone variables are used to guide path search, and are updated as a result of the found paths (ant agents need to physically retrace, hop by hop, the travelled path from source to destination) § Challenge: Ideally, the more ant agents are generated, the more paths are sampled, and the better the pheromone values can be estimated. Unfortunately, generating many ants creates traffic congestion and make the network unusable à Finding a good tradeoff between path sampling and bandwidth usage is necessary

14

ANTHOCNET: ACO FOR ROUTING IN MANETS

§ Hybrid algorithm: § Reactive: paths are only setup at the starting of a session (~ controlled flooding, high bandwidth usage) § Proactive: during the course of a session paths are proactively improved and extended with a low impact on bandwidth § AntHocNet integrates two mechanisms for distributed adaptive learning: § Monte Carlo sampling and updating of full paths with ants (ACO) § Periodic exchange of routing information among the nodes, and local incremental updating of the estimated pheromone values using a mechanism equivalent to Dynamic programming / Bellman-Ford § What exactly pheromone variables represent? The quality of a path in a MANET depends on many aspects: associated end-to-end delay, but also the number of hops, interference at each hop, degree of mobility of path nodes § 𝑅= Low delays AND High expected stability and reliability of the path § 𝑅-. = 𝐺(SNR, end-to-end delay, hops, mobility, … )

15

REACTIVE PATH SETUP

§ Node 𝑡, at the start of a new data session to node 𝑒: § Pheromone information is available (i.e., the local routing table contains information about how to reach 𝑒) à There’s (likely) a known routing path from 𝑡 to 𝑒, therefore, then the path information is used to forward the packets from the data session § No pheromone information is available: a path needs to be discovered, if it exists, and the following reactive (on-demand) path setup process is started

• 1. Forward ants are broadcast from 𝑡 to all its neighbors (i.e., nodes in radio range)
• 2. Each neighbor 𝑜 that receives a forward ant, should forward it towards 𝑒: if 𝑜 has

pheromone information (i.e., path information) about 𝑒, then the ant is forwarded to a neighbor selected based on an additive ant-routing rule and an 𝜁-greedy

• policy. If no path information is available, the ant is again broadcast
• 3. At all intermediate nodes only the first arriving ant agent is forwarded further
• 4. The first ant arriving at the destination node 𝑒 retraces its forward path (it

becomes a backward ant), and at each node along the way it sets up the path to 𝑒 by adding/updating pheromone variables in their routing tables

• 5. → A single path from 𝑡 to 𝑒 is made available for the data session

16

PROACTIVE IMPROVEMENT / EXTENSIONS

§ At low frequency, each node 𝒌 broadcasts a pheromone information message with the best pheromone value 𝑅-

3. 4

for each destination 𝑒 known by 𝑘 § 𝑅-

3. 4 is the value of the pheromone variable associated to the best neighbor, 𝑜

3, to reach destination 𝑒 from 𝑘. The numeric value 𝑅-

3. 4

is a measure of a generalized distance to 𝒆 from 𝒌 via 𝒐 9 § A 𝑘’s neighbor, 𝑗, on receiving a best pheromone message: § Update / Refresh its neighbor view § Bootstrap on received pheromone values to update local 𝑅: values for the already known destinations: 𝑅4.

: ← 𝛽 𝑅- 3. 4 + 𝑅44 :

+ (1 − 𝛽)𝑅4.

:

§ Add new destinations/pheromone to the routing table (in the case 𝑒 was not a known destination to 𝑗

17

PROACTIVE IMPROVEMENT / EXTENSIONS

§ The bootstrapping process is equivalent to a multi-step diffusion process (node 𝑗, in turn, will publish its new pheromone updates to its neighbors, and they will do the same). The process is however slow and prone to errors § → New found paths are not used by data before validation § à Pheromone-following proactive ants are sent out to test the new paths and/or update the estimates over the old ones § → As a result, a bundle of multiple paths are made dynamically available and kept up-to-date

18

STOCHASTIC DATA ROUTING

§ Available multiple paths can be used both as routing and backuppaths for data transmission (because of continual changes it’s necessary to have backup paths) § Routing decisions for data packets are based on a stochastic policy which gives strong preference to the next hops with higher pheromone values § If pheromone is kept up-do-date, this leads to improved robustness to failures and better resources utilization

19

ANTHOCNET AT WORK

§ Animations showing for a sample urban scenario: § The set of dynamically discovered paths and their pheromone value (low, medium, high) § The subset of these paths used to route data (blue lines) § Running statistics averaged over the last 5 seconds for delay, throughput, and jitter for both AntHocNet and AODV § The scenario is composed of 350 nodes § The color inside each node shows the congestion level it is experiencing (low, medium, high) due to constant active data sessions between nodes § One sample session is shown at a time

20

ANTHOCNET AT WORK

https://www.dropbox.com/s/cmxxpgtr 0epnlas/openSpace.mp4?dl=0

Open space Urban scenario, boat sudden congestion

https://www.dropbox.com/s/l23bdn2 2ohkwr4z/truck-depot.mp4?dl=0

Urban scenario, truck - depot

https://www.dropbox.com/s/8ncxdt stf11yrkn/boat.mp4?dl=0

21

INTEGER PROGRAMMING PROBLEMS

§ Back to stationary problems: ACO has shown good performance as a meta-heuristic for integer / binary optimization problems (IP)

min

𝒚

𝑔 𝒚 s.t. 𝒚 ∈ ℱ, 𝒚 ∈ 𝑌- ⊆ ℤ-

Rosenbrock function in 2D

x1 x2

1 2 3 1 2 4 5 3

x1 + 0.8x2 = 5.8 x1 = 5 z = −x1 − 2x2 x1 + 8x2 = 26 x1 = 1 x1 − 0.8x2 = 0.2

Integer Linear Program (ILP)

min

𝒚 −𝑦K − 2𝑦M

𝑡. 𝑢. 𝑦K≥ 1 𝑦K ≤ 5 𝑦K + 0.8𝑦M ≤ 5.8 𝑦K − 0.8𝑦M ≥ 0.2 𝑦K + 8𝑦M ≤ 26 𝑦K,𝑦M ∈ ℤT

22

COMBINATORIAL OPTIMIZATION PROBLEMS

§ A Combinatorial Optimization Problem (COP) is an IP

• ptimization problem in which we seek to find a solution in a finite

set of solutions § à Integer variables 𝒚 have bounded ranges § TSP, VRP, QAP, Set covering, Assignment, Knapsack, …

min

𝒚

−𝑦K − 2𝑦M 𝑡.𝑢. 𝑦K≥ 1 𝑦K + 0.8𝑦M ≤ 5.8 𝑦K − 0.8𝑦M ≥ 0.2 𝑦K + 8𝑦M ≤ 26 𝑦K ≤ 5, 𝑦M≤ 4 𝑦K,𝑦M ∈ ℤT

§ Any bounded integer, 0 ≤ 𝑦 ≤ 𝑣, can be converted to a set of 0-1 variables, 2W ≤ 𝑣 ≤ 2WTK § 0 ≤ 𝑦 ≤ 20, 𝑦 = 2X𝑧0 + 2K𝑧1 + 2M𝑧2 + 2Z𝑧3 + 2\𝑧4

§ A COP can be formulated as a 0-1 integer program: 𝒚 ∈ {0,1}n

HOW DO WE APPROACH AN ILP?

23

§ Exact algorithm: provably optimal solution, deal with NP-hardness § Heuristic algorithm: apply heuristics and metaheuristics to find good solutions in a reasonably short time, no guarantees à PSO, ACO, GA, TS, … § Approximation algorithm: derive an approximation giving a bound on the solution value

24

TYPICAL PROBLEMS (MIPLIB2010)

15781 Fall 2016: Lecture 13

IP SOLVERS

25

IBM ILOG CPLEX Gurobi

http://scip.zib.de