Interactions sur le fonctionnement dans les systmes multi-agents - - PowerPoint PPT Presentation
Interactions sur le fonctionnement dans les systmes multi-agents ouverts et htrognes Interactions about Actions in Open and Heterogenous Multi-Agent Systems Soutenance d'Habilitation Diriger des Recherches Nicolas Sabouret Lundi
1/47
Soutenance d'Habilitation à Diriger des Recherches
Lundi 20 novembre 2009
2/47
Explainations
Real World situations (ex: Ambient Computing)
3/47
Distributed
System behaviour ← entities + interactions Need to combine functionalities
Open
Loosely coupled
Heterogeneous
Inconsistent models for data & actions Agent interactions and Human-Agent interactions
4/47
Explicit goal → dynamic choregraphy Implicit goal → learning interactions
Very simple problems often turn out very difficult to solve...
Incompatible representations → dynamic semantic interpretation
5/47
Related work in…
Service composition Semantic heterogeneity Reinforcement learning & interactions
The VDL model Service composition Learning interactions Semantic heterogeneity Conclusion & future work
6/47
7/47
[Peltz, 03] → service orchestration & choreography [OWL-S, 04] [WSDL, 03] [Moreau, 08] → syntactic service orchestration [Durfee, 01] → task-oriented [Shehory, 99] [Aknine, 02] [Müller, 06] → workflow description [Ermolayev, 03] → goal description
[Traversore, 04] → planing on service ontology [Wu, 03] → H TaskNets
[Paurobally, 05]
8/47
Service description
Tasks decomposition
9/47
Existing work
A priori task decomposition A priori known set of possible actions
Open MAS
Discover tasks at runtime
10/47
11/47
[Laera, 07] → MAS protocol for
Thesaurus [WN, 98] Semantic Networks Ontologies [OWL, 04] Structure-based Instance-based Reference
Semantic negociation [Morge, 07] [Breitman, 05] [Ichise, 03] [van Diggelen, 06] (Anemone) [Aleksowski, 06] [Valencia, 04]
12/47
Reference ontology
Semantic negociation
Open & loosely coupled MAS
Dynamic understanding of concepts
13/47
14/47
Interaction protocols for learning
Open MAS
Learning when to interact
Learning what to interact
? Open MAS
15/47
MDPs POMDPs Memory [Dutech, 03]
SMDPs Asynchronism Other agents Delegation
Asynchronous & open
16/47
17/47
18/47
Language Layer Interaction Layer
→ softbody
→ events
→ questions about actions! Anchoring
Cognitive Layer
Interaction control Runtime control Observers
19/47
XML tree rewriting
Data
Ontology
typeof and includes
Other relations
20/47
Actions
Preconditions – effets
Effects on data
Message sending
Preconditions
Events: evt(x1(val1),...,xn(valn)) Event patterns
Boolean preconditions
Context Context-Structure Structure
21/47
Capacities = acceptable events
evale(p,evt) → true iff p∈Ps∪Pc
s is true under evt
evalc(p) → true iff p∈Pc is true
VDL code introspection (using precondition and data structure)
22/47
FIPA-ACL based
snd rcv
Specific performatives
query, inform, unknown request, agree impossible, assert-cannot assert-can, clarify, suggest what-can query-contraint not-understood, error
23/47
Agt1 Agt2
query(v) inform(v=val)
ALT
unknown(v) Query & al.
Request & al.
Agt1 Agt2
request(e) agree
ALT
impossible(np(e)) assert-cannot(e) assert-can(e'∈F) clarify(E)
24/47
25/47
Initial request
Service discovery Dynamic choreography Final answer → initiator agent
2 1 3 4
26/47
Initiator - participants Delegation to all participants
Waiting for answers Convergence in EXPTIME
VDL ?
27/47
init part
query-constraint inform
ALT
unknown
OPT
query init part
request assert-can
ALT
agree
OPT
query assert-can * part1 part2
query inform
ALT
unknown
n k m kmn
part1 part2
assert-can inform
OPT
query OR Agent
inform
unknown
history
answer to m0 ?
query-constraint request assert-can
[IAT, 07]
28/47
Implemented in Java on the VDL platform (2006)
[RIA, 08]
29/47
30/47
Goal → reward function Problems:
Asynchronous → learn to wait Non-observable → POMDPs Delegation (request) → Memory
Limited to…
1 learning agent Performatives query & request
Internal action Send message Wait Acquire requests Acquire queries
Q-table Memory
Environment
reward action Lastest requests
Temperature
31/47
Acquiring requests Acquiring queries
Agt1 Agt2
request impossible query
add(query)
vars p, p∈NP Agt1 Agt2
what-can suggest request
add(request)
E
+ timeouts + timeouts
32/47
Memory
State + latest request(s) or query-result(s)
[McCallum, 96]
Iterative construction memory
s1: a1>a2 s4: a4>a1 s2: a3>a1>a4
0 slot memory 1 slot memory 2 slot memory etc
s1: a2>a3 s4: a4>a1 s2: a3>a1>a4 a1 s1: a1>a2 a2 s1: a2>a3 s4: a1>a3 s2: a3>a1>a4 a1,a2 s1: a1>a2 a2 a1,a1 s1: a1>a2 a4
33/47
adjust W
At each step
Q s, a/T
Qs ,a/T
OR send message query OR send message request OR perform action store answer store action adjust Q(s,a)
ambs=wait s1 3rang s [ups]rang s[ 1
∣As∣∑a∈As qa]rang s
−1[qa1−qa2]
N cycles
Add memory to k most ambiguous states:
[MFI, 09]
34/47
Asynchronous learning of commands (requests)
Learner agent
Explorer agent (twice slower)
commands No memory 1 slot memory 2 slots memory → no progress
35/47
Learning of interactions (requests & queries)
Learner agent
random state change (depending on heater) heater
stir/remove (request) / observe (query) more → no improvement 1 slot memory 0 slot memory no query
36/47
37/47
Incomplete alignment
Limited to requests Architecture:
OpenNLP WordNet Alignment Human user Agent event
code & context capacities generation capacities selection answer manager semantic relatedness measure
38/47
Generalization to other Unique Type Pathes
[ISWC, 08]
39/47
appe req,ecap=Q ereq× max S∈Creq ,C cap∑cr , cc∈S relcr ,cc
capacities generation capacities selection answer manager semantic relatedness measure
Concepts anchoring
evt(c1(val1),...cn(valn)) External event (aligned) er e
q
Internal capacity ec
a p
evt(c1(val1),...cn(valn)) Set of possible couples (filtered using the VDL structure) Quality of alignement or NL analysis
40/47
E reqrcv ={ecap∈E rcv∨ pE−appereq ,e cap } F reqrcv={e cap∈F rcv∨ pF−appereq ,ecap}
Build the set of best capacities Two thresholds [Maes, 94]
[WIAS, 08]
41/47
Cube-world based
Two agents, shared world, 2 representations Impossible/incomplete alignement
Take red polygon
42/47
Implemented on the VDL platform in 2007
43/47
44/47
Reasoning about actions and interactions
Agent-Agent interactions Human-Agent interactions
Instrospection of agents' capacities
Service composition Learning interactions Semantic heterogeneity
Openness, heterogeneity, loosely couple,
45/47
Composition vs Semantic heterogeneity
Message not understood → 2 possible methods Heuristics to choose?
Learning interactions
Discover data unseen by request-impossible New memory structure Semantic interpretation
46/47
Extend the current model
Service composition with advanced constraints Semantic heterogeneity for other performatives Generalisation of learned behaviours
Reasoning about time
Past and future interactions Other performatives
Environment (Eric Platon)
Unknown environment Indirect interactions in the composition process
Communicate about unknown actions and data
47/47
Eric Yasmine Laurent Shirley 2011