GRAS: a Research and Development Framework for Grid and P2P - - PowerPoint PPT Presentation

gras a research and development framework for grid and
SMART_READER_LITE
LIVE PREVIEW

GRAS: a Research and Development Framework for Grid and P2P - - PowerPoint PPT Presentation

GRAS: a Research and Development Framework for Grid and P2P Infrastructures Martin Quinson martin.quinson@loria.fr Nancy University / LORIA (France) Parallel and Distributed Computing and Systems (PDCS 2006) November 13, 2006 Context


slide-1
SLIDE 1

GRAS: a Research and Development Framework for Grid and P2P Infrastructures

Martin Quinson martin.quinson@loria.fr

Nancy University / LORIA (France)

Parallel and Distributed Computing and Systems (PDCS 2006)

November 13, 2006

slide-2
SLIDE 2

Context

Modern computational platforms

◮ Grid and P2P systems aggregate distributed resources

Almost infinite potential Difficult to use

Main characteristic: large scale

◮ Range from a few dozen nodes to millions

⇒ Heterogeneous: hardware, software, even administrative orientations ⇒ Dynamic: quantitative (bandwidth variation) and qualitative (resource churn)

Difficulties

◮ Theoretical: Heuristics mandatory for NP-hard problems (scheduling, routing, etc)

But dynamicity hinders experiment reproductability! ⇒ Temptation of a simulator, but we want more than prototyping

◮ Technical: Setup a development and experimentation environment difficult

⇒ Need for an adapted runtime environment

Martin Quinson Research and Development of Grid and P2P Infrastructures Introduction 1/17

slide-3
SLIDE 3

Motivation Applications need to rely on infrastructures

A lot of middlewares and infrastructures exist already

◮ Remote execution (NetSolve, DIET, APST, Condor) ◮ Platform monitoring (NWS) and discovery (ENV) ◮ A bit of everything (Globus)

Infrastructures are themselves difficult to develop, assess and tune

◮ Can be seen as large-scaled distributed applications ◮ Several entities placed on nodes interacting with application level protocol ◮ Specificity: Distribution must not be masked

The GRAS project

◮ Aims at easing the development of network-aware applications ◮ Development on simulator, deployment without modification

Martin Quinson Research and Development of Grid and P2P Infrastructures Introduction 2/17

slide-4
SLIDE 4

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

Development

rewrite

Without GRAS

Code Simulation Application Code

Research

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-5
SLIDE 5

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

Simulation Application Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research ◮ Framework for Rapid Development of Distributed Infrastructure

◮ Develop and tune on the simulator; Deploy in situ without modification Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-6
SLIDE 6

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

SimGrid GRDK GRE API

Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research ◮ Framework for Rapid Development of Distributed Infrastructure

◮ Develop and tune on the simulator; Deploy in situ without modification

How: One API, two implementations

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-7
SLIDE 7

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

SimGrid GRDK GRE API

Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research

GRAS

◮ Framework for Rapid Development of Distributed Infrastructure

◮ Develop and tune on the simulator; Deploy in situ without modification

How: One API, two implementations

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-8
SLIDE 8

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

SimGrid GRDK GRE API

Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research

GRAS

◮ Framework for Rapid Development of Distributed Infrastructure

◮ Develop and tune on the simulator; Deploy in situ without modification

How: One API, two implementations

◮ Efficient Grid Runtime Environment (result = application = prototype)

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-9
SLIDE 9

Goals of the GRAS project Easing infrastructure development

Development of real distributed applications using a simulator

SimGrid GRDK GRE API

Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research

GRAS

◮ Framework for Rapid Development of Distributed Infrastructure

◮ Develop and tune on the simulator; Deploy in situ without modification

How: One API, two implementations

◮ Efficient Grid Runtime Environment (result = application = prototype)

◮ Performance concern: efficient communication of structured data

How: Efficient wire protocol (avoid data conversion)

◮ Portability concern: because of grid heterogeneity

How: ANSI C + autoconf + no dependency

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 3/17

slide-10
SLIDE 10

Presentation outline

Introduction The GRAS project Project goals The SimGrid simulator Offered API Efficient communication of structured data Emulation and Virtualization Experimental evaluation Assessing communication performance Assessing API simplicity Conclusion and Perspectives

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 4/17

slide-11
SLIDE 11

The simulator used: SimGrid [Casanova, Legrand]

Standard simulator for grid application studies SimGrid functionalities

◮ Complex and realistic platforms (with availability variations and resource churn) ◮ Based on fluid models (unlike packet based network simulator like NS2)

◮ Fast: simulates several hours per second ◮ Thousands of simulated processes on a single host ◮ Satisfying (ongoing) validation

GRAS and SimGrid

◮ GRAS hides SimGrid details: applications just ”work” on the simulator ◮ Actually, GRAS is now part of the SimGrid toolbox

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 5/17

slide-12
SLIDE 12

Main concepts of the GRAS API

Agents (acting entities) Messages (what gets exchanged between agents) Callbacks (code to execute when a message is received)

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 6/17

slide-13
SLIDE 13

Main concepts of the GRAS API

Agents (acting entities)

◮ Code (C function) ◮ Private data ◮ Location (hosting computer)

Messages (what gets exchanged between agents)

◮ Semantic: Message type ◮ Payload described by data type description (fixed for a given type)

Callbacks (code to execute when a message is received)

◮ Also possible to explicitly wait for given messages

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 6/17

slide-14
SLIDE 14

Conveying structured data

GRAS message payload can be any valid C type

◮ Structure, enumeration, array, pointer, . . . ◮ Classical garbage collection algorithm to deep-copy it

Describing a data type to GRAS

◮ Can be manually described, or automatically parsed from C type declaration

GRAS wire protocol: NDR (Native Data Representation)

Avoid data conversion when possible:

◮ Sender write data on socket as they are in memory ◮ If receiver’s architecture does match, no conversion ◮ Receiver able to convert from any architecture

Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 7/17

slide-15
SLIDE 15

Emulation and Virtualization

Same code runs without modification both in simulation and in situ

◮ In simulation, agents run as threads within a single process ◮ In situ, each agent runs within its own process

⇒ Agents are threads, which can run as separate processes

Emulation issues

◮ How to get the current time? How to get the process sleeping?

◮ System calls are virtualized: gras os time; gras os sleep

◮ How to report computation time into the simulator?

◮ Asked explicitly by user, using provided macros ◮ Time to report can be benchmarked automatically Martin Quinson Research and Development of Grid and P2P Infrastructures The GRAS project 8/17

slide-16
SLIDE 16

Presentation outline

Introduction The GRAS project Project goals The SimGrid simulator Offered API Efficient communication of structured data Emulation and Virtualization Experimental evaluation Assessing communication performance Assessing API simplicity Conclusion and Perspectives

Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 9/17

slide-17
SLIDE 17

Assessing communication performance

Only communication performance studied since computation are not mediated

◮ Experiment: timing ping-pong of structured data (a message of Pastry) typedef struct { int id, row_count; double time_sent; row_t *rows; int leaves[MAX_LEAFSET]; } welcome_msg_t; typedef struct { int which_row; int row[COLS][MAX_ROUTESET]; } row_t ; ◮ Tested solutions

◮ GRAS ◮ PBIO (uses NDR) ◮ OmniORB (classical CORBA solution) ◮ MPICH (classical MPI solution) ◮ XML (Expat parser + handcrafted communication)

◮ Platform

◮ Scale: intra-machine; on LAN; on WAN ◮ Architectures: x86, PPC, sparc (all under Linux) Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 10/17

slide-18
SLIDE 18

Intra-machine results

10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 1.1ms 0.7ms 2.0ms n/a 14.8ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 9.8ms 0.7ms 7.4ms 7.3ms 62.6ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 1.3ms 0.8ms 1.4ms 1.4ms 10.6ms

ppc sparc x86 Discussion

◮ Portability: PBIO broken on PPC ◮ Performance: XML ways slower (extra conversions + verbose wire encoding)

Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 11/17

slide-19
SLIDE 19

LAN results (column: sender; row: receiver)

ppc sparc x86 ppc

10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 4.3ms 0.8ms 8.2ms n/a 22.7ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 3.9ms 2.4ms 7.7ms n/a 40.0ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 3.1ms n/a 5.4ms n/a 17.9ms

sparc

10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 6.3ms 1.6ms 26.8ms n/a 42.6ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 4.8ms 2.5ms 7.7ms 7.0ms 55.7ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 5.7ms n/a 20.7ms 6.9ms 38.0ms

x86

10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 3.4ms n/a 5.2ms n/a 18.0ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 2.9ms n/a 5.4ms 5.6ms 34.3ms 10-4 10-3 10-2 XML PBIO OmniORB MPICH GRAS 2.3ms 0.5ms 3.8ms 2.2ms 12.8ms

◮ MPICH twice as fast as GRAS, but cannot mix little- and big-endian Linux ◮ GRAS second best solution

Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 12/17

slide-20
SLIDE 20

WAN Results (California → France)

10-4 10-3 10-2 10-1 100 XML PBIO OmniORB MPICH GRAS 1.10s n/a 1.75s n/a 1.87s 10-4 10-3 10-2 10-1 100 XML PBIO OmniORB MPICH GRAS 0.94s n/a 1.49s 1.02s 1.70s 10-4 10-3 10-2 10-1 100 XML PBIO OmniORB MPICH GRAS 0.98s n/a 1.38s 1.09s 1.69s

ppc sparc x86

◮ Less performance difference on WAN ◮ Portability: failed to use MPICH on WAN

Experiment conclusion GRAS is the better compromise between performance and portability

Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 13/17

slide-21
SLIDE 21

Assessing API simplicity

Experiment: ran code complexity measurements on code for previous experiment

GRAS MPICH PBIO OmniORB XML McCabe Cyclomatic Complexity 8 10 10 12 35 Number of lines of code 48 65 84 92 150

Results discussion

◮ XML complexity may be artefact of Expat parser (but fastest) ◮ MPICH: manual marshaling/unmarshalling ◮ PBIO: automatic marshaling, but manual type description ◮ OmniORB: automatic marshaling, IDL as type description ◮ GRAS: automatic marshaling & type description (IDL is C)

Conclusion GRAS is the less demanding solution from developer perspective

Martin Quinson Research and Development of Grid and P2P Infrastructures Experimental evaluation 14/17

slide-22
SLIDE 22

Presentation outline

Introduction The GRAS project Project goals The SimGrid simulator Offered API Efficient communication of structured data Emulation and Virtualization Experimental evaluation Assessing communication performance Assessing API simplicity Conclusion and Perspectives

Martin Quinson Research and Development of Grid and P2P Infrastructures Conclusion and Perspectives 15/17

slide-23
SLIDE 23

Conclusion: GRAS eases infrastructure development

SimGrid GRDK GRE API

Code

Research & Development

With GRAS

Development

rewrite

Without GRAS

Code Simulation Application Code

Research

GRDK: Grid Research & Development Kit

◮ API for (explicitly) distributed applications ◮ Uses a fast yet realistic simulator (SimGrid)

GRE: Grid Runtime Environment

◮ Efficient: twice as slow as MPICH, faster than OmniORB, PBIO, XML ◮ Portable: Linux (11 CPU archs); Windows; Mac OS X; Solaris; IRIX; AIX ◮ Simple and convenient:

◮ API simpler than classical communication libraries ◮ Easy to deploy: C ANSI; no dependency; autotools; <400kb ◮ Extensive toolbox: data containers, logs, config, exceptions, test units, etc. Martin Quinson Research and Development of Grid and P2P Infrastructures Conclusion and Perspectives 16/17

slide-24
SLIDE 24

Perspectives Future work on GRAS

◮ Performance: type precompilation, communication taming and compression ◮ GRASPE (GRAS Platform Expender) for automatic deployment ◮ Model-checking as third mode along with simulation and in-situ execution

Ongoing applications

◮ Comparison of P2P protocols (Pastry, Chord, etc) ◮ Network mapper (ALNeM): capture platform descriptions for simulator ◮ Large scale mutual exclusion service

SimGrid Distribution

◮ LGPL, 30 000 lines of code ◮ http://gforge.inria.fr/projects/simgrid/ ◮ Examples, documentation and tutorials on the web page

Martin Quinson Research and Development of Grid and P2P Infrastructures Conclusion and Perspectives 17/17

slide-25
SLIDE 25

Thanks for attending

Any question?

Martin Quinson Research and Development of Grid and P2P Infrastructures Conclusion and Perspectives 17/17

slide-26
SLIDE 26

Presentation outline

Introduction The GRAS project Project goals The SimGrid simulator Offered API Efficient communication of structured data Emulation and Virtualization Experimental evaluation Assessing communication performance Assessing API simplicity Conclusion and Perspectives

slide-27
SLIDE 27

Appendix

Example of code: Ping-Pong Describing data types to GRAS Network simulators vs SimGrid Simulation kernel details Details on the emulation support Visualizing the simulation Model-checking

Martin Quinson Research and Development of Grid and P2P Infrastructures 17/17

slide-28
SLIDE 28

Example of code: ping-pong (1/2)

Common to client and server

#include "gras.h" XBT_LOG_NEW_DEFAULT_CATEGORY(Ping,"Messages specific to this example"); static void register_messages(void) { gras_msgtype_declare("ping", gras_datadesc_by_name("int")); gras_msgtype_declare("pong", gras_datadesc_by_name("int")); }

Client code

int client(int argc,char *argv[ ]) { gras_socket_t peer=NULL, from ; int ping=1234, pong; gras_init(&argc, argv); gras_os_sleep(1); /* Wait for the server startup */ peer=gras_socket_client("127.0.0.1",4000); register_messages(); gras_msg_send(peer, gras_msgtype_by_name("ping"), &ping); INFO3("PING(%d) -> %s:%d",ping, gras_socket_peer_name(peer), gras_socket_peer_port(peer)); gras_msg_wait(6000,gras_msgtype_by_name("pong"),&from,&pong); gras_exit(); return 0; }

Martin Quinson Research and Development of Grid and P2P Infrastructures Example of code: Ping-Pong 17/17

slide-29
SLIDE 29

Example of code: ping-pong (2/2)

Server code

typedef struct { /* Global private data */ int endcondition; } server_data_t; int server (int argc,char *argv[ ]) { server_data_t *globals; gras_init(&argc,argv); globals = gras_userdata_new(server_data_t); globals->endcondition=0; gras_socket_server(4000); register_messages(); gras_cb_register(gras_msgtype_by_name("ping"),&server_cb_ping_handler); gras_msg_handle(600.0); if (!globals->endcondition) WARN0("An error occured: endcondition != 0"); free(globals); gras_exit(); return 0; } static int server_cb_ping_handler(gras_socket_t expeditor, void *payload_data) { int msg=*(int*)payload_data; /* Get the payload */ server_data_t *globals=(server_data_t*)gras_userdata_get(); /* Get the globals */ /* Send data back as payload of a pong message to the ping’s expeditor */ gras_msg_send(expeditor, gras_msgtype_by_name("pong"), &msg); globals->endcondition = 1; }

Martin Quinson Research and Development of Grid and P2P Infrastructures Example of code: Ping-Pong 17/17

slide-30
SLIDE 30

Conveying structured data (details)

Manual description (excerpt)

gras_datadesc_type_t gras_datadesc_struct(const char *name); void gras_datadesc_struct_append(gras_datadesc_type_t s, char*name, gras_datadesc_type_t field); void gras datadesc struct close(gras datadesc t struct type);

Automatic description of a classical matrix type

GRAS_DEFINE_TYPE(s_matrix, struct s_matrix { int rows; int cols; double *data GRAS_ANNOTE(size,rows*cols); } );

C declaration stored into a char* variable to be parsed at runtime

Existing wire protocols

◮ XDR Everything converted to a ”common language” ◮ CDR Scalar converted on need only; Structured data converted anyway ◮ NDR Scalar and structures converted on need only (used by GRAS)

Martin Quinson Research and Development of Grid and P2P Infrastructures Describing data types to GRAS 17/17

slide-31
SLIDE 31

Network and Grid Simulators

Usual Network Simulators

◮ Goals:

◮ Understand networks behavior, routing protocols, QoS, . . . ◮ Identify and overcome limitations of network protocols

⇒ Precise simulation of the packet movements along links

◮ Examples: NS, DaSSF, OMNeT++

Our needs:

◮ Network behavior as experienced by applications: no need for packets ◮ Something fast (during debugging phase): no need for all details ◮ Must consider CPU resource

Solution: SimGrid [Casanova, Legrand]

Martin Quinson Research and Development of Grid and P2P Infrastructures Network simulators vs SimGrid 17/17

slide-32
SLIDE 32

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-33
SLIDE 33

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-34
SLIDE 34

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-35
SLIDE 35

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

Schedule(T3,P1) GetPrediction(P2) GetPrediction(P1) Simulate(ANY TASK)

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-36
SLIDE 36

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

Schedule(T3,P1) GetPrediction(P2) GetPrediction(P1) Simulate(ANY TASK)

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-37
SLIDE 37

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

Simulate() res ← T2 Schedule(T3,P1) GetPrediction(P2) GetPrediction(P1) Simulate(ANY TASK)

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-38
SLIDE 38

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

Simulate() res ← T2 Schedule(T3,P1) GetPrediction(P2) GetPrediction(P1) Simulate(ANY TASK)

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-39
SLIDE 39

Under the hood: the simulation kernel

◮ Main objects:

◮ Resource: Availability trace (CPU, BW) + Latency trace + Sharing policy ◮ Task: Amount of work

◮ Tasks are scheduled onto resources ◮ Simulator computes task ending time T: Work =

t0+T

t=t0+L(t0)

B(t)dt

◮ Example:

done Simulate() res ← T2 Schedule(T3,P1) GetPrediction(P2) GetPrediction(P1) Simulate(ANY TASK)

10s

Schedule(T1,P1) Simulate(10 s) Schedule(T2,P2)

P1 P2

◮ Sharing policies: Sequential, Shared or TCP

TCP sharing modeled as Max-Min fairness (proportional soon

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-40
SLIDE 40

Under the hood: implementing multiple agents

◮ Several execution contexts:

◮ One context per agent ◮ A maestro context

◮ Control flow

◮ Agent runs gras msg send() and similar ⇒ control passed to maestro ◮ Maestro schedules the task within simulation kernel ◮ Maestro asks simulation kernel which scheduled tasks are done ◮ Maestro passes control to corresponding agent 10s P1 P2

◮ Implementation: pthreads or ucontexts (lighter, faster but unix98)

Some hundreds or thousands of simulated processes on a single host

Martin Quinson Research and Development of Grid and P2P Infrastructures Simulation kernel details 17/17

slide-41
SLIDE 41

Details on the emulation support

◮ We have a collection of macro to automatically report computation time into

the simulator:

Run on host? Bench’ed? Time reported? GRAS BENCH ALWAYS BEGIN() GRAS BENCH ALWAYS END() Each time Each time Each time GRAS BENCH ONCE RUN ONCE BEGIN() GRAS BENCH ONCE RUN ONCE END() First time First time Each time GRAS BENCH ONCE RUN ALWAYS BEGIN() GRAS BENCH ONCE RUN ALWAYS END() Each time First time Each time ◮ Other problems to solve:

◮ What about global data? ◮ Agent status placed in a specific structure, ad-hoc manipulation API ◮ How to write the main()? ◮ Use another name (as usual with threads, the real main() is generated) Martin Quinson Research and Development of Grid and P2P Infrastructures Details on the emulation support 17/17

slide-42
SLIDE 42

Visualizing the simulation: SimGrid generates Paj´

e traces

Martin Quinson Research and Development of Grid and P2P Infrastructures Visualizing the simulation 17/17

slide-43
SLIDE 43

Model-checking GRAS application (still to be done)

Motivation

◮ GRAS allows to debug an application on simulator and deploy it when it works ◮ Problem: when to decide that it works?

◮ Demonstrate a theorem → conversion to C difficult ◮ Test some cases → may still fail on other cases

Model-checking

◮ Given an initial situation (”we have three nodes”),

test all possible executions (”A gets first message first”, ”B does”, ”C does”, . . . )

◮ Combinatorial search in the tree of possibilities ◮ Fight combinatorial explosion: cycle detection, symmetry, abstraction

Model-checking in GRAS

◮ First difficulty: Checkpoint simulated processes (to rewind simulation)

We know how to marshal C datatypes

◮ Second difficulty: Fight against combinatorial explosion

Simulation can easily be distributed (is it enough?)

Martin Quinson Research and Development of Grid and P2P Infrastructures Model-checking 17/17