Tomasz Buchert Lucas Nussbaum INRIA, LORIA, Universit e de - - PowerPoint PPT Presentation

▶
tomasz buchert lucas nussbaum
SMART_READER_LITE
LIVE PREVIEW

Tomasz Buchert Lucas Nussbaum INRIA, LORIA, Universit e de - - PowerPoint PPT Presentation

Mar 12, 2023 855 likes •1.12k views

Leveraging business workflows in distributed systems research for the orchestration of reproducible and scalable experiments Tomasz Buchert Lucas Nussbaum INRIA, LORIA, Universit e de Lorraine, CNRS T. Buchert, L. Nussbaum Business

slide-1
SLIDE 1

Leveraging business workflows in distributed systems research for the orchestration of reproducible and scalable experiments

Tomasz Buchert Lucas Nussbaum

INRIA, LORIA, Universit´ e de Lorraine, CNRS

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 1 / 21

slide-2
SLIDE 2

What is a distributed system?

A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable. Leslie Lamport

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 2 / 21

slide-3
SLIDE 3

What is a distributed system? (2)

Examples of well-known distributed systems: DNS, BitTorrent, Gmail.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 3 / 21

slide-4
SLIDE 4

What is a distributed system? (3)

Distributed systems are: complex, erroneous, difficult to control, nondeterministic. Moreover, they: span many networks, span many geographical locations, consist of thousands of machines.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 4 / 21

slide-5
SLIDE 5

Research in distributed systems

Can we make better BitTorrent, Gmail, etc.? Solution: experimentation. The “classical way” is difficult, but other solutions exist: simulation, benchmarking, emulation.

Real program Modeled program Real platform Standard approach Benchmarking Modeled platform Emulation Simulation

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 5 / 21

slide-6
SLIDE 6

Existing tools

Many solutions exist (hardware and software based): Grid’5000, Emulab, PlanetLab, OMF, Expo, Plush, g5k-campaign, ... among the others. How can one evaluate or grade them?

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 6 / 21

slide-7
SLIDE 7

Our final goal

To improve the research, the experimentation framework has to: improve descriptiveness of the experiments, handle unexpected, but inevitable errors, ensure scalability of experiments, ensure reproducibility of the results. In the end, we want to

improve experimentation on testbeds like Grid’5000 and PlanetLab.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 7 / 21

slide-8
SLIDE 8

Agenda

Introduction Issues with distributed systems research Approach – a framework to control experiments Our contributions:

1

Identify requirements for an experimental framework

2

A new approach based on Business Process Management

Conclusions

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 8 / 21

slide-9
SLIDE 9

Contribution 1

Let’s define properties an experimentation framework must have. That way we can evaluate existing and future solutions. We distinguished 12 features in 3 categories.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 9 / 21

slide-10
SLIDE 10

Design

To improve the process of designing the experiment we need: descriptiveness, modularity, reusability, maintainability, support for common patterns.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 10 / 21

slide-11
SLIDE 11

Execution

Required features of experiment execution are: snapshotting, error handling, integration with lower-level tools, human interaction.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 11 / 21

slide-12
SLIDE 12

Monitoring

Finally, to monitor the experiment, we need: monitoring, instrumentation, data analysis.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 12 / 21

slide-13
SLIDE 13

By the way...

The partitioning of the properties is not arbitrary. The three categories: design, execution, monitoring, map to phases in Business Process Management Lifecycle.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 13 / 21

slide-14
SLIDE 14

Contribution 2

None of existing solutions covers all our requirements. Can we provide them all at once? Our approach is based on Business Process Management (BPM).

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 14 / 21

slide-15
SLIDE 15

Business Process Management

Traditionally, Business Process Management is about: understanding how an organization works, modeling its activities in a language of workflows, executing processes and monitoring their progress, identifying ways to improve activities in an organization, redesigning processes to make them:

cheaper, faster, less defective.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 15 / 21

slide-16
SLIDE 16

Toyota Production System

Toyota Corporation is (2010): the largest producer of cars world-wide, the most profitable car-making company, well known for its socio-technical system of practices. The fundamental objective is to eliminate waste or muda (無駄):

  • verproduction, waiting, transportation, processing, stock at hand,

movement and defective products.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 16 / 21

slide-17
SLIDE 17

BPM and experimentation

There is an analogy between production and experimentation: a car Ð Ñ a finished experiment, an assembly line Ð Ñ execution of an experiment, parts of the assebly line Ð Ñ parts of an experiment, eliminated waste (muda) Ð Ñ efficiency and reliability.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 17 / 21

slide-18
SLIDE 18

Example of a workflow

Create configuration files III ‘ Update system and add gLite repositories III ‘ Create VO & CA and set up VOMS Configure sites III a Evaluate TORQUE

+

Configure worker nodes III ‘ Prepare configuration

+

Start services

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 18 / 21

slide-19
SLIDE 19

How BPM fulfills the requirements?

Create configuration files III ‘ Update system and add gLite repositories III ‘ Create VO & CA and set up VOMS Configure sites III a Evaluate TORQUE

+

Configure worker nodes III ‘ Prepare configuration

+

Start services

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 19 / 21

slide-20
SLIDE 20

How BPM fulfills the requirements?

Create configuration files III ‘ Update system and add gLite repositories III ‘ Create VO & CA and set up VOMS Configure sites III a Evaluate TORQUE

+

Configure worker nodes III ‘ Prepare configuration

+

Start services

Design

Descriptiveness Modularity Reusability Maintainability Support for common patterns

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 19 / 21

slide-21
SLIDE 21

How BPM fulfills the requirements?

Create configuration files III ‘ Update system and add gLite repositories III ‘ Create VO & CA and set up VOMS Configure sites III a Evaluate TORQUE

+

Configure worker nodes III ‘ Prepare configuration

+

Start services

Design

Descriptiveness Modularity Reusability Maintainability Support for common patterns

Execution

Snapshotting Error handling Integration with lower-level tools Human interaction

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 19 / 21

slide-22
SLIDE 22

How BPM fulfills the requirements?

Create configuration files III ‘ Update system and add gLite repositories III ‘ Create VO & CA and set up VOMS Configure sites III a Evaluate TORQUE

+

Configure worker nodes III ‘ Prepare configuration

+

Start services

Design

Descriptiveness Modularity Reusability Maintainability Support for common patterns

Execution

Snapshotting Error handling Integration with lower-level tools Human interaction

Monitoring

Monitoring Instrumentation Data analysis

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 19 / 21

slide-23
SLIDE 23

Summary

To sum up, we: introduced problems of research in distributed systems, described our contributions:

1

requirements for an experimentation engine,

2

a new approach to experimentation based on BPM,

showed how our approach meets the defined requirements.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 20 / 21

slide-24
SLIDE 24

Future work

In the near future, we plan to: implement a workflow-based experiment engine, evaluate it using a set of well-chosen experiments, find more analogies between BPM and experimental science. http://www.loria.fr/ ~buchert/

Thank you for your attention.

  • T. Buchert, L. Nussbaum

Business workflows in distributed systems research 21 / 21