EMF Models getting XXL? An overview of available solutions Jonathan - - PowerPoint PPT Presentation

▶
emf models getting xxl an overview of available solutions
SMART_READER_LITE
LIVE PREVIEW

EMF Models getting XXL? An overview of available solutions Jonathan - - PowerPoint PPT Presentation

Jan 10, 2024 913 likes •1.31k views

EMF Models getting XXL? An overview of available solutions Jonathan Lasalle Benoit Viaud ARTAL Technologies 227 rue Pierre Gilles de Gennes Ensemble La Rue Bt. 9 31670 Labge, France 05 61 00 39 30 artal@artal.fr The

slide-1
SLIDE 1

ARTAL Technologies – 227 rue Pierre Gilles de Gennes – Ensemble La Rue – Bât. 9 – 31670 Labège, France – 05 61 00 39 30 – artal@artal.fr

EMF Models getting XXL? An overview of available solutions

Jonathan Lasalle Benoit Viaud

slide-2
SLIDE 2

2

N E B U L A S I R I U S A C C E L E O E D A P T O C L

A framework for designing simulations

  • f large & complex systems

The context

OCL

« Excel-like » tables « Box-and-arrows » diagrams Metamodel evolution Query / export in various format Validation of data

Functional requirements

slide-3
SLIDE 3

3

The context

slide-4
SLIDE 4

4

A first naive solution EMF resources serialized into plain XMI files

  • Lazy-loading is possible (at the grain of a resource)
  • Unloading is a bit more complex…

In the end: all the model in memory

The -XMX race for memory … … until you reach the

PERFORMANCES

Swap !

slide-5
SLIDE 5

5

CDO Design

Database (H2) RCP Application CDO Server Persistency Layer

Features :

  • Easy integration
  • Client / Server
  • Transaction based
  • Branching
  • Auditing
  • Lazy loading
  • Unloading
  • Query mode (SQL)

RCP Application Persistency Layer

slide-6
SLIDE 6

6

Limitations:

  • Insertion cost (Linear but costly)
  • Deletion cost (DB schema / XRef cleaning)
  • RAM consumption / release (Opaque behavior)

Explanation :

  • Client / Server Management
  • Constraints on database schema
  • Several management layers

CDO Limitations

slide-7
SLIDE 7

7

CDO: what else ? Ligthweight basic EMF:

  • Lazy-loading using resources
  • Using Minimal EObjects

Pro:

  • Easy to implement
  • Light impact on source code

Cons:

  • RAM blast: transversal model queries
slide-8
SLIDE 8

8

CDO: what else ? Improved CDO:

  • Merge management layers
  • Optimization of database schema

Pro:

  • No impacts on source code

Cons:

  • Only few experts can do it
  • Opacity / Complexity
slide-9
SLIDE 9

9

CDO: what else ? Teneo:

  • Use of Teneo instead of CDO

Pro:

  • Out-of-the-box solution

Cons:

  • Opacity
  • No longer maintained
  • No unloading
slide-10
SLIDE 10

10

CDO: what else ? Hibernate / Java Persistence API (JPA)

  • Hibernate for data storage
  • JPA for data management (POJO)
  • Conversion from POJO to EObject

Pro:

  • Hand-made solution (to be tailored)

Cons:

  • Hand-made solution
slide-11
SLIDE 11

11

CDO: what else ? Neo EMF:

  • Promising solution
  • Graph-based database

Pro:

  • Out-of-the-box solution

Cons:

  • No mature enough
  • No more place on the podium…
slide-12
SLIDE 12

12

CDO ++ Hibernate / JPA Import process (750 000 objects) (Insertion only) 1’00” 29” Update process (Coupling) 10’35” 37” Data Export 18” 16” Delete 1 item 25” 0,002” Delete 100 items 7’24” 0,100”

Benchmark results

11%

99%

slide-13
SLIDE 13

The Hibernate/JPA solution

Database (H2) JPA Processes EMF Processes JPA Layer EMF Layer (Texo) Converter Data Import RCP Application Data Deletion Nebula Editor Sirius Editor Data Export OCL Validation Sync Complex Algorithm Sync

slide-14
SLIDE 14

14

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

slide-15
SLIDE 15

15

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

slide-16
SLIDE 16

16

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

slide-17
SLIDE 17

17

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

  • Quite heavy database mecanism
slide-18
SLIDE 18

18

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

  • rg.eclipse.emf.cdo.util.DanglingReferenceException
  • Quite heavy database mecanism
slide-19
SLIDE 19

19

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Bus_Signal Bus_ID Signal_ID B1 S1 Signal ID Container … S1 B1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

How to explain gain: CDO deletion

  • Quite heavy database mecanism
  • Must clean non-containment references

(e.g. using CDO XReferencer) => Get and build all impacted Eobjects (even if they were not loaded) => Clean associated features (or delete items)

slide-20
SLIDE 20

20

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

Bus_Signal Bus_ID Signal_ID B1 S1 CDO_Objects ID Type B1 Bus S1 Signal SP1 SimuParam

slide-21
SLIDE 21

21

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

Foreign Key FK

slide-22
SLIDE 22

22

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

Foreign Key FK

slide-23
SLIDE 23

23

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

Foreign Key FK

slide-24
SLIDE 24

24

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

Foreign Key FK

slide-25
SLIDE 25

25

Bus ID … B1 SimuParam ID Simulate … SP1 S1 Signal ID Container … S1 B1

How to explain gain: Hib/JPA deletion

  • Optimized database mechanism

(using Hibernate native features)

  • Non-containment references automatically

cleaned (or deleted)

  • Must synchronize only loaded EObjects

Foreign Key FK

slide-26
SLIDE 26

From benchmark to real product

  • Success recipe:

– Take a JPA-compliant EMF Metamodel – Add a pinch of Texo – Spread it on an Hibernate pastry

slide-27
SLIDE 27

From benchmark to real product

  • Success recipe:

– Take a JPA-compliant EMF Metamodel – Add a pinch of Texo – Spread it on an Hibernate pastry – For a faster deletion: stuff it with a maximum of foreign keys – Stir it with a transparent implementation

  • f a lazy loading mechanism

– Filter the unused EObject from RAM

slide-28
SLIDE 28

From benchmark to real product

  • Success recipe:

– Take a JPA-compliant EMF Metamodel – Add a pinch of Texo – Spread it on an Hibernate pastry – For a faster deletion: stuff it with a maximum of foreign keys – Stir it with a transparent implementation

  • f a lazy loading mechanism

– Filter the unused EObject from RAM – Dust some query-DSL to lighten some algorithms

slide-29
SLIDE 29

29

Sirius Connectors EMF Layer JPA Layer Query API Converter

The Result: a technical framework

Deletion Helper Lazy Mode Database (H2) EObject Cache Sharing

slide-30
SLIDE 30

30

Sirius Connectors EMF Layer JPA Layer Query API Converter

The Result: a technical framework

Deletion Helper Lazy Mode Database (H2) EObject Cache Sharing

slide-31
SLIDE 31

31

Step 0: Identify your requirements regarding perf

  • Consider using CDO if

– You need out-of-the-box advanced features: concurrent access, branching & merging, etc – You don’t need extra performances

  • Consider using a Hibernate/JPA-flavor solution if

– Your context allows to release some constraints on features – Performances are never good enough…

A step by step solution to improve perf

  • n an existing EMF app
slide-32
SLIDE 32

32

Step 1: Introduce the technical framework Cost : low

  • Very low if resources management is already identified

Benefits

  • Lazy loading
  • Faster Deletion => reduce CPU usage

A step by step solution to improve perf

  • n an existing EMF app
slide-33
SLIDE 33

33

Step 2.1: Refactor the Meta-Model Cost : low to medium

  • If you know what you are doing/expecting

Benefits

  • Unloading => reduce memory usage
  • Extra-fast deletion

A step by step solution to improve perf

  • n an existing EMF app
slide-34
SLIDE 34

34

Step 2.2: Refactor algorithms & HMI Cost : from low to high

  • Requires dual expertise: Hibernate/JPA & EMF/TEXO

Benefits

  • Allow creation of data at lower level
  • Allow optimized queries alla SQL
  • with the support of higher level language Query DSL

A step by step solution to improve perf

  • n an existing EMF app
slide-35
SLIDE 35

To go further

What we can do for

  • Related Technology providers:
  • Publish (EPL) our benchmark challenge & framework

Propose your solution

  • S/W products developers / integrators:
  • Publish our solution as a ready-to-use package

Test your requirements

You are there

Contact us to go further

slide-36
SLIDE 36
  • 1 0 +1

Sign in and vote at eclipsecon.org

slide-37
SLIDE 37