Common Execution Infrastructure (CEI) Subsystem OOI CI System - - PowerPoint PPT Presentation
Ocean Observatories Initiative Common Execution Infrastructure (CEI) Subsystem OOI CI System Architecture Team: R3 Kickoff Meeting 1 CEI Developers CEI Developer Patrick Armstrong CEI Senior Developer University of Chicago Pierre Riteau
R3 Kickoff Meeting
1
R3 Kickoff Meeting
2
10/18/12 2 CEI Developer John Bresnahan Argonne National Lab (part-time) CEI Developer Patrick Armstrong University of Chicago CEI Developer Pierre Riteau University of Chicago (part-time) CEI Senior Developer Pierre Riteau University of Chicago
R3 Kickoff Meeting
3
R3 Kickoff Meeting
4
R3 Kickoff Meeting
– Implement elastic computing services to provide on-demand scaling and high availability.
– Working with operations and ITV to develop and refine tools to upload and sync the different deployable type representations adapted to each site.
– Provide the management services for policy-based process execution within specified deployable types intended to support the data distribution services; as such the processes are sequential and require primarily a process to resource match.
– The Process Catalog and Repository Services maintain process definitions as well as lists active processes.
– Provide the capability to deploy OOI processing on the Amazon cloud services as well as academic clouds
5
R3 Kickoff Meeting
10/1 8/12 6
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
R3 Kickoff Meeting
10/1 8/12 7
EPU Management Monitor and regulate set properties based on system-specific and application-specific metrics
R3 Kickoff Meeting
8
R3 Kickoff Meeting
9
AMQP Other
R3 Kickoff Meeting
EPU Management
9
Decision Engine
AMQP Other
R3 Kickoff Meeting
EPU Management
9
Decision Engine Provisioner create instance
AMQP Other
R3 Kickoff Meeting
EPU Management
9
Decision Engine Provisioner create instance
AMQP Other
DTRS
R3 Kickoff Meeting
EPU Management
9
Decision Engine Provisioner IaaS create instance
AMQP Other
DTRS
R3 Kickoff Meeting
EE ioncore 1.3 EPU Management
9
EE ioncore 1.2
context-agent
EE matlab 6.1
context-agent
Decision Engine
context-agent
Provisioner IaaS create instance
AMQP Other
DTRS
R3 Kickoff Meeting
EE ioncore 1.3 EPU Management
9
EE ioncore 1.2
context-agent
EE matlab 6.1
context-agent
Decision Engine
context-agent
Provisioner IaaS create instance
AMQP Other
DTRS CB
R3 Kickoff Meeting
EE ioncore 1.3 EPU Management
9
EE ioncore 1.2
context-agent
EE matlab 6.1
context-agent
Decision Engine
context-agent
Provisioner IaaS create instance
AMQP Other
DTRS CB
R3 Kickoff Meeting
EE ioncore 1.3 EPU Management EPU Management EPU Management
9
EE ioncore 1.2
context-agent
EE matlab 6.1
context-agent
Decision Engine
context-agent
Provisioner IaaS create instance
AMQP Other
DTRS CB
R3 Kickoff Meeting
EE ioncore 1.3 EPU Management EPU Management EPU Management
9
EE ioncore 1.2
context-agent
EE matlab 6.1
context-agent
Decision Engine
context-agent
Provisioner IaaS create instance
AMQP Other
DTRS CB
R3 Kickoff Meeting
AMQP Other
R3 Kickoff Meeting
Bootstrap EPU Dedicated DE
AMQP Other
R3 Kickoff Meeting
Bootstrap EPU Dedicated DE Provisioner/DTRS create instance
AMQP Other
R3 Kickoff Meeting
EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker Bootstrap EPU Dedicated DE Provisioner/DTRS IaaS create instance
AMQP Other
R3 Kickoff Meeting
EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker Bootstrap EPU Dedicated DE Provisioner/DTRS IaaS create instance
AMQP Other
R3 Kickoff Meeting
EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker EPU Worker Bootstrap EPU Dedicated DE Provisioner/DTRS IaaS create instance
AMQP Other
R3 Kickoff Meeting
R3 Kickoff Meeting
12
Process Dispatcher EE type A instance ee-agent Decision Engine
AMQP Other
R3 Kickoff Meeting
12
Process Dispatcher EE type A instance request to activate process X ee-agent Decision Engine
AMQP Other
R3 Kickoff Meeting
12
Process Definition Registry Process Dispatcher EE type A instance request to activate process X ee-agent Decision Engine lookup
AMQP Other
R3 Kickoff Meeting
12
Process Definition Registry Process Dispatcher EE type A instance request to activate process X ee-agent Decision Engine lookup launch
AMQP Other
R3 Kickoff Meeting
12
Process Definition Registry Process Dispatcher EE type A instance Process Instance Registry request to activate process X ee-agent Decision Engine lookup launch enter
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Process Instance Registry Decision Engine
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Process Instance Registry request to activate process X Decision Engine
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Process Instance Registry request to activate process X Decision Engine lookup
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher EPU Management Process Instance Registry request to activate process X Decision Engine lookup request instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS EPU Management Process Instance Registry request to activate process X Decision Engine lookup request instance create instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EPU Management Process Instance Registry request to activate process X Decision Engine lookup request instance create instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EE type A instance EPU Management Process Instance Registry request to activate process X ee-agent Decision Engine lookup request instance create instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EE type A instance EPU Management Process Instance Registry request to activate process X ee-agent Decision Engine lookup request instance create instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EE type A instance EPU Management Process Instance Registry request to activate process X ee-agent Decision Engine lookup launch request instance create instance
AMQP Other
R3 Kickoff Meeting
13
Process Definition Registry Process Dispatcher Provisioner/DTRS IaaS EE type A instance EPU Management Process Instance Registry request to activate process X ee-agent Decision Engine lookup launch enter request instance create instance
AMQP Other
R3 Kickoff Meeting
14
EE type A instance context-agent
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance
14
EE type A instance context-agent
supervisord C CMR
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance
14
EE type A instance context-agent
supervisord C CMR EPU Management
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C CMR CMR EPU Management
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C CMR CMR Process Dispatcher EPU Management
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR Process Dispatcher EPU Management
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR Process Dispatcher EPU Management
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR Process Dispatcher EPU Management Package Server
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR CMK Process Dispatcher EPU Management Package Server process (adapter) 1
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR CMK datastream subscription Process Dispatcher EPU Management Package Server process (adapter) 1
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord C C M CMR CMR CMK datastream subscription result Process Dispatcher EPU Management Package Server process (adapter) 1
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
CC instance CC instance
14
EE type A instance context-agent ee-agent
supervisord supervisord supervisord Matlab script C C M CMR CMR CMK CMKO CMKO datastream subscription result Process Dispatcher EPU Management Package Server process (adapter) 1
AMQP Other
C – create M – monitor R – restart K – kill O – I/O
R3 Kickoff Meeting
10/1 8/12 15
R3 Kickoff Meeting
10/1 8/12 15
Mean time between failures
R3 Kickoff Meeting
10/1 8/12 15
Mean time between failures Mean time to repair
R3 Kickoff Meeting
10/1 8/12 15
Mean time between failures Mean time to repair
R3 Kickoff Meeting
10/1 8/12 15
Mean time between failures Mean time to repair TTS: preliminary results for 2,000 VMs provisioned on AWS EC2
R3 Kickoff Meeting
16
R3 Kickoff Meeting
17
R3 Kickoff Meeting
18