NEON TF-Storage, March 5Th 2010, Utrecht Maarten Koopmans for - - PowerPoint PPT Presentation

▶

Nov 23, 2023 150 likes •386 views

NEON TF-Storage, March 5Th 2010, Utrecht Maarten Koopmans for UNINETT Sigma maarten@vrijheid.net Goals state-of-the art of cloud computing; cost of moving and running non-HPC jobs on a cloud computing environment;

SLIDE 1

NEON

TF-Storage, March 5Th 2010, Utrecht Maarten Koopmans for UNINETT Sigma maarten@vrijheid.net

SLIDE 2

Goals

state-‑of-‑the ¡art ¡of cloud computing;
cost of moving and running non-HPC jobs on a

cloud computing environment;

how to do this in prac-ce;
a list of iden-fied ¡risks/benefits ¡on a short/long

perspective.

SLIDE 3

Areas

SLIDE 4

...and management

SLIDE 5

Shortlists

SLIDE 6

Why again?

SLIDE 7

Let’s zoom in on storage

SLIDE 8

Storage: requirements

No client to access the data.
Transparent versioning.
Transparent encryption - both transport

and storage

AAI integration
Allow sharing of resources

SLIDE 9

Current Cloud storage

API based - complex for end users

SLIDE 10

No AAI integration at all

SLIDE 11

So...

SLIDE 12

AAI: enrollment

AAI Cloud backed storage initial request access granted token returned (rotating?) authenticate user user authenticated

SLIDE 13

“Just WebDAV”

Webdav daemon Encryption Resource naming Versioning Storage cloud

Continue Map resource to hash code version = 1 Stream data metered through encryption ...........into the cloud

Metering

SLIDE 14

...Locking...

Webdav daemon Locking Resource naming Versioning

Return lock Map resource to hash code Get reference to most recent version Acquire lock

SLIDE 15

...downloading...

Webdav daemon Decryption Resource naming Versioning Storage cloud

Continue Get most recent version = 1 Stream data metered via decryption ...........from the cloud

Metering

SLIDE 16

Stand on the shoulders

f giants
WebDAV widely

deployed, lots of 3rd party clients.

Service on top of Java

Scala (integration

language)

Cloud access libraries

(often Java based)

Apache ZooKeeper

(configuration management, locking)

Apache Cassandra or

HBase (metering)

AAI integration

components

SLIDE 17

Lessons learnt so far

WebDAV is a nice start for client-less access to file based resources.
OS X requires DAV level 2 and does a lot of locking. But: from 10.5.x onwards it also does HTTP 1.1

chunked encoding; that broke a lot of servers.

Windows works best with digest authentication.
Linux seems to be most forgiving and least demanding.
all clients support SSL.
WebDAV's XML is relatively simple but the usage may differ per client type.
CPU intensive due to the encryption per "stream". A language (model) with concurrency support is a big
plus. This breaks the trend of asynchronous I/O based network services.
Stand on the shoulders of giants: Apache Zookeeper, Bookkeeper, the JVM, Scala language, libraries for

cloud access

SLIDE 18

Questions so far?

SLIDE 19

Cloud management

SLIDE 20

Computing

Torque/PBS on AWS via RightScale
OSGi on Eucalyptus
Challenge: Matlab, BLAST etc - how to deal with licensing?

SLIDE 21

Where we’ll be

TF Storage (March 5Th)
OGF session (leading, March 16Th)
NOTUR workshop (leading, May 19Th) -

first NEON WORKSHOP

TNC 2010 (May 31St)
Two more workshops in H2-2010
More presentations.... who knows?

SLIDE 22