SLIDE 1 Aake
Edlund,
PhD
KTH
and
SNIC,
Sweden
Northern
Europe
Cloud
(Project
Manager)NEW
Bal4cGrid
(Project
Director)
Bal4cGrid
Innova4on
Lab
and
Bal4cCloud
(Ini<ator,
Manager)
Bal4c
Cloud
1
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 2 2
– Started
1st
of
May,
2008,
dura<on
‐
2
years
– Con<nues
the
Bal<cGrid
project
(2005‐2008)
– 13
partners
from
Bal<c
States,
Belarus,
Poland,
Sweden,
Switzerland
– Financing
–
3
MEUR
– Collabora<ng
project
for
EGEE
– Suppor<ng
NGIs
and
their
collabora<on
Normalized
compu<ng
hours
for
the
Bal<cGrid
project
and
that
of
the
first
six
months
of
Bal<cGrid‐II
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Bal4cGrid
‐
the
Founda4on
SLIDE 3 Mo4va4on
for
Bal4cCloud
When
we
started,
cloud
was
considered
immature,
and
only
for
startups
and
possibly
prototyping.
This
suited
us
well,
knowing
that
we
had
not
been
very
successful
in
a\rac<ng
bigger
companies.
Startup
and
prototyping
focus
3
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 4
- Ini<ated
in
January
20,
2009
- A
program
for
early
stage
companies
in
our
region,
- by
invi<ng
them
to
a
hands‐on
course
in
grid
and
cloud
compu<ng.
– A
course
prepared
and
run
by
us,
on
our
resources
as
well
as
on
donated
dito
from
e.g.
Amazon.
- The
aim
with
the
course
is
to
help
early
stage
high‐tech
Internet
based
companies
to
try
their
services
on
new
pla_orms,
resul<ng
in
early
proof
- f
concepts
and
later
exploita<on
of
grid
and
cloud
in
the
region.
- On
top
of
the
course
we'll
build
a
network
of
innova<ve
companies
in
the
region.”
4
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
The
Vision
A
new
way
to
aGract
startups
bgin.wordpress.com
SLIDE 5 Mo4va4on
for
Bal4cCloud
‐ Low
number
of
applica<ons
and
applica<on
areas
in
BG.
‐ Need
of
elas<cy
‐
volume
on
demand
‐
a
way
to
handle
capacity
on
the
fly,
to
balance
the
usage
‐ Too
steep
learning
curve
for
grid
users
‐ Low
acceptance
within
industry
–
so
far
‐ Complicated
to
adapt
new
applica<ons,
typically,
need
a
“project”
to
do
that
Complement
to
current
grid
services
5
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 6 Mo4va4on
for
Bal4cCloud
One
more,
hopefully,
easier
access
point
to
our
Bal<cGrid
resources.
So,
one
more
access
point
6
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 7 Bal4c
Cloud
How
it
all
fits
together
Bal<cGrid
Planned
ac<vi<es
SA1‐3,
NA1‐4,
JRA
BGi
‐
Bal<cGrid
Innova<on
Lab
Bal<cCloud
Courses
SME
connec<vity
BC
7
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 8
- Bal<cGrid
asked
one
center
per
country
to
install
a
cloud
instance
- SNIC
(Swedish
Na<onal
Infrastructure
for
Compu<ng)
supported
the
coordina<on
of
the
work
- Learning
by
doing
‐
what's
in
it
for
eScience,
really?
Is
industry
interested?
- Started
informa<on
gathering
and
sharing
through
our
site;
went
to
cloud
expos
and
conferences
- Took
early
contact
with
open
source
cloud
vendors,
decided
to
focus
on
Eucalyptus
8
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Prac4cali4es &
Learning
by
doing
SLIDE 9 First
users
First
usage:
- 1. Virtual
world
installa<ons
(Wonderland),
for
the
Immersive
Educa<on
organiza<on
- 2. Film
rendering,
for
Estonian
film
company
Virtual
worlds
and
movies
July
17,
2009
9
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 10 Open
source
“cloud
in
a
box”
Our (main) choice:
- Lead by Rich Wolski, UCSB.
- Integrated with
- Integrated with
10
BC
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 11
- Will
the
open
source
alterna<ves
be
supported
in
a
sa<sfactory
way?
– “Not
invented
here”
easily
becomes
“Not
supported
here”
- Early
success
open
source
alterna<ves
branching
off
a
private
version
–
think
Xen,
Eucalyptus,…
–
which
branch
will
get
the
best
support?
- Missing
standards
for
interoperability,
c
f
grids.
- Lock‐in
and
de
facto
lock‐in,
the
bidding‐on‐a‐dollar
syndrome
Major
Concerns
11
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 12 Status
–
Bal4c
Cloud
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
12
See
hGp://cloud.bal4cgrid.eu
SLIDE 13 Status
–
Estonia
‐
NICPB
(1/3)
KVM‐based
setup
with
Eucalyptus
interface
Evalua<ng:
- other
virtualisa<on
technologies,
e.g.
vmware
solu<on
- crea<on
of
the
virtual
private
networks
using
Eucalyptus,
OpenVPN
and
dynamic
DNS
‐
to
be
able
to
provide
something
like
the
recently
rolled
out
Amazon
Virtual
Private
Cloud
(h\p://aws.amazon.com/vpc/)
Usage:
- primarily
NICPB
- ImmersiveEduca<on
project
with
the
developement
and
demo
resources
for
running
the
Project
Wonderland‐based
virtual
worlds
13
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 14 Status
–
Estonia
‐
NICPB
(2/3)
The
KVM
infrastructure
is
also
used
as
a
pla_orm
for
the
ongoing
SME
collabora4on
‐
on‐demand
render
farm
crea4on.
The
plans
are
to
make
it
controllable
by
Eucalyptus,
but
so
far
it
is
not
capable
of
suppor<ng
Windows
machines
out
of
the
box
(ongoing).
Inves4ga4ng
the
crea4on
of
the
private
virtual
networks
(using
OpenVPN
and
dynamic
DNS
solu<ons)
to
be
able
to
provide
something
like
the
recently
rolled
out
Amazon
Virtual
Private
Cloud
(h\p://aws.amazon.com/vpc/).
14
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 15 Status
–
Estonia
‐
NICPB
(3/3)
Storage:
– testbed
installa<on
of
the
hadoopFS
in
NICPB
with
SRM
interfaces.
Trying
to
migrate
the
storage
infrastructure
from
dCache
to
HDFS.
– Wins:
easier
administra<on,
cloud
flavor
‐
can
run
hadoop
jobs
much
more
efficiently.
Problems:
– Eucalyptus
sw
is
s<ll
too
alpha/pre‐beta.
The
1.6
_should_
be
be\er,
incl.
support
for
mul<ple
data
centers
(availability
zones)
and
more
robust
opera<ons,
but
it's
not
out
yet.
– Accoun<ng
of
consumed
resources
is
complicated
–
the
virtualisa<on
tools
are
not
making
an
accent
on
that.
15
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 16 Status
‐
Latvia
‐
RTU
Installa4on:
Eucalyptus
installa<on,
xen,
4
cpu
for
users.
HW
updates
expected
(user
request).
Usage:
Submi\ed
simple
tests,
including
students
tes<ng
cloud
Issues:
System
not
quite
stable.
“imho
so
far
not
usable
for
serious
project”.
16
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 17 Status
‐
Belarus
–
UIIP
NASB
(1/3)
Installa4on:
– HW:
6
CPU,
24
cores
(enough
for
present
tasks);
– Virtualisa<on:
Xen;
– Cloud
technolgies:
Eucalyptus,
Walrus;
– How
to
reach:
h\ps://80.94.168.123:8443/
– Status:
3
GNU/Linux
images
available,
commonly
running
from
2
to
10
instances.
Usage:
‐ verified
by
regular
manual
monitoring
of
hosts
and
services
running
within
the
cloud;
‐ used
by
UIIP
NASB
grid
team
for
designing
dynamic
grid
resource
reserva<on
and
adjustment
mechanisms
for
UNICORE
sites
17
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 18 Status
‐
Belarus
–
UIIP
NASB
(2/3)
Usage:
pilot
project
of
many
users
‐ we
have
prepared
a
pilot
project
proposal
on
deploying
one
of
the
advanced
informa<on
systems
of
the
State
Commi\ee
for
Science
and
Technologies
of
the
Republic
of
Belarus
(SCST)
into
the
cloud
infrastructure
provided
by
the
datacenter
of
the
UIIP
NASB.
‐ The
informa<on
system
to
be
cloudified
is
created
and
administered
on
behalf
of
the
SCST
by
Republican
Centre
for
Technology
Transfer
(h\p:// ic\.by).
The
system
has
thousands
of
visitors
and
hundreds
of
ac<ve
users
who
regularly
post
and
update
their
Technology
Offers
and
Requests,
disseminate
news,
upload
technology
transfer
related
applica<ons,
graphical
and
video
data.
‐ The
primary
goal
of
the
pilot
project
is
proving
the
concept
of
cloud
compu<ng
being
a
good
way
to
go
not
only
for
SMEs
but
for
some
governmental
agencies
as
well.
18
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 19 Status
‐
Belarus
–
UIIP
NASB
(3/3)
Plans:
UIIP
NASB
have
plans
for
cloud
usage
growth,
expansion,
connec<on
to
the
- utside
world
and
to
other
countries.
[e.g.
right
now
we
are
buying
hardware
for
the
expansion
of
our
cloud
infrastructure
]
Issues:
To
increase
cloud
infrastructure
stability
we
use
custom
kernel
builds
for
Xen
virtual
machine
images
deployed
in
the
cloud.
19
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 20 Status
‐
Lithuania
–
VGTU
(1/7)
Eucalyptus
Cloud
Scheme
20
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 21 Cluster
installa<on
is
done
using
Rocks
Clusters
5.2
(Chimichanga)
VGTU
cloud
instance
is
available
at
h\ps://euca.cloud.vgtu.lt:8443
Eucalyptus
v1.5.2
is
installed
XEN
hypervisor
is
used
for
paravirtualiza<on
MANAGED
network
mode
is
configured
i.e.
machine
instances
can
be
accessed
directly
from
internet
Available
two
virtual
machine
images
compiled
manually:
CentOS
5.3
(Base
setup)
SLC
53
(Scien<fic
Linux
Cern)
(Base
setup)
Status
‐
Lithuania
–
VGTU
(2/7)
Cloud
cluster
setup
21
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 22 Frontend
(Cluster
controler,
Cloud
controler):
1
x
Pen<um
4
CPU,
2
GB
RAM,
200
GB
HDD,
1
Gbps
Nodes
(Node
controler)
4
x
Pen<um
4
CPU,
2
GB
RAM,
200
GB
HDD,
1
Gbps
2
x
Quad
Qore
CPU,
4
GB
RAM,
320
GB
HDD,
1
Gbps
Availability
zone
DEBESELIS
AVAILABILITYZONE
DEBESELIS
UP
euca.cloud.vgtu.lt
AVAILABILITYZONE
|‐
vm
types
free
/
max
cpu
ram
disk
AVAILABILITYZONE
|‐
m1.small
0016
/
0016
1
128
3
AVAILABILITYZONE
|‐
c1.medium
0016
/
0016
1
256
5
AVAILABILITYZONE
|‐
m1.large
0016
/
0016
1
512
5
AVAILABILITYZONE
|‐
m1.xlarge
0016
/
0016
1
768
5
AVAILABILITYZONE
|‐
c1.xlarge
0008
/
0008
2
1024
5
Status
‐
Lithuania
–
VGTU
(3/7)
Available
Resources
Total
12
Cores
22
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 23 Status
‐
Lithuania
–
VGTU
(4/7)
Technical
ISSUES
(problems
and
limita<ons)
In
recent
eucalyptus
version
1.5.2
famous
error:
“Error
500
message
expired”
that
prevented
graphical
tools
like
Elas<cfox,
Rightscale,
Ylas<c,
EC2Dream
from
working
with
eucalyptus
instance,
was
not
fixed
We
have
fixed
this
issue
manually:
Added
this
code
to:
HMACQuerySecurityHandler.java
as
a
temporary
fix
‐:
tsFormat.setTimeZone(TimeZone.getTimeZone("EEST"));
//
Add
this
Line
and
import
TimeZone
class
ts.setTime(
tsFormat.parse(
<mestamp
)
);
And
then
Eucalyptus
rpms
were
rebuilt.
23
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 24 Status
‐
Lithuania
–
VGTU
(5/7)
We
have
installed
and
use
Elas<cFox
for
Eucalyptus
management.
24
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 25 We
have
tried
EC2Dream
for
Eucalyptus
management
as
well.
25
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Status
‐
Lithuania
–
VGTU
(6/7)
SLIDE 26 “Cloud computing” activity is included into LitGrid programme 2009. Walrus based storage will be installed.
Status
‐
Sweden
–
PDC
Installa<on
(ongoing):
4
quad
core
machines,
Eucalyptus,
XEN
Usage:
for
tes<ng
and
learning,
to
prepare
larger
installa<on
based
on
older
HPC
resources
26
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Status
‐
Lithuania
–
VGTU
(7/7)
Nearest
future
SLIDE 27 Status
‐
Lithuania
‐
VU
Installa4on:
Small
eucaluptus
intalla<on
(might
be
increased
if
demand
arrises),
xen
based,
hardware
is
only
32
bit.
Doing
some
tests
with
vmware
and
MS
hyper‐v
Usage:
Small
tests,
nothing
in
produc<on
Plans:
Thinking
of
expanding
(have
to
think
more
about
which
technology),
maybe
some
services
to
some
other
ins<tu<ons
Issues:
Current
system
is
not
very
stable,
some
accidental
crashes
and
system
hangings
were
no<ced.
27
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 28 Status
‐
Poland
–
PSNC
Installa4on:
– two
dual
CPU
machines
(one
for
cloud
controller
one
for
cloud
nodes),
2x
dual
core
Intel
Xeon
,
8GM
ram/each
– 147
GB
SAS
hdd
– two
1Gbit
Eth
network
card/each
virtualiza<on
:
kvm
or
xen
(
on
demand
)
– cloud
stack
:
eucalyptus
1.5.2
– url
:
h\ps://bgcloud.man.poznan.pl:8443
,
currently
available
only
from
PSNC
network
Usage:
test
stage
Plans
for
usage
and
expansion
:
– as
part
of
Bal<c
Cloud
– for
local
users
(PSNC
developers
and
research
team),
educa<on
ins<tu<on
in
Poznan
– expansion
according
to
load
and
u<liza<on
we
can
provide
addi<onal
cloudnode
controllers
28
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 29
- VGTU:
Bachelor
thesis
on
cloud
compu<ng
(2009):
“Deployment
of
Service
Oriented
IT
Infrastructure”
- VGTU:
Cloud
compu<ng
is
part
of
the
MSc
course
“High
performance
systems”
at
VGTU
- BG:
general
presenta<ons,
e.g.
at
ISSGC09
- NICPB:
A
course
in
IT
college
(in
Tallinn)
about
the
distributed
systems.
Cloud
compu<ng
is
part
of
the
course,
labs
will
be
run
on
the
BC
resources
- NICPB:
Tartu
cloud
and
startup
school
–
May,
2009
29
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Educa4on
SLIDE 30
- Connec<ng
the
Bal<c
Cloud
with
the
Northern
Europe
Cloud‐
Sweden,
Norway,
Denmark,
Estonia,
Latvia,
Lithuania,
Belarus,
Poland
+
Iceland(?)
+
…
- Availability
zones
–
connec<ng
the
clouds
- Connec<ng
with
other
clouds,
e.g.
in
the
Netherlands
- See
more
cloud
compu<ng
being
used
by
startups
and
in‐house
quick
prototyping
- See
more
cloud
compu<ng
being
used
in
academia
–
esp.
HPC
flavours
- See
more
open
source
alterna<ves,
and
alterna<ve
cloud
providers
–
also
in
academia
- Improved
solu<ons
addressing
the
dangers
in
cloud
compu<ng.
- Low
expecta<ons
on
standard
APIs
July
17,
2009
30
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Future
Expecta4ons
on
2009
SLIDE 31
- Bal<cCloud
have
learned
quite
much
about
the
current
open
source
cloud
technologies
–
what
there
is
to
use,
what
is
missing,
the
quality
level
- The
user
side
is
evolving,
now
that
we
have
some
basic
courses
and
an
clouds
to
play
with
- Next
level,
beyond
this
pilot
project,
is
to
be
part
of
the
Northern
Europe
Cloud
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
31
Cloud
compu4ng,
Virtualisa4on
&
the
Future
Summary
SLIDE 32
?
32
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 33 Thank
you!
Aake
Edlund
ake.edlund@gmail.com
www.pdc.kth.se/Members/edlund
www.bal<cgrid.eu
(Bal<c
Grid)
cloud.bal<cgrid.eu
(Bal<c
Cloud)
bgin.wordpress.com
(BG
Innova<on
Lab)
33
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 34 Basics
Cloud
Interface
Cloud
Controller
Cluster
Controller
Node
Controller
34
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
SLIDE 35 Cloud
controller
– Java
- Handles
distribu<on
of
VM
instance
control
events
to
clusters
of
resources
Cloud
controller
SLA
Engine
Instance
Scheduler
Other
Schedulers?
35
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Cloud
Interface
Cloud
Controller
Cluster
Controller
Node
Controller
SLIDE 36 Cluster
controller
– C
based
- Handles
distribu<on
of
VM
instance
control
events
to
individual
of
resources
- Inves<ga<on:
what
are
the
performance,
invasiveness
and
func<onality
tradeoffs
- f
various
cloud
networking
strategies?
Cluster
controller
Virtual
Network
Instance
Scheduler
Other
Network/Scheduler
solu<ons?
36
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Cloud
Interface
Cloud
Controller
Cluster
Controller
Node
Controller
SLIDE 37 Cloud
interface
– Wri\en
in
Java
- Handles
user
event
to
cloud
VM
instance
control
transla<on
- Experiment:
instance
crea<on
<me
Cloud
interface
EC2
compa<ble
user
interface
Web‐based
admin
interface
Other
interfaces?
37
4th
Terena
TF
Storage
Mee<ng,
Copenhagen,
2009.Sept.15,
Bal<cCloud
Cloud
Interface
Cloud
Controller
Cluster
Controller
Node
Controller
