Taming Deployment with SmartFrog Steve Loughran Julio Guijarro HP Laboratories, Bristol, UK steve.loughran at hpl.hp.com julio.guijarro at hpl.hp.com
About Us Steve Loughran Research scientist at HP Laboratories on Grid-Scale Deployment Apache Ant & Axis committer Co-author of Java Development with Ant Writing the 2 nd “Ant1.7” edition; Julio Guijarro Research scientist at HP Laboratories on Grid-Scale Deployment Leads the SmartFrog open source effort Taming Deployment with SmartFrog www.smartfrog.org Page 2
The goal of our HPLabs research • How to host big applications across distributed resources – Automatically / Repeatably – Dynamically – Correctly – Securely • How to manage them from installation to removal • How to make grid fabrics useful for classic server-side apps Taming Deployment with SmartFrog www.smartfrog.org Page 3
Deployment: why does it always go wrong? Because – it gets ignored – configuration is half the problem – nobody ever automates it – the tools are inadequate – it always goes wrong just before you go live Deployment is unreliable, unrepeatable and doesn't scale Taming Deployment with SmartFrog www.smartfrog.org Page 4
Configuration causes the problems • It’s the difference between configurations that hurt • All those things that need to be consistent – configuration files – registry settings – router bindings – firewall – database – run-time values • Trying to track down mismatches is hard Taming Deployment with SmartFrog www.smartfrog.org Page 5
Choreography is “tricky” App Server Database Network server filestore router #0 #2 #3 #3: app server #4: webapp (50% availability OK) Domain #1 Controller & DNS Server Taming Deployment with SmartFrog www.smartfrog.org Page 6
Deployment through development configure for diagnostics & testing host on developers' boxes or local servers development “near-live” configuration. Host on cut-down cluster; visible to partners managed by operations & staging dev teams ; 5x12 remote installation broadly accessible - secure High Availability/Fault Tolerant live system Scale on demand operations team on call 7x24 redistribution location/user specific configuration e.g. IP addresses, passwords, ... Self-diagnosis Taming Deployment with SmartFrog www.smartfrog.org Page 7
Configuration is deployment Imagine a file that could declare the desired configuration state of a configuration description distributed system – Define templates and extend them to describe different configurations – Cross-referencing to eliminate duplication errors – Composition for bigger systems Create reality to match – configure the declared items – start/stop them – adapt to failure or changing load ��������� Taming Deployment with SmartFrog www.smartfrog.org Page 8
Imagine: SmartFrog • Distributed Deployment System • LGPL licensed • Written in Java • SourceForge hosted • http://smartfrog.org/ Taming Deployment with SmartFrog www.smartfrog.org Page 9
SmartFrog (Smart Framework for Object Groups) A framework for describing, deploying and managing distributed service components. •A description language for specifying configuration •A runtime for realising the descriptions •A component model for managing service lifecycle • Components to deploy specific things sfConfig extends WebService { WebServer extends LAZY Apache { port 8080; } AppServer extends Jboss; } Taming Deployment with SmartFrog www.smartfrog.org Page 10
SmartFrog Description Language • A declarative, data description language – Describes the configuration of a system • templates for deployment – Prototypes to fill in with real values – Extend, override, combine • Service descriptions are interpreted by components hosted by the runtime – Semantics are not implemented in the language – Can accommodate wide range of services and models Taming Deployment with SmartFrog www.smartfrog.org Page 11
SmartFrog Deployment Engine • Distributed, decentralized, secure deployment engine • Loads and instantiates the components making up each service Description / Code Repositories • Supplies the correct configuration data to each component Deploy Descriptions RMI / (SOAP) SmartFrog SmartFrog SmartFrog Node Node Node SmartFrog SmartFrog SmartFrog Daemon Daemon Daemon RMI RMI SmartFrog Components SmartFrog Components SmartFrog Components Taming Deployment with SmartFrog www.smartfrog.org Page 12
A complex template can cover everything Template parameters access edge routers • transaction rate tier routing • response times switches authentication, DNS, intrusion detect, VPN 1st level firewall • constructed from templates for load balancing web switches • web server tier • min/max no. of web • application server servers web servers web page storage • … (NAS) 2nd level firewall • example of multiple domains application switches tier • (sub-)system templates require • min no. of app application servers servers strong notion of validation files • specific EJB’s (NAS) switches • collections of sub-templates are a common feature database database tier SQL servers • size of data, storage area • no. of tables network (SAN) Taming Deployment with SmartFrog www.smartfrog.org Page 13
Goal: two tier app App Server MySQL database Tomcat +WebApp Tomcat server WAR application Two hosts Database server MySQL Taming Deployment with SmartFrog www.smartfrog.org Page 14
MySQL svr1 MySQLTemplate extends Prim { sfClass "org.sf.mysql"; port TBD; } service service service service sfConfig extends Compound { port=80 port 80; mySql extends MySQLTemplate { mySql mySql mySql mySql sfProcessHost "svr1"; port ATTRIB:port; port=80 db "myDB"; username "user"; password LAZY securePassW; } } $ sfstart mySQL.sf svr1 service $ sfterminate svr1 service $ sfstart mySQL.sf svr1 service sfterminate svr1 service $ Taming Deployment with SmartFrog www.smartfrog.org Page 15
MySQL Demo Taming Deployment with SmartFrog www.smartfrog.org Page 16
Tomcat svr2 TomcatTemplate extends Prim { sfClass "org.sf.tomcat"; port TBD; peer TBD; service service service service } port=80 sfConfig extends Compound { port 80; tomcat extends TomcatTemplate { sfProcessHost "svr2"; port ATTRIB:port; tomcat peer LAZY svr1; } port=80 $ sfstart tomcat.sf svr2 service } $ sfterminate svr2 service $ sfstart service.sf svr2 service sfstart tomcat.sf svr2 service sfterminate svr2 service $ Taming Deployment with SmartFrog www.smartfrog.org Page 17
Demo: Tomcat + Web Application Taming Deployment with SmartFrog www.smartfrog.org Page 18
Integration: Deploying a Service svr2 svr1 Service extends Compound { sfClass "org.sf.service"; port TBD; } service service service service sfConfig extends Service { port=80 port 80; mySql extends MySQLTemplate { mySql mySql mySql mySql sfProcessHost "svr1"; port ATTRIB:port; port=80 } tomcat extends TomcatTemplate { sfProcessHost "svr2"; port ATTRIB:port; tomcat peer LAZY mySql; } port=80 $ sfstart service.sf svr1 service peer } $ sfterminate svr1 service $ sfstart service.sf svr1 service sfstart service.sf svr1 service sfterminate svr1 service $ Taming Deployment with SmartFrog www.smartfrog.org Page 19
Integration: Deploying everything Demo Taming Deployment with SmartFrog www.smartfrog.org Page 20
Components are like Ant tasks: they do the heavy lifting Ant SmartFrog Runtime Ant SmartFrog Daemon Unit of execution Project System Unit of work Task Component IntrospectionHelper sfResolve() Binding execute() Lifespan Lifecycle methods Failure Halt the build or ignore Report to container/ping Taming Deployment with SmartFrog www.smartfrog.org Page 21
Implementing a component import com.hp.smartfrog.Prim.*; import java.rmi.*; public class Example extends PrimImpl implements Remote { private String hostname; extend base class public Example() throws RemoteException { implement a Remote interface } public void sfDeploy() throws Exception { super.sfDeploy(); hostname=sfResolve("hostname","",true); } public void sfStart() throws Exception { lifecycle methods super.sfStart(); called by the runtime sfReplaceAttribute("Started",new java.util.Date()); } public void sfTerminateWith(TerminationRecored tr) { /* any component specific termination code */ super.sfTerminateWith(tr); } } Taming Deployment with SmartFrog www.smartfrog.org Page 22
How to write a new one? Describing components MyExample extends { initial template sfClass "Example"; hostname "localhost"; } something extends MyExample { component location sfProcessHost "192.168.2.1"; sfProcessName "subproc-2"; hostname "laptop"; other configuration data timestamp LAZY:Started; } sfDeployWith(ComponentDescription) Parse instantiated sfDeploy() failed terminated initialized sfStart() running sfTerminateWith(TerminationRecord ) Taming Deployment with SmartFrog www.smartfrog.org Page 23
Recommend
More recommend
