Every SAS Cloud has a Silver Lining Letting your data reign in the cloud
DSS SAS SYSTEM Current Single Virtual Server unit with 16 cores upgraded to 32 cores 256 Gb RAM 150 registered users Data collector from other government departments and external bodies Runs Models and builds reports from this data. Processing time – Considerably long due to model running Goal Grow with increased data collection Integrate with new tools Deliver reports in shorter time frame Increased analytical capabilities Able to operate in the new DSS Infrastructure Hosting Zone (on premises cloud)
DATA IS JUST DATA ISN’T IT? Structured data captured from operational systems Customer interaction directly with company Purchase detail Usage detail Addition of 3 rd party data to add value to operational data The beginning of CRM systems Beginning of Data Mining Big Data Social media Non customer interaction Unstructured data Anything that generates some form of data in large volumes
IN 1999, THE WORLD PRODUCED ABOUT 1.5 EXABYTES OF UNIQUE INFORMATION, OR ABOUT 250 MEGABYTES FOR EVERY MAN, WOMAN, AND CHILD ON EARTH.
IN 2008 THE WORLD PRODUCED 14.7 EXABYTES OF NEW INFORMATION, NEARLY TRIPLE THE VOLUME OF INFORMATION IN 2003.
WHAT WE DO WITH THE DATA 1990 Descriptive Analytics – What happened and when did it happen? Standard reports Ad – hoc reports Simple drill down queries 2000 - onwards Predictive Analytics – based on what happened, what is likely to happen? Statistical Analysis Forecasting Predictive Modelling Optimisation
ADVANCED ANALYTICS What happened? Why did it happen? When did it happen? What caused it to happen? Who made it happen? Will it happen again? What should we do to make happen /stop it from happening again? What is the outcome of this event?
WHAT DO WE WANT FROM THE DATA? What do we want to know? Why do we want to know? When do we want to know? Who will use the what we now know? How will this knowledge be used? What is the benefit of knowing this?
THE HARDWARE It’s more than an abacus, coloured pencils, ruler, and graph paper
GETTING THE RIGHT BALANCE For today and tomorrow
Sunway TaihuLight (10.65 million cores). Probably Not 'It would take 7.2 billion people using electronic calculators 32 years, or two million desktop computers working together for one minute, to do the same calculation the computer can solve in just 60 seconds.'
How about this? Ummm, Yeah, No
THINGS TO CONSIDER You’ll always get more data More analytics and reporting More, More, More Efficiency Fixed resources Elastic/Scalable resources Coding Scheduling Tools $$$$$$ ROI Lost Time
JUST HOW FAST ARE THE $$$$ ? Fixed resources Scalable/Elastic Resources One User Many Users
IN BUSINESS The difference between a great system and a bad system is Being No. 1 Or Being No 1
IN GOVERNMENT The difference between a good system and a bad system is Wise use of taxpayers money Or Being asked questions by the Auditor General
GRID SERVER VS VIYA Grid Server (on Premises or Cloud) Viya (Cloud) Scalable (with limited CPU available) Elastic (Virtually unlimited CPU) On or off premises Mainly off premises location Integrate other tools Integrate other tools E.g. Hadoop etc. E.g. Hadoop etc. In house systems need to be Minimal maintenance required (Host) maintained / upgraded Security by cloud host and In house-Security by organisation organisation
Yes, I stole this from SAS
BEST OPTION SAS Grid Server configuration in an on premises cloud Resources in house Meets government security protocols (ASD) Internal systems and Infrastructure meet needs now and future Integration with tools currently used by DSS Hadoop Federation Server Easy to expand nodes due to virtual server environment Node configuration to suit business needs Advanced analytical capabilites
Data Tier SAS Server Tier Client Tier Source and Target Grid Server (Node N+1) Environment Administrators Grid Server (Controller) Yarn Node Manager Grid Server (Node N) Yarn Resource Manager RDBMS SAS Management Console Yarn Node Manager SAS Foundation Grid Server (Node 1) OOZIE Yarn Node Manager SAS Connect SAS Foundation HIVE SAS Environment Manager Analytical data HUE Warehouse/ marts SAS Connect SAS Foundation SAS Grid Node HttpFS SAS Connect SAS Object Spawner SAS Grid Node Hadoop Administration HDFS Name Node SAS Object Spawner SAS Grid Node SAS Workspace Server SAS Foundation SAS Object Spawner SAS Stored Process Server SAS Workspace Server ADFS Access Administration SAS Connect SAS Stored Process Server SAS Workspace Server SAS Pooled Workspace Server SAS Grid Server SAS Stored Process Server SAS Data Step Batch Server SAS Pooled Workspace Server SAS Object Spawner End User SAS Data Step Batch Server SAS Pooled Workspace Server SAS Workspace Server HDFS Data Node Hadoop SAS Stored Process Server SAS Enterprise Guide SAS Data Step Batch Server HDFS Data Node SAS Pooled Workspace Server HDFS Data Node SAS Addin for Microsoft ERP, applications SAS Data Step Batch Server Office & other Sources SAS Enterprise Miner SASGSUB Clustered Middle Tier Server Shared Storage Clustered Metadata Server SAS Web Server HDFS Storage SAS Metadata Server User Home Directories Shared Projects SAS Web Application Servers SAS Data SAS Grid Work Libraries SAS Depot SAS Studio Mid-Tier SAS Web Infrastructure Hadoop Client jars and SAS SPDE SPDS Data Platform SAS Config Files Hadoop config and client Jars Config Files OK I took this from SAS too, but modified it a little
THANK YOU Peter Stagg peter@decision-informatics.com www.linkedin.com/in/pstagg https://www.facebook.com/decisionInformatics
Recommend
More recommend
