10/30/17 1
Today’s Objec3ves
- Phil’s Talk Review
- Amazon Web Services
Ø Elas3c Map Reduce (EMR)
Oct 30, 2017 1 Sprenkle - CSCI325
Phil’s Talk
Oct 30, 2017 Sprenkle - CSCI325 2
Todays Objec3ves Phils Talk Review Amazon Web Services Elas3c Map - - PDF document
10/30/17 Todays Objec3ves Phils Talk Review Amazon Web Services Elas3c Map Reduce (EMR) Oct 30, 2017 Sprenkle - CSCI325 1 Phils Talk Oct 30, 2017 Sprenkle - CSCI325 2 1 10/30/17 AMAZON WEB SERVICES (AWS) Oct 30, 2017
Oct 30, 2017 1 Sprenkle - CSCI325
Oct 30, 2017 Sprenkle - CSCI325 2
Oct 30, 2017 Sprenkle - CSCI325 3
4
Oct 30, 2017 Sprenkle - CSCI325
Ø Region: Geographic loca3on, price, laws, network locality. Ø wholly contained within a single country and all of its data and services stay within the designated Region.
Ø dis3nct data centers providing AWS services Ø isolated from each other to prevent outages from spreading between Zones Ø 44 availability zones
5 Oct 30, 2017 Sprenkle - CSCI325
https://aws.amazon.com/about-aws/global-infrastructure/
Oct 30, 2017 Sprenkle - CSCI325 6
Oct 30, 2017 Sprenkle - CSCI325 7
Oct 30, 2017 Sprenkle - CSCI325 8
http://aws.amazon.com/tools/
http://aws.amazon.com/cli/ http://aws.amazon.com/console/
Oct 30, 2017 Sprenkle - CSCI325 9
http://aws.amazon.com/s3/ http://aws.amazon.com/ec2/ http://aws.amazon.com/ebs/
http://aws.amazon.com/vpc/
Oct 30, 2017 Sprenkle - CSCI325 10
hcp://aws.amazon.com/dynamodb/ hcps://aws.amazon.com/emr/ hcp://aws.amazon.com/elas3cbeanstalk/ hcp://aws.amazon.com/rds/
DynamoDB
Beanstalk
Oct 30, 2017 Sprenkle - CSCI325 11
Oct 30, 2017 Sprenkle - CSCI325 12
Oct 30, 2017 Sprenkle - CSCI325 13
public public static static class class TokenizerMapper TokenizerMapper extends extends Mapper<Object, Text, Text, Mapper<Object, Text, Text, IntWritable IntWritable> { > { private private final final static static IntWritable IntWritable one
= new new IntWritable IntWritable(1); (1); private private Text Text word word = = new new Text(); Text(); public public void void map(Object map(Object key key, Text , Text value value, Context , Context context context) throws throws IOException IOException, , InterruptedException InterruptedException { { StringTokenizer itr = new new StringTokenizer StringTokenizer(value value.toString .toString()); ()); while while ( (itr itr.hasMoreTokens .hasMoreTokens()) { ()) { word.set(itr.nextToken()); context.write(word, one
); } } }
Oct 30, 2017 Sprenkle - CSCI325 14
public public static static class class IntSumReducer IntSumReducer extends extends Reducer<Text, Reducer<Text, IntWritable IntWritable, Text, , Text, IntWritable IntWritable> { > { private private IntWritable IntWritable result result = = new new IntWritable IntWritable(); (); public public void void reduce(Text reduce(Text key key, , Iterable Iterable<IntWritable IntWritable> > values values, Context , Context context context) throws throws IOException IOException, , InterruptedException InterruptedException { { int int sum sum = 0; = 0; for for ( (IntWritable IntWritable val val : : values values) { ) { sum += val.get(); } result.set(sum); context.write(key, result); } }
Oct 30, 2017 Sprenkle - CSCI325 15
Oct 30, 2017 Sprenkle - CSCI325 16
Oct 30, 2017 Sprenkle - CSCI325 17
Input file InputSplit InputSplit InputSplit InputSplit Input file RecordReader RecordReader RecordReader RecordReader Mapper (intermediates) Mapper (intermediates) Mapper (intermediates) Mapper (intermediates) InputFormat
Oct 30, 2017 Sprenkle - CSCI325 19
Reducer Reducer Reducer RecordWriter RecordWriter RecordWriter
OutputFormat
Oct 30, 2017 Sprenkle - CSCI325 21