SNS Topics IN TRODUCTION TO AW S BOTO IN P YTH ON Maksim - - PowerPoint PPT Presentation

▶
sns topics
SMART_READER_LITE
LIVE PREVIEW

SNS Topics IN TRODUCTION TO AW S BOTO IN P YTH ON Maksim - - PowerPoint PPT Presentation

Nov 24, 2023 722 likes •1.48k views

SNS Topics IN TRODUCTION TO AW S BOTO IN P YTH ON Maksim Pecherskiy Data Engineer! SNS INTRODUCTION TO AWS BOTO IN PYTHON Understanding SNS INTRODUCTION TO AWS BOTO IN PYTHON Understanding SNS INTRODUCTION TO AWS BOTO IN PYTHON

slide-1
SLIDE 1

SNS Topics

IN TRODUCTION TO AW S BOTO IN P YTH ON

Maksim Pecherskiy

Data Engineer!

slide-2
SLIDE 2

INTRODUCTION TO AWS BOTO IN PYTHON

SNS

slide-3
SLIDE 3

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-4
SLIDE 4

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-5
SLIDE 5

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-6
SLIDE 6

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-7
SLIDE 7

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-8
SLIDE 8

INTRODUCTION TO AWS BOTO IN PYTHON

Understanding SNS

slide-9
SLIDE 9

INTRODUCTION TO AWS BOTO IN PYTHON

Accessing SNS

slide-10
SLIDE 10

INTRODUCTION TO AWS BOTO IN PYTHON

SNS Dashboard

slide-11
SLIDE 11

INTRODUCTION TO AWS BOTO IN PYTHON

SNS Topics

slide-12
SLIDE 12

INTRODUCTION TO AWS BOTO IN PYTHON

SNS Topics

slide-13
SLIDE 13

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SNS Topic

sns = boto3.client('sns', region_name='us-east-1', aws_access_key_id=AWS_KEY_ID, aws_secret_access_key=AWS_SECRET)

slide-14
SLIDE 14

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SNS Topic

response = sns.create_topic(Name='city_alerts')

slide-15
SLIDE 15

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SNS Topic

slide-16
SLIDE 16

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SNS Topic

topic_arn = response['TopicArn']

Or... a shortcut

sns.create_topic(Name='city_alerts')['TopicArn']

slide-17
SLIDE 17

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SNS Topic

slide-18
SLIDE 18

INTRODUCTION TO AWS BOTO IN PYTHON

Permissions

slide-19
SLIDE 19

INTRODUCTION TO AWS BOTO IN PYTHON

Listing topics

response = sns.list_topics()

slide-20
SLIDE 20

INTRODUCTION TO AWS BOTO IN PYTHON

Listing topics

slide-21
SLIDE 21

INTRODUCTION TO AWS BOTO IN PYTHON

Deleting topics

sns.delete_topic(TopicArn='arn:aws:sns:us-east-1:320333787981:city_alerts')

slide-22
SLIDE 22

INTRODUCTION TO AWS BOTO IN PYTHON

Review

slide-23
SLIDE 23

INTRODUCTION TO AWS BOTO IN PYTHON

Review

Create SNS Client

sns = boto3.client('sns', region_name='us-east-1', aws_access_key_id=AWS_KEY_ID, aws_secret_access_key=AWS_SECRET)

Create a topic

response = sns.create_topic(Name='city_alerts') topic_arn = response['TopicArn']

slide-24
SLIDE 24

INTRODUCTION TO AWS BOTO IN PYTHON

Review

List Topics

response = sns.list_topics() topics = response['Topics']

Delete a topic

sns.delete_topic(TopicArn='arn:aws:sns:us-east-1:320333787981:city_alerts')

slide-25
SLIDE 25

Let's practice!

IN TRODUCTION TO AW S BOTO IN P YTH ON

slide-26
SLIDE 26

SNS Subscriptions

IN TRODUCTION TO AW S BOTO IN P YTH ON

Maksim Pecherskiy

Data Engineer

slide-27
SLIDE 27

INTRODUCTION TO AWS BOTO IN PYTHON

Subscription Listing

slide-28
SLIDE 28

INTRODUCTION TO AWS BOTO IN PYTHON

Subscription Listing

slide-29
SLIDE 29

INTRODUCTION TO AWS BOTO IN PYTHON

Subscription Listing

slide-30
SLIDE 30

INTRODUCTION TO AWS BOTO IN PYTHON

Subscription Listing

slide-31
SLIDE 31

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an SMS subscription.

sns = boto3.client('sns', region_name='us-east-1', aws_access_key_id=AWS_KEY_ID, aws_secret_access_key=AWS_SECRET) response = sns.subscribe( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Protocol = 'SMS', Endpoint = '+13125551123')

slide-32
SLIDE 32

INTRODUCTION TO AWS BOTO IN PYTHON

Create an SMS subscription.

slide-33
SLIDE 33

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an email subscription

response = sns.subscribe( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Protocol='email', Endpoint='max@maksimize.com')

slide-34
SLIDE 34

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an email subscription

slide-35
SLIDE 35

INTRODUCTION TO AWS BOTO IN PYTHON

Creating an email subscription

Conrmed email address

slide-36
SLIDE 36

INTRODUCTION TO AWS BOTO IN PYTHON

Listing subscriptions by Topic

sns.list_subscriptions_by_topic( TopicArn='arn:aws:sns:us-east-1:320333787981:city_alerts')

slide-37
SLIDE 37

INTRODUCTION TO AWS BOTO IN PYTHON

Listing subscriptions

slide-38
SLIDE 38

INTRODUCTION TO AWS BOTO IN PYTHON

Listing subscriptions

sns.list_subscriptions()['Subscriptions']

slide-39
SLIDE 39

INTRODUCTION TO AWS BOTO IN PYTHON

Deleting subscriptions

sns.unsubscribe( SubscriptionArn='arn:aws:sns:us-east-1:320333787981:city_alerts:9f2dad1d-8844-4fe8-86f )

slide-40
SLIDE 40

INTRODUCTION TO AWS BOTO IN PYTHON

Deleting multiple subscriptions

Get list of subscriptions

response = sns.list_subscriptions_by_topic( TopicArn='arn:aws:sns:us-east-1:320333787981:city_alerts') subs = response['Subscriptions']

Unsubscribe SMS subscriptions

for sub in subs: if sub['Protocol'] == 'sms': sns.unsubscribe(sub['SubscriptionArn'])

slide-41
SLIDE 41

INTRODUCTION TO AWS BOTO IN PYTHON

Review

SMS

Protocol='sms' Endpoint='+13122334433' Status: 'confirmed'

Email

Protocol='email' Endpoint='email@address.com' Status: 'confirmed' Status: 'pending confirmation'

slide-42
SLIDE 42

INTRODUCTION TO AWS BOTO IN PYTHON

Review

Create a subscription

response = sns.subscribe( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Protocol = 'sms', Endpoint = '+13125551123')

List subscriptions by topic

response = sns.list_subscriptions_by_topic( TopicArn='arn:aws:sns:us-east-1:320333787981:city_alerts') subs = response['Subscriptions']

slide-43
SLIDE 43

INTRODUCTION TO AWS BOTO IN PYTHON

Review

List subscriptions

sns.list_subscriptions()['Subscriptions']

Delete a subscription

sns.unsubscribe( SubscriptionArn='arn:aws:sns:us-east-1:320333787981:city_alerts:9f2dad1d-8844-4fe8-86f )

slide-44
SLIDE 44

Let's practice!

IN TRODUCTION TO AW S BOTO IN P YTH ON

slide-45
SLIDE 45

Sending messages

IN TRODUCTION TO AW S BOTO IN P YTH ON

Maksim Pecherskiy

Data engineer

slide-46
SLIDE 46

INTRODUCTION TO AWS BOTO IN PYTHON

Publishing to a Topic

response = sns.publish( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Message = 'Body text of SMS or e-mail', Subject = 'Subject Line for Email' )

slide-47
SLIDE 47

INTRODUCTION TO AWS BOTO IN PYTHON

Publishing to a Topic

slide-48
SLIDE 48

INTRODUCTION TO AWS BOTO IN PYTHON

Publishing to a Topic

slide-49
SLIDE 49

INTRODUCTION TO AWS BOTO IN PYTHON

Sending custom messages

num_of_reports = 137 response = client.publish( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Message = 'There are {} reports outstanding'.format(num_of_reports), Subject = 'Subject Line for Email' )

slide-50
SLIDE 50

INTRODUCTION TO AWS BOTO IN PYTHON

Sending a single SMS

response = sns.publish( PhoneNumber = '+13121233211', Message = 'Body text of SMS or e-mail' )

slide-51
SLIDE 51

INTRODUCTION TO AWS BOTO IN PYTHON

Not a good long term practice

One-off texts = getting stuff done T

  • pics and subscribers = maintenability
slide-52
SLIDE 52

INTRODUCTION TO AWS BOTO IN PYTHON

Publish to Topic vs Single SMS

Publish to a topic Have to have a topic Our topic has to have subscriptions Better for multiple receivers Easier list management Send a single SMS Don't need a topic Don't need subscriptions Just sends a message to a phone number Email option not available

slide-53
SLIDE 53

INTRODUCTION TO AWS BOTO IN PYTHON

Review

Publish to a topic

response = sns.publish( TopicArn = 'arn:aws:sns:us-east-1:320333787981:city_alerts', Message = 'Body text of SMS or e-mail', Subject = 'Subject Line for Email' )

Send a single SMS

response = sns.publish( PhoneNumber = '+13121233211', Message = 'Body text of SMS or e-mail' )

slide-54
SLIDE 54

Let's practice!

IN TRODUCTION TO AW S BOTO IN P YTH ON

slide-55
SLIDE 55

Case Study: Building a notication system

IN TRODUCTION TO AW S BOTO IN P YTH ON

Maksim Pecherskiy

Data Engineer

slide-56
SLIDE 56

INTRODUCTION TO AWS BOTO IN PYTHON

Final product

slide-57
SLIDE 57

INTRODUCTION TO AWS BOTO IN PYTHON

Final product

slide-58
SLIDE 58

INTRODUCTION TO AWS BOTO IN PYTHON

Final product

slide-59
SLIDE 59

INTRODUCTION TO AWS BOTO IN PYTHON

Final product

slide-60
SLIDE 60

INTRODUCTION TO AWS BOTO IN PYTHON

Building the notication system

Topic Set Up Create the topic Download the contact list csv Create topics for each service Subscribe the contacts to their respective topics

slide-61
SLIDE 61

INTRODUCTION TO AWS BOTO IN PYTHON

Building the notication system

Get the aggregated numbers Download the monthly get it done report Get the count of Potholes Get the count of Illegal dumping notications Send Alerts If potholes exceeds 100, send alert If illegal dumping exceeds 30, send alert

slide-62
SLIDE 62

INTRODUCTION TO AWS BOTO IN PYTHON

Topic set up

Initialize SNS client

sns = boto3.client('sns', region_name='us-east-1', aws_access_key_id=AWS_KEY_ID, aws_secret_access_key=AWS_SECRET)

Create topics and store their ARNs

trash_arn = sns.create_topic(Name="trash_notifications")['TopicArn'] streets_arn = sns.create_topic(Name="streets_notifications")['TopicArn']

slide-63
SLIDE 63

INTRODUCTION TO AWS BOTO IN PYTHON

Topic set up

slide-64
SLIDE 64

INTRODUCTION TO AWS BOTO IN PYTHON

Subscribing users to topics

contacts = pd.read_csv('http://gid-staging.s3.amazonaws.com/contacts.csv')

slide-65
SLIDE 65

INTRODUCTION TO AWS BOTO IN PYTHON

Subscribing users to topics

contacts.csv

?Name Email Phone Department John Smith js@fake.com +11224567890 trash Fanny Mae fannyma3@fake.com +11234597890 trash Janessa Goldsmith whoami@fake.com +11534567890 streets Evelyn Monroe Evely@fake.com +11234067890 streets Max Pe max@maksimize.com +11234517890 streets

slide-66
SLIDE 66

INTRODUCTION TO AWS BOTO IN PYTHON

Subscribing users to topics

Create subscribe_user method

def subscribe_user(user_row): if user_row['Department'] == 'trash': sns.subscribe(TopicArn = trash_arn, Protocol='sms', Endpoint=str(user_row['Phone'])) sns.subscribe(TopicArn = trash_arn, Protocol='email', Endpoint=user_row['Email']) else: sns.subscribe(TopicArn = streets_arn, Protocol='sms', Endpoint=str(user_row['Phone'])) sns.subscribe(TopicArn = streets_arn, Protocol='email', Endpoint=user_row['Email'])

Apply the subscribe_user method to every row

contacts.apply(subscribe_user, axis=1)

slide-67
SLIDE 67

INTRODUCTION TO AWS BOTO IN PYTHON

Subscribing users to topics

slide-68
SLIDE 68

INTRODUCTION TO AWS BOTO IN PYTHON

Get the aggregated numbers

Load January's report into a DataFrame

df = pd.read_csv('http://gid-reports.s3.amazonaws.com/2019/feb/final_report.csv')

slide-69
SLIDE 69

INTRODUCTION TO AWS BOTO IN PYTHON

Get the aggregated numbers

service_name count Illegal Dumping 2580 Potential Missed Collection 150 Pothole 1170 Trafc Sign - Maintain 210 Trafc Signal Head Turned 60 Trafc Signal Light Out 120

slide-70
SLIDE 70

INTRODUCTION TO AWS BOTO IN PYTHON

Get the aggregated numbers

Set the index so we can access counts by service name directly

df.set_index('service_name', inplace=True)

Get the aggregated numbers

trash_violations_count = df.at['Illegal Dumping', 'count'] streets_violations_count = df.at['Pothole', 'count']

slide-71
SLIDE 71

INTRODUCTION TO AWS BOTO IN PYTHON

Send Alerts

if trash_violations_count > 100: # Construct the message to send message = "Trash violations count is now {}".format(trash_violations_count) # Send message sns.publish(TopicArn = trash_arn, Message = message, Subject = "Trash Alert")

slide-72
SLIDE 72

INTRODUCTION TO AWS BOTO IN PYTHON

Send alerts

if streets_violations_count > 30: # Construct the message to send message = "Streets violations count is now {}".format(streets_violations_count) # Send message sns.publish(TopicArn = streets_arn, Message = message, Subject = "Streets Alert")

slide-73
SLIDE 73

INTRODUCTION TO AWS BOTO IN PYTHON

Final Result

slide-74
SLIDE 74

Let's practice!

IN TRODUCTION TO AW S BOTO IN P YTH ON