Orchestrating Security Tooling With AWS Step Functions 1 - - PowerPoint PPT Presentation

▶
orchestrating security tooling with aws step functions
SMART_READER_LITE
LIVE PREVIEW

Orchestrating Security Tooling With AWS Step Functions 1 - - PowerPoint PPT Presentation

Nov 11, 2022 351 likes •1.2k views

Orchestrating Security Tooling With AWS Step Functions 1 Background Jules Denardou Justin Massey @Pod_Sec @jmassey09 @JulesDT @th3r3p0 Security Engineers at Datadog Product Security team Improve security of product without

slide-1
SLIDE 1

Orchestrating Security Tooling With AWS Step Functions

1

slide-2
SLIDE 2

Background

  • Security Engineers at Datadog
  • Product Security team
  • Improve security of product without detrimental impact to the developers

workflows Justin Massey @jmassey09 @th3r3p0 Jules Denardou @Pod_Sec @JulesDT

2

slide-3
SLIDE 3

3

slide-4
SLIDE 4

Goals:

4

Developers own the security of their app

slide-5
SLIDE 5

Goals:

5

Developers own the security of their app Find bugs early. Fix bugs early.

slide-6
SLIDE 6

6

Security Tooling and CI/CD A Love/Hate Relationship

slide-7
SLIDE 7

Security tooling and CI/CD

  • Security tools often don’t integrate with CI/CD

○ Long running jobs ○ False positives ○ Each tool has its own way of giving a report

7

slide-8
SLIDE 8

Security tooling and CI/CD

  • Security tools often don’t integrate with CI/CD

○ Long running jobs ○ False positives ○ Each tool has its own way of giving a report

8

slide-9
SLIDE 9

Security tooling and CI/CD

  • Security tools often don’t integrate with Dev Workflows

○ Blocking deployment pipelines ○ Reports in a third party website

9

slide-10
SLIDE 10

Security tooling and CI/CD

  • Security tools often don’t integrate with Dev Workflows

○ Blocking deployment pipelines ○ Reports in a third party website

10

slide-11
SLIDE 11

Plugin Based Workflow

11

slide-12
SLIDE 12

Plugin Based Workflow Triggers

12

slide-13
SLIDE 13

Plugin Based Workflow Processors Triggers

13

slide-14
SLIDE 14

Plugin Based Workflow Processors Triggers Outputs

14

slide-15
SLIDE 15

15

Our Solution

slide-16
SLIDE 16

16

Github PR

Our Solution

slide-17
SLIDE 17

17

Webhook API Gateway Github PR

Our Solution

slide-18
SLIDE 18

Lambda Function

18

Webhook API Gateway Github PR

Our Solution

slide-19
SLIDE 19

Security Scanning

Lambda Function API

19

Webhook API Gateway Github PR

Our Solution

slide-20
SLIDE 20

Lambda Function

20

Webhook API API Gateway Github PR Github PR Comment API

Security Scanning

Our Solution

slide-21
SLIDE 21

Issues With This Solution

Time

  • Some scans take > 30 min
  • Lambdas timeout out after 300 seconds

21

slide-22
SLIDE 22

Issues with 1st Solution

Time

  • Some scans take > 30 min
  • Lambdas timeout out after 300 seconds

22

slide-23
SLIDE 23

Issues with 1st Solution

Time

  • Some scans take > 30 min
  • Lambdas timeout out after 300 seconds

Notifications

  • Some scans have no way to notify you when complete

23

slide-24
SLIDE 24

Design

  • First implementation

24

slide-25
SLIDE 25

Design

  • First implementation

25

AWS Lambdas

slide-26
SLIDE 26

Design

  • First implementation

26

slide-27
SLIDE 27

Design

  • First implementation

○

AWS Lambdas

27

slide-28
SLIDE 28
  • Alternative: AWS Step Functions

Design

28

slide-29
SLIDE 29
  • Alternative: AWS Step Functions

○ State Machine ◻ Amazon States Language ○ Individual States ○ Step Function console = State Machine GUI

Design

29

slide-30
SLIDE 30
  • States

○ Task ○ Choice ○ Pass ○ Delay ○ Parallel ○ Success or Failure

Design

30

slide-31
SLIDE 31

Design

State Machine Individual State

31

slide-32
SLIDE 32

Design

32

slide-33
SLIDE 33

Demo !

33

slide-34
SLIDE 34

34

slide-35
SLIDE 35

35

Design

slide-36
SLIDE 36
  • Integrating Github & AWS: Incompatible by default

Design

36

slide-37
SLIDE 37
  • Integrating Github & AWS: Incompatible by default

○ Github uses HMAC signed events

Design

37

slide-38
SLIDE 38
  • Integrating Github & AWS: Incompatible by default

○ Github uses HMAC signed events ○ AWS API Gateway uses an “Authorizer”

Design

38

slide-39
SLIDE 39
  • Integrating Github & AWS: Incompatible by default

○ Implement a custom authorizer as a first step of the Step function

Design

Yes No

39

slide-40
SLIDE 40
  • Integrating Github & AWS: Incompatible by default

○ Implement a custom authorizer as a first step of the Step function

Design

40

slide-41
SLIDE 41

Design

41

slide-42
SLIDE 42

Design

42

slide-43
SLIDE 43

Design

43

slide-44
SLIDE 44

Design

44

slide-45
SLIDE 45

Design

45

slide-46
SLIDE 46

Design

46

slide-47
SLIDE 47

Design

47

slide-48
SLIDE 48

Design

48

slide-49
SLIDE 49

Design

49

slide-50
SLIDE 50

Design

50

slide-51
SLIDE 51

Design

51

slide-52
SLIDE 52

Design

52

slide-53
SLIDE 53

Design

53

slide-54
SLIDE 54

Design

54

slide-55
SLIDE 55

Design

55

slide-56
SLIDE 56

56

Sifting through logs...

slide-57
SLIDE 57

57

slide-58
SLIDE 58

58

slide-59
SLIDE 59

Searching for an ID unique to the state machine: print(event) print(context)

59

slide-60
SLIDE 60

It

60

slide-61
SLIDE 61

It doesn’t

61

slide-62
SLIDE 62

It doesn’t exist

62

slide-63
SLIDE 63

63

slide-64
SLIDE 64

64

Solution

1. Generate a Unique ID in the first lambda (state_id)

slide-65
SLIDE 65

65

Solution

1. Generate a Unique ID in the first lambda (state_id) 2. Override `logging.Filter.filter()`

slide-66
SLIDE 66

Solution

66

1. Generate a Unique ID in the first lambda 2. Override `logging.Filter.filter()` 3. Use custom filter

slide-67
SLIDE 67

Design and Implement

67

slide-68
SLIDE 68

68

slide-69
SLIDE 69

69

Developer feedback

slide-70
SLIDE 70
  • Important part of the project: the developer

○ Developer engagement is crucial

Developer feedback

70

slide-71
SLIDE 71
  • Important part of the project: the developer

○ Developer engagement is crucial ○ We need to give the results in the best way possible

Developer feedback

71

slide-72
SLIDE 72
  • Important part of the project: the developer

○ Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is

Developer feedback

72

slide-73
SLIDE 73
  • Important part of the project: the developer

○ Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is ○ Make issues found actionable

Developer feedback

73

slide-74
SLIDE 74
  • Important part of the project: the developer

○ Developer engagement is crucial ○ We need to give the results in the best way possible ○ Low-noise, easy to read, where the developer is ○ Make issues found actionable

  • The developer is our “customer”

Developer feedback

74

slide-75
SLIDE 75
  • We reach out to them

○ Send organization wide emails asking for feedback ○ Application Security focus group ○ Direct discussion with developers

Developer feedback

75

slide-76
SLIDE 76

Developer feedback

76

  • We reach out to them

○ Send organization wide emails asking for feedback ○ Application Security focus group ○ Direct discussion with developers

slide-77
SLIDE 77
  • Follow their workflows

○ Use the same CI/CD tools ○ Try to use the same technologies

  • Helps understanding their constraints

Developer feedback

77

slide-78
SLIDE 78
  • Our current solution

○ Comment on the PR This is where the dev is looking

Developer feedback

78

slide-79
SLIDE 79
  • Our current solution

○ Comment on the PR This is where the dev is looking

  • Our plan for the future

○ All the scans in a single comment ○ “Auto-fix” PR (when possible) created and linked for review

Developer feedback

79

slide-80
SLIDE 80

Deployment

80

slide-81
SLIDE 81
  • GitHub Authorizer
  • GitHub Trigger
  • Plugin that uses Go Security Scanner
  • GitHub PR Commenter
  • Slack Output

Deployment

81

slide-82
SLIDE 82
  • Lambdas

○ Serverless framework

Deployment

82

slide-83
SLIDE 83

Deployment

83

  • Lambdas

○ Serverless framework

  • IAM, Step Function, API Gateway

○ Terraform

slide-84
SLIDE 84

Justin Massey @jmassey09 @th3r3p0 Jules Denardou @Pod_Sec @JulesDT

84

We are hiring: Paris, New York, and remote!

Thank you! Questions?