FORGET ME PLEASE? EVENT SOURCING & THE GDPR Michiel Rook - - - PowerPoint PPT Presentation

▶
forget me please event sourcing the gdpr
SMART_READER_LITE
LIVE PREVIEW

FORGET ME PLEASE? EVENT SOURCING & THE GDPR Michiel Rook - - - PowerPoint PPT Presentation

Aug 27, 2022 11 likes •1.36k views

FORGET ME PLEASE? EVENT SOURCING & THE GDPR Michiel Rook - @michieltcs DISCLAIMER: I AM NOT A LAWYER GDPR GENERAL DATA PROTECTION REGULATION ' Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016

slide-1
SLIDE 1

FORGET ME PLEASE?
 EVENT SOURCING & THE GDPR

Michiel Rook - @michieltcs

slide-2
SLIDE 2

DISCLAIMER: I AM NOT A LAWYER

slide-3
SLIDE 3

GDPR

slide-4
SLIDE 4

GENERAL DATA PROTECTION REGULATION

slide-5
SLIDE 5

'

Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (Data Protection Directive)

  • General Data Protection Regulation
slide-6
SLIDE 6

A SHORT HISTORY

slide-7
SLIDE 7

1995 Data Protection Directive

slide-8
SLIDE 8

1995 Data Protection Directive 2012 GDPR proposal

slide-9
SLIDE 9

1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted

slide-10
SLIDE 10

1995 Data Protection Directive 2012 GDPR proposal 2016 GDPR adopted 25 May 2018 GDPR enforceable

slide-11
SLIDE 11

REGULATION

slide-12
SLIDE 12

PROTECTS EU CITIZENS

slide-13
SLIDE 13

DATA PROTECTION ACT

slide-14
SLIDE 14

BROAD & VAGUE

slide-15
SLIDE 15

PRIVACY BY DESIGN

slide-16
SLIDE 16

'

The controller shall implement appropriate technical and organisational measures for ensuring that, by default,

  • nly personal data which are necessary for

each specific purpose of the processing are processed.

  • GDPR, Article 25
slide-17
SLIDE 17

DATA PROTECTION OFFICER

slide-18
SLIDE 18

SUPERVISORY AUTHORITY

slide-19
SLIDE 19

FINES

slide-20
SLIDE 20

€20 MILLION OR 4% OF ANNUAL TURNOVER

slide-21
SLIDE 21

YOU

slide-22
SLIDE 22

RAISE YOUR HAND

IF YOU HAVE

slide-23
SLIDE 23

read CQRS / Event Sourcing theory

RAISE YOUR HAND

IF YOU HAVE

slide-24
SLIDE 24

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project

RAISE YOUR HAND

IF YOU HAVE

slide-25
SLIDE 25

read CQRS / Event Sourcing theory followed a tutorial, built a hobby project used it in production

RAISE YOUR HAND

IF YOU HAVE

slide-26
SLIDE 26

Axon Framework Spring Boot

slide-27
SLIDE 27

QUICK RECAP CQRS + EVENT SOURCING

slide-28
SLIDE 28

CQRS

slide-29
SLIDE 29

COMMAND QUERY RESPONSIBILITY SEGREGATION

slide-30
SLIDE 30

STORAGE SIDE
 VS.
 QUERY SIDE

slide-31
SLIDE 31

UI

@michieltcs

slide-32
SLIDE 32

Domain UI

Command

commands

Aggregates

@michieltcs

slide-33
SLIDE 33

Domain UI

Command Repository

Event Store

commands events

Aggregates

@michieltcs

slide-34
SLIDE 34

Domain UI

Event Bus Event Handlers Command Repository Database Database

Event Store

commands events events

Aggregates

@michieltcs

slide-35
SLIDE 35

Domain UI

Event Bus Event Handlers Command Repository Data Layer Database Database

Event Store

commands events events queries DTOs

Aggregates

@michieltcs

slide-36
SLIDE 36

EVENT SOURCING

slide-37
SLIDE 37

'

Event Sourcing ensures that all changes to application state are stored as a sequence of events.

  • Martin Fowler
slide-38
SLIDE 38

ACTIVE RECORD VS. EVENT SOURCING

Account Id Account number Balance 1234 12345678 €50,00 ... ... ... Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678

@michieltcs

slide-39
SLIDE 39

COMMANDS TO EVENTS

Deposit Money Account Id 1234 Amount €100,00

@michieltcs

1 @Value 2 public class DepositMoney { 3 @TargetAggregateIdentifier 4 String accountId; 5 BigDecimal amount; 6 }

slide-40
SLIDE 40

COMMANDS TO EVENTS

Deposit Money Account Id 1234 Amount €100,00

command
 handler

@michieltcs

1 @CommandHandler 2 public void depositMoney(DepositMoney command) { 3 apply(new MoneyDeposited( 4 command.getAccountId(), 5 command.getAmount(), 6 ZonedDateTime.now())); 7 }

slide-41
SLIDE 41

COMMANDS TO EVENTS

Deposit Money Account Id 1234 Amount €100,00 Money Deposited Account Id 1234 Amount €100,00

command
 handler

@michieltcs

1 @Value 2 public class MoneyDeposited { 3 String accountId; 4 BigDecimal amount; 5 ZonedDateTime timestamp; 6 }

slide-42
SLIDE 42

AGGREGATES

@michieltcs

an Aggregate handles Commands and generates Events based on the current state

slide-43
SLIDE 43

AGGREGATES

@michieltcs

1 class BankAccount { 2 @AggregateIdentifier 3 private String accountId; 4 private String accountNumber; 5 private BigDecimal balance; 6 7 // ... 8 @EventHandler 9 public void accountOpened(AccountOpened event) { 10 this.accountId = event.getAccountId(); 11 this.accountNumber = event.getAccountNumber(); 12 this.balance = BigDecimal.valueOf(0); 13 } 14 15 @EventHandler 16 public void moneyDeposited(MoneyDeposited event) { 17 this.balance = this.balance.add(event.getAmount()); 18 } 19 }

slide-44
SLIDE 44

AGGREGATE STATE

Account number Balance 12345678 €0,00 Account number Balance 12345678 €100,00 Account number Balance 12345678 €50,00

event
 handler event
 handler event
 handler

@michieltcs

Money Withdrawn Account Id 1234 Amount €50,00 Money Deposited Account Id 1234 Amount €100,00 Account Opened Account Id 1234 Account number 12345678

slide-45
SLIDE 45

VALIDATING COMMANDS

@michieltcs

1 @CommandHandler 2 public void withdrawMoney(WithdrawMoney command) throws 3 OverdraftDetectedException { 4 if (balance.compareTo(command.getAmount()) >= 0) { 5 apply(new MoneyWithdrawn( 6 command.getAccountId(), 7 command.getAmount(), 8 ZonedDateTime.now())); 9 } else { 10 throw new OverdraftDetectedException(accountNumber, balance, command. 11 getAmount()); 12 } 13 }

slide-46
SLIDE 46

TESTING AGGREGATES

@michieltcs

1 public class BankAccountTest { 2 private FixtureConfiguration<BankAccount> fixture; 3 4 @Before 5 public void createFixture() { 6 fixture = new AggregateTestFixture<>(BankAccount.class); 7 } 8 9 @Test 10 public void noOverdraftsOnEmptyAccount() { 11 fixture.given(new AccountOpened(ACCOUNT_ID, ACCOUNT_NUMBER)) 12 .when(new WithdrawMoney(ACCOUNT_ID, new BigDecimal(20))) 13 .expectException(OverdraftDetectedException.class); 14 } 15 16 private final static String ACCOUNT_ID = "accountId"; 17 private final static String ACCOUNT_NUMBER = "accountNumber"; 18 }

slide-47
SLIDE 47

EVENT SOURCING
 & GDPR

slide-48
SLIDE 48

CONSENT

slide-49
SLIDE 49

'

Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data.

  • GDPR, Article 7
slide-50
SLIDE 50

REGISTERING CONSENT

slide-51
SLIDE 51

'

...the request for consent shall be presented in a manner which is clearly distinguishable from the other matters...

  • GDPR, Article 7
slide-52
SLIDE 52

REVOKING CONSENT

slide-53
SLIDE 53

'

The data subject shall have the right to withdraw his or her consent at any

  • time. ... It shall be as easy to withdraw

as to give consent.

  • GDPR, Article 7
slide-54
SLIDE 54

WHY USE EVENT SOURCING / CQRS?

slide-55
SLIDE 55

CAPTURE INTENT

slide-56
SLIDE 56

DEMONSTRATING CONSENT

slide-57
SLIDE 57

EVENT LOG
 AS AUDIT LOG

slide-58
SLIDE 58

NEW READ MODELS

slide-59
SLIDE 59

EASIER DEBUGGING

slide-60
SLIDE 60

EVENT LOG AS AUDIT LOG

@michieltcs

ConsentedToNewsletters

slide-61
SLIDE 61

EVENT LOG AS AUDIT LOG

@michieltcs

ConsentedToNewsletters ConsentedToDataGathering

slide-62
SLIDE 62

EVENT LOG AS AUDIT LOG

@michieltcs

ConsentedToNewsletters ConsentedToDataGathering RevokedConsentToNewsletters

slide-63
SLIDE 63

"RIGHT TO ACCESS"

slide-64
SLIDE 64

'

The data subject shall have the right to

  • btain from the controller

confirmation as to whether or not personal data ... are being processed, and ... access to the personal data ...

  • GDPR, Article 15
slide-65
SLIDE 65

"RIGHT TO ERASURE"

slide-66
SLIDE 66

'

The data subject shall have the right to

  • btain from the controller the erasure
  • f personal data concerning him or

her without undue delay

  • GDPR, Article 17
slide-67
SLIDE 67

PERSONALLY IDENTIFIABLE INFORMATION

slide-68
SLIDE 68

'

‘personal data’ means any information relating to an identified or identifiable natural person; an identifiable natural person is one who can be identified, directly or indirectly

  • GDPR, Article 4
slide-69
SLIDE 69

GROUNDS

slide-70
SLIDE 70

'

.. the personal data are no longer necessary .. the data subject withdraws consent

  • n which the processing is based
  • GDPR, Article 17
slide-71
SLIDE 71

EXCEPTIONS

slide-72
SLIDE 72

'

.. to comply with a legal obligation .. for the establishment, exercise or defence of legal claims (*)

  • GDPR, Article 17
slide-73
SLIDE 73

UNDUE DELAY

slide-74
SLIDE 74

INFORM 3RD PARTIES

slide-75
SLIDE 75

BACKUPS?

slide-76
SLIDE 76

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked

slide-77
SLIDE 77

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked Notify 3rd parties

slide-78
SLIDE 78

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked Remove from read models Notify 3rd parties

slide-79
SLIDE 79

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked Remove from event store Remove from read models Notify 3rd parties

slide-80
SLIDE 80

PROCESSING GDPR ART. 17 REQUESTS

@michieltcs

RightToErasureInvoked Remove from event store

?

Remove from read models Notify 3rd parties

slide-81
SLIDE 81

IMMUTABLE EVENTS?

slide-82
SLIDE 82

COMPENSATING ACTIONS

slide-83
SLIDE 83

@michieltcs

Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00

slide-84
SLIDE 84

@michieltcs

Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00

slide-85
SLIDE 85

@michieltcs

Ledger Entry Aug 14 Inventory €15600,00 Accounts Payable €15600,00 Ledger Entry Aug 14 Inventory €16500,00 Accounts Payable €16500,00 Ledger Correction Entry Aug 14 Inventory €900,00 Accounts Payable €900,00

slide-86
SLIDE 86

COMPENSATING ACTIONS

class MoneyWithdrawn {
 String accountId;
 BigDecimal amount;
 } class WithdrawalRolledBack {
 String accountId;
 BigDecimal amount;
 } Typo: too much withdrawn!

slide-87
SLIDE 87

COMPENSATING ACTIONS

class AccountOpened {
 String accountId;
 String accountNumber;
 } class DuplicateAccountClosed {
 String accountId;
 } Duplicate account number!

slide-88
SLIDE 88

GDPR?

slide-89
SLIDE 89

STRATEGIES

slide-90
SLIDE 90

ONLY REMOVE FROM PROJECTION?

slide-91
SLIDE 91

LEGAL DEFENCE?

slide-92
SLIDE 92

'

.. adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’)

  • GDPR, Article 5
slide-93
SLIDE 93

UPCASTING?

slide-94
SLIDE 94

UPCASTING

Event Store Event_V1 Upcaster Event_V2 Event Handler

@michieltcs

slide-95
SLIDE 95

UPCASTING

Event Store Event_V1 Upcaster Event_V2 Event Handler

@michieltcs

Event_V2 = f(Event_V1)

slide-96
SLIDE 96

UPCASTING

Event Store Event_V1 Upcaster Event_V2 Event Handler

@michieltcs

Event_V2 = f(Event_V1)

slide-97
SLIDE 97

DELETING EVENTS

slide-98
SLIDE 98

DELETING EVENTS

slide-99
SLIDE 99

MODIFYING EVENTS

slide-100
SLIDE 100

MODIFYING EVENTS

slide-101
SLIDE 101

COPY & FILTER

slide-102
SLIDE 102

VERSIONED EVENT STORE

slide-103
SLIDE 103

VERSIONED EVENT STORE

events_v1 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "1.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ]

@michieltcs

slide-104
SLIDE 104

COPY & REPLACE

slide-105
SLIDE 105

VERSIONED EVENT STORE

Loop over existing events Apply upcaster Add queued events Use new event store New events Queue

@michieltcs

slide-106
SLIDE 106

VERSIONED EVENT STORE

events_v2 [
 {
 "id": "12345678",
 "type": "AccountOpened",
 "aggregateType": "Account",
 "aggregateIdentifier": "1234",
 "sequenceNumber": 0,
 "payloadRevision": "2.0",
 "payload": { ... },
 "timestamp": ...
 ...
 },
 ...
 ]

@michieltcs

slide-107
SLIDE 107

STORE PII EXTERNALLY

slide-108
SLIDE 108

STORE PII EXTERNALLY

@michieltcs

1 @Value 2 public class AccountOpened { 3 String accountId; 4 String accountNumber; 5 String name; 6 }

slide-109
SLIDE 109

STORE PII EXTERNALLY

@michieltcs

1 @Value 2 public class AccountOpened { 3 String accountId; 4 }

slide-110
SLIDE 110

STORE PII EXTERNALLY

@michieltcs

AccountOpened External Storage

1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 John Doe ... ... ...

slide-111
SLIDE 111

STORE PII EXTERNALLY

@michieltcs

AccountOpened External Storage

1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...

slide-112
SLIDE 112

STORE PII EXTERNALLY

@michieltcs

AccountOpened External Storage

1 @Value 2 public class AccountOpened { 3 String accountId; 4 } Account Id Account number Name 1234 12345678 ANON ... ... ...

slide-113
SLIDE 113

CRYPTO ERASURE

slide-114
SLIDE 114

ENCRYPT EVENTS

slide-115
SLIDE 115

DECRYPT EVENTS

slide-116
SLIDE 116

ENCRYPT FIELD VALUES

slide-117
SLIDE 117

DECRYPT FIELD VALUES

slide-118
SLIDE 118

ENCRYPTING EVENTS

@michieltcs

<org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>NL00ABNA012345678</accountNumberIban>
 <firstName>Foo</firstName>
 <lastName>Bar</lastName> ...
 </org.demo.AccountOpened>

slide-119
SLIDE 119

ENCRYPTING EVENTS

@michieltcs

<org.demo.AccountOpened>
 <accountId>80f49161</accountId>
 <accountNumberIban>2dqjHkY8Mc8+cek4vs/9hzgkob4J3fZJNIJh2sAXlJ0=</accountNumberIban>
 <firstName>N5Y27vd0UbKo6FIu5c7QGQ==</firstName>
 <lastName>OSKrzfuuuayuUNXYS5YUug==</lastName> ...
 </org.demo.AccountOpened>

slide-120
SLIDE 120

ENCRYPTING EVENTS

Generate event Find / create encryption key Encrypt payload values Store
 event

@michieltcs

slide-121
SLIDE 121

DECRYPTING EVENTS

Load
 event Find associated encryption key Decrypt payload values Process
 event

@michieltcs

slide-122
SLIDE 122

SHEDDING THE KEY

Load
 event Find associated encryption key Decrypt payload values Process
 event

@michieltcs

X

slide-123
SLIDE 123

AXON GDPR MODULE

@michieltcs

1 @Value 2 public class AccountOpened { 3 @DataSubjectId 4 String accountId; 5 6 @PersonalData 7 String accountNumberIban; 8 9 @PersonalData 10 String firstName; 11 12 @PersonalData 13 String lastName;
 14 }

slide-124
SLIDE 124

KEY MANAGEMENT

slide-125
SLIDE 125

PERFORMANCE

slide-126
SLIDE 126

RE-ENCRYPT DATA AT REST

slide-127
SLIDE 127

CLOSING WORDS

slide-128
SLIDE 128

GDPR

slide-129
SLIDE 129

CHALLENGES

slide-130
SLIDE 130

FRAMEWORK SUPPORT

slide-131
SLIDE 131

(IM)MUTABILITY

slide-132
SLIDE 132

AUDIT TRAIL

slide-133
SLIDE 133

DEMONSTRATING CONSENT

slide-134
SLIDE 134

FUTURE?

slide-135
SLIDE 135

THANK YOU!

@michieltcs / michiel@michielrook.nl
 
 www.michielrook.nl