SLIDE 1
Outbox Pattern with Debezium
www.thoughts-on-java.org
Outbox Event Router
- Goals
- Monitor only outboxevent table
- Transform events
- Remove before state
- Define message key
- Change routing
www.thoughts-on-java.org
Outbox Pattern with Debezium www.thoughts-on-java.org Outbox Event - - PowerPoint PPT Presentation
Outbox Pattern with Debezium www.thoughts-on-java.org Outbox Event - - PowerPoint PPT Presentation
Outbox Pattern with Debezium www.thoughts-on-java.org Outbox Event Router Goals Monitor only outboxevent table Transform events Remove before state Define message key Change routing www.thoughts-on-java.org Outbox Table
SLIDE 2
SLIDE 3
Outbox Table
www.thoughts-on-java.org
SLIDE 4
Outbox Event Router
www.thoughts-on-java.org curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "order-outbox-connector", "config": { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "postgres", "database.dbname" : "order", "database.server.name": "localhost", "tombstones.on.delete" : "false", "table.whitelist" : "public.outboxevent", "transforms" : "outbox", "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter"} }'
SLIDE 5
Code Samples
www.thoughts-on-java.org
SLIDE 6
Configuration
- table.field.event.id
- Default: id
- Id of event in outboxevent table
- table.field.event.key
- Default: aggregateid
- Key of event in outboxevent table
www.thoughts-on-java.org
SLIDE 7
Configuration
- table.field.event.type
- Default: type
- Event type in outboxevent table
- table.field.event.timestamp
- Default: Kafka message timestamp
- Event timestamp in outbox event table
www.thoughts-on-java.org
SLIDE 8
Configuration
- table.field.event.payload
- Default: payload
- Payload of event in outboxevent table
- table.field.event.payload.id
- Default: aggregateid
- Payload id in outboxevent table
www.thoughts-on-java.org
SLIDE 9
Configuration
- table.fields.additional.placement
- Adding extra fields to event
format: <column_name>:<header or envelope>:<alias> e.g.: aggregateid:envelope:id
- table.field.event.schema.version
- Kafka Connect schema version
www.thoughts-on-java.org
SLIDE 10
Configuration
- route.by.field
- Default: aggregatetype
- Column used for routing
- route.topic.regex
- Default: (?<routedByValue>.*)
- Regex to replace topic name
www.thoughts-on-java.org
SLIDE 11
Configuration
- route.topic.replacement
- Default: outbox.event.${routedByValue}
- Name of the topic
- route.tombstone.on.empty.payload
- Default: false
- Shall empty event cause a tombstone event
www.thoughts-on-java.org
SLIDE 12
Configuration
- debezium.op.invalid.behavior
- Options: warn, error, fatal
- Default: warn
- If update event occurs, log warning/error or stop
process
www.thoughts-on-java.org
SLIDE 13
Customized Event Router
www.thoughts-on-java.org curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d '{ "name": "order-outbox-connector", "config": { "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "postgres", "database.port": "5432", "database.user": "postgres", "database.password": "postgres", "database.dbname" : "order", "database.server.name": "localhost", "tombstones.on.delete" : "false", "table.whitelist" : "public.outboxevent", "transforms" : "outbox", "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter", "transforms.outbox.route.topic.replacement" : "bookstore.events", "transforms.outbox.table.fields.additional.placement" : "aggregateid:envelope:id"} }'
SLIDE 14
Code Samples
www.thoughts-on-java.org
SLIDE 15
Summary
- Connect outbox table with Apache Kafka
- Configurable event transformation
- No code necessary
- Requires complex infrastructure
www.thoughts-on-java.org