Writing Stream Processors in Kotlin KotlinConf 2018 - Amsterdam - - PowerPoint PPT Presentation

Writing Stream Processors in Kotlin

KotlinConf 2018 - Amsterdam

Who of you uses ?

A quick look

• ver the shoulder

3

Kafk fka in in a Nutshell

Source: Wikipedia

5

What is is so special about Kafka?

• Persistent
• Time and Log compaction
• Guaranteed order per partition

Messages

• Have a byte[] key
• Have a byte[] value
• Contain Metadata like a timestamp and the partition

Image from http://kafka.apache.org/documentation/

Event Streams

• null-key and a bare or serialized value
• Compaction of the topic by time
• Examples: Clickstreams, Logs, IoT events

Image from http://cloudurable.com/blog/kafka-architecture-log-compaction/index.html

Log Compaction

Stream Processors

• Imagine you could do Kotlin Sequences on an infinite list of messages
• Stateful
• Parallel
• And the output is one or more Kafka topic(s)

What now?

• I do home automation, a lot
• I use homee, which aggregates multiple protocols

(Z-Wave, ZigBee, etc)

• It has no official API, yet
• But … They (unintentionally) made one of their repos public on Github
• And now I have an API Blueprint file ☺
• Guess what? It’s a websocket!

Let’s get started

• The websocket issues text messages when something happens
• Attribute Changes of any connected device

(motion, temperature, light, door opens, etc.)

• System Events (user created, system update, etc.)
• We use Kafka Connect to build what is called a “source” and pipe these events

into Kafka.

Let’s look at some code

(no live demo, sorry)

Now, why Kotlin?

The Java code did not look that bad…

An engineering vie iew

• Less verbose, better to read
• Extension functions enable composable, high level abstractions
• Extensions can be called from the Java folks
• One can write libraries with business abstractions, reusable within

the company, that blend into the original API

A team le leads vie iew

• Onboarding people coming from non-JVM languages to Kotlin is easier than

Java

• The education material available is very rich, based on current teaching

standards

• People love the language and the feeling of productivity
• Unmodifiable / Immutable helps a lot of people to write more stable code,
• resulting in better outcome and higher motivation to go on with learning
• because “it just works es expected”
• You can write good code even without knowing the language in-depth

Kotlin at tri rivago

• More than 80% of our Kafka related source code is Kotlin
• Kafka Connect Sinks, Transforms, Converters
• Stream Processors
• Custom Solutions, based on Spring Boot 2, Spring Kafka, Spring Integration
• My current team writes client facing REST and GRPC services based on Spring Boot 2

entirely in Kotlin

• Our recently relaunched Android App is written entirely in Kotlin
• Some of our B2B focused teams started to rewrite their PHP applications in Kotlin

Curious about how we use Apache Kafka?

Kafka and Debezium at trivago (code.talks 2017) https://speakerdeck.com/xenji/kafka-and-debezium-at-trivago-code-dot-talks-2017-edition