SLIDE 1
Scalaz-Stream Masterclass
Rúnar Bjarnason, Verizon Labs @runarorama NEScala 2016, Philadelphia
Scalaz-Stream (FS2)
Functional Streams for Scala
https://github.com/functional-streams-for-scala/fs2
Scalaz-Stream Masterclass Rnar Bjarnason, Verizon Labs @ runarorama - - PowerPoint PPT Presentation
Scalaz-Stream Masterclass Rnar Bjarnason, Verizon Labs @ runarorama - - PowerPoint PPT Presentation
Scalaz-Stream Masterclass Rnar Bjarnason, Verizon Labs @ runarorama NEScala 2016 , Philadelphia Scalaz-Stream (FS2) F unctional S treams for S cala https://github.com/functional-streams-for-scala/fs2 Disclaimer This library is
SLIDE 2
SLIDE 3
Disclaimer
This library is changing. We’ll talk about the current version (0.8). Scalaz 7.1
SLIDE 4
Scalaz-Stream (FS2)
a purely functional streaming I/O library for Scala
SLIDE 5
- Streams are essentially “lazy lists”
- f data and effects.
- Naturally pull-based
- Immutable and
referentially transparent
SLIDE 6
Design goals
- compositional
- expressive
- resource-safe
- comprehensible
SLIDE 7
Takeaway: No magic
SLIDE 8
import scalaz.stream._ import scalaz.concurrent.Task val converter: Task[Unit] = io.linesR("testdata/fahrenheit.txt") .filter(s => !s.trim.isEmpty && !s.startsWith("//")) .map(line => fahrenheitToCelsius(line.toDouble).toString) .intersperse("\n") .pipe(text.utf8Encode) .to(io.fileChunkW("testdata/celsius.txt")) .run val u: Unit = converter.run
SLIDE 9
scalaz.concurrent.Task
- Asynchronous
- Compositional
- Purely functional
SLIDE 10
a Task is a first-class program
SLIDE 11
a Task is a list of instructions
SLIDE 12
Task is a monad
SLIDE 13
a Task doesn’t do anything until you call .run
SLIDE 14
Constructing Tasks
SLIDE 15
Task.delay(readLine): Task[String] Task.now(42): Task[Int] Task.fail( new Exception("oops!") ): Task[Nothing]
SLIDE 16
fut: scala.concurrent.Future[Int] Task.async(fut.onComplete): Task[Int]
SLIDE 17
Task.async { k => fut.onComplete { case Success(a) => k(\/.right(a)) case Fail(a) => k(\/.left(e)) } }
SLIDE 18
a: Task[A] pool: java.util.concurrent.ExecutorService Task.fork(a)(pool): Task[A]
SLIDE 19
Combining Tasks
SLIDE 20
a: Task[A] b: Task[B] val c: Task[(A,B)] = Nondeterminism[Task].both(a,b)
SLIDE 21
a: Task[A] f: A => Task[B] val b: Task[B] = a flatMap f
SLIDE 22
val program: Task[Unit] = for { _ <- delay(println("What's your name?")) n <- delay(scala.io.StdIn.readLine) _ <- delay(println(s"Hello $n")) } yield ()
SLIDE 23
Running Tasks
SLIDE 24
a: Task[A] a.run: A
SLIDE 25
a: Task[A] k: (Throwable \/ A) => Unit a runAsync k: Unit
SLIDE 26
Handling errors
SLIDE 27
Task.delay { throw new Exception("oops") } Task.fail {
new Exception("oops") }
SLIDE 28
t: Task[A] t.attempt: Task[Throwable \/ A]
SLIDE 29
scalaz.stream.Process
SLIDE 30
Process[+F[_],+A]
SLIDE 31
Process[Task,A]
SLIDE 32
val halt: Process[Nothing,Nothing] def emit[O](o: O): Process[Nothing,O] def await[F[_],I,O]( req: F[I])( recv: I => Process[F,O]): Process[F,O]
Stream primitives
SLIDE 33
foo: F[A] Process.eval(foo): Process[F,A]
SLIDE 34
foo: F[A] await(foo)(emit): Process[F,A]
SLIDE 35
Process.eval( Task.delay(readLine) ): Process[Task,String]
SLIDE 36
def IO[A](a: => A): Process[Task,A] = Process.eval(Task.delay(a))
SLIDE 37
Combining Processes
SLIDE 38
p1: Process[F,A] p2: Process[F,A] val p3: Process[F,A] = p1 ++ p2
SLIDE 39
p1: Process[F,A] p2: Process[F,A] val p3: Process[F,A] = p1 append p2
SLIDE 40
val twoLines: Process[Task,String] = IO(readLine) ++ IO(readLine)
SLIDE 41
val stdIn: Process[Task,String] = IO(readLine) ++ stdIn
SLIDE 42
val stdIn: Process[Task,String] = IO(readLine).repeat
SLIDE 43
val cat: Process[Task,Unit] = stdIn flatMap { s => IO(println(s)) }
SLIDE 44
val cat: Process[Task,Unit] = for { s <- stdIn _ <- IO(println(s)) } yield ()
SLIDE 45
def grep(r: Regex): Process[Task,Unit] = { val p = r.pattern.asPredicate.test _ def out(s: String) = IO(println(s)) stdIn filter p flatMap out }
SLIDE 46
Running Processes
SLIDE 47
F: Monad p: Process[F,A] p.run: F[Unit]
SLIDE 48
p: Process[F,A] p.runLog: F[List[A]]
SLIDE 49
p: Process[F,A] B: Monoid f: A => B p runFoldMap f: F[B]
SLIDE 50
Pipes
SLIDE 51
Process.await1[A]: Process1[A,A]
SLIDE 52
def take[I](n: Int): Process1[I,I] = if (n <= 0) halt else await1[I] ++ take(n - 1)
SLIDE 53
as: Process[F,A] p: Process1[A,B] as pipe p: Process[F,B]
SLIDE 54
as: Process[F,A] val p = process1.chunk(10) as pipe p: Process[F,Vector[A]]
SLIDE 55
as: Process[F,A] as.chunk(10): Process[F,Vector[A]]
SLIDE 56
def distinct[A]: Process1[A,A] = { def go(seen: Set[A]): Process1[A,A] = Process.await1[A].flatMap { a => if (seen(a)) go(seen) else Process.emit(a) ++ go(seen + a) } go(Set.empty) }
SLIDE 57
Process1[A,B] ~= Process[(A=>?),O]
SLIDE 58
Multiple sources
SLIDE 59
scalaz.stream.tee
SLIDE 60
val f1 = scalaz.stream.io.linesR("/tmp/foo.txt") val f2 = scalaz.stream.io.linesR("/tmp/bar.txt") type Source[A] = Process[Task,A] f1 zip f2: Source[(String,String)] f1 interleave f2: Source[String] f1 until f2.map(_ == "stop"): Source[String]
SLIDE 61
f1 zip f2 f1 interleave f2 f1 until f2.map(_ == "stop")
SLIDE 62
f1.tee(f2)(tee.zip) f1.tee(f2)(tee.interleave) f1.map(_ == "stop").tee(f2)(tee.until)
SLIDE 63
as: Process[F,A] bs: Process[F,B] t: Tee[A,B,C] (as tee bs)(t): Process[F,C]
SLIDE 64
val add: Tee[Int,Int,Int] = { for { x <- awaitL[Int] y <- awaitR[Int] } yield x + y }.repeat val sumEach = (p1 tee p2)(add)
SLIDE 65
Tee[A,B,O] ~= Process[λ[x] = (A=>x) \/ (B=>x), O]
SLIDE 66
scalaz.stream.wye
SLIDE 67
val f1 = IO(System.in.read).repeat val f2 = io.linesR("/tmp/foo.txt") type Source[A] = Process[Task,A] f1 either f2: Source[Int \/ String] f1.map(_.toChar.toString) merge f2: Source[String] f1.map(_ => true))(f2)(wye.interrupt): Source[String]
SLIDE 68
as: Process[F,A] bs: Process[F,B] y: Wye[A,B,C] (as wye bs)(y): Process[F,C]
SLIDE 69
Wye[A,B,O] ~= Process[λ[x] = (A=>x, B=>x, (A,B)=>x), O]
SLIDE 70
scalaz.stream.merge
SLIDE 71
ps: Process[F,Process[F,A]] merge.mergeN(ps): Process[F,A]
SLIDE 72
nondeterminism.njoin(maxOpen, maxQueued)(ps)
SLIDE 73
Sinks
SLIDE 74
x : Process[F,A] y : Sink[F,A] x to y : Process[F,Unit]
SLIDE 75
import scalaz.stream.io io.stdInLines: Process[Task,String] io.stdOutLines: Sink[Task,String] val cat = io.stdInLines to io.stdOutLines
SLIDE 76
A sink is just a stream of functions
SLIDE 77
type Sink[F[_],A] = Process[F, A => Task[Unit]]
SLIDE 78
val stdOut: Sink[Task,String] = IO { s => Task.delay(println(s)) }.repeat
SLIDE 79
Channels
SLIDE 80
x : Process[F,A] y : Channel[F,A,B] x through y : Process[F,B]
SLIDE 81
A channel is just a stream of functions
SLIDE 82
type Channel[F[_],A,B] = Process[F, A => F[B]]
SLIDE 83
type Sink[F[_],A] = Channel[F,A,Unit]
SLIDE 84
s: java.io.InputStream io.chunkR(s): Channel[Task,Int,ByteVector]
SLIDE 85
scalaz.stream.async
SLIDE 86
Queues & Signals
SLIDE 87
trait Queue[A] { ... def enqueue: Sink[Task,A] def dequeue: Process[Task,A] ... }
SLIDE 88
import scalaz.stream.async._ def boundedQueue[A](n: Int): Queue[A] def unboundedQueue[A]: Queue[A] def circularBuffer[A](n: Int): Queue[A]
SLIDE 89
val pool = java.util.concurrent.Executors.newFixedThreadPool(16) implicit val S = scalaz.concurrent.Strategy.Executor(pool)
SLIDE 90
trait Signal[A] { ... def get: Task[A] def set(a: A) Task[Unit] ... }
SLIDE 91
trait Signal[A] { ... def discrete: Process[Task,A] def continuous: Process[Task,A] ... }
SLIDE 92
Demo: Internet Relay Chat
SLIDE 93