Algebraic Effects on the JVM Jonathan Immanuel Brachthuser - PowerPoint PPT Presentation

Ξ Effekt Calling an Effect OperaOon b-studios.de/scala-effekt 16 0

Ξ Effekt Calling an Effect OperaOon We can think of effect operaKons as uninterpreted constructors of an effect-language. An effec[ul program then could be represented as a b-studios.de/scala-effekt tree of operaKons: 16 O p 1 (args…, res 1 ⟹ 0 Op 2 (args…, res 2 ⟹ … Pure(value)))

Ξ Effekt Calling an Effect OperaOon We can think of effect operaKons as uninterpreted constructors of an effect-language. An effec[ul program then could be represented as a b-studios.de/scala-effekt tree of operaKons: 16 O p 1 (args…, res 1 ⟹ 0 Op 2 (args…, res 2 ⟹ … Pure(value))) we can write a recursive, paMern matching recursive interpreter to provide semanKcs to effec[ul operaKons. In PL terms: a deep embedding of effect operaKons.

Ξ Effekt Shallow Embedding of Effect Handlers In Scala Effekt, effect operaKons are immediately called on effect handlers. Schema1cally : b-studios.de/scala-effekt handler. op 1 (args…, res 1 ⟹ 17 handler. op 2 (args…, res 2 ⟹ 0 …))

Ξ Effekt Shallow Embedding of Effect Handlers In Scala Effekt, effect operaKons are immediately called on effect handlers. Schema1cally : b-studios.de/scala-effekt handler. op 1 (args…, res 1 ⟹ 17 handler. op 2 (args…, res 2 ⟹ 0 …)) Technical Insights

Ξ Effekt Shallow Embedding of Effect Handlers In Scala Effekt, effect operaKons are immediately called on effect handlers. Schema1cally : b-studios.de/scala-effekt handler. op 1 (args…, res 1 ⟹ 17 handler. op 2 (args…, res 2 ⟹ 0 …)) Technical Insights (a) Shallow embedding of effect handlers simplifies typing – 
 no GADTs are necessary!

Ξ Effekt Shallow Embedding of Effect Handlers In Scala Effekt, effect operaKons are immediately called on effect handlers. Schema1cally : b-studios.de/scala-effekt handler. op 1 (args…, res 1 ⟹ 17 handler. op 2 (args…, res 2 ⟹ 0 …)) Technical Insights (a) Shallow embedding of effect handlers simplifies typing – 
 no GADTs are necessary! (b) PaMern matching is replaced by dynamic dispatch – 
 benefits performance on the JVM.

Ξ Effekt Shallow Embedding of Effect Handlers In Scala Effekt, effect operaKons are immediately called on effect handlers. Schema1cally : b-studios.de/scala-effekt handler. op 1 (args…, res 1 ⟹ 17 handler. op 2 (args…, res 2 ⟹ 0 …)) Technical Insights (a) Shallow embedding of effect handlers simplifies typing – 
 no GADTs are necessary! (b) PaMern matching is replaced by dynamic dispatch – 
 benefits performance on the JVM. (c) Direct call to corresponding handler – no need to lookup handler.

Ξ Effekt Part II Algebraic b-studios.de/scala-effekt Effects as 18 0 Libraries for Java / JVM

Ξ Effekt Key Specs: JVM / Java Effekt - shallow embedding vs. deep embedding of handlers - "handler passing style" b-studios.de/scala-effekt - shallow handlers vs. deep handlers 19 0 - user defined effects ✔ - dynamic effect instances ✔ - modular and extensible effect signatures and handlers ( ✔ ) - safety (capabiliKes can leak) ✘ - user programs are wriMen in direct style ✔ - performance: compeKKve with JVM conKnuaKon libraries ✔

Ξ Effekt Key Specs: JVM / Java Effekt - shallow embedding vs. deep embedding of handlers - "handler passing style" b-studios.de/scala-effekt - shallow handlers vs. deep handlers 19 0 - user defined effects ✔ - dynamic effect instances ✔ - modular and extensible effect signatures and handlers ( ✔ ) - safety (capabiliKes can leak) ✘ - user programs are wriMen in direct style ✔ - performance: compeKKve with JVM conKnuaKon libraries ✔

Ξ Effekt Overview of JVM Effekt b-studios.de/scala-effekt 20 0 Programs are wriMen in direct style, but CPS translated via bytecode transformaOon - Translated programs use a separate Stack interface for effec[ul frames - Delimited control is implemented as a library, implemenKng the Stack interface - We redesigned the algebraic effects library to only require simple generics - RestricOon : We only transform the terms, not types / signatures -

Ξ Effekt Replacing the JVM Stack For effec1ul methods, we maintain our own custom stack, which allows us to manipulate it (searching, slicing, copying). b-studios.de/scala-effekt 21 Example.plain:27 Example.effectOp2:13 Example.plain:38 Example.effectOp1:5 .plain:27 Example.plain:29 Effekt.run:13 .effectOp2:13 Example.main:55 .main:55 .effectOp1:5 JVM Stack JVM Stack Effekt Stack

Ξ Effekt Example: Drunk Coin Flipping b-studios.de/scala-effekt 22 String drunkFlip(Amb amb, Exc exc) throws Effects { 00 if (amb.flip()) { return exc.raise("too drunk"); } else { return amb.flip() ? "heads" : "tails"; } }

Ξ Effekt Example: Drunk Coin Flipping interface Amb { boolean flip() throws Effects; } interface Exc { <A> A raise(String msg) throws Effects; } b-studios.de/scala-effekt 22 String drunkFlip(Amb amb, Exc exc) throws Effects { 00 if (amb.flip()) { return exc.raise("too drunk"); } else { return amb.flip() ? "heads" : "tails"; } }

Ξ Effekt Handling Effects class AmbList<R> extends Handler<R, List<R>> implements Amb { List<R> pure(R r) { return Lists.singleton(r); } b-studios.de/scala-effekt boolean flip() throws Effects { 23 return use(k -> Lists.concat(k.resume( true ), k.resume( false ))); 00 } } handle( new AmbList<Optional<String>>(), amb -> handle( new Maybe<String>(), exc -> drunkFlip(amb, exc))) > [Optional["heads"], Optional["tails"], Optional.empty]

Ξ Effekt Stateful / Parametrized Handlers interface Reader<In> { In read() throws Effects; } b-studios.de/scala-effekt class StringReader<R> extends Handler<R, R> implements Reader<Char> { 24 00 final String input; int pos = 0; Char read() throws Effects { return input.charAt(pos++) } }

Ξ Effekt Stateful / Parametrized Handlers interface Reader<In> { In read() throws Effects; } b-studios.de/scala-effekt class StringReader<R> extends Handler<R, R> implements Reader<Char>, 25 Stateful<Integer> { 00 final String input; int pos = 0; Char read() throws Effects { return input.charAt(pos++) } Integer exportState() { return pos; } void importState(Integer n) { pos = n; } }

Ξ Effekt Design Decisions - EffecQul methods are marked with a special, checked excepKon Effects - Effect signatures are interfaces that contain effec[ul methods b-studios.de/scala-effekt - Effect handlers are implementaKons of those interfaces. 26 0 - Users need to manually follow the capability passing style . - Effect handlers can extend the library class Handler to capture the conOnuaOon (but don't need to). - We use the handler instances as prompt markers .

Ξ Effekt Bytecode TransformaOon Example (CPS) String drunkFlip(Amb amb, Exc exc) throws Effects { Effekt.push(() -> drunkFlip1(amb, exc)); amb.flip(); b-studios.de/scala-effekt return null; } 27 0

Ξ Effekt Bytecode TransformaOon Example (CPS) String drunkFlip(Amb amb, Exc exc) throws Effects { Effekt.push(() -> drunkFlip1(amb, exc)); amb.flip(); b-studios.de/scala-effekt return null; } 27 void drunkFlip1(Amb amb, Exc exc) throws Effects { 0 boolean caught = Effekt.result(); if (Effekt.result()) { exc.raise("too drunk"); } else { Effekt.push(() -> drunkFlip2(amb, exc, caught)); amb.flip(); } }

Ξ Effekt Bytecode TransformaOon Example (CPS) String drunkFlip(Amb amb, Exc exc) throws Effects { Effekt.push(() -> drunkFlip1(amb, exc)); amb.flip(); b-studios.de/scala-effekt return null; } 27 void drunkFlip1(Amb amb, Exc exc) throws Effects { 0 boolean caught = Effekt.result(); if (Effekt.result()) { exc.raise("too drunk"); } else { Effekt.push(() -> drunkFlip2(amb, exc, caught)); amb.flip(); } } void drunkFlip2(Amb amb, Exc exc, boolean caught) throws Effects { Effekt.returnWith(Effekt.result() ? "heads" : "tails"); }

Ξ Effekt AlternaOve TransformaOons CPS Effekt.push(() -> drunkFlip1(amb, exc)); b-studios.de/scala-effekt amb.flip(); 28 return DUMMY ; 0

Ξ Effekt AlternaOve TransformaOons CPS Gen. Stack InspecOon / Bubble Sem. Effekt.push(() -> drunkFlip1(amb, exc)); b-studios.de/scala-effekt Effekt.beforeCall(); amb.flip(); amb.flip(); 28 return DUMMY ; if (Effekt.isImpure()) { 0 Effekt.push(() -> drunkFlip1(amb, exc)); return DUMMY; }

Ξ Effekt AlternaOve TransformaOons CPS Gen. Stack InspecOon / Bubble Sem. Effekt.push(() -> drunkFlip1(amb, exc)); b-studios.de/scala-effekt Effekt.beforeCall(); amb.flip(); amb.flip(); 28 return DUMMY ; if (Effekt.isImpure()) { 0 Effekt.push(() -> drunkFlip1(amb, exc)); return DUMMY; } all effect calls are tail calls - cont. is constructed eagerly and - immediately available unnecessary push/pop/enter cycles - full reificaKon of the stack -

Ξ Effekt AlternaOve TransformaOons CPS Gen. Stack InspecOon / Bubble Sem. Effekt.push(() -> drunkFlip1(amb, exc)); b-studios.de/scala-effekt Effekt.beforeCall(); amb.flip(); amb.flip(); 28 return DUMMY ; if (Effekt.isImpure()) { 0 Effekt.push(() -> drunkFlip1(amb, exc)); return DUMMY; } all effect calls are tail calls two ways to leave a method, - - disKnguished by a flag cont. is constructed eagerly and - immediately available cont. is constructed on demand - unnecessary push/pop/enter cycles reduced overhead for pure code - - full reificaKon of the stack prompt markers are trampolines - -

Ξ Effekt AlternaOve TransformaOons (Performance) b-studios.de/scala-effekt 29 0

Ξ Effekt AlternaOve TransformaOons (Performance) b-studios.de/scala-effekt 29 0

Ξ Effekt AlternaOve TransformaOons (Performance) b-studios.de/scala-effekt 29 0 CorouKnes (hMps://github.com/ocynull/corouKnes) - Quasar (hMp://docs.paralleluniverse.co/quasar) - Javaflow (hMps://github.com/vsilaev/tascalate-javaflow) - Eff (hMps://github.com/atnos-org/eff) -

Ξ Effekt Part III Even More b-studios.de/scala-effekt Extensible 30 0 Effects

Ξ Effekt The (Effect) Expression Problem Original Expression Problem Effect Expression Problem b-studios.de/scala-effekt vs. Variant of a Datatype Effect OperaKon 31 vs. (Recursive) OperaKon Handler ImplementaKon 0

Ξ Effekt The (Effect) Expression Problem Original Expression Problem Effect Expression Problem b-studios.de/scala-effekt vs. Variant of a Datatype Effect OperaKon 31 vs. (Recursive) OperaKon Handler ImplementaKon 0 We rephrase the expression problem in context of algebraic effects as: Modularly being able to a) implement new handlers for an effect signature. b) add new effect opera2ons to an exis1ng effect signature

Ξ Effekt Extensibility supported by Effekt b-studios.de/scala-effekt 32 0

Ξ Effekt Extensibility supported by Effekt a) implement new handlers for an effect signature. trait ExcOption[ R ] extends Exc with Handler[ R , Option[ R ]] { … } b-studios.de/scala-effekt trait ExcEither[ R ] extends Exc with Handler[ R , Either[String, R ]] { … } 32 0

Ξ Effekt Extensibility supported by Effekt a) implement new handlers for an effect signature. trait ExcOption[ R ] extends Exc with Handler[ R , Option[ R ]] { … } b-studios.de/scala-effekt trait ExcEither[ R ] extends Exc with Handler[ R , Either[String, R ]] { … } 32 0 b) add new effect opera2ons … … by adding a new signature (like Amb and Exc) … by adding operaKons to a signature

Ξ Effekt Extensibility supported by Effekt a) implement new handlers for an effect signature. trait ExcOption[ R ] extends Exc with Handler[ R , Option[ R ]] { … } b-studios.de/scala-effekt trait ExcEither[ R ] extends Exc with Handler[ R , Either[String, R ]] { … } 32 0 b) add new effect opera2ons … … by adding a new signature (like Amb and Exc) … by adding operaKons to a signature trait AmbChoose extends Amb { def choose [ A ]( choices : List[ A ]): Op [ A ] } trait AmbChooseList[ R ] extends AmbChoose with AmbList[ R ] { def choose [ A ]( choices : List[ A ]): Op [ A ] = … }

Ξ Effekt Extensibility supported by Effekt (2) Handling two effects with one handler: trait ExcList[ R ] extends Exc with Handler[ R, List[ R ]] { b-studios.de/scala-effekt def raise [ A ]( msg : String): Op [ A ] = resume ⟹ pure(List.empty) 33 } 0 trait ExcAmbList[ R ] extends ExcList[R] with AmbList[R] {} ExcAmbList { drunkFlip } > List("heads", "tails")

Ξ Effekt Extensibility supported by Effekt (2) Handling two effects with one handler: trait ExcList[ R ] extends Exc with Handler[ R, List[ R ]] { b-studios.de/scala-effekt def raise [ A ]( msg : String): Op [ A ] = resume ⟹ pure(List.empty) 33 } 0 trait ExcAmbList[ R ] extends ExcList[R] with AmbList[R] {} ExcAmbList { drunkFlip } > List("heads", "tails") Desugares to: ExcAmbList { both ⟹ drunkFlip(both, both) }

Ξ Effekt Part IV Effect Typing b-studios.de/scala-effekt and OO: 34 00 A Problem Statement

Ξ Effekt Object Oriented Programming The mantra of OOP: - subtyping and Liskov's subsKtuKon principle b-studios.de/scala-effekt - hiding implementaOon details behind interfaces 35 - implementaKon is existenOally hidden - InformaKon hiding happens on the granularity of single objects

Ξ Effekt Subtyping & InformaOon Hiding trait Person { b-studios.de/scala-effekt def greet(): Unit } 36 0 trait IOPerson extends Person { def greet(): Unit using Console } trait AlertPerson extends Person { def greet(): Unit using GUI }

Ξ Effekt Subtyping & InformaOon Hiding trait Person { b-studios.de/scala-effekt def greet(): Unit } 37 0 trait IOPerson extends Person { Not a subtype! def greet(): Unit using Console } trait AlertPerson extends Person { def greet(): Unit using GUI }

Ξ Effekt SoluOon A_empt 1 Users of Person now also need to be trait Person[ E ] { b-studios.de/scala-effekt effect polymorphic! def greet(): Unit using E } def user[ E ](p: Person[ E ]) : Int using E 38 0 trait IOPerson extends Person[Console] trait AlertPerson extends Person[GUI]

Ξ Effekt SoluOon A_empt 1 trait Person { b-studios.de/scala-effekt Effect types are now path dependent: type E def greet(): Unit using E def user( p : Person) : Int using p.E 39 } 0 Only works for stable trait IOPerson extends Person { type E = Console values! } trait AlertPerson extends Person { type E = GUI }

Ξ Effekt SoluOon A_empt 2 trait Person { The effect now is truly hidden b-studios.de/scala-effekt def greet(): Control[Unit] def user(p: Person) : Control[Int] } 40 0 trait IOPerson extends Person { implicit val console: Cap[Console] } trait AlertPerson extends Person { implicit val gui: Cap[GUI] }

Ξ Effekt Capability Safety b-studios.de/scala-effekt def leaking ( implicit amb : Cap[Amb]): Control[String] = { 41 0 pure("hello world") }

Ξ Effekt Capability Safety var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit amb : Cap[Amb]): Control[String] = { c = amb; 42 0 pure("hello world") }

Ξ Effekt Capability Safety var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit amb : Cap[Amb]): Control[String] = { c = amb; 42 0 pure("hello world") } AmbList { leaking }.run()

Ξ Effekt Capability Safety var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit amb : Cap[Amb]): Control[String] = { c = amb; 42 0 pure("hello world") } AmbList { leaking }.run() { flip()(c) }.run()

Ξ Effekt Capability Safety var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit amb : Cap[Amb]): Control[String] = { c = amb; 43 0 pure("hello world") } { flip()(c) }.run() AmbList { leaking }.run()

Ξ Effekt Possible SoluOon Make (capability) objects second class again b-studios.de/scala-effekt 44 0

Ξ Effekt Second Class Values in Scala Escape var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit @local amb : Cap[Amb]): Control[String] = { c = amb; 45 0 pure("hello world") }

Ξ Effekt Second Class Values in Scala Escape var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit @local amb : Cap[Amb]): Control[String] = { c = amb; 45 0 pure("hello world") } Error: local value amb cannot be assigned to variable c since it would leave the scope of funcKon leaking .

Ξ Effekt Second Class Values in Scala Escape var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit @local amb : Cap[Amb]): Control[String] = { c = amb; 45 0 pure("hello world") } Error: local value amb cannot be assigned to variable c since it would leave the scope of funcKon leaking . Restricts scope of capabiliKes so that they can be stack allocated.

Ξ Effekt Second Class Values in Scala Escape var c: Cap[Amb] = null b-studios.de/scala-effekt def leaking ( implicit @local amb : Cap[Amb]): Control[String] = { c = amb; 45 0 pure("hello world") } Error: local value amb cannot be assigned to variable c since it would leave the scope of funcKon leaking . Restricts scope of capabiliKes so that they can be stack allocated. This perfectly fits algebraic effects.

Ξ Effekt SoluOon A_empt 2 trait Person { b-studios.de/scala-effekt def greet(): Control[Unit] } 46 0 trait IOPerson extends Person { implicit val console: Cap[Console] } trait AlertPerson extends Person { implicit val gui: Cap[GUI] }

Ξ Effekt SoluOon A_empt 2 trait Person { b-studios.de/scala-effekt def greet(): Control[Unit] } 46 Object lifetime < Capability lifetime 0 trait IOPerson extends Person { implicit val console: Cap[Console] } trait AlertPerson extends Person { implicit val gui: Cap[GUI] }

Ξ Effekt The Root of Evil There is a simple connecKon: - Algebraic effects and delimited conKnuaKons are all about the stack b-studios.de/scala-effekt - Object oriented programming is all about heap allocated objects 47 0 ConflicKng requirements: - capabiliKes should be stack allocated, objects don't - but object lifeKme should not be coupled to capability lifeKme - in parKcular, objects should be able to escape the handler scope 
 losing the capabiliKes

b-studios.de/scala-effekt 00 48 Ξ Effekt Syntax Effec[ul Part V

Towards NaturalisCc EDSLs using Algebraic Effects EffecQul Syntax LinguisKc phenomena like anaphora, scoping, quanKficaKon, implicature, focus and more can be modeled uniformly using algebraic effects. b-studios.de/scala-effekt LinguisKcs 49 00 Algebraic Effects Jiří Maršík and Maxime Amblard, 2016 PL

Towards NaturalisCc EDSLs using Algebraic Effects EffecQul Syntax LinguisKc phenomena like anaphora, scoping, quanKficaKon, implicature, focus and more can be modeled uniformly using algebraic effects. b-studios.de/scala-effekt LinguisKcs 49 00 EffecQul Syntax Algebraic Effects This Talk Jiří Maršík and Maxime Amblard, 2016 PL - Support linguisOc phenomena in EDSLs using algebraic effects - Use (algebraic) effects for AST construcOon

Towards NaturalisCc EDSLs using Algebraic Effects ○ Kny.cc/effec[ul-syntax Example 1: The Speaker Effect b-studios.de/scala-effekt val s 1 : Sentence using Speaker = john said { mary loves me } 50 0

Towards NaturalisCc EDSLs using Algebraic Effects ○ Kny.cc/effec[ul-syntax Example 1: The Speaker Effect Effect Signature Groups effect operaKons in a type b-studios.de/scala-effekt val s 1 : Sentence using Speaker = john said { mary loves me } Speaker me 51 0 Effect OperaKons SemanKcs of the operaKons is leP open

Towards NaturalisCc EDSLs using Algebraic Effects ○ Kny.cc/effec[ul-syntax Example 1: The Speaker Effect Effect Signature Groups effect operaKons in a type b-studios.de/scala-effekt val s 1 : Sentence using Speaker = john said { mary loves me } Speaker me 52 0 Effect OperaKons SemanKcs of the operaKons is leP open pete saidQuote { s 1 }

Towards NaturalisCc EDSLs using Algebraic Effects ○ Kny.cc/effec[ul-syntax Example 1: The Speaker Effect Effect Signature Groups effect operaKons in a type b-studios.de/scala-effekt val s 1 : Sentence using Speaker = john said { mary loves me } Speaker me 52 0 Effect Handlers Effect OperaKons Provide semanKcs to effect operaKons SemanKcs of the operaKons is leP open pete saidQuote { s 1 }

Towards NaturalisCc EDSLs using Algebraic Effects ○ Kny.cc/effec[ul-syntax Example 1: The Speaker Effect Effect Signature Groups effect operaKons in a type b-studios.de/scala-effekt val s 1 : Sentence using Speaker = john said { mary loves me } Speaker me 52 0 Effect Handlers Effect OperaKons Provide semanKcs to effect operaKons SemanKcs of the operaKons is leP open pete saidQuote { s 1 } > Said(Pete, Said(John, Loves(Mary, Pete)))

Towards NaturalisCc EDSLs using Algebraic Effects Example 2: The Scope Effect b-studios.de/scala-effekt val s 2 : Sentence using Scope = john saidQuote { every(woman) loves me } 53 0