Migrating to Scala 2.13 Ju Julien Richar ard-Fo Foy , Scala Center - - PowerPoint PPT Presentation

Migrating to Scala 2.13

Ju Julien Richar ard-Fo Foy, Scala Center St Stefan Zeiger, Lightbend

Scala 2.13

• Simplifying the collections
• Compiler performance
• Modularizing the standard library
• User-friendliness

https://github.com/scala/scala-dev/issues/324

Scala 2.13 Roadmap

• Scala 2.13.0-M4 released one month ago during ScalaDays Berlin
• First milestone with the new collections library
• Final milestone release: M5 (August 10)
• Minor API changes
• Bug fixes
• Performance improvements
• Compatibility improvements

Current Status

• Project that works on Scala 2.12
• Targeting Scala 2.13.0-M4 or higher
• sbt build (for examples shown)

Starting Point

Prepare for migration

• Remove deprecated calls
• Many deprecated APIs were removed in 2.13 (e.g. JavaConversions)
• A clean build without deprecations on 2.12 makes the migration easier
• build.sbt:

scalacOptions in Compile += "-deprecation"

Clean up on 2.12: Deprecations

• Compiler flags that modify the language are being removed in 2.13
• Only -Xsource from now on
• Compile without the following flags:
• -Yno-adapted-args
• -Xstrict-inference
• -Xfull-lubs
• -Yoverride-objects
• -Yoverride-vars
• -Yinfer-argument-types
• -Yvirtpatmat
• See https://github.com/scala/scala/pull/6505

Clean up on 2.12: Compiler flags

• Argument adaptation cannot be turned off selectively any more
• -Yno-adapted-args removed
• Dotty has more restrictive adaptation
• Goal: Always allow safe adaptation, prohibit unsafe (accidental) cases
• Scala 2.14 will align with Dotty rules
• Automatic eta-expansion of zero-arg methods already disabled in 2.13

under -Xsource:2.14 (see https://github.com/scala/scala/pull/6475)

• But more aggressive eta-expansion of other methods

Clean up on 2.12: Argument adaptation / Auto-tupling

• scala-library-all has been removed
• Depend on the required modules individually
• Parallel collections have been moved into a separate module
• Add a dependency to scala-parallel-collections on 2.13 and see

https://github.com/scala/scala-parallel-collections/issues/22 for cross- building

• Not available for M4, will be published again for M5
• scala-xml is no longer a transitive dependency of scala-compiler
• Not used by scaladoc anymore
• Depend on it directly if necessary

Clean up on 2.12: Modules

Build on 2.13

crossScalaVersions := Seq("2.13.0-M4", "2.12.6") scalaVersion := crossScalaVersions.value.head

Cross-building: Basic setup

scalacOptions ++= (CrossVersion.partialVersion(scalaVersion.value) match { case Some((2, n)) if n >= 13 => Seq("-Xsource:2.14") case _ => Seq("-Yno-adapted-args") })

Cross-building: Compiler options

• Expect some breakage due to the new collections library
• Singleton types and minor type inference changes can influence types and

implicit resolution in corner cases

• Other small changes in source compatibility (see https://github.com/scala/scala-

dev/issues/470)

• Try -Xsource:2.12

Build on 2.13

• The new collections library is mostly source compatible
• Many old methods and types are deprecated
• e.g. Traversable, TraversableOnce, Stream, ...
• Some features could not be added back in a 2.12-compatible way
• Instead scala-collection-compat provides the 2.13 syntax on 2.11 & 2.12
• https://github.com/scala/scala-collection-compat/

scala-collection-compat

• build.sbt:

libraryDependencies += "org.scala-lang.modules" %% "scala-collection-compat" % "0.1.0"

• Old Scala code:

xs.to[List]

• New Scala code:

import scala.collection.compat._ xs.to(List)

scala-collection-compat: Example

New collections

• This is not the topic of this talk, but in short:
• Simpler user-facing API (no CanBuildFrom)
• Correct operation implementations for non-strict collections
• Simpler internal hierarchy (no Gen… types)

Goals of the collections redesign

• scala.Seq is now scala.collection.immutable.Seq
• Same for IndexedSeq
• Consistent with Set and Map
• Decide on a case-by-case basis which one to use
• Use s.c.Seq or s.c.i.Seq explicitly when cross-building

Major collection incompatibilities: Immutable Seq

• Varargs use scala.Seq, so also immutable now
• Wrapped Java varargs pretend the array is immutable
• Using the new s.c.i.ArraySeq
• You can do the same with ArraySeq.unsafeWrapArray
• A deprecated implicit conversion makes a copy of the array

Major collection incompatibilities: Immutable Seq

method copyArrayToImmutableIndexedSeq in class LowPriorityImplicits2 is deprecated (since 2.13.0): Implicit conversions from Array to immutable.IndexedSeq are implemented by copying; Use the more efficient non-copying ArraySeq.unsafeWrapArray or an explicit toIndexedSeq call

• CanBuildFrom replaced by BuildFrom
• BuildFrom requires an instance of the source collection:

trait BuildFrom[-From, -A, +C] { def fromSpecificIterable(from: From)(it: Iterable[A]): C def newBuilder(from: From): Builder[A, C] }

• Collection methods like flatMap no longer need it
• Overloaded to produce different results depending on source collection type
• Used in methods like Future.sequence

Major collection incompatibilities: CanBuildFrom

• Factory allows target type-driven building (like CanBuild in 2.12)
• Used in Iterable.to
• Like BuildFrom but does not use a source collection:

trait Factory[-A, +C] { def fromSpecific(it: IterableOnce[A]): C def newBuilder: Builder[A, C] }

• General rule: Use BuildFrom to rebuild with the best matching type of an

existing source collection, otherwise use Factory

Major collection incompatibilities: CanBuildFrom

• Without CanBuildFrom, there is no breakOut
• Use an Iterator: xs.iterator.map(…).to(Vector)
• Methods with BuildFrom or Factory can be called with a companion object

instead (via implicit conversion to BuildFrom / Factory):

• Old:

val xs: List[Future[Int]] = … Future.sequence(xs)(breakOut, implicitly): Future[Vector[Int]]

• New:

val xs: List[Future[Int]] = … Future.sequence(xs)(Vector, implicitly)

Major collection incompatibilities: breakOut

Doesn't actually work in 2.12

• Views in the new collections are reified Iterator operations
• View vs Iterator is similar to Stream vs Spliterator in Java 8

streams

• Views don't remember source collection types
• Use an explicit to… operation instead of force to build the desired type
• mapValues and filterKeys on Map now return MapView
• It was always a lazy View-like object but pretended to be a Map
• Add .toMap if necessary

Major collection incompatibilities: Views

• Only very simple collection implementations are source compatible
• General rule: Use two different source files in different source directories to

cross-build

• Type hierarchy simplified
• Less boilerplate
• All methods have alphabetic names, symbolic operators are aliases
• E.g. override concat instead of ++
• Overloading instead of CanBuildFrom

Major collection incompatibilities: Custom collections

crossScalaVersions := Seq("2.13.0-M4", "2.12.6") scalaVersion := crossScalaVersions.value.head // Add src/main/scala-2.13+ for Scala 2.13 and newer // and src/main/scala-2.12- for Scala versions older than 2.13 unmanagedSourceDirectories in Compile += { val sourceDir = (sourceDirectory in Compile).value CrossVersion.partialVersion(scalaVersion.value) match { case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+" case _ => sourceDir / "scala-2.12-" } }

Cross-building: Source directories

sbt already gives you:

• scala
• scala-2.12
• scala-2.13.0-M4

• See https://github.com/scala/collection-strawman/wiki/FAQ for a more

comprehensive list

Major collection incompatibilities

Automated migration

2.12 2.13

Scalafix

manual work

Migrating an application

Rules under development – more in M5

• plugins.sbt:

addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.5.7")

• From your shell:

$ sbt > scalafixEnable > scalafix github:scala/scala-collection- compat/NewCollections

• build.sbt:

scalaVersion := "2.13.0-M4"

• And then:

> ;reload ;compile

• Documentation: https://github.com/scala/scala-collection-compat

Running the migration rules

Step 1: Add sbt-scalafix to your project Step 2: Run the migration rule Step 3: Update the scalaVersion

• Renamings
• Stream(1, 2, 3).append(Stream(4, 5, 6))
• LazyList(1, 2, 3).lazyAppendedAll(LazyList(4, 5, 6))
• Expression rewritings
• xs.copyToBuffer(b)
• b ++= xs
• The more complex the expression, the harder to implement the rewrite rule

Scope of the migration rules

Has a lazy head

• Custom collection implementations
• Advanced usage of CanBuildFrom
• scala.Seq usage
• Comprehensive list of supported rewrites:

https://github.com/scala/collection-strawman/wiki/FAQ

Not in the scope of the migration rules

2.11 2.12 2.11 2.12 2.13

Scalafix

scala-collection-compat

manual work

Migrating a library

Rules not yet implemented

• "The Architecture of Scala 2.13’s Collections": https://docs.scala-

lang.org/overviews/core/architecture-of-scala-213-collections.html

• Demo project for cross-building:

https://github.com/szeiger/new-collections-demo

• FAQ: https://github.com/scala/collection-strawman/wiki/FAQ
• Previous talks on the new collections:
• The new collections library for Scala 2.13 and Dotty
• The next() collections
• @julienrf
• @StefanZeiger

Links