Compilation and optimization pipeline

From Scala source files to optimized JavaScript code, there are a few steps which are described in this document.

  • compiling: compile .scala files to one .sjsir file per class using the scalac compiler with the Scala.js compiler plugin.

For libraries, it stops here. The compiled .sjsir files are put together with .class files in the binary jars that can be published to an Ivy or Maven repository.

For the application project, one more mandatory step creates one or three .js files consumable by browsers. Then an optional step can be used to optimize the result.

  • linking all the .sjsir files together, including those in transitive dependencies. This is either one of:
    • fastoptimizing: apply a type-aware inter-method global dead code elimination that results in one preoptimized .js file.
    • packaging: concatenate blindly all compiled .sjsir files in three big .js files. This used to be the default in earlier versions of Scala.js. Packaging is discouraged and might fall away in future versions since the fast optimization is now faster.
  • optimizing (optional, with the Closure Compiler): apply the Closure Compiler to the linked .js file(s).

Compilation

Compilation is similar to using the Scala compiler daily. .scala source files are compiled with incremental compilation by the Scala compiler. They can depend on external libraries or other projects in the build, as usual.

The Scala.js compiler plugin produces .sjsir files in addition to the .class files generated by scalac. Roughly one .sjsir file is produced for each .class file. (Sometimes, fewer .sjsir files are necessary when the compiler can optimize anonymous function classes as JavaScript functions.)

An .sjsir file contains a Scala.js specific intermediate representation. You may use scalajsp from the Scala.js CLI to display it in a human readable form.

The .sjsir files are bundled together with .class files in jars published on Ivy or Maven repositories.

This step completely supports separate compilation and incremental compilation, just like the regular Scala compiler.

Fast Optimizing

The size of the JavaScript files produced by blindly concatenating .sjsir files are huge. They are always bigger than 20 MB because they include the full Scala standard library. But typically, only a small fraction of the libraries you depend on are actually used.

Fast optimizing is a task that identifies all the classes and methods reachable from a given set of entry points, and removes everything else. Entry points are classes and methods that are exported to JavaScript.

The fast optimizer uses information stored by the compilation step in .sjsir to derive a reachability graph from the entry points. Then it produces a single .js file containing all the code that is actually useful (or, since in theory the algorithm computes an over-approximation, all the code that could not be proved to be useless).

In sbt, the fast optimizing task is called fastOptJS. The result of fast optimization is typically between 1.5 MB and 2.5 MB.

The fast optimizer can also be called programmatically using the class ScalaJSOptimizer in the Scala.js toolbox.

Packaging

Packaging is discouraged, since fast optimizing has become faster. This section is for information only, as packaging can be useful for very specific use-cases.

Packaging is the simplest form of linking. Conceptually, it simply takes the JavaScript code in the .sjsir files generated by the compilation step for all the transitive dependencies of the project and concatenates them. There are however two subtleties: a decomposition in three files for performance, and an ordering for correctness.

In sbt, the packaging task is called packageJS.

Decomposition in three files

Concatenating everything every time one changes one source file in the project takes time, because this is a heavily I/O-bound operation. However, typically most of the code is concentrated in the external dependencies of the project (in sbt terminology, i.e., the libraries referenced in libraryDependencies), and these tend to change rarely (definitely not in a quick edit-compile-reload cycle). Next, much less code is located in the internal dependencies, i.e., the other projects in the same build referenced via dependsOn). Finally, the code that changes on every single dev cycle are in the project itself, which are the exported products of the project.

The packaging takes advantage of this obversation by producing three .js files instead of just one: app-pack-extdeps.js contains the external dependencies, app-pack-intdeps.js contains the internal dependencies, and app-pack-app.js contains the exported products. app-pack-extdeps.js is typically very large (the Scala standard library itself accounts for more than 20 MB of JavaScript code), but need not be repackaged often. app-pack-app.js has to be repackaged on every single dev cycle, but contains relatively much fewer code, and so this is fast.

Ordering

The internal structure of the produced .js files imposes a constraint on the order in which they are packaged. A class must always be packaged after its superclass (and hence, transitively, all its superclasses).

To do so, the .sjsir file contains (among other things), the number of class ancestors of each class. Since a subclass A of a superclass B has at least one more ancestor than B (namely, B), it is sufficient, to guarantee the required ordering, that classes with more ancestors are ordered after classes with less ancestors.

Optimizing using Closure

Scala.js emits code that is always guaranteed to follow the restrictions imposed by the Advanced Optimizations of the Google Closure Compiler. Hence, we can optimize the linked JavaScript files (either fast optimized or packaged) using Closure.

In sbt, the optimizing task is called fullOptJS and is applied on the result of fastOptJS. The result optimization is typically between 150 KB and a few hundreds of KB.

The optimizer can also be called programmatically using the class ScalaJSClosureOptimizer in the Scala.js toolbox.