chore: add the `scala-library` projects (non-bootstrapped and bootstrapped) #23510

hamzaremmal · 2025-07-11T13:13:21Z

Add the configuration for the scala-library project in sbt.
For now, this configuration is disabled and doesn't publish anything until we enable it in 3.8.0

hamzaremmal · 2025-07-11T13:26:07Z

project/Build.scala

+   *        - The only commands you should run on this project are `publish` and `publishLocal`
+   *        - This project produces no artifacts and has no sources
+   */
+  lazy val `scala-library` = project.in(file("scala-library"))


FYI: publishSettings are missing here, but I prefer to add them when we are ready to release

hamzaremmal · 2025-07-16T12:50:55Z

library/src/scala/Conversion.scala

-  @experimental
-  opaque type into[+T] >: T = T
+  //@experimental
+  //opaque type into[+T] >: T = T

  /** Unwrap an `into` */
-  extension [T](x: into[T])
-    @experimental def underlying: T = x
+  //extension [T](x: into[T])
+  //  @experimental def underlying: T = x


Obviously this must be fixed before merging the PR. This triggers a stale symbol only in the new stdlib...

hamzaremmal · 2025-07-18T13:02:26Z

project/Build.scala

+      version       := dottyVersion,
+      versionScheme := Some("semver-spec"),
+      crossPaths    := false, // org.scala-lang:scala-library doesn't have a crosspath
+      // NOTE: The only difference here is that we drop `-Werror` and semanticDB for now


semanticDB fails because the definitions are not present in Predef, which will be fixed in another PR and -Werror is self explanatory. Too many warnings for now that will have to be fixed later (some even in scala 2)

hamzaremmal · 2025-07-18T13:02:55Z

project/Build.scala

+      // Add the source directories for the stdlib (non-boostrapped)
+      Compile / unmanagedSourceDirectories := Seq(baseDirectory.value / "src"),
+      Compile / unmanagedSourceDirectories += baseDirectory.value / "src-non-bootstrapped",
+      // NOTE: The only difference here is that we drop `-Werror` and semanticDB for now


See https://github.com/scala/scala3/pull/23510/files#r2215995517 for details

hamzaremmal · 2025-07-18T13:03:57Z

project/Build.scala

+        val externalLibraryDeps = (`scala3-library` / Compile / externalDependencyClasspath).value.map(_.data).toSet
+        val externalCompilerDeps = (`scala3-compiler` / Compile / externalDependencyClasspath).value.map(_.data).toSet
+
+        // IMPORTANT: We need to use actual jars to form the ScalaInstance and not
+        // just directories containing classfiles because sbt maintains a cache of
+        // compiler instances. This cache is invalidated based on timestamps
+        // however this is only implemented on jars, directories are never
+        // invalidated.
+        val tastyCore = (`tasty-core` / Compile / packageBin).value
+        val scala3Library = (`scala3-library` / Compile / packageBin).value
+        val scala3Interfaces = (`scala3-interfaces` / Compile / packageBin).value
+        val scala3Compiler = (`scala3-compiler` / Compile / packageBin).value
+
+        val libraryJars = Array(scala3Library) ++ externalLibraryDeps
+        val compilerJars = Seq(tastyCore, scala3Interfaces, scala3Compiler) ++ (externalCompilerDeps -- externalLibraryDeps)
+
+        Defaults.makeScalaInstance(
+          scalaVersion.value,
+          libraryJars = libraryJars,
+          allCompilerJars = compilerJars,
+          allDocJars = Seq.empty,
+          state.value,
+          scalaInstanceTopLoader.value
+        )
+      },
+      scalaCompilerBridgeBinaryJar := {
+        Some((`scala3-sbt-bridge` / Compile / packageBin).value)
+      },


This still uses the old compiler that uses the old stdlib, in another PR, when I add the versions of the compiler with the new stdlib as a dependency, we will proceed to the swap.

hamzaremmal

Note that many other settings are required to be added before moving to prod. This is just one step to move forward in the build for now.

hamzaremmal · 2025-07-18T13:28:07Z

Also, I'm explicitly splitting and duplicating the settings in each project as want to know exactly the configuration and not to start follow methods to see what gets configured and what doesn't. This is a choice, not a bad implementation.

hamzaremmal · 2025-07-18T14:23:00Z

.github/workflows/stdlib.yaml

+        uses: actions/setup-java@v4
+        with:
+          distribution: 'temurin'
+          java-version: 17


We are already compiling the migration of stdlib with 17, ahead of the switch to Java 17 in 3.8.0

WojciechMazur · 2025-07-18T15:29:08Z

.github/workflows/stdlib.yaml

+permissions:
+  contents: read
+
+jobs:


Tests CI directives e.g. [skip_ci] or [compile_nonboostrapped] are missing, currently we have already multiple jobs that would execute even though they're told to not be executed. If possible we should gradually try to unify all of GHA jobs

I prefer not to have them for multiple reasons:

This is a temporary files

CI will have to be redesigned anyways (subsequent PR)

I prefer always checking this during the transition period (until we actually proceed to the swap). I don't want to have any surprises down the road.

project/Build.scala

WojciechMazur · 2025-07-18T15:47:54Z

project/Build.scala

+      Compile / unmanagedSourceDirectories := Seq(baseDirectory.value / "src"),
+      Compile / unmanagedSourceDirectories += baseDirectory.value / "src-bootstrapped",
+      // NOTE: The only difference here is that we drop `-Werror` and semanticDB for now
+      Compile / scalacOptions :=  Seq("-deprecation", "-feature", "-unchecked", "-encoding", "UTF8", "-language:implicitConversions"),


NIT: we're using here 2 ways of providing settings: using seperate strings ("-encoding", "UTF8"), and using a single argument with separator (-language:implicitConversions).
Maybe let's pick a single style and use it?
Personally I'm in favour of replacing -encoding:UTF8 change as it's easier to ready when all data is kept in single token.

WojciechMazur · 2025-07-18T15:52:28Z

project/Build.scala

+      Compile / scalacOptions := Seq("-deprecation", "-feature", "-unchecked", "-encoding", "UTF8", "-language:implicitConversions"),
+      (Compile / scalacOptions) ++= Seq(
+        // Needed so that the library sources are visible when `dotty.tools.dotc.core.Definitions#init` is called
+        "-sourcepath", (Compile / sourceDirectories).value.map(_.getCanonicalPath).distinct.mkString(File.pathSeparator),
+      ),
+      // Only publish compilation artifacts, no test artifacts
+      Compile / publishArtifact := true,
+      Test    / publishArtifact := false,
+      // Do not allow to publish this project for now
+      publish / skip := true,
+      // Project specific target folder. sbt doesn't like having two projects using the same target folder


I understand the argument of explicit, duplicated, configuration, but maybe let's create a Def.settings directly above these project definitions with the common parts that are known to be always shared between the two?
Otherwise, sooner or later these 2 would diverge.

So stuff like the marked settings, crossPaths, versionScheme, moduleName can be defined in a single place

I have a branch with these kind of changes but it's coming later. For now, I just need the base project to unblock some rebases like explicitly null checking the stdlib and stuff

hamzaremmal requested review from tgodzik and WojciechMazur July 11, 2025 13:13

hamzaremmal assigned tgodzik and WojciechMazur Jul 11, 2025

hamzaremmal enabled auto-merge July 11, 2025 13:13

hamzaremmal force-pushed the scalalib-sbt branch from cc869c3 to 78060b3 Compare July 11, 2025 13:20

hamzaremmal commented Jul 11, 2025

View reviewed changes

hamzaremmal force-pushed the scalalib-sbt branch from 78060b3 to b1ddc81 Compare July 11, 2025 13:27

hamzaremmal disabled auto-merge July 14, 2025 11:19

hamzaremmal unassigned tgodzik and WojciechMazur Jul 14, 2025

hamzaremmal removed request for tgodzik and WojciechMazur July 14, 2025 11:19

hamzaremmal force-pushed the scalalib-sbt branch from b1ddc81 to 618ad57 Compare July 16, 2025 12:49

hamzaremmal requested a review from a team as a code owner July 16, 2025 12:49

hamzaremmal commented Jul 16, 2025

View reviewed changes

hamzaremmal force-pushed the scalalib-sbt branch from 618ad57 to f0967f9 Compare July 18, 2025 11:42

hamzaremmal added 2 commits July 18, 2025 13:59

chore: add the scala-library project

c6785a9

chore: add CI to verify that the non-bootstrapped stdlib compiles

f9921f5

hamzaremmal force-pushed the scalalib-sbt branch 2 times, most recently from 266a732 to 6e3e584 Compare July 18, 2025 13:01

hamzaremmal commented Jul 18, 2025

View reviewed changes

hamzaremmal added 2 commits July 18, 2025 14:21

chore: add scala-library-bootstrapped

12a784c

chore: add CI to verify that the bootstrapped stdlib compiles

a19c2b0

hamzaremmal force-pushed the scalalib-sbt branch from 6e3e584 to a19c2b0 Compare July 18, 2025 13:22

hamzaremmal changed the title ~~chore: add the scala-library project~~ chore: add the scala-library projects (non-bootstrapped and bootstrapped) Jul 18, 2025

hamzaremmal mentioned this pull request Jul 18, 2025

Add missing definitions in the stdlib (from stdLibPatches) #23562

Open

hamzaremmal requested a review from WojciechMazur July 18, 2025 14:16

hamzaremmal assigned WojciechMazur Jul 18, 2025

hamzaremmal commented Jul 18, 2025

View reviewed changes

hamzaremmal enabled auto-merge July 18, 2025 14:32

WojciechMazur reviewed Jul 18, 2025

View reviewed changes

WojciechMazur approved these changes Jul 18, 2025

View reviewed changes

hamzaremmal merged commit 5899730 into scala:main Jul 18, 2025
31 checks passed

hamzaremmal deleted the scalalib-sbt branch July 18, 2025 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore: add the `scala-library` projects (non-bootstrapped and bootstrapped) #23510

chore: add the `scala-library` projects (non-bootstrapped and bootstrapped) #23510

Uh oh!

hamzaremmal commented Jul 11, 2025

Uh oh!

hamzaremmal Jul 11, 2025

Uh oh!

hamzaremmal Jul 16, 2025

Uh oh!

hamzaremmal Jul 18, 2025

Uh oh!

hamzaremmal Jul 18, 2025

Uh oh!

hamzaremmal Jul 18, 2025

Uh oh!

hamzaremmal left a comment

Uh oh!

hamzaremmal commented Jul 18, 2025

Uh oh!

hamzaremmal Jul 18, 2025 •

edited

Loading

Uh oh!

WojciechMazur Jul 18, 2025

Uh oh!

hamzaremmal Jul 18, 2025

Uh oh!

Uh oh!

WojciechMazur Jul 18, 2025

Uh oh!

WojciechMazur Jul 18, 2025 •

edited

Loading

Uh oh!

hamzaremmal Jul 18, 2025

Uh oh!

Uh oh!

Uh oh!

chore: add the scala-library projects (non-bootstrapped and bootstrapped) #23510

chore: add the scala-library projects (non-bootstrapped and bootstrapped) #23510

Uh oh!

Conversation

hamzaremmal commented Jul 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hamzaremmal left a comment

Choose a reason for hiding this comment

Uh oh!

hamzaremmal commented Jul 18, 2025

Uh oh!

hamzaremmal Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WojciechMazur Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chore: add the `scala-library` projects (non-bootstrapped and bootstrapped) #23510

chore: add the `scala-library` projects (non-bootstrapped and bootstrapped) #23510

hamzaremmal Jul 18, 2025 •

edited

Loading

WojciechMazur Jul 18, 2025 •

edited

Loading