Any interest in scala-native (and scala.js) ?

[Voltir]

It appears that cats-actors is pretty close to being able to be run natively and in scala.js since its minimal dependencies (ie cats-effect) can mostly run on those platforms now.

I did a proof of concept fork here:
https://github.com/Voltir/cats-actors/tree/topic/scala-native-poc

The upstream dependencies are a bit in unstable right now - looks like cats-effect is still working through some issue working with scala-native 0.5.7.

Additionally, scala-futures don't play great in the test code. Instead I used https://github.com/typelevel/cats-effect-testing which seems pretty compatiable with the current tests if you are willing to drop unsafeRunSync calls in favor of the extra test dependency. But with a few minor other changes, I was able to get the first couple of AskSpec tests to run natively, which is pretty cool :)

[cloudmark]

This would be amazing!

…

On Thu, 08 May 2025 at 00:25, Nick Childers ***@***.***> wrote: *Voltir* created an issue (cloudmark/cats-actors#40) <#40> It appears that cats-actors is pretty close to being able to be run natively and in scala.js since its minimal dependencies (ie cats-effect) can mostly run on those platforms now. I did a proof of concept fork here: https://github.com/Voltir/cats-actors/tree/topic/scala-native-poc The upstream dependencies are a bit in unstable right now - looks like cats-effect is still working through some issue working with scala-native 0.5.7. Additionally, scala-futures don't play great in the test code. Instead I used https://github.com/typelevel/cats-effect-testing which seems pretty compatiable with the current tests if you are willing to drop unsafeRunSync calls in favor of the extra test dependency. But with a few minor other changes, I was able to get the first couple of AskSpec tests to run natively, which is pretty cool :) — Reply to this email directly, view it on GitHub <#40>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANKHJRWZLUGPJ5GPYCLWJ325KB7BAVCNFSM6AAAAAB4VASQ7CVHI2DSMVQWIX3LMV43ASLTON2WKOZTGA2DOMRZHA2DINA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Voltir]

@cloudmark - alright :)

Ill go update more of the tests and see if anything breaks then - and if that works, ill open a PR if/when the upstream dependencies stabilize. Also, my current thinking is that for the js/native builds, only targeting scala 3 - im leaving the 2.13 dependencies as is for the JVM build.

Also, just wanted to get your opinion on:
main...Voltir:cats-actors:topic/scala-native-poc#diff-28eccc967d9d4e9ffb29eacabbe52f054205b5019696aea129b4291f66035c75R115-R118

As you might imagine, java reflection is a no go in scala.native / scala.js land - luckily it seems like its usage is pretty minimal. I might have missed something, but I think the goal was to initialize a private val. Im wondering if just passing it in as an arg and giving the companion object a new constructor is a more straightforward way to achieve that. The code that is there is just a var bashed into place to make it compile, and apparently that works as well.

[cloudmark]

Hey @Voltir, LGTM! Just to make sure I’m not missing anything, the main changes here are to the dependencies and build config in build.sbt, right?

I noticed a couple of unrelated changes in the PR, like the IdGen update and the folder structure tweaks. I’d prefer to keep this PR focused just on the Scala Native build changes so the scope stays clear and the diff is easier to follow.

Would you be up for moving those other changes to a separate PR? Also, if there's a reason for the IdGen change, happy to discuss it there.

[Voltir]

Right, so all these changes basically are still in service of making scala.native / scala.js work. Step 1 really starts with the plugins:

addSbtPlugin("org.portable-scala" % "sbt-scalajs-crossproject" % "1.3.2")
addSbtPlugin("org.portable-scala" % "sbt-scala-native-crossproject" % "1.3.2")
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.19.0")
addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.5.6")

Im most familiar (having spent time with scala.js) with the CrossType.Full style of shared libraries:

.
├── js
├── jvm
├── native
└── shared

That is everything you want to work across all platforms goes into shared, and then the platform specific stuff goes in the respective directories. There are other ways to set it up I could explore if you prefer some other layout, but what this means is that I did yolo mv all the stuff in src/ into /shared/src as step 2.

The next thing I did was made sure the jvm tests passed - 'sbt catsActorJVM/test` worked. Then I tried doing the same for native and thats were the fun link time errors show up. So the stuff you see with IdGen is because java.util.UUID isnt available on scala.native and scala.js - hence the shim and some small platform specific code to make it work. Im currently trying to make more of the tests work and see there is another issue around IO.sleep on scala.native, so it looks like I will need some more shim code, at least at the unit test level.

[cloudmark]

Awesome, thanks for the info!

…

On Thu, May 8, 2025 at 8:04 PM Nick Childers ***@***.***> wrote: *Voltir* left a comment (cloudmark/cats-actors#40) <#40 (comment)> Right, so all these changes basically are still in service of making scala.native / scala.js work. Step 1 really starts with the plugins: addSbtPlugin("org.portable-scala" % "sbt-scalajs-crossproject" % "1.3.2") addSbtPlugin("org.portable-scala" % "sbt-scala-native-crossproject" % "1.3.2") addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.19.0") addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.5.6") Im most familiar (having spent time with scala.js) with the CrossType.Full style of shared libraries: . ├── js ├── jvm ├── native └── shared That is everything you want to work across all platforms goes into shared, and then the platform specific stuff goes in the respective directories. There are other ways to set it up I could explore if you prefer some other layout, but what this means is that I did yolo mv all the stuff into source into /shared/src as step 2. The next thing I did was made sure the jvm tests passed - 'sbt catsActorJVM/test` worked. Then I tried doing the same for native and thats were the fun link time errors show up. So the stuff you see with IdGen is because java.util.UUID isnt available on scala.native and scala.js - hence the shim and some small platform specific code to make it work. Im currently trying to make more of the tests work and see there is another issue around IO.sleep on scala.native, so it looks like I will need some more shim code, at least at the unit test level. — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANKHJRIVKKHMVJTAIH6KML25OMDDAVCNFSM6AAAAAB4VASQ7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNRTHA2TSNRUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Voltir]

Made some progress on porting tests - all but a few seem to be working. There are a couple of test failures on scala.native that are non-determinstic / timeout / race condition failures, eg:

[info] - should wait for mailboxes to be empty before progressing *** FAILED ***
[info]   java.lang.IllegalStateException: Wait timeout exceeded, consider re-writing the test so that its not so slow.  If this is expected behaviour increase the maxTimeout
[info]

[Voltir]

Alright, I think all the tests pass now - there was some timing differences that happen in scala.native, for example in FSMTimingSpec:

          _ <- proxyActor ! Send(Tick, fsm)
          _ <- expectMsgs(testActor, 500.millis)(List(Tick))

That first messages was being processed and completed before expectMsgs had a chance to check, behaving as if tell was more like a normal blocking function call. However, if I make the following change to expectMsgs:

    expectMsgInternal(
      actor,
      timeout,
      startQ => // actor.messageBuffer.map(_._2 == startQ._2 ++ messages),
        actor.messageBuffer.map(_._2.endsWith(messages)),
      messages
    )

Then by using endsWith, then it no longer matters if the message comes in "early" with respect to the check - which I think is more or less the intended goal. With that change, all the tests pass in both JVM and native.

[Voltir]

Ah, shoot life is a little harder with making it run in scala.js - there are some additional java dependencies that are not implemented on that platform - the most critical seems to be:

[error] Referring to non-existent class java.util.concurrent.LinkedTransferQueue
[error]   called from constructor com.suprnation.actor.dispatch.mailbox.Mailboxes$$anon$2.<init>(cats.effect.kernel.Async,cats.effect.std.Semaphore)void

If we still wanted to push on, my initial thought is to look into replacing the java queue with scala / cats queue - im not entirely sure how important the choice of the LinkTransferQueue was in terms of semantics or performance.

There is also an instance of reflection that scala.js flagged:

[error] Referring to non-existent class java.lang.reflect.InvocationTargetException
[error]   called from com.suprnation.actor.SupervisionStrategy.logFailure(com.suprnation.actor.ActorContext,com.suprnation.actor.ReplyingActorRef,scala.Option,com.suprnation.actor.SupervisorStrategy$Directive)java.lang.Object
[error]   called from com.suprnation.actor.SupervisionStrategy.handleFailure(com.suprnation.actor.ActorContext,com.suprnation.actor.ReplyingActorRef,java.lang.Throwable,com.suprnation.actor.ChildRestartStats,scala.collection.immutable.List)java.lang.Object

That can probably be worked around through stubbing out some platform specific handlers, but for now I just commented out the one case where that was being checked.

--- update ---
Migrating to the cats-effect queue, eg:

  def createMailbox[F[+_]: Console: Async, SystemMessage, A](
      name: String
  ): F[Mailbox[F, SystemMessage, A]] = {
    for {
      lock <- Semaphore[F](1)
      userQueue <- Queue.unbounded[F, A]
      systemQueue <- Queue.unbounded[F, SystemMessage]
    } yield {
...

seems to work :)
Assuming unbounded semantics are good enough..

[cloudmark]

Great progress man! From my experience the cats-queue was significantly slower (tried a couple of queues when I had implemented this and that was the most performant)

…

On Tue, May 13, 2025 at 6:36 PM Nick Childers ***@***.***> wrote: *Voltir* left a comment (cloudmark/cats-actors#40) <#40 (comment)> Ah, shoot life is a little harder with making it run in scala.js - there are some additional java dependencies that are not implemented on that platform - the most critical seems to be: [error] Referring to non-existent class java.util.concurrent.LinkedTransferQueue [error] called from constructor com.suprnation.actor.dispatch.mailbox.Mailboxes$$anon$2.<init>(cats.effect.kernel.Async,cats.effect.std.Semaphore)void If we still wanted to push on, my initial thought is to look into replacing the java queue with scala / cats queue - im not entirely sure how important the choice of the LinkTransferQueue was in terms of semantics or performance. There is also an instance of reflection that scala.js flagged: [error] Referring to non-existent class java.lang.reflect.InvocationTargetException [error] called from com.suprnation.actor.SupervisionStrategy.logFailure(com.suprnation.actor.ActorContext,com.suprnation.actor.ReplyingActorRef,scala.Option,com.suprnation.actor.SupervisorStrategy$Directive)java.lang.Object [error] called from com.suprnation.actor.SupervisionStrategy.handleFailure(com.suprnation.actor.ActorContext,com.suprnation.actor.ReplyingActorRef,java.lang.Throwable,com.suprnation.actor.ChildRestartStats,scala.collection.immutable.List)java.lang.Object That can probably be worked around through stubbing out some platform specific handlers, but for now I just commented out the one case where that was being checked. — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANKHJWKTIZRWK2NEIKFCBT26INQBAVCNFSM6AAAAAB4VASQ7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZXGIZTQMZSGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Voltir]

Ah, that's good to know. I can spend some time trying to make platform specific mailboxes if you like, but it would be nice to have some concrete benchmarks if the goal is performance - what was your workflow for choosing which queue implementation to go with?

[cloudmark]

The way I benchmarked this before was to put 2mi messages in a rabbit queue and drain this queue onto an actor system which shards by user. Unfortunately, I do not have the benchmark results anymore.

…

On 13 May 2025 at 23:27:10, Nick Childers ***@***.***) wrote: *Voltir* left a comment (cloudmark/cats-actors#40) <#40 (comment)> Ah, that's good to know. I can spend some time trying to make platform specific mailboxes if you like, but it would be nice to have some concrete benchmarks if the goal is performance - what was your workflow for choosing which queue implementation to go with? — Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANKHJQAKQ2YR2HI3QVDUID26JPS5AVCNFSM6AAAAAB4VASQ7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNZXHE4DQOBYGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[rlavolee]

Hi @Voltir @cloudmark ,

i'll also be interesting in Cats-Actor Native. I'm not trying to rush you but is it still happening ?

Regards

[cloudmark]

Hi there @rlavolee, I myself do not have time atm but I'm happy to collaborate on this with others

[rlavolee]

Do we know what remains to be done? I'm confused about what's left.