Pointless compiled to native machine code

[owaismohsin001]

Hello. I have been experimenting with various backends for Pointless. The first mildly successful attempt at writing one of these was this compiler that compiles Pointless to Lua but as I was working on it, I hit a realization that most of this stuff could be just as well represented in a statically typed and compiled language. This set me looking for good statically typed backends for Pointless and then I explored options like the JVM but had to abandon it since it quite famously doesn't support TCO. Finally the best compilation target for it, in the statically-typed world, in my opinion, is Haskell.

So I have spent some time creating this Pointless to Haskell compiler. It still doesn't implement the entire language, for instance, it's missing many of the language fields that equip Pointless with marketable IO, the inclusion of prelude in a separate scope, and still isn't as flexible with its module system as the interpreter. Its performance will also be improved by implementing more efficient visual representations of available types. Anyways, the point is that now Pointless can be compiled to Haskell and then be compiled to native machine code by GHC with free optimizations.

I would like to know your opinion on the implementation and would also like to get some advice on if interop with Haskell would be appropriate for Pointless because, if it gets to interop with Haskell it can leverage Haskell's entire ecosystem of libraries such as Parsec, Network, GTK, and similar stuff with bindings.

[averynortonsmith]

Hey! I've been meaning to get back to your last post on the other issue thread. I'd love to discuss this stuff with you more -- I don't know if you use video-call software but if you do we could set up a time to chat. It might be more efficient than writing out messages like this. If not no worries, I'll take time tomorrow to look at the links and share my thoughts!

[owaismohsin001]

Sorry as I don't really have a video chat software setup. just let me know when you get around having a look at these implementation links.

[averynortonsmith]

I get a 16x speedup on the following code with the Haskell compiler over the Dart interpreter! (11 seconds vs 3 minutes)

fact(n, d, res) = cond {
  case (n == 1) res
  case (n % d == 0) fact(n / d, d, [d] ++ res)
  else fact(n, d + 1, res)
}

factors = fact(1000000000001, 2, [])
output = [factors]

(I had to modify the compiler to use Double for numbers instead of Float)

[owaismohsin001]

Cool. What compiler flags were used? Because with O2 Haskell aggressively inlines functions across modules just like it does inside modules without O2 and due to that(and other optimizations), on my machine when I ran all the tests I have for this compiler the speed of the compiled code increases by, around 1.5x.

Also, what would your suggestions and opinions regarding FFI with Haskell be? There are several ways I can think of, that can be used to interop. The immediate one that comes to mind is having an import of a file named "ffi.hs" which would be the source of all the bindings being imported. I honestly have little experience designing bindings with other languages so any sort of suggestion or even a paper would be very helpful for the design.

Edit: About doubles, yeah I think it makes more sense for doubles to be used for the entire thing instead of floats because of more precision so now I have updated that repository to have doubles.

[averynortonsmith]

I got 11 seconds with O2, without optimizations the code ran in a little under a minute.

FFI is a new area for me as well. I did a little reading and found this article which outlines the basic strategies for FFI (modify caller functions in the original language, modify callee functions in the target language, or write a wrapper layer). I'm assuming that you'd want to interact with Haskell code through the Haskell-Pointless runtime -- interacting with other pre-compiled Haskell would be another level of complexity.

If you think about it, PtlsRuntime.hs is already a sort of FFI, since you have Pointless functions calling Haskell functions. The only caveat is that your Haskell functions have to expect Pointless types as input. But you could add FFI capabilities in this way, perhaps as fields on some special label like Haskell.!someFunction.

If you want to interface with Haskell functions that take normal Haskell arguments, then you could write a wrapper function converts Ptls types to Haskell types and another to convert back at the end. Or you could use an existing interchange format like JSON (you'd need a JSON parse for Pointless -- I'd be happy to make one if you like).

Sorry I can't give you much concrete advice here. I think there are just a lot of ways to approach the problem of FFI. You could also look at how some other small languages do it [1] [2] I've been wanting to add more Dart FFI to Pointless so maybe once I do we can keep bouncing ideas off each other's work.

[owaismohsin001]

First I would like to ask your opinion on a --prod option, in theory, it would use a separate runtime and a different code generator. This runtime would not track any positions so, it might lessen the runtime overhead for Haskell. Idea is that just like Nim's --opt-speed(or at least that's what I believe it's called), it would be an opt-in way of sacrificing the debuggability of a build in order to boost its speed.

After reading the things you have sent I have a few options here, and all of them have a Haskell function that takes a Haskell vector of Values and returns a Value. Like this

addNums :: Vector Value -> Value
addNums v = add (v ! 0) (v ! 1)

Now both approaches use different call syntax, the first one would be having a foreign keyword/symbol before a call which would let Pointless know that you accessing a Haskell function. Something along the lines of

foreign f(2)

or

import "stuff.ptls" as stuff
foreign {f, g}
import "otherStuff.ptls" as otherStuff

f(2)

or maybe just import a reserved module FFI/Foreign/Haskell and when it's imported have the compiler create that file for you with a basic template(somewhat like the one that follows) in the output directory.

module Ffi (export) where

import PtlsRuntime

type Values = Vector Value

addNums :: Values -> Value
addNums vs = add (vs ! 0) (vs ! 1)

export = [ ("addNums", addNums) ]

while in the main Pointless file we could now just write

import "FFI" as ffi
-- No file named FFI should exist

list = [1, 2, 3, 4, 5]
output = map(ffi.addNumber, list)

and FFI would work. Any ideas on if this would work would be nice. Personally, I like the last he most because, in my opinion, it has a good balance between explicitness, elegance, idiomaticness, and terseness.