Switch from CSTParser to something else (tree-sitter or JuliaSyntax)

[gcv]

By switching to tree-sitter, built into Emacs as of version 29, all the questions Snail asks of CSTParser can be answered without leaving Elisp. This will be a significant benefit, not only by eliminating a dependency on CSTParser and its potentially unstable API, but also because Julia startup time will improve.

In general, it'll be great to ditch all Julia-side dependencies for core Snail functionality and leave them to extensions.

[MasonProtter]

By the way, JuliaSyntax.jl is now bundled inside of modern versions of Julia (since 1.9, and the new LTS is going to be v1.10), so you could also just use JuliaSyntax.jl instead of TreeSitter, but if there's other benefits to TreeSitter then I guess we should go with that.

[linwaytin]

I also feel JuliaSyntax.jl should be a better option than TreeSitter. The progress of tree-sitter-julia is slow, and I do not think it is supported officially from the Julia community.

[gcv]

At this point, I’m also leaning in the direction of JuliaSyntax. I’m a little concerned about yanking the rug out from any users who need Julia versions <1.10, but it should be okay, as long as the docs point out that older versions of Snail can be installed using straight.el or Elpaca.

[linwaytin]

Also I think JuliaSyntax can be manually installed for version < 1.10, so it should not be a big problem to use this parser.

[MasonProtter]

Yes, JuliaSyntax is compatible all the way back to julia v1.0 and is on the general registry.

[gcv]

@MasonProtter, @linwaytin, @danielmatz, @karthink, @dahtah, @alecStewart1, @norci, @orialb: I now have a branch with CSTParser replaced with JuliaSyntax: https://github.com/gcv/julia-snail/tree/juliasyntax

Unless someone persuades me otherwise, I will not bother supporting Julia <1.10 at this point. I like that Snail doesn't have anything in its Project.toml now.

It could use some testing before I merge into the main branch and break everything for everyone on MELPA. 😀

[linwaytin]

Thanks for working on this.

I tried it but I keep getting errors when typing. A *julia* error buffer pops up showing

MethodError: no method matching pathat(::Nothing, ::Int64)

Closest candidates are:
  pathat(!Matched::Base.JuliaSyntax.SyntaxNode, ::Any, !Matched::Any)
   @ Main.JuliaSnail ~/new-emacs.d/elpa/julia-snail/JuliaSnail.jl:533
  pathat(!Matched::Base.JuliaSyntax.SyntaxNode, ::Any)
   @ Main.JuliaSnail ~/new-emacs.d/elpa/julia-snail/JuliaSnail.jl:533


moduleat(encodedbuf::String, byteloc::Int64) at JuliaSnail.jl:634
top-level scope at none:1
eval at boot.jl:385 [inlined]
eval_in_module(fully_qualified_module_name::Vector{Symbol}, expr::Expr) at JuliaSnail.jl:245
(::Main.JuliaSnail.var"#30#36"{Sockets.TCPSocket})() at JuliaSnail.jl:1014

Also in the *julia* buffer there is Base.JuliaSyntax.ParseError.

It seems the package tries to parse the code in the middle when I'm typing, but since the code is incomplete JuliaSyntax cannot parse it successfully.