Support building on Windows

[lpil]

Hello!

Currently CI runs on OSX and Linux, and there's plans to add #292 .

It would be great to see Windows added to this list! It's an extremely popular OS and GitHub actions has free Windows support.

Traditionally Erlang neglects Windows support so it would be great to have it as a first class citizen of Lumen, taking full advantage of the cross-platform strengths of Rust and LLVM.

Thanks,
Louis

[bcardarella]

Per Dec 9 standup: supporting Windows is in scope for Lumen. @bitwalker will give it a shot in the future but the priority is the Mac and Linux builds for the time being.

[lpil]

I'm on Windows without WSL these days so shoot me a message if you would like me to serve as a test guinea pig.

[bitwalker]

@lpil Just wanted to chime in and let you know that I'll take a look once I merge my current branch of work that has been in progress for a few months, we should be able to get the build working on Windows. The switch to cargo-make was in part intended to get rid of shell scripts we were depending on.

That said, there is also the matter of making our fiber model work on Windows, which differs from the *NIX implementation, so while we can probably get Lumen building on Windows in relatively short order, it'll probably take a bit longer to build native executables for Windows until that's done. It's not a huge amount of effort, but I'm aiming to get our WebAssembly target fully supported first.

[lpil]

Spooky, we were just talking about this in the Gleam discord. 🔮

Thanks for the update! Fantastic work, I am excited to try it in future. Being able to compile to wasm from Windows would also be very useful even if targeting Windows directly isn't possible.

[johns10]

I'm attempting to do this on wsl as we speak. Instructions were effective until
make llvm-shared

Then I got

CHECK 'chrpath': NO
CHECK 'file': YES
CHECK 'objdump': YES
CHECK 'ninja': YES
Required utility programs are missing! Cannot proceed.

fixed with apt-get install chrpath

Now I've got

# env CC=cc CXX=g++ cmake -GNinja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_TARGETS_TO_BUILD=X86;AArch64;ARM;WebAssembly -DLLVM_ENABLE_PROJECTS=llvm;mlir;lld;lldb -DLLVM_ENABLE_RUNTIMES= -DLLVM_CCACHE_BUILD=ON -DLLVM_OPTIMIZED_TABLEGEN=ON -DBUILD_SHARED_LIBS=OFF -DLLVM_DYLIB_COMPONENTS=all -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON -DLLVM_INSTALL_TOOLCHAIN_ONLY=OFF -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_PARALLEL_COMPILE_JOBS=8 -DLLVM_VERSION_SUFFIX=-lumen-12.0.0 -DLLVM_INCLUDE_DOCS=OFF -DLLVM_BUILD_DOCS=OFF -DLLVM_ENABLE_DOXYGEN=OFF -DLLVM_ENABLE_SPHINX=OFF -DLLVM_ENABLE_BINDINGS=OFF -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_BUILD_TESTS=OFF -DLLVM_INCLUDE_GO_TESTS=OFF -DLLVM_INCLUDE_BENCHMARKS=OFF -DLLVM_ENABLE_ZLIB=OFF -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_ENABLE_LIBEDIT=OFF -DLLVM_ENABLE_Z3_SOLVER=OFF -DLLVM_ENABLE_LIBXML2=ON -DLLDB_NO_DEBUGSERVER=ON -DLLDB_CODESIGN_IDENTITY="" -DCMAKE_EXE_LINKER_FLAGS="-Wl,-Bsymbolic -static-libstdc++" -DLLVM_ENABLE_LLD=ON -DLLVM_BUILD_UTILS=ON -DLLVM_INSTALL_UTILS=ON  /home/johns10/Temp/llvm-project/llvm
env: ‘cmake’: No such file or directory
make: *** [Makefile:21: llvm-shared] Error 127

fixed with apt-get install cmake

Then I got

...
-- Performing Test CXX_SUPPORTS_NO_VLA_EXTENSION
-- Performing Test CXX_SUPPORTS_NO_VLA_EXTENSION - Success
-- LLDB version: 13.0.0-lumen-12.0.0
CMake Error at /home/johns10/Temp/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:271 (message):
  Expected directory for clang-resource headers not found:
Call Stack (most recent call first):
  /home/johns10/Temp/llvm-project/lldb/CMakeLists.txt:21 (include)


-- Configuring incomplete, errors occurred!
See also "/home/johns10/Temp/llvm-project/build/host_shared/stage2/RelWithDebInfo/stage2-12.0.0.obj/CMakeFiles/CMakeOutput.log".
See also "/home/johns10/Temp/llvm-project/build/host_shared/stage2/RelWithDebInfo/stage2-12.0.0.obj/CMakeFiles/CMakeError.log".
make: *** [Makefile:21: llvm-shared] Error 1

Error log says

Run Build Command(s):/usr/bin/ninja cmTC_24386 && [1/2] Building CXX object CMakeFiles/cmTC_24386.dir/src.cxx.o
[2/2] Linking CXX executable cmTC_24386
FAILED: cmTC_24386
: && /usr/bin/g++  -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -fdiagnostics-color -DLINKER_SUPPORTS_COLOR_DIAGNOSTICS  -Wl,--color-diagnostics CMakeFiles/cmTC_24386.dir/src.cxx.o  -o cmTC_24386  -lm && :
/usr/bin/ld: unrecognized option '--color-diagnostics'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

I'm assuming that's g++ failing because of the --color diagnostics

It could be associated with NASA-SW-VnV/ikos#48

It's throwing a NUMBER of missing requirements. Trying to get those resolved:

-- Could NOT find Lua (missing: LUA_LIBRARIES LUA_INCLUDE_DIR) (Required is at least version "5.3")
-- Could NOT find LuaAndSwig (missing: LUA_LIBRARIES LUA_INCLUDE_DIR)
-- Enable Lua scripting support in LLDB: FALSE
-- Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Development) (found version "3.8.5")
-- Could NOT find PythonAndSwig (missing: Python3_INCLUDE_DIRS)
-- Enable Python scripting support in LLDB: FALSE
-- Could NOT find LibXml2 (missing: LIBXML2_LIBRARY LIBXML2_INCLUDE_DIR) (Required is at least version "2.8")
-- Enable Libxml 2 support in LLDB: FALSE

Installed lua5.2-dev
Installed python3-dev
Installed libxml2-dev

No more obvious errors, still getting

FAILED: CMakeFiles/cmTC_56c9a.dir/CheckSymbolExists.c.o
/usr/bin/cc -D_GNU_SOURCE  -fPIC -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -o CMakeFiles/cmTC_56c9a.dir/CheckSymbolExists.c.o   -c CheckSymbolExists.c
CheckSymbolExists.c:2:10: fatal error: os/signpost.h: No such file or directory
    2 | #include <os/signpost.h>
      |          ^~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Maybe http://openbsd-archive.7691.n7.nabble.com/Ports-LLVM-10-s-CMake-files-trying-to-use-macOS-os-signpost-h-during-configure-time-td396435.html ?

Is os/signpost.h an osx file?

I think so

https://fossies.org/linux/llvm/cmake/modules/HandleLLVMOptions.cmake

From the error, it looks like LLDB_EXTERNAL_CLANG_RESOURCE_DIR isn't getting set properly (because there's no output in the error)
Error:

CMake Error at /home/johns10/Temp/llvm-project/lldb/cmake/modules/LLDBConfig.cmake:271 (message):
  Expected directory for clang-resource headers not found:

Codes

<!--StartFragment-->  if (NOT LLDB_EXTERNAL_CLANG_RESOURCE_DIR)    message(FATAL_ERROR "Expected directory for clang-resource headers not found: ${LLDB_EXTERNAL_CLANG_RESOURCE_DIR}")  endif()<!--EndFragment-->

https://github.com/llvm/llvm-project/blob/main/lldb/cmake/modules/LLDBConfig.cmake#L271

[johns10]

I ran this on my ubuntu machine too, and get stuck at that error.

[fire]

Can someone help me understand what is blocking this?

[bitwalker]

The issue mentioned above is one I just recently fixed in the past day or so, an issue with the LLVM build script, but I haven't had the chance to verify other platforms like Windows yet. You should be able to pull the most recent version of the firefly branch of our GetFirefly/llvm-project repo and build under WSL - let me know if you encounter any issues.