rust: 1.31.0 -> 1.32.0 by lblasc · Pull Request #54323 · NixOS/nixpkgs

lblasc · 2019-01-19T15:01:35Z

Motivation for this change

Bump Rust to latest release, also closes #54183

Notes:

removed disable-test-inherit-env.patch - this test got removed in upstream
remove building with jemalloc (solves Rust 1.31.0 shared libs are bundled with jemalloc #54183) https://blog.rust-lang.org/2019/01/17/Rust-1.32.0.html#jemalloc-is-removed-by-default
rustc changed vendor dir location from src/vendor to just vendor

Things done

ghost · 2019-01-19T18:30:42Z

blog.rust-lang.org/2019/01/17/Rust-1.32.0.html#jemalloc-is-removed-by-default

I think this is not relevant there. rustc still compiled with jemalloc by default

lblasc · 2019-01-19T21:21:53Z

That's true for rustc, it gets jemalloc from crate now, only for selected architectures. What is relevant (see rust-lang/rust@61e8944) that they removed bundled jemalloc and option to build with external one.

Mic92 · 2019-01-20T02:18:00Z

Have you tried building a create?
I currently get a segfault when I build fd on x86_64-linux: https://gist.github.com/Mic92/d67b19bad239b9c7669450d2fef7da56

lblasc · 2019-01-20T09:49:31Z

Hello @Mic92, thanks for testing.

Yes, tried with random crates from the nixpkgs and relatively big internal rust project. And it worked fine. I got the same segfault with fd crate. When I disabled bundled LLVM crate builded without the problem.

  diff --git a/pkgs/development/compilers/rust/default.nix b/pkgs/development/compilers/rust/default.nix
index 9640cd9b577..19ab8b71171 100644
--- a/pkgs/development/compilers/rust/default.nix
+++ b/pkgs/development/compilers/rust/default.nix
@@ -24,7 +24,7 @@ in rec {
       #./patches/stdsimd-disable-doctest.patch
     ];
 
-    withBundledLLVM = false;
+    withBundledLLVM = true;
 
     configureFlags = [ "--release-channel=stable" ];

Maybe it will be good idea to use bundled llvm by default and leave a option for building with system llvm. Stability should be more important than closure size. :) What do you think?

Mic92 · 2019-01-20T13:38:16Z

cc @symphorien

symphorien · 2019-01-20T14:38:13Z

It seems avoiding the bundled llvm is necessary at least on darwin:
#49708 descbribes breakage which is fixed by using the system llvm #49557

lblasc · 2019-01-21T09:19:43Z

Will this change suffice ?

diff --git a/pkgs/development/compilers/rust/default.nix b/pkgs/development/compilers/rust/default.nix
index 9640cd9b577..511d6e7956e 100644
--- a/pkgs/development/compilers/rust/default.nix
+++ b/pkgs/development/compilers/rust/default.nix
@@ -24,7 +24,7 @@ in rec {
       #./patches/stdsimd-disable-doctest.patch
     ];
 
-    withBundledLLVM = false;
+    withBundledLLVM = if stdenv.isDarwin then false else true;
 
     configureFlags = [ "--release-channel=stable" ];

Mic92 · 2019-01-21T12:07:38Z

Btw. I get the exact same failure when compiling fd-find with archlinux that also uses the system llvm...

error: failed to compile `fd-find v7.2.0`, intermediate artifacts can be found at `/tmp/cargo-installCT8S32`

Caused by:
  Could not compile `fd-find`.

Caused by:
  process didn't exit successfully: `rustc --crate-name fd /root/.cargo/registry/src/github.com-1ecc6299db9ec823/fd-find-7.2.0/src/main.rs --color always --crate-type bin --emit=dep-info,link -C opt-level=3 -C lto -C codegen-units=1 -C metadata=a3a6a7cb919634a5 -C extra-filename=-a3a6a7cb919634a5 --out-dir /tmp/cargo-installCT8S32/release/deps -L dependency=/tmp/cargo-installCT8S32/release/deps --extern ansi_term=/tmp/cargo-installCT8S32/release/deps/libansi_term-da0b45e138ff353b.rlib --extern atty=/tmp/cargo-installCT8S32/release/deps/libatty-d255fe7ee8d55655.rlib --extern clap=/tmp/cargo-installCT8S32/release/deps/libclap-f83ca700ee891f8b.rlib --extern ctrlc=/tmp/cargo-installCT8S32/release/deps/libctrlc-03b9fe7f29c143b5.rlib --extern humantime=/tmp/cargo-installCT8S32/release/deps/libhumantime-67b9f25c854b0215.rlib --extern ignore=/tmp/cargo-installCT8S32/release/deps/libignore-1259b5bca495d9c6.rlib --extern lazy_static=/tmp/cargo-installCT8S32/release/deps/liblazy_static-7deee2bf7505e8c9.rlib --extern libc=/tmp/cargo-installCT8S32/release/deps/liblibc-e3278554887eedae.rlib --extern num_cpus=/tmp/cargo-installCT8S32/release/deps/libnum_cpus-f24ed7872633de1c.rlib --extern regex=/tmp/cargo-installCT8S32/release/deps/libregex-dcae60647554bb97.rlib --extern regex_syntax=/tmp/cargo-installCT8S32/release/deps/libregex_syntax-16497b92e03853d9.rlib --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
[root@4bb391331530 /]#

Mic92 · 2019-01-21T12:26:00Z

I also reported the error upstream: rust-lang/rust#57762

Mic92 · 2019-01-21T13:26:03Z

It seems to affect also other projects that have lto=true enabled.

- also accept the full url of pull request i.e. NixOS#54323 - stream-parse nix-env's eval output - fix logs for failed builds issued by remote builders - prepare code for overlays like: NixOS#53934

Mic92 · 2019-01-21T18:22:46Z

@lblasc can you try to apply llvm-mirror/llvm@da1fb72 using fetchpatch?

lblasc · 2019-01-21T21:57:37Z

@Mic92 applied patch:

diff --git a/pkgs/development/compilers/llvm/7/llvm.nix b/pkgs/development/compilers/llvm/7/llvm.nix
index bff89812cae..4d87107eed5 100644
--- a/pkgs/development/compilers/llvm/7/llvm.nix
+++ b/pkgs/development/compilers/llvm/7/llvm.nix
@@ -10,6 +10,7 @@
 , version
 , release_version
 , zlib
+, fetchpatch
 , buildPackages
 , debugVersion ? false
 , enableManpages ? false
@@ -51,6 +52,13 @@ in stdenv.mkDerivation (rec {
 
   propagatedBuildInputs = [ ncurses zlib ];
 
+  patches =
+    [ (fetchpatch {
+        url = "https://github.com/llvm-mirror/llvm/commit/da1fb72bb305d6bc1f3899d541414146934bf80f.patch";
+        sha256 = "0p81gkhc1xhcx0hmnkwyhrn8x8l8fd24xgaj1whni29yga466dwc";
+      })
+    ];
+
   postPatch = optionalString stdenv.isDarwin ''
     substituteInPlace cmake/modules/AddLLVM.cmake \
       --replace 'set(_install_name_dir INSTALL_NAME_DIR "@rpath")' "set(_install_name_dir)" \

But llvm failed to build:

[100%] Running all regression tests
/nix/store/l95nkqp7bdimqnz9ixay1aahljzsz7vc-python-2.7.15/bin/python2.7 /build/llvm/build/./bin/llvm-lit -sv /build/llvm/build/utils/lit /build/llvm/build/test
llvm-lit: /build/llvm/build/utils/lit/tests/lit.cfg:62: warning: Could not import psutil. Some tests will be skipped and the --timeout command line argument will not work.
-- Testing: 26996 tests, 48 threads --
Testing: 0 .. 10
FAIL: LLVM :: Assembler/debug-info.ll (3773 of 26996)
******************** TEST 'LLVM :: Assembler/debug-info.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /build/llvm/build/bin/llvm-as < /build/llvm/test/Assembler/debug-info.ll | /build/llvm/build/bin/llvm-dis | /build/llvm/build/bin/llvm-as | /build/llvm/build/bin/llvm-dis | /build/llvm/build/bin/FileCheck /build/llvm/test/Assembler/debug-info.ll
: 'RUN: at line 2';   /build/llvm/build/bin/verify-uselistorder /build/llvm/test/Assembler/debug-info.ll
--
Exit Code: 1

Command Output (stderr):
--
/build/llvm/test/Assembler/debug-info.ll:86:15: error: CHECK-NEXT: expected string not found in input
; CHECK-NEXT: !33 = !DICompositeType(tag: DW_TAG_variant_part, name: "A", scope: !14, size: 64)
              ^
<stdin>:39:1: note: scanning from here
!33 = !DICompositeType(tag: DW_TAG_variant_part, name: "A", scope: !14, size: 64, discriminator: !34)
^

--

********************
Testing: 0 .. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..

1 warning(s) in tests.
Testing Time: 152.24s
********************
Failing Tests (1):
    LLVM :: Assembler/debug-info.ll
  
  Expected Passes    : 17124
  Expected Failures  : 56
  Unsupported Tests  : 9815
  Unexpected Failures: 1
make[3]: *** [CMakeFiles/check-all.dir/build.make:58: CMakeFiles/check-all] Error 1
make[3]: Leaving directory '/build/llvm/build'
make[2]: *** [CMakeFiles/Makefile2:365: CMakeFiles/check-all.dir/all] Error 2
make[2]: Leaving directory '/build/llvm/build'
make[1]: *** [CMakeFiles/Makefile2:372: CMakeFiles/check-all.dir/rule] Error 2
make[1]: Leaving directory '/build/llvm/build'
make: *** [Makefile:251: check-all] Error 2
builder for '/nix/store/r0r6ahdlfh7zlr9hqrrav1c4gi377f4z-llvm-7.0.1.drv' failed with exit code 2
cannot build derivation '/nix/store/7d4868z0q0642jcrqs0sjlivdrcrrc9s-rustc-1.32.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/7d4868z0q0642jcrqs0sjlivdrcrrc9s-rustc-1.32.0.drv', '/nix/store/dz1magqgml5j3xa1af7snbir8vgwjp04-cargo-1.32.0.drv' failed

lblasc · 2019-01-21T22:42:42Z

llvm-mirror/llvm@cc1f2a5 is also needed for tests to pass.

diff --git a/pkgs/development/compilers/llvm/7/llvm.nix b/pkgs/development/compilers/llvm/7/llvm.nix
index bff89812cae..5fdc82faf2e 100644
--- a/pkgs/development/compilers/llvm/7/llvm.nix
+++ b/pkgs/development/compilers/llvm/7/llvm.nix
@@ -10,6 +10,7 @@
 , version
 , release_version
 , zlib
+, fetchpatch
 , buildPackages
 , debugVersion ? false
 , enableManpages ? false
@@ -51,6 +52,17 @@ in stdenv.mkDerivation (rec {
 
   propagatedBuildInputs = [ ncurses zlib ];
 
+  patches =
+    [ (fetchpatch {
+        url = "https://github.com/llvm-mirror/llvm/commit/da1fb72bb305d6bc1f3899d541414146934bf80f.patch";
+        sha256 = "0p81gkhc1xhcx0hmnkwyhrn8x8l8fd24xgaj1whni29yga466dwc";
+      })
+      (fetchpatch {
+        url = "https://github.com/llvm-mirror/llvm/commit/cc1f2a595ead516812a6c50398f0f3480ebe031f.patch";
+        sha256 = "0k6k1p5yisgwx417a67s7sr9930rqh1n0zv5jvply8vjjy4b3kf8";
+      })
+    ];
+
   postPatch = optionalString stdenv.isDarwin ''
     substituteInPlace cmake/modules/AddLLVM.cmake \
       --replace 'set(_install_name_dir INSTALL_NAME_DIR "@rpath")' "set(_install_name_dir)" \

nix-build -A fd build passes now!

Mic92 · 2019-01-21T23:05:05Z

Ah. Sorry I forgot we were running llvm tests also.

Mic92 · 2019-01-21T23:06:37Z

@lblasc can you add this change and move the pull request to the staging branch?

See: rust-lang/rust#57762

lblasc · 2019-01-22T08:22:19Z

Rebased on latest staging, build is running fine.

lblasc · 2019-01-23T15:42:23Z

@Mic92 anything else needed?

Mic92 · 2019-01-23T16:33:58Z

I tested the following platforms:

aarch64-linux
x86_64-linux

@grahamc or @alyssais could you test the darwin build (nix-shell -p fd)?

grahamc · 2019-01-23T17:19:34Z

on it

grahamc · 2019-01-23T18:08:42Z

Failed to build:

make  check-TESTS
make[1]: Entering directory '/private/tmp/nix-build-libuv-1.24.1.drv-0/source'
1..292
ok 1 - platform_output

[snip]

not ok 110 - hrtime
# exit code 6
# Output from process `hrtime`:
# Assertion failed in test/test-hrtime.c on line 50: diff < (uint64_t) 80 * NANOSEC / MILLISEC

[snip]

FAIL: test/run-tests
======================================================
1 of 1 test failed
Please report to https://github.com/libuv/libuv/issues
======================================================
make[1]: *** [Makefile:4928: check-TESTS] Error 1
make[1]: Leaving directory '/private/tmp/nix-build-libuv-1.24.1.drv-0/source'
make: *** [Makefile:5188: check-am] Error 2
builder for '/nix/store/5whbiq00jjw9jkpnv3x5vpg0whdr593c-libuv-1.24.1.drv' failed with exit code 2
cannot build derivation '/nix/store/3l00iv04iiw1si9mildd67jrn7j5yy43-cmake-3.12.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/j0hh5rr36vkk1k1ryjsjxmpjg263s9b2-libgit2-0.26.6.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6l471jlhm5qwif45kjib9xlxcrlrdk6h-llvm-7.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yp3yf9mi3d9cwqlwld3xx4qwziaisyvx-rustc-1.32.0.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/cr3df5hx291sw2bf78j92zsdnc5gkq3i-cargo-1.32.0.drv': 3 dependencies couldn't be built
cannot build derivation '/nix/store/0xdd7b12s9sshn2n7fndijjam680pqkl-fd-7.2.0.drv': 2 dependencies couldn't be built
error: build of '/nix/store/0xdd7b12s9sshn2n7fndijjam680pqkl-fd-7.2.0.drv' failed

Mic92 · 2019-01-23T18:18:53Z

Oh, this way before we even had a chance to build rust.

Mic92 · 2019-01-23T18:22:42Z

The test failure in libuv does not seem nixpkgs-specific: libuv/libuv#2117

Mic92 · 2019-01-23T18:24:25Z

This seems not like good idea on hydra anyway: https://github.com/libuv/libuv/blob/v1.x/test/test-hrtime.c

Mic92 · 2019-01-23T18:27:14Z

@grahamc in case you have time, can you apply https://github.com/libuv/libuv/commit/d9f42a0fee63d094e35a1ed7d3a8a5dc3f73b90c.patch to libuv?

grahamc · 2019-01-23T18:34:39Z

Mind pushing a commit to this PR (or a different branch) applying that patch?

grahamc · 2019-01-23T18:43:42Z

nevermind, I applied it

grahamc · 2019-01-23T18:55:34Z

That patch didn't seem to make a difference -- same test fails.

Mic92 · 2019-01-23T21:34:03Z

Ah. Sorry, we are building libuv with autotools of course: https://github.com/libuv/libuv/commit/5d6ab57e582d9713fa4a209f90551d4c1c14fcf8.patch

grahamc · 2019-01-23T22:06:17Z

Undefined symbols for architecture x86_64:
  "_run_test_hrtime", referenced from:
      _TASKS in run_tests-run-tests.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)

lblasc · 2019-01-28T14:21:13Z

@Mic92 @grahamc any progress on this? I'll be glad to help but I don't have any darwin machines available..

Mic92 · 2019-01-28T16:15:22Z

I will proceed without testing rust on macOS. We have to fix the issue with libuv anyway.

- also accept the full url of pull request i.e. #54323 - stream-parse nix-env's eval output - fix logs for failed builds issued by remote builders - prepare code for overlays like: #53934 (cherry picked from commit f6d6e5d)

lblasc requested review from LnL7 and Mic92 as code owners January 19, 2019 15:01

GrahamcOfBorg added the 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. label Jan 19, 2019

GrahamcOfBorg requested review from cstrahan, globin, madjar, wizeman and wkennington January 19, 2019 15:17

GrahamcOfBorg added 10.rebuild-darwin: 101-500 This PR causes between 101 and 500 packages to rebuild on Darwin. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Jan 19, 2019

Mic92 mentioned this pull request Jan 21, 2019

Rustc 1.32 segfaults with llvm 7.0.1 while building fd-find and lto=true enabled rust-lang/rust#57801

Closed

Mic92 mentioned this pull request Jan 21, 2019

nix-review: 1.0.2 -> 1.0.4 #54422

Merged

10 tasks

This was referenced Jan 21, 2019

Unable to compile from git master alacritty/alacritty#2023

Closed

rustc 1.32 using system LLVM crashes while building the resvg crate rust-lang/rust#57762

Closed

lblasc requested a review from matthewbauer as a code owner January 22, 2019 08:03

lblasc changed the base branch from master to staging January 22, 2019 08:04

rust: 1.31.0 -> 1.32.0

cc5b55f

GrahamcOfBorg added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 22, 2019

llvm7: backport patches, fix building rust crates with lto

c340704

See: rust-lang/rust#57762

lblasc force-pushed the rust132 branch from 598ab37 to c340704 Compare January 22, 2019 08:21

GrahamcOfBorg removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 22, 2019

Mic92 merged commit 6f69004 into NixOS:staging Jan 28, 2019

7c6f434c referenced this pull request Feb 6, 2019

rustc: 1.31.0 -> 1.32.0

56dba36

timokau mentioned this pull request Feb 7, 2019

Revert "rustc: 1.31.0 -> 1.32.0" #55379

Merged

10 tasks

frol mentioned this pull request Feb 12, 2019

not ok - fs_event_watch_dir_recursive and hrtime (MacOS) libuv/libuv#2117

Closed

Uh oh!

Conversation

lblasc commented Jan 19, 2019

Motivation for this change

Things done

Uh oh!

ghost commented Jan 19, 2019

Uh oh!

lblasc commented Jan 19, 2019

Uh oh!

Mic92 commented Jan 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lblasc commented Jan 20, 2019

Uh oh!

Mic92 commented Jan 20, 2019

Uh oh!

symphorien commented Jan 20, 2019

Uh oh!

lblasc commented Jan 21, 2019

Uh oh!

Mic92 commented Jan 21, 2019

Uh oh!

Mic92 commented Jan 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mic92 commented Jan 21, 2019

Uh oh!

Mic92 commented Jan 21, 2019

Uh oh!

lblasc commented Jan 21, 2019

Uh oh!

lblasc commented Jan 21, 2019

Uh oh!

Mic92 commented Jan 21, 2019

Uh oh!

Mic92 commented Jan 21, 2019

Uh oh!

lblasc commented Jan 22, 2019

Uh oh!

lblasc commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

Mic92 commented Jan 23, 2019

Uh oh!

grahamc commented Jan 23, 2019

Uh oh!

lblasc commented Jan 28, 2019

Uh oh!

Mic92 commented Jan 28, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Mic92 commented Jan 20, 2019 •

edited

Loading

Mic92 commented Jan 21, 2019 •

edited

Loading