Update jemalloc to 4.5.0 by arthurprs · Pull Request #41861 · rust-lang/rust

arthurprs · 2017-05-09T18:18:38Z

pointing jemalloc submodule to the 4.5.0 HEAD + generated configure script
removed the now unnecessary extra config for windows. jemalloc is now much better at configuring itself overall.

Let's see what the CI think.

rust-highfive · 2017-05-09T18:18:50Z

r? @brson

(rust_highfive has picked a reviewer for you, use r? to override)

arthurprs · 2017-05-09T20:19:10Z

cc @alexcrichton rust-lang/jemalloc#17

alexcrichton · 2017-05-09T21:08:28Z

Thanks! Do you have a pointer to the commit for the disable-lazy-lock business? Or a pointer to the source which disables it on Windows today?

(just curious)

arthurprs · 2017-05-09T21:19:53Z

Apparently it involves multiple commits jemalloc/jemalloc#310 (comment)

Changelog: https://github.com/jemalloc/jemalloc/blob/31baedbbb9d1701b13312415b59d2b6240bb18e4/ChangeLog#L120

alexcrichton · 2017-05-09T21:25:01Z

@bors: r+

Sounds great!

bors · 2017-05-09T21:25:02Z

📌 Commit 8f420b4 has been approved by alexcrichton

…ichton Update jemalloc to 4.5.0 * pointing jemalloc submodule to the 4.5.0 HEAD + generated configure script * removed the now unnecessary extra config for windows. jemalloc is now much better at configuring itself overall. Let's see what the CI think.

TimNN · 2017-05-10T07:16:35Z

I believe that this has caused the failure of #41872 (maybe an interaction with #41863?):

[00:55:02] ===============================================================================
[00:55:02] running: "make" "build_lib_static" "-j" "4"
[00:55:02] /android/ndk/arm64-21/bin/aarch64-linux-android-gcc -std=gnu11 -Wall -Werror=declaration-after-statement -Wsign-compare -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -c -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -D_GNU_SOURCE -D_REENTRANT -I/checkout/src/liballoc_jemalloc/../jemalloc/include -Iinclude -o src/jemalloc.o /checkout/src/liballoc_jemalloc/../jemalloc/src/jemalloc.c
[00:55:02] /android/ndk/arm64-21/bin/aarch64-linux-android-gcc -std=gnu11 -Wall -Werror=declaration-after-statement -Wsign-compare -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -c -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -D_GNU_SOURCE -D_REENTRANT -I/checkout/src/liballoc_jemalloc/../jemalloc/include -Iinclude -o src/arena.o /checkout/src/liballoc_jemalloc/../jemalloc/src/arena.c
[00:55:02] /android/ndk/arm64-21/bin/aarch64-linux-android-gcc -std=gnu11 -Wall -Werror=declaration-after-statement -Wsign-compare -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -c -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -D_GNU_SOURCE -D_REENTRANT -I/checkout/src/liballoc_jemalloc/../jemalloc/include -Iinclude -o src/base.o /checkout/src/liballoc_jemalloc/../jemalloc/src/base.c
[00:55:02] /android/ndk/arm64-21/bin/aarch64-linux-android-gcc -std=gnu11 -Wall -Werror=declaration-after-statement -Wsign-compare -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -c -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -D_GNU_SOURCE -D_REENTRANT -I/checkout/src/liballoc_jemalloc/../jemalloc/include -Iinclude -o src/atomic.o /checkout/src/liballoc_jemalloc/../jemalloc/src/atomic.c
[00:55:02] Makefile:293: recipe for target 'src/arena.o' failed
[00:55:02] /android/ndk/arm64-21/bin/aarch64-linux-android-gcc -std=gnu11 -Wall -Werror=declaration-after-statement -Wsign-compare -pipe -g3 -fvisibility=hidden -O3 -funroll-loops -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -c -O2 -ffunction-sections -fdata-sections -fPIC -ffunction-sections -fdata-sections -fPIC -D_GNU_SOURCE -D_REENTRANT -I/checkout/src/liballoc_jemalloc/../jemalloc/include -Iinclude -o src/bitmap.o /checkout/src/liballoc_jemalloc/../jemalloc/src/bitmap.c
[00:55:02] 
[00:55:02] 
[00:55:02] command did not execute successfully: "make" "build_lib_static" "-j" "4"
[00:55:02] expected success, got: exit code: 2
[00:55:02] 
[00:55:02] 
[00:55:02] 
[00:55:02] --- stderr
[00:55:02] In file included from /android/ndk/arm64-21/sysroot/usr/include/sys/syscall.h:36:0,
[00:55:02]                  from /checkout/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/jemalloc_internal_decls.h:13,
[00:55:02]                  from include/jemalloc/internal/jemalloc_internal.h:5,
[00:55:02]                  from /checkout/src/liballoc_jemalloc/../jemalloc/src/arena.c:2:
[00:55:02] /checkout/src/liballoc_jemalloc/../jemalloc/src/arena.c: In function 'init_thp_initially_huge':
[00:55:02] /checkout/src/liballoc_jemalloc/../jemalloc/src/arena.c:3812:20: error: '__NR_open' undeclared (first use in this function)
[00:55:02]   fd = (int)syscall(SYS_open,
[00:55:02]                     ^
[00:55:02] /checkout/src/liballoc_jemalloc/../jemalloc/src/arena.c:3812:20: note: each undeclared identifier is reported only once for each function it appears in
[00:55:02] make: *** [src/arena.o] Error 1
[00:55:02] make: *** Waiting for unfinished jobs....
[00:55:02] 
[00:55:16] error: build failed
[00:55:16] 
[00:55:16] 
[00:55:16] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "-j" "4" "--target" "aarch64-linux-android" "--release" "--locked" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml"

arthurprs · 2017-05-10T08:05:06Z

I'm looking into it.

arthurprs · 2017-05-10T08:46:30Z

Once we merge rust-lang/jemalloc#18 I'll update the submodule reference here.

frewsxcv · 2017-05-10T13:03:33Z

@bors r-

arthurprs · 2017-05-10T19:41:13Z

Submodule updated. Shall we give it another shot?

Mark-Simulacrum · 2017-05-10T19:43:26Z

@bors r=alexcrichton

bors · 2017-05-10T19:43:27Z

📌 Commit 65d0be3 has been approved by alexcrichton

bors · 2017-05-10T20:02:08Z

⌛ Testing commit 65d0be3 with merge 583c025...

bors · 2017-05-10T21:32:31Z

💔 Test failed - status-travis

Mark-Simulacrum · 2017-05-10T21:36:54Z

Looks like 32-bit segfaults?

[01:20:39] error: process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/i686-unknown-linux-musl/release/deps/std-116dfe5a1b0286e7` (signal: 11, SIGSEGV: invalid memory reference)

arthurprs · 2017-05-11T07:23:45Z

😞

How can I simulate this build locally? I tried to follow the CI setup but I got lost real quick.

Mark-Simulacrum · 2017-05-11T13:07:40Z

@alexcrichton Could you help out here? We should provide some instructions on how to use Rust's CI containers locally on the forge...

frewsxcv · 2017-05-11T13:16:11Z

I'm on my phone, but you should be able toodify one of the CI scripts to whitelist the machines you want to test on in a script somewhere and then push to this branch

arthurprs · 2017-05-11T13:22:22Z

After posting that I tried running the container with src/ci/docker/run.sh dist-i586-gnu-i686-musl but I can't get it to compile the compiler_builtins crate.

Here's the output with -v flag active

Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
running: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "-j" "4" "--target" "x86_64-unknown-linux-gnu" "-v" "--release" "--locked" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml"
       Fresh libc v0.2.21
       Fresh core v0.0.0 (file:///checkout/src/libcore)
       Fresh filetime v0.1.10
       Fresh gcc v0.3.45
       Fresh rand v0.0.0 (file:///checkout/src/librand)
       Fresh alloc v0.0.0 (file:///checkout/src/liballoc)
       Fresh cmake v0.1.22
       Fresh libc v0.0.0 (file:///checkout/src/rustc/libc_shim)
       Fresh build_helper v0.1.0 (file:///checkout/src/build_helper)
       Fresh std_unicode v0.0.0 (file:///checkout/src/libstd_unicode)
       Fresh collections v0.0.0 (file:///checkout/src/libcollections)
       Fresh panic_abort v0.0.0 (file:///checkout/src/libpanic_abort)
       Fresh unwind v0.0.0 (file:///checkout/src/libunwind)
   Compiling compiler_builtins v0.0.0 (file:///checkout/src/libcompiler_builtins)
       Fresh panic_unwind v0.0.0 (file:///checkout/src/libpanic_unwind)
     Running `/checkout/obj/build/bootstrap/debug/rustc --crate-name compiler_builtins /checkout/src/libcompiler_builtins/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=863b57a66ba6c3e1 -C extra-filename=-863b57a66ba6c3e1 --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/release/deps --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libcore-bfaa82017ca17cb2.rlib -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/native/compiler-rt/. -l static=compiler-rt`
       Fresh alloc_system v0.0.0 (file:///checkout/src/liballoc_system)
       Fresh rustc_tsan v0.0.0 (file:///checkout/src/librustc_tsan)
       Fresh rustc_msan v0.0.0 (file:///checkout/src/librustc_msan)
       Fresh alloc_jemalloc v0.0.0 (file:///checkout/src/liballoc_jemalloc)
       Fresh rustc_lsan v0.0.0 (file:///checkout/src/librustc_lsan)
       Fresh rustc_asan v0.0.0 (file:///checkout/src/librustc_asan)
error: could not find native static library `compiler-rt`, perhaps an -L flag is missing?

error: Could not compile `compiler_builtins`.

Caused by:
  process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name compiler_builtins /checkout/src/libcompiler_builtins/lib.rs --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=863b57a66ba6c3e1 -C extra-filename=-863b57a66ba6c3e1 --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/release/deps --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-std/x86_64-unknown-linux-gnu/release/deps/libcore-bfaa82017ca17cb2.rlib -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/native/compiler-rt/. -l static=compiler-rt` (exit code: 101)


command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "-j" "4" "--target" "x86_64-unknown-linux-gnu" "-v" "--release" "--locked" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/checkout/src/libstd/Cargo.toml"
expected success, got: exit code: 101

alexcrichton · 2017-05-11T14:07:14Z

Ah yeah the README has some more info but looks like you found it. That's odd though that it failed to build, can you try blowing away local state, updating submodules, and trying again?

alexcrichton · 2017-05-11T20:43:29Z

@bors: retry

maybe a musl segfault? Spurious segfault in tests for musl #38618

arthurprs · 2017-05-12T11:12:13Z

Bors is on a strike?

Mark-Simulacrum · 2017-05-12T13:01:52Z

Well, we had one PR take like 6 hours to test since I think a builder didn't start or was really slow (no clue why) and then the rollup I created failed, so... we're waiting :). See https://buildbot2.rust-lang.org/homu/queue/rust if you want to follow progress.

arthurprs · 2017-05-12T13:04:49Z

I wasn't aware of that queue page, thanks!

bors · 2017-05-13T09:12:16Z

⌛ Testing commit 65d0be3 with merge 453cad6...

Update jemalloc to 4.5.0 * pointing jemalloc submodule to the 4.5.0 HEAD + generated configure script * removed the now unnecessary extra config for windows. jemalloc is now much better at configuring itself overall. Let's see what the CI think.

bors · 2017-05-13T11:35:56Z

☀️ Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing 453cad6 to master...

arthurprs · 2017-05-13T19:02:57Z

🎉

alexcrichton · 2017-06-07T21:36:38Z

Unfortunately I think this causes a regression on OSX 10.12.

@arthurprs would you be up for submitting a PR to revert?

arthurprs · 2017-06-08T08:41:02Z

Will do.

@alexcrichton

Update jemalloc to 4.5.0 Second try, including the fix for osx deadlock jemalloc/jemalloc#895. cc #41861 @alexcrichton @RalfJung

rust-highfive assigned brson May 9, 2017

frewsxcv mentioned this pull request May 10, 2017

Rollup of 11 pull requests #41869

Closed

frewsxcv mentioned this pull request May 10, 2017

Rollup of 9 pull requests #41872

Closed

arthurprs mentioned this pull request May 10, 2017

CherryPick: Use openat syscall if available rust-lang/jemalloc#18

Merged

Update jemalloc to 4.5.0

65d0be3

arthurprs force-pushed the update-jemalloc branch from 8f420b4 to 65d0be3 Compare May 10, 2017 16:36

TimNN added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label May 10, 2017

bors merged commit 65d0be3 into rust-lang:master May 13, 2017

arthurprs deleted the update-jemalloc branch May 13, 2017 19:03

BusyJay mentioned this pull request May 29, 2017

update jemalloc gnzlbg/jemallocator#9

Merged

alexcrichton mentioned this pull request Jun 7, 2017

Deadlock with multithreaded fork on OSX 10.12 jemalloc/jemalloc#895

Closed

Mark-Simulacrum mentioned this pull request Jun 11, 2017

Revert "Update jemalloc to 4.5.0" #42566

Merged

petrochenkov mentioned this pull request Jun 14, 2017

Rustbuild: No rule to make "rust/src/liballoc_jemalloc/../jemalloc/include/jemalloc/internal/ph.h" #42652

Closed

arthurprs mentioned this pull request Jul 27, 2017

CheryPick: Fix deadlock in multithreaded fork in OS X. rust-lang/jemalloc#19

Merged

arthurprs mentioned this pull request Aug 16, 2017

Update jemalloc to 4.5.0 #43911

Merged

bors added a commit that referenced this pull request Aug 20, 2017

Auto merge of #43911 - arthurprs:update-jemalloc, r=alexcrichton

5fa54ee

Update jemalloc to 4.5.0 Second try, including the fix for osx deadlock jemalloc/jemalloc#895. cc #41861 @alexcrichton @RalfJung

ashie mentioned this pull request Jul 28, 2021

macOS version doesn't use jemalloc fluent/fluent-package-builder#323

Open

Uh oh!

Conversation

arthurprs commented May 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented May 9, 2017

Uh oh!

arthurprs commented May 9, 2017

Uh oh!

alexcrichton commented May 9, 2017

Uh oh!

arthurprs commented May 9, 2017

Uh oh!

alexcrichton commented May 9, 2017

Uh oh!

bors commented May 9, 2017

Uh oh!

TimNN commented May 10, 2017

Uh oh!

arthurprs commented May 10, 2017

Uh oh!

arthurprs commented May 10, 2017

Uh oh!

frewsxcv commented May 10, 2017

Uh oh!

arthurprs commented May 10, 2017

Uh oh!

Mark-Simulacrum commented May 10, 2017

Uh oh!

bors commented May 10, 2017

Uh oh!

bors commented May 10, 2017

Uh oh!

bors commented May 10, 2017

Uh oh!

Mark-Simulacrum commented May 10, 2017

Uh oh!

arthurprs commented May 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mark-Simulacrum commented May 11, 2017

Uh oh!

frewsxcv commented May 11, 2017

Uh oh!

arthurprs commented May 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton commented May 11, 2017

Uh oh!

alexcrichton commented May 11, 2017

Uh oh!

arthurprs commented May 12, 2017

Uh oh!

Mark-Simulacrum commented May 12, 2017

Uh oh!

arthurprs commented May 12, 2017

Uh oh!

bors commented May 13, 2017

Uh oh!

bors commented May 13, 2017

Uh oh!

arthurprs commented May 13, 2017

Uh oh!

alexcrichton commented Jun 7, 2017

Uh oh!

arthurprs commented Jun 8, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

arthurprs commented May 9, 2017 •

edited

Loading

arthurprs commented May 11, 2017 •

edited

Loading

arthurprs commented May 11, 2017 •

edited

Loading