Skip to content

Fix debuginfo compression in bootstrap#158169

Open
Kobzol wants to merge 6 commits into
rust-lang:mainfrom
Kobzol:fix-cc-rs-flag-if-supported
Open

Fix debuginfo compression in bootstrap#158169
Kobzol wants to merge 6 commits into
rust-lang:mainfrom
Kobzol:fix-cc-rs-flag-if-supported

Conversation

@Kobzol

@Kobzol Kobzol commented Jun 20, 2026

Copy link
Copy Markdown
Member

View all comments

Found through https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/weird.20code.20in.20fill_target_compiler/with/604588655.

This PR solves a few issues with debuginfo compression in bootstrap.

The main issue was that debuginfo compression through the -gz flag wasn't enabled, by mistake.

When cc-rs checks if a compiler flag is supported, it tries to read out_dir to generate the source file in. However, when this option is not set, then the check silently fails and the flag is considered to be unsupported. Since we didn't set out_dir, all such added flags were simply ignored.

Normally, it reads OUT_DIR, which is fine if cc-rs is used within a build script. But if it is used elsewhere, then this is a pretty gnarly footgun in cc-rs, because the failure is completely silent. CC @madsmtm or @NobodyXu in case you have thoughts on that :)

After that was fixed, I had to change the used compression flag from -gz to --compress-debug-sections, to support both BFD and LLD linkers.

And to solve it properly, and allow dist CI jobs to configure debuginfo compression for all targets that they build, I added a new bootstrap option to configure debuginfo compression.

I wanted the flag to be configurable both globally and per target. At the same time, the flag applies both to C/C++ and Rust. We currently don't have such config area in bootstrap.toml, and [build] didn't seem like the right place, so I shoved it into [rust] (while documenting that it also applies to C/C++).

r? @bjorn3

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jun 20, 2026
@bjorn3

bjorn3 commented Jun 20, 2026

Copy link
Copy Markdown
Member

Thanks!

@bors r+

@rust-bors

rust-bors Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

📌 Commit 1bc29dc has been approved by bjorn3

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 20, 2026
@rust-bors

This comment has been minimized.

rust-bors Bot pushed a commit that referenced this pull request Jun 20, 2026
Fix cc rs flag if supported

Found in https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/weird.20code.20in.20fill_target_compiler/with/604588655.

When `cc-rs` checks if a compiler flag is supported, it tries to read `out_dir` to generate the source file in. However, when this option is not set, then the check silently fails and the flag is considered to be unsupported. Since we didn't set `out_dir`, all such added flags were simply ignored.

Normally, it reads `OUT_DIR`, which is fine if `cc-rs` is used within a build script. But if it is used elsewhere, then this is a pretty gnarly footgun in `cc-rs`, because the failure is [completely silent](https://github.com/rust-lang/cc-rs/blob/main/src/lib.rs#L1483).  CC @madsmtm or @NobodyXu in case you have thoughts on that :)

r? @bjorn3
@rust-log-analyzer

This comment has been minimized.

@rust-bors rust-bors Bot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 20, 2026
@rust-bors

rust-bors Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

💔 Test for b946733 failed: CI. Failed job:

@rust-bors rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 20, 2026
@rustbot rustbot added A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jun 20, 2026
@Kobzol

Kobzol commented Jun 20, 2026

Copy link
Copy Markdown
Member Author

@bors try jobs=x86_64-gnu-distcheck

@rust-bors

This comment has been minimized.

rust-bors Bot pushed a commit that referenced this pull request Jun 20, 2026
Fix cc rs flag if supported


try-job: x86_64-gnu-distcheck
@NobodyXu

Copy link
Copy Markdown
Contributor

Thanks, I've opened an issue #1765, I think this is something we should fix.

We could use a tempfile instead for it

@ChrisDenton

Copy link
Copy Markdown
Member

Thanks, I've opened an issue #1765

I think you meant rust-lang/cc-rs#1765 😉

@rust-bors

rust-bors Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

☀️ Try build successful (CI)
Build commit: d269546 (d269546b8320f5f3255b5c7e6268def1a4808374)
Base parent: 4008bbd (4008bbdf34b583ce44a467043aae3703afc0a76f)

@Kobzol

Kobzol commented Jun 20, 2026

Copy link
Copy Markdown
Member Author

@bors r=bjorn3

Let's try again.

@rust-bors

rust-bors Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

📌 Commit d59253e has been approved by bjorn3

It is now in the queue for this repository.

@Kobzol

Kobzol commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Oh, it is hardcoded 🤦 This keeps getting better.

So maybe we can detect if the linker is lld, and in that case use -compress-debug-section=zlib instead?

@bjorn3

bjorn3 commented Jun 21, 2026

Copy link
Copy Markdown
Member

Yeah, should work.
Edit: Fat fingered the close button.

@bjorn3 bjorn3 closed this Jun 21, 2026
@rust-bors rust-bors Bot added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jun 21, 2026
@rustbot rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 21, 2026
@rust-bors

rust-bors Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

This pull request was unapproved due to being closed.

@bjorn3 bjorn3 reopened this Jun 21, 2026
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 21, 2026
@rustbot

rustbot commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@Kobzol Kobzol force-pushed the fix-cc-rs-flag-if-supported branch from 6dcec99 to ead43fb Compare June 21, 2026 11:47
@Kobzol Kobzol changed the title Fix cc rs flag if supported Fix debuginfo compression in bootstrap Jun 21, 2026
@Kobzol

Kobzol commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

I decide to leave this in a better state than it was, and added an option to bootstrap for debuginfo compression.

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rust-bors

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 21, 2026
rust-bors Bot pushed a commit that referenced this pull request Jun 21, 2026
@rust-log-analyzer

This comment has been minimized.

@rust-bors

rust-bors Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

☀️ Try build successful (CI)
Build commit: 9372e13 (9372e132b33ebc6bda95a4477c349388793efbd5)
Base parent: a774017 (a7740170e5e2f733b32a8206d5bb439c8e8c2fce)

@rust-timer

This comment has been minimized.

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (9372e13): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking means the PR may be perf-sensitive. Consider adding rollup=never if this change is not fit for rolling up.

@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This perf run didn't have relevant results for this metric.

Max RSS (memory usage)

Results (primary -2.1%, secondary -2.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
3.3% [3.3%, 3.3%] 1
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-3.9% [-5.9%, -1.8%] 3
Improvements ✅
(secondary)
-2.8% [-2.8%, -2.8%] 1
All ❌✅ (primary) -2.1% [-5.9%, 3.3%] 4

Cycles

Results (secondary 2.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
2.7% [2.7%, 2.7%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 482.353s -> 481.6s (-0.16%)
Artifact size: 390.76 MiB -> 386.51 MiB (-1.09%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 21, 2026
@Kobzol

Kobzol commented Jun 21, 2026

Copy link
Copy Markdown
Member Author

Nice artifact size wins! Looks like we weren't compressing libstd nor Cargo (I confirmed it using readelf -S --wide locally).

@Kobzol Kobzol requested a review from jieyouxu June 21, 2026 15:02
@jieyouxu jieyouxu self-assigned this Jun 22, 2026

@jieyouxu jieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, bootstrap changes look good to me, r=me with or without nit.

View changes since this review

}
}
StringOrBool::String(value) => CompressDebuginfo::from_str(&value)
.map_err(|_| D::Error::invalid_value(Unexpected::Str(&value), &"zlib or off"))?,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit:

Suggested change
.map_err(|_| D::Error::invalid_value(Unexpected::Str(&value), &"zlib or off"))?,
.map_err(|_| D::Error::invalid_value(Unexpected::Str(&value), &"`zlib` or `off`"))?,

@jieyouxu jieyouxu removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 23, 2026
Comment thread src/ci/run.sh
fi

# Compress any remaining debuginfo to make the resulting artifacts slightly smaller
RUST_CONFIGURE_ARGS="$RUST_CONFIGURE_ARGS --set rust.compress-debuginfo=zlib"

@bjorn3 bjorn3 Jun 23, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be in the dist profile?

View changes since the review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants