Optimizing docs generation

[jcotton42]

I had brought up the size of the generated rustdoc for windows-rs on the Rust Community Discord and rustdoc team member @jyn514 mentioned that adding the #![doc(html_no_source)] to the crate should suppress the HTML rendering of the crate source.

https://doc.rust-lang.org/stable/rustdoc/the-doc-attribute.html#html_no_source

[kennykerr]

Thanks!

[jyn514]

I can't build the docs on linux:

  = note: /usr/bin/ld: cannot find -loleaut32
          /usr/bin/ld: cannot find -lkernel32
          /usr/bin/ld: cannot find -loleaut32
          /usr/bin/ld: cannot find -lkernel32
          /usr/bin/ld: cannot find -lole32
          /usr/bin/ld: cannot find -lkernel32
          /usr/bin/ld: cannot find -lkernel32
          /usr/bin/ld: cannot find -lkernel32
          /usr/bin/ld: cannot find -loleaut32

I'd be interested how long rustdoc takes to run after you add html_no_source. Could someone generate some timing info for me?

$ rustup install nightly
$ time -v cargo +nightly rustdoc -- -Z time-passes

and then upload windows-PID.mm_profdata.
(time -v tells the total memory usage; if it's not on windows anything else that gives the max RSS works too)

[kennykerr]

Hey @jyn514 thanks for the help! @rylev may have some tips for how to build on Linux - I have no idea.

Note that you need to build a separate repo. The docs are too huge that I didn't want to include them inside the windows-rs repo. Instead, you can clone the https://github.com/microsoft/windows-docs-rs repo and build the crates/bindings package. Something like this:

C:\git\windows-docs-rs\crates\bindings>cargo doc --no-deps

This takes about 30 minutes and 20GB of memory. I updated the lib.rs file to look like this:

#![doc(html_no_source)]
::windows::include_bindings!();

Note that time -v is not available on Windows. I ran the following:

cargo +nightly rustdoc -- -Z time-passes

It took about 52 mins and it's peak working set (memory) is over 21GB.

I could not find a windows-PID.mm_profdata anywhere.

[jyn514]

Hmm, it should be there somewhere - do you see any file ending in .mm_profdata? It may be in the workspaces root, not the directory you ran cargo in.

If not, what version of rustdoc are you using? I think the format of the self-profile files changed a few months ago.

[kennykerr]

Not from what I can tell:

[kennykerr]

C:\git\windows-docs-rs>rustdoc --version
rustdoc 1.49.0 (e1884a8e3 2020-12-29)

[jyn514]

Oh shoot, I'm sorry, it should have been -Z self-profile, not -Z time-passes. It's ok, time-passes gave some useful info too. In particular it shows most of the time is spent on get_blanket_impls, which I've been meaning to improve for a while. It doesn't show where the memory usage is coming from, though, it already uses 18 GB before looking at impls.

You posted the default version of rustdoc - can you post the nightly version instead? rustdoc +nightly --version.

[kennykerr]

C:\git\windows-docs-rs>rustdoc +nightly --version
rustdoc 1.51.0-nightly (a4cbb44ae 2021-01-20)

OK, I'll rerun with self-profile.

[kennykerr]

Hmm, it's rather large. 😉

[kennykerr]

Not sure if it will work, but I've uploaded to OneDrive. Let me know if you can download it.

[jyn514]

That worked, thanks! Downloading it now, I'll let you know what I find :)

[jyn514]

By the way, you can analyze it yourself with measureme: https://github.com/rust-lang/measureme/