Skip to content

rustdoc: rename the doc parts metadata params #159415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rust-bors merged 1 commit into from
Jul 17, 2026
Merged

rustdoc: rename the doc parts metadata params #159415

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 20 additions & 9 deletions src/doc/rustdoc/src/unstable-features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,15 +197,26 @@ themselves marked as unstable. To use any of these options, pass `-Z unstable-op
the flag in question to Rustdoc on the command-line. To do this from Cargo, you can either use the
`RUSTDOCFLAGS` environment variable or the `cargo rustdoc` command.

### `--merge`, `--parts-out-dir`, and `--include-parts-dir`
### `--write-doc-meta-dir`, and `--read-doc-meta-dir`

These options control how rustdoc handles files that combine data from multiple crates.

By default, they act like `--merge=shared` is set, and `--parts-out-dir` and `--include-parts-dir`
are turned off. The `--merge=shared` mode causes rustdoc to load the existing data in the out-dir,
combine the new crate data into it, and write the result. This is very easy to use in scripts that
manually invoke rustdoc, but it's also slow, because it performs O(crates) work on
every crate, meaning it performs O(crates<sup>2</sup>) work.
By default, rustdoc will read the doc meta from the doc output dir itself and merge them together.
This is very easy to use in scripts that manually invoke rustdoc, but it's also slow, because it
performs O(crates) work on every crate, meaning it performs O(crates<sup>2</sup>) work. When
`--write-doc-meta-dir` and/or `--read-doc-meta-dir` are supplied, this is turned off.

When `--write-doc-meta-dir` is supplied, rustdoc will write the crate's metadata to that directory.
If this parameter is supplied but `--read-doc-meta-dir` isn't, it runs in *intermediate mode*:
some pages may be written to the output dir, but there is a lot of functionality that won't work
until rustdoc is run in *finalize mode*.

When `--read-doc-meta-dir` is supplied, rustdoc runs in *finalize mode*. It will read the data from
the supplied directory, and will write it to the doc output directory in the form that the web
frontend will use.

If both `--write-doc-meta-dir` and `--read-doc-meta-dir` are specified, the crate metadata will be
written to both the HTML `--out-dir` and to the supplied `--write-doc-meta-dir`.
Comment on lines +209 to +219

@camelid camelid Jul 17, 2026 •
edited by rustbot
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like these names (better than the previous and what I'd proposed), and I think this is a good interface given the current state of CCI. Long-term, once full metadata is implemented, I don't want rustdoc to emit HTML at all until its last invocation in a workspace (i.e. the last crate, or maybe even a step after that). However, that's obviously a ways off, so we shouldn't block CCI on that. But I would like to be forwards-compatible with it as much as possible.

So, I'm wondering if we should forbid the combination of --write-doc-meta-dir and --read-doc-meta-dir. Right now, there is no use for it as far as I know. Notice that the original --merge flag only had three options, not four, for this reason.

To be clear, my long-term vision for rustdoc usage:

  • no flags: legacy shared mode
  • just write: produce only metadata for the current crate, no HTML -- but the metadata includes all of clean and anything else needed to generate HTML later
  • read and write: read in metadata for dependencies so anything that needs info about them, e.g., inlining will work. produce only metadata for the current crate, no HTML
  • just read: read in metadata for all documented crates in the workspace, and produce a static HTML site for all of them
    To implement this in a way that is backwards-compatible with CCI, we'd probably need to add a --defer-html flag that switches to the mode I just described. Note that this mode would require feeding in metadata to all intermediate crates, not just the final crate like in CCI.

The reason to delay HTML generation until the end is then we have complete information about the workspace and can remove all of our hacks that use JS to inject downstream implementors, etc. Rustdoc really wants to operate at the workspace level, not the crate level where it actually operators, which is why we have so much pain from cross-crate inlining and other features.

View changes since the review

@notriddle notriddle Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want rustdoc to emit HTML at all until its last invocation in a workspace (i.e. the last crate

It sounds like you’re expecting rustdoc to perform finalization at the same time that it documents the last crate? That makes sense, and that’s why the current system lets you do read and write-doc-meta-dir at the same time.

It was designed that way because the build system might schedule a different crate to be documented last in a different run. For example, with Cargo, you would supply the -p parameter. To make sure this doesn’t cause problems, you’re expected to write doc metadata for every crate in your workspace, even the one that gets documented last and used for finalization.

One thing that does concern me, though, is that Cargo always does finalization as a separate step, because the finalization step isn’t parallelized. It might not make sense to worry about doing finalization and documentation at the same time, if no builds systems want to do that.

@camelid camelid Jul 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly. I would prefer that finalization happens as a separate step (like you said Cargo does) after the last crate. In the future, once finalization will do all the HTML generation too, this would also get rid of our weird quirk where the last documented crate is considered the main one.

I suppose supporting --read and --write together is fine for CCI. The future version of rustdoc that defers all HTML generation until the end will need to be gated behind a new flag anyway, so we can change the interface when that flag is passed.

@camelid camelid Jul 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, if we don't need to support simultaneous documenting-last-crate and finalizing, it'd be better to save ourselves the trouble and just disallow it. I'm a bit rusty on the details of the CCI implementation, but it sounds like rustdoc already supports finalizing as a separate step like you said?

Sorry if this all seems a bit pedantic, I just want to make sure we don't paint ourselves into a corner.

@notriddle notriddle Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's possible. I'll do that, then.

@camelid camelid Jul 17, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! And thanks for taking action to make this PR, much appreciated.


```console
$ rustdoc crate1.rs --out-dir=doc
Expand All @@ -217,13 +228,13 @@ rd_("fcrate1fcrate2")
```

To delay shared-data merging until the end of a build, so that you only have to perform O(crates)
work, use `--merge=none` on every crate except the last one, which will use `--merge=finalize`.
work, use `--write-doc-meta-dir` on every crate, and the last will use `--read-doc-meta-dir`.

```console
$ rustdoc +nightly crate1.rs --merge=none --parts-out-dir=crate1.d -Zunstable-options
$ rustdoc +nightly crate1.rs --write-doc-meta=crate1.d -Zunstable-options
$ cat doc/search.index/crateNames/*
cat: 'doc/search.index/crateNames/*': No such file or directory
$ rustdoc +nightly crate2.rs --merge=finalize --include-parts-dir=crate1.d -Zunstable-options
$ rustdoc +nightly crate2.rs --read-doc-meta=crate1.d -Zunstable-options
$ cat doc/search.index/crateNames/*
rd_("fcrate1fcrate2")
```
Expand Down
104 changes: 62 additions & 42 deletions src/librustdoc/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -601,34 +601,66 @@ impl Options {

let (lint_opts, describe_lints, lint_cap) = get_cmd_lint_options(early_dcx, matches);

let input = if describe_lints {
InputMode::HasFile(make_input(early_dcx, ""))
} else {
match matches.free.as_slice() {
[] if matches.opt_str("merge").as_deref() == Some("finalize") => {
InputMode::NoInputMergeFinalize
}
[] => dcx.fatal("missing file operand"),
[input] => InputMode::HasFile(make_input(early_dcx, input)),
_ => dcx.fatal("too many file operands"),
}
};

let externs = parse_externs(early_dcx, matches, &unstable_opts);
let extern_html_root_urls = match parse_extern_html_roots(matches) {
Ok(ex) => ex,
Err(err) => dcx.fatal(err),
};

let parts_out_dir =
match matches.opt_str("parts-out-dir").map(PathToParts::from_flag).transpose() {
let mut parts_out_dir =
match matches.opt_str("write-doc-meta-dir").map(PathToParts::from_flag).transpose() {
Ok(parts_out_dir) => parts_out_dir,
Err(e) => dcx.fatal(e),
};
let include_parts_dir = match parse_include_parts_dir(matches) {
let mut include_parts_dir = match parse_read_doc_meta(matches, "read-doc-meta-dir") {
Ok(include_parts_dir) => include_parts_dir,
Err(e) => dcx.fatal(e),
};
let mut should_merge = compute_should_merge(matches);
if parts_out_dir.is_none() && include_parts_dir.is_empty() {
// we'll need to get rid of this stuff once Cargo stops using them
parts_out_dir =
match matches.opt_str("parts-out-dir").map(PathToParts::from_flag).transpose() {
Ok(parts_out_dir) => parts_out_dir,
Err(e) => dcx.fatal(e),
};
include_parts_dir = match parse_read_doc_meta(matches, "include-parts-dir") {
Ok(include_parts_dir) => include_parts_dir,
Err(e) => dcx.fatal(e),
};
should_merge = match matches.opt_str("merge").as_deref() {
None => ShouldMerge { read_rendered_cci: true, write_rendered_cci: true },
Some("none") => ShouldMerge { read_rendered_cci: false, write_rendered_cci: false },
Some("shared") => ShouldMerge { read_rendered_cci: true, write_rendered_cci: true },
Some("finalize") => {
ShouldMerge { read_rendered_cci: false, write_rendered_cci: true }
}
Some(_) => dcx.fatal("argument to --merge must be `none`, `shared`, or `finalize`"),
};
} else if matches.opt_str("parts-out-dir").is_some() {
dcx.fatal(
"deprecated version of write-doc-meta-dir is used with new doc-meta-dir stuff",
);
} else if matches.opt_str("include-parts-dir").is_some() {
dcx.fatal(
"deprecated version of read-doc-meta-dir is used with new doc-meta-dir stuff",
);
} else if matches.opt_str("merge").is_some() {
dcx.fatal("deprecated parameter merge is used with new doc-meta-dir stuff");
}

let input = if describe_lints {
InputMode::HasFile(make_input(early_dcx, ""))
} else {
match matches.free.as_slice() {
[] if !include_parts_dir.is_empty() && should_merge.write_rendered_cci => {
InputMode::NoInputMergeFinalize
}
[] => dcx.fatal("missing file operand"),
[input] => InputMode::HasFile(make_input(early_dcx, input)),
_ => dcx.fatal("too many file operands"),
}
};

let default_settings: Vec<Vec<(String, String)>> = vec![
matches
Expand Down Expand Up @@ -853,10 +885,6 @@ impl Options {
let extern_html_root_takes_precedence =
matches.opt_present("extern-html-root-takes-precedence");
let html_no_source = matches.opt_present("html-no-source");
let should_merge = match parse_merge(matches) {
Ok(result) => result,
Err(e) => dcx.fatal(format!("--merge option error: {e}")),
};
let merge_doctests = parse_merge_doctests(matches, edition, dcx);
tracing::debug!("merge_doctests: {merge_doctests:?}");

Expand Down Expand Up @@ -1051,7 +1079,7 @@ impl PathToParts {
// check here is for diagnostics
if path.exists() && !path.is_dir() {
Err(format!(
"--parts-out-dir and --include-parts-dir expect directories, found: {}",
"--write-doc-meta-dir and --read-doc-meta-dir expect directories, found: {}",
path.display(),
))
} else {
Expand All @@ -1061,15 +1089,15 @@ impl PathToParts {
}
}

/// Reports error if --include-parts-dir is not a directory
fn parse_include_parts_dir(m: &getopts::Matches) -> Result<Vec<PathToParts>, String> {
/// Reports error if --read-doc-meta-dir is not a directory
fn parse_read_doc_meta(m: &getopts::Matches, name: &str) -> Result<Vec<PathToParts>, String> {
let mut ret = Vec::new();
for p in m.opt_strs("include-parts-dir") {
for p in m.opt_strs(name) {
let p = PathToParts::from_flag(p)?;
// this is just for diagnostic
if !p.0.is_dir() {
return Err(format!(
"--include-parts-dir expected {} to be a directory",
"--read-doc-meta-dir expected {} to be a directory",
p.0.display()
));
}
Expand All @@ -1089,23 +1117,15 @@ pub(crate) struct ShouldMerge {

/// Extracts read_rendered_cci and write_rendered_cci from command line arguments, or
/// reports an error if an invalid option was provided
fn parse_merge(m: &getopts::Matches) -> Result<ShouldMerge, &'static str> {
match m.opt_str("merge").as_deref() {
// default = read-write
None => Ok(ShouldMerge { read_rendered_cci: true, write_rendered_cci: true }),
Some("none") if m.opt_present("include-parts-dir") => {
Err("--include-parts-dir not allowed if --merge=none")
}
Some("none") => Ok(ShouldMerge { read_rendered_cci: false, write_rendered_cci: false }),
Some("shared") if m.opt_present("parts-out-dir") || m.opt_present("include-parts-dir") => {
Err("--parts-out-dir and --include-parts-dir not allowed if --merge=shared")
}
Some("shared") => Ok(ShouldMerge { read_rendered_cci: true, write_rendered_cci: true }),
Some("finalize") if m.opt_present("parts-out-dir") => {
Err("--parts-out-dir not allowed if --merge=finalize")
}
Some("finalize") => Ok(ShouldMerge { read_rendered_cci: false, write_rendered_cci: true }),
Some(_) => Err("argument to --merge must be `none`, `shared`, or `finalize`"),
fn compute_should_merge(m: &getopts::Matches) -> ShouldMerge {
match (m.opt_present("read-doc-meta-dir"), m.opt_present("write-doc-meta-dir")) {
// shared mode
(false, false) => ShouldMerge { read_rendered_cci: true, write_rendered_cci: true },
// intermediate mode
(false, true) => ShouldMerge { read_rendered_cci: false, write_rendered_cci: false },
// finalize mode
(true, false) => ShouldMerge { read_rendered_cci: false, write_rendered_cci: true },
(true, true) => ShouldMerge { read_rendered_cci: false, write_rendered_cci: true },
}
}

Expand Down
35 changes: 24 additions & 11 deletions src/librustdoc/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -613,28 +613,41 @@ fn opts() -> Vec<RustcOptGroup> {
Unstable,
Opt,
"",
"merge",
"Controls how rustdoc handles files from previously documented crates in the doc root\n\
none = Do not write cross-crate information to the --out-dir\n\
shared = Append current crate's info to files found in the --out-dir\n\
finalize = Write current crate's info and --include-parts-dir info to the --out-dir, overwriting conflicting files",
"none|shared|finalize",
"write-doc-meta-dir",
"Writes trait implementations and other info for the current crate to provided path",
"path/to/doc.meta",
),
opt(
Unstable,
Multi,
"",
"read-doc-meta-dir",
"Includes trait implementations and other crate info from provided path",
"path/to/doc.meta",
),
opt(
Unstable,
Opt,
"",
"parts-out-dir",
"Writes trait implementations and other info for the current crate to provided path. Only use with --merge=none",
"path/to/doc.parts/<crate-name>",
"Deprecated synonym of write-doc-meta-dir",
"path/to/doc.meta",
),
opt(
Unstable,
Multi,
"",
"include-parts-dir",
"Includes trait implementations and other crate info from provided path. Only use with --merge=finalize",
"path/to/doc.parts/<crate-name>",
"Deprecated synonym of read-doc-meta-dir",
"path/to/doc.meta",
),
opt(
Unstable,
Opt,
"",
"merge",
"Deprecated option to specify read/write-doc-meta-dir mode",
"none, shared, finalize",
),
opt(Unstable, Flag, "", "html-no-source", "Disable HTML source code pages generation", ""),
opt(
Expand Down Expand Up @@ -758,7 +771,7 @@ fn run_renderer<

/// Renders and writes cross-crate info files, like the search index. This function exists so that
/// we can run rustdoc without a crate root in the `--merge=finalize` mode. Cross-crate info files
/// discovered via `--include-parts-dir` are combined and written to the doc root.
/// discovered via `--read-doc-meta-dir` are combined and written to the doc root.
fn run_merge_finalize(opt: config::RenderOptions) -> Result<(), error::Error> {
assert!(
opt.should_merge.write_rendered_cci,
Expand Down
26 changes: 11 additions & 15 deletions tests/run-make/rustdoc-default-output/output-default.stdout
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -176,23 +176,19 @@ Options:
--scrape-tests Include test code when scraping examples
--with-examples path to function call information (for displaying examples in the documentation)

--merge none|shared|finalize
Controls how rustdoc handles files from previously
documented crates in the doc root
none = Do not write cross-crate information to the
--out-dir
shared = Append current crate's info to files found in
the --out-dir
finalize = Write current crate's info and
--include-parts-dir info to the --out-dir, overwriting
conflicting files
--parts-out-dir path/to/doc.parts/<crate-name>
--write-doc-meta-dir path/to/doc.meta
Writes trait implementations and other info for the
current crate to provided path. Only use with
--merge=none
--include-parts-dir path/to/doc.parts/<crate-name>
current crate to provided path
--read-doc-meta-dir path/to/doc.meta
Includes trait implementations and other crate info
from provided path. Only use with --merge=finalize
from provided path
--parts-out-dir path/to/doc.meta
Deprecated synonym of write-doc-meta-dir
--include-parts-dir path/to/doc.meta
Deprecated synonym of read-doc-meta-dir
--merge none, shared, finalize
Deprecated option to specify read/write-doc-meta-dir
mode
--html-no-source
Disable HTML source code pages generation
--doctest-build-arg ARG
Expand Down
28 changes: 10 additions & 18 deletions tests/run-make/rustdoc-merge-directory-alias/rmake.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Running --merge=finalize without an input crate root should not trigger ICE.
// Running --read-doc-meta-dir without an input crate root should not trigger ICE.
// Issue: https://github.com/rust-lang/rust/issues/146646

//@ needs-target-std
Expand All @@ -14,71 +14,63 @@ fn main() {
.input("dep1.rs")
.out_dir(&out_dir)
.arg("-Zunstable-options")
.arg(format!("--parts-out-dir={}", parts_out_dir.display()))
.arg("--merge=none")
.arg(format!("--write-doc-meta-dir={}", parts_out_dir.display()))
.run();
assert!(parts_out_dir.join("dep1.json").exists());

let output = rustdoc()
.arg("-Zunstable-options")
.out_dir(&out_dir)
.arg(format!("--include-parts-dir={}", parts_out_dir.display()))
.arg("--merge=finalize")
.arg(format!("--read-doc-meta-dir={}", parts_out_dir.display()))
.run();
output.assert_stderr_not_contains("error: the compiler unexpectedly panicked. this is a bug.");

rustdoc()
.input("dep2.rs")
.out_dir(&out_dir)
.arg("-Zunstable-options")
.arg(format!("--parts-out-dir={}", parts_out_dir.display()))
.arg("--merge=none")
.arg(format!("--write-doc-meta-dir={}", parts_out_dir.display()))
.run();
assert!(parts_out_dir.join("dep2.json").exists());

let output2 = rustdoc()
.arg("-Zunstable-options")
.out_dir(&out_dir)
.arg(format!("--include-parts-dir={}", parts_out_dir.display()))
.arg("--merge=finalize")
.arg(format!("--read-doc-meta-dir={}", parts_out_dir.display()))
.run();
output2.assert_stderr_not_contains("error: the compiler unexpectedly panicked. this is a bug.");

rustdoc()
.input("dep1.rs")
.out_dir(&out_dir)
.arg("-Zunstable-options")
.arg(format!("--parts-out-dir={}", parts_out_dir.display()))
.arg("--merge=none")
.arg(format!("--write-doc-meta-dir={}", parts_out_dir.display()))
.run();
assert!(parts_out_dir.join("dep1.json").exists());

let output3 = rustdoc()
.arg("-Zunstable-options")
.out_dir(&out_dir)
.arg(format!("--include-parts-dir={}", parts_out_dir.display()))
.arg("--merge=finalize")
.arg(format!("--read-doc-meta-dir={}", parts_out_dir.display()))
.run();
output3.assert_stderr_not_contains("error: the compiler unexpectedly panicked. this is a bug.");

// dep_missing is different, because --parts-out-dir is not supplied
// dep_missing is different, because --write-doc-meta-dir is not supplied
rustdoc().input("dep_missing.rs").out_dir(&out_dir).run();
assert!(parts_out_dir.join("dep2.json").exists());

rustdoc()
.input("dep1.rs")
.out_dir(&out_dir)
.arg("-Zunstable-options")
.arg(format!("--parts-out-dir={}", parts_out_dir.display()))
.arg("--merge=none")
.arg(format!("--write-doc-meta-dir={}", parts_out_dir.display()))
.run();
assert!(parts_out_dir.join("dep1.json").exists());

let output4 = rustdoc()
.arg("-Zunstable-options")
.out_dir(&out_dir)
.arg(format!("--include-parts-dir={}", parts_out_dir.display()))
.arg("--merge=finalize")
.arg(format!("--read-doc-meta-dir={}", parts_out_dir.display()))
.run();
output4.assert_stderr_not_contains("error: the compiler unexpectedly panicked. this is a bug.");

Expand Down
Loading
Loading