feat(generate-lockfile): Add unstable --publish-time flag #16265

epage · 2025-11-14T21:40:25Z

What does this PR try to resolve?

Implementation for #5221, #15491

Tracking issues: #16270, #16271

Use cases:

Improved reproduction steps using cargo scripts without including a
lockfile
Debugging issues in the past
Manual stop gap for Minimum age for dependencies #15973

This seemed like the shortest path to testing the proposed pubtime
index entries (#15491) so we can unblock other work on it like #15973.
While this has some big caveats, the design is straightforward.
In contrast, #15973 has a lot more design questions (is this a
resolver-wide setting or a per-registry setting, what formats are
accepted, etc) that could slow down development and testing pubtime.

How to test and review this PR?

Unresolved questions (deferred to the tracking issues):

How do we compensate for the caveats, with one option being to leave
this perma-unstable?
How should we deal with Summary pubtime parse errors?
How strict should we be on the Summary pubtime format?
Should we offer a convenient way of getting a compatible timestamp?

Future possibilities:

Ability to lock to a timestamp with cargo update updating the
timestamp (could be useful for cargo scripts)

rustbot · 2025-11-14T21:40:30Z

r? @ehuss

rustbot has assigned @ehuss.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

epage · 2025-11-14T21:51:59Z

tests/testsuite/cargo_generate_lockfile/help/stdout.term.svg

While the long value_name makes the format very clear, the length is annoying

epage · 2025-11-14T22:13:18Z

src/bin/cargo/commands/generate_lockfile.rs

+fn datetime_completer(current: &std::ffi::OsStr) -> Vec<CompletionCandidate> {
+    let mut completions = vec![];
+    let Some(current) = current.to_str() else {
+        return completions;
+    };
+
+    if current.is_empty() {
+        // While not likely what people want, it can at least give them a starting point to edit
+        let timestamp = jiff::Timestamp::now();
+        completions.push(CompletionCandidate::new(timestamp.to_string()));
+    } else if let Ok(date) = current.parse::<jiff::civil::Date>() {
+        if let Ok(zoned) = jiff::Zoned::default().with().date(date).build() {
+            let timestamp = zoned.timestamp();
+            completions.push(CompletionCandidate::new(timestamp.to_string()));
+        }
+    }
+    completions
+}


Didn't feel like writing my own parser for the sake of handling incomplete date, so I threw this together to at least make it easier to get the format correct so people can modify them from here.

weihanglo · 2025-11-14T22:14:03Z

src/bin/cargo/commands/generate_lockfile.rs

+    let mut ws = args.workspace(gctx)?;
+    if let Some(publish_time) = publish_time {
+        gctx.cli_unstable()
+            .fail_if_stable_opt("--publish-time", 5221)?;


Do we want to close the original issue and create a new one for tracking?
Benefit of this:

Cargo team has full control over the issue. Original author won't delete that accidentally.

Sometimes original issue has too many discussion. And we tend to use tracking issue for, well, tracking progress only.

Sometimes original issue has too many discussion. And we tend to use tracking issue for, well, tracking progress only.

I know the compiler team is concerned about this but it hasn't really felt like it has been too much of a problem for us.

As an end-user, it can be annoying to have to follow the problem you are tracking go through different issues for different stages of the process.

It also splits up the conversation and history.

The part I worry most about, which is less of a problem in this case, is when we close the original issue as the tracking issue may be for one particular solution which we might later reject and then we need to remember to re-open the original issue (I guess that is a reason to create a tracking issue and then close the original on stabilization?)

I lean towards creating a tracking issue but leaving the original open. Does that make sense?

Sounds good to me.

weihanglo · 2025-11-14T22:14:43Z

src/bin/cargo/commands/generate_lockfile.rs

+        .arg(
+            clap::Arg::new("publish-time")
+                .long("publish-time")
+                .value_name("yyyy-mm-ddThh:mm:ssZ")


mm is ambiguous here 😆.
(no need to change)

I had considered doing MM vs mm but

People likely pick it up from context

I wonder how many people remember which is which

I wanted to emphasize the difference between literals (upper case) and substitutions (lower case)

weihanglo · 2025-11-14T22:22:50Z

src/cargo/core/resolver/version_prefs.rs

+        self.publish_time = Some(publish_time);
+    }
+
    /// Sort (and filter) the given vector of summaries in-place


Probably expand this doc comment?

weihanglo · 2025-11-14T22:24:44Z

src/doc/man/cargo-generate-lockfile.md

+
+This is a best-effort filter on allowed packages, including:
+- packages from unsupported registries are always accepted
+- only the latest yank status is respected


I had a hard time understanding this. Do need a test with yanked versions?

I had a hard time understanding this.

We do not record each yank or unyank operation and timestamp them, instead only whether it is currently yanked or not. So if a package was yanked before publish-time and then unyanked after, we'll resolve with it unyanked despite that not being the state of the index as of publish-time.

Do need a test with yanked versions?

I'm trying to decide what a test would look like that would be meaningful. it would basically be a test showing that yanking works. Even if we later track yank history, we'd need to update our testing infrastructure to support it and it is unlikely it would be done in a way for us to know to update that test.

If a package was yanked but this yank happened after the time passed to --publish-time, will it be considered not yanked by the resolver?

cargo generate-lockfile will only look at the current yank status, independent of --publish-time. In your scenario, it is currently yanked so that is the state that will be used.

Wouldn't part of the value of having --publish-time at all be to be able to generate a working lockfile even if all compatible versions of a crate are yanked?

If your concern is more about wanting to bypass yanked status, we have other places where we are discussing ways to deal with that.

For the given use case of "lock to a specific time", for it to be accurate to the past, we would need the yank status of as of that time. I recognized that this is a shortcoming of this solution by documenting it in the help, calling it out in the PR description ("big caveats'), and leaving it as an unresolved question for the tracking issue. As I said in the PR description, this is more about providing a testing ground for the pubtime Index Summary entry. Even then, I think there are still use cases for this feature as-is.

only the latest yank status is respected

Should we expand this a bit to say that yank time is not tracked? It is not obvious to users (even me) what "latest yank status" means here, as they may not even know yank is a mutating operation.

Made a small tweak but likely not covering what you are looking for. Maybe it is curse of knowledge but I'm having a hard time thinking of how to further clarify this. maybe provide an example?

Looks good to me. Let's tweak if any other confusion pops up.

weihanglo · 2025-11-15T01:09:40Z

crates/cargo-util-schemas/src/index.rs

    /// incorrectly select a new package that uses a new index schema. A
    /// workaround is to downgrade any packages that are incompatible with the
    /// `--precise` flag of `cargo update`.
    pub v: Option<u32>,


We should bump a index version, no?

iiuc the index version represents incompatible versions of the schema. This change is forward and backwards compatible so it shouldn't need it bumped.

How were version 2 and 3 (see the comment) incompatible versions? I probably forgot the policy. Looks like every field additions should be treated as the same version?

Also, have we dealt with CURRENT_CACHE_VERSION the on disk summary cache? If a cache was generated by older cargo, and them cargo generate-lockfile --publish-time read from it, would it not respect the pubtime field because of missing?

I'm unsure why bindeps bumps the version but it does declare an incompatible version, see

cargo/src/cargo/sources/registry/index/mod.rs

Lines 682 to 699 in b5354b5

let v = index.v.unwrap_or(1);

tracing::trace!("json parsed registry {}/{}", index.name, index.vers);

let v_max = if bindeps {

INDEX_V_MAX + 1

} else {

INDEX_V_MAX

};

if v_max < v {

Ok(IndexSummary::Unsupported(summary, v))

} else if !valid {

Ok(IndexSummary::Invalid(summary))

} else if index.yanked.unwrap_or(false) {

Ok(IndexSummary::Yanked(summary))

} else {

Ok(IndexSummary::Candidate(summary))

}

As this new field is set on every Summary, if we bump .v for this, then registries providing this information would break support for all older versions of Cargo.

We did not bump it when we added rust_version.

Also, have we dealt with CURRENT_CACHE_VERSION the on disk summary cache? If a cache was generated by older cargo, and them cargo generate-lockfile --publish-time read from it, would it not respect the pubtime field because of missing?

That cache is for what we get back from the registry, before we turn it into Cargo-native types and possibly lose fields, so if the server has it, we'll preserve it.

I do not think we bump this field when doing backports but detect that we need to overwrite it. We didn't do anything for it for rust_version.

I do not think we bump this field when doing backports but detect that we need to overwrite it. We didn't do anything for it for rust_version.

Yeah I thought about this as well. crates.io index backport shouldn't touch the index version.

Seems reasonable to me.

crates/cargo-util-schemas/src/index.rs

rustbot · 2025-11-17T19:58:15Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Implementation for rust-lang#5221 Use cases: - Improved reproduction steps using cargo scripts without including a lockfile - Debugging issues in the past - Manual stop gap for rust-lang#15973 Unresolved questions: - How do we compensate for the caveats, with one option being to leave this perma-unstable? - How should we deal with Summary `pubtime` parse errors? - How strict should we be on the Summary `pubtime` format? - Should we offer a convenient way of getting a compatible timestamp? Future possibilities: - Ability to lock to a timestamp with `cargo update` updating the timestamp (could be useful for cargo scripts) This seemed like the shortest path to testing the proposed `pubtime` index entries so we can unblock other work on it like rust-lang#15973. While this has some big caveats, the design is straightforward. In contrast, rust-lang#15973 has a lot more design questions (is this a resolver-wide setting or a per-registry setting, what formats are accepted, etc) that could slow down development and testing `pubtime`.

weihanglo

This adds an unstable --publish-time flag mainly for demonstrating the use of the pubtime field in the crates.io index files. We don't have any plan to stabilize the flag yet.

View changes since this review

weihanglo · 2025-11-18T17:42:58Z

crates/cargo-util-schemas/src/index.rs

    /// incorrectly select a new package that uses a new index schema. A
    /// workaround is to downgrade any packages that are incompatible with the
    /// `--precise` flag of `cargo update`.
    pub v: Option<u32>,


I do not think we bump this field when doing backports but detect that we need to overwrite it. We didn't do anything for it for rust_version.

Yeah I thought about this as well. crates.io index backport shouldn't touch the index version.

Seems reasonable to me.

weihanglo · 2025-11-18T17:45:00Z

src/doc/man/cargo-generate-lockfile.md

+
+This is a best-effort filter on allowed packages, including:
+- packages from unsupported registries are always accepted
+- only the latest yank status is respected


Looks good to me. Let's tweak if any other confusion pops up.

epage added 2 commits November 14, 2025 10:59

chore(schemas): Bump to v0.11

a83ebb9

feat(index): Add unstable pubtime field

c802c3e

rustbot assigned ehuss Nov 14, 2025

epage mentioned this pull request Nov 14, 2025

registry-index: Add release date to Index Format #15491

Open

epage commented Nov 14, 2025

View reviewed changes

epage force-pushed the pubtime branch from 74e6375 to f5d465a Compare November 14, 2025 22:12

epage commented Nov 14, 2025

View reviewed changes

weihanglo reviewed Nov 14, 2025

View reviewed changes

weihanglo reviewed Nov 15, 2025

View reviewed changes

epage commented Nov 15, 2025

View reviewed changes

crates/cargo-util-schemas/src/index.rs Show resolved Hide resolved

epage force-pushed the pubtime branch from f5d465a to 30e868e Compare November 17, 2025 19:58

This was referenced Nov 17, 2025

Tracking Issue for _pubtime_ #16270

Open

Tracking Issue for _lockfile-publish-time_ #16271

Open

epage added 3 commits November 17, 2025 14:23

feat(summary): Add unstable pubtime field

1340d22

test(generate-lockfile): Add test to show --publish-time

b9bdb38

epage force-pushed the pubtime branch from 30e868e to 665150d Compare November 17, 2025 20:25

epage mentioned this pull request Nov 17, 2025

Cargo time machine (generate lock files based on old registry state) #5221

Open

epage force-pushed the pubtime branch 2 times, most recently from 1828830 to 57d0592 Compare November 18, 2025 16:49

weihanglo approved these changes Nov 18, 2025

View reviewed changes

weihanglo added this pull request to the merge queue Nov 18, 2025

Merged via the queue into rust-lang:master with commit 6ce5357 Nov 18, 2025
28 checks passed

rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 18, 2025

weihanglo mentioned this pull request Nov 19, 2025

Update cargo submodule rust-lang/rust#149073

Open

	let v = index.v.unwrap_or(1);
	tracing::trace!("json parsed registry {}/{}", index.name, index.vers);

	let v_max = if bindeps {
	INDEX_V_MAX + 1
	} else {
	INDEX_V_MAX
	};

	if v_max < v {
	Ok(IndexSummary::Unsupported(summary, v))
	} else if !valid {
	Ok(IndexSummary::Invalid(summary))
	} else if index.yanked.unwrap_or(false) {
	Ok(IndexSummary::Yanked(summary))
	} else {
	Ok(IndexSummary::Candidate(summary))
	}

feat(generate-lockfile): Add unstable --publish-time flag #16265

feat(generate-lockfile): Add unstable --publish-time flag #16265

Uh oh!

Conversation

epage commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR try to resolve?

How to test and review this PR?

Uh oh!

rustbot commented Nov 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rustbot commented Nov 17, 2025

Uh oh!

weihanglo left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

epage commented Nov 14, 2025 •

edited

Loading

weihanglo left a comment •

edited by rustbot

Loading