Auto merge of #146233 - jieyouxu:run-make-fission, r=Kobzol

Split `run-make` into two {`run-make`,`run-make-cargo`} test suites

## Summary

Split `tests/run-make` into two test suites, to make it faster and more convenient for contributors to run run-make tests that do not need in-tree `cargo`.

| New test suites        | Explanation                                                                                                                                                                                                                              |
| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tests/run-make`       | The "fast path" test suite intended for run-make tests that do not need in-tree `cargo`. These tests may not use `cargo`.                                                                                                                |
| `tests/run-make-cargo` | The "slow path" test suite that requires checking out `cargo` submodule and building in-tree `cargo`, and thus will have access to in-tree `cargo`. In practice, these constitute a very small portion of the original `run-make` tests. |

This PR carries out [MCP 847: Split run-make test suite into slower-building test suite with suitably-staged cargo and faster-building test suite without cargo](rust-lang/compiler-team#847).
Fixes rust-lang/rust#135573 (for the tests that do not need in-tree `cargo`).
Fixes rust-lang/rust#134109.

## Remarks

- I considered if we want to split by in-tree tools previously. However, as discussed rust-lang/rust#134109, in practice `rustdoc` is not very slow to build, but `cargo` takes a good few minutes. So, the partition boundary was determined to be along in-tree `cargo` availability.
- The `run-make` tests previously that wanted to use `cargo` cannot just use the bootstrap `cargo`, otherwise they would run into situations where bootstrap `cargo` can significantly diverge from in-tree `cargo` (see rust-lang/rust#130642).

---

try-job: aarch64-msvc-1
try-job: test-various
try-job: x86_64-gnu-debug
try-job: aarch64-gnu-debug
try-job: aarch64-apple
try-job: dist-various-1

Author: bors

src/tests/adding.md

@@ -29,6 +29,8 @@ guidelines:
     suites.
   - Need to inspect the resulting binary in some way? Or if all the other test
     suites are too limited for your purposes? Then use `run-make`.
+    - Use `run-make-cargo` if you need to exercise in-tree `cargo` in conjunction
+      with in-tree `rustc`.
   - Check out the [compiletest] chapter for more specialized test suites.
 
 After deciding on which kind of test to add, see [best

src/tests/best-practices.md

@@ -83,10 +83,10 @@ related tests.
     - E.g. for an implementation of RFC 2093 specifically, we can group a
       collection of tests under `tests/ui/rfc-2093-infer-outlives/`. For the
       directory name, include what the RFC is about.
-- For the [`run-make`] test suite, each `rmake.rs` must be contained within an
-  immediate subdirectory under `tests/run-make/`. Further nesting is not
-  presently supported. Avoid including issue number in the directory name too,
-  include that info in a comment inside `rmake.rs`.
+- For the [`run-make`]/`run-make-support` test suites, each `rmake.rs` must
+  be contained within an immediate subdirectory under `tests/run-make/` or
+  `tests/run-make-cargo/` respectively. Further nesting is not presently
+  supported. Avoid using _only_ an issue number for the test name as well.
 
 ## Test descriptions
 

src/tests/compiletest.md

@@ -397,13 +397,19 @@ your test, causing separate files to be generated for 32bit and 64bit systems.
 
 ### `run-make` tests
 
-The tests in [`tests/run-make`] are general-purpose tests using Rust *recipes*,
-which are small programs (`rmake.rs`) allowing arbitrary Rust code such as
-`rustc` invocations, and is supported by a [`run_make_support`] library. Using
-Rust recipes provide the ultimate in flexibility.
+The tests in [`tests/run-make`] and [`tests/run-make-cargo`] are general-purpose
+tests using Rust *recipes*, which are small programs (`rmake.rs`) allowing
+arbitrary Rust code such as `rustc` invocations, and is supported by a
+[`run_make_support`] library. Using Rust recipes provide the ultimate in
+flexibility.
 
 `run-make` tests should be used if no other test suites better suit your needs.
 
+The `run-make-cargo` test suite additionally builds an in-tree `cargo` to support
+use cases that require testing in-tree `cargo` in conjunction with in-tree `rustc`.
+The `run-make` test suite does not have access to in-tree `cargo` (so it can be the
+faster-to-iterate test suite).
+
 #### Using Rust recipes
 
 Each test should be in a separate directory with a `rmake.rs` Rust program,
@@ -476,6 +482,7 @@ Then add a corresponding entry to `"rust-analyzer.linkedProjects"`
 ```
 
 [`tests/run-make`]: https://github.com/rust-lang/rust/tree/master/tests/run-make
+[`tests/run-make-cargo`]: https://github.com/rust-lang/rust/tree/master/tests/run-make-cargo
 [`run_make_support`]: https://github.com/rust-lang/rust/tree/master/src/tools/run-make-support
 
 ### Coverage tests

src/tests/directives.md

@@ -52,14 +52,14 @@ not be exhaustive. Directives can generally be found by browsing the
 
 See [Building auxiliary crates](compiletest.html#building-auxiliary-crates)
 
-| Directive             | Explanation                                                                                           | Supported test suites | Possible values                               |
-|-----------------------|-------------------------------------------------------------------------------------------------------|-----------------------|-----------------------------------------------|
-| `aux-bin`             | Build a aux binary, made available in `auxiliary/bin` relative to test directory                      | All except `run-make` | Path to auxiliary `.rs` file                  |
-| `aux-build`           | Build a separate crate from the named source file                                                     | All except `run-make` | Path to auxiliary `.rs` file                  |
-| `aux-crate`           | Like `aux-build` but makes available as extern prelude                                                | All except `run-make` | `<extern_prelude_name>=<path/to/aux/file.rs>` |
-| `aux-codegen-backend` | Similar to `aux-build` but pass the compiled dylib to `-Zcodegen-backend` when building the main file | `ui-fulldeps`         | Path to codegen backend file                  |
-| `proc-macro`          | Similar to `aux-build`, but for aux forces host and don't use `-Cprefer-dynamic`[^pm].                | All except `run-make` | Path to auxiliary proc-macro `.rs` file       |
-| `build-aux-docs`      | Build docs for auxiliaries as well.  Note that this only works with `aux-build`, not `aux-crate`.     | All except `run-make` | N/A                                           |
+| Directive             | Explanation                                                                                           | Supported test suites                  | Possible values                               |
+|-----------------------|-------------------------------------------------------------------------------------------------------|----------------------------------------|-----------------------------------------------|
+| `aux-bin`             | Build a aux binary, made available in `auxiliary/bin` relative to test directory                      | All except `run-make`/`run-make-cargo` | Path to auxiliary `.rs` file                  |
+| `aux-build`           | Build a separate crate from the named source file                                                     | All except `run-make`/`run-make-cargo` | Path to auxiliary `.rs` file                  |
+| `aux-crate`           | Like `aux-build` but makes available as extern prelude                                                | All except `run-make`/`run-make-cargo` | `<extern_prelude_name>=<path/to/aux/file.rs>` |
+| `aux-codegen-backend` | Similar to `aux-build` but pass the compiled dylib to `-Zcodegen-backend` when building the main file | `ui-fulldeps`                          | Path to codegen backend file                  |
+| `proc-macro`          | Similar to `aux-build`, but for aux forces host and don't use `-Cprefer-dynamic`[^pm].                | All except `run-make`/`run-make-cargo` | Path to auxiliary proc-macro `.rs` file       |
+| `build-aux-docs`      | Build docs for auxiliaries as well.  Note that this only works with `aux-build`, not `aux-crate`.     | All except `run-make`/`run-make-cargo` | N/A                                           |
 
 [^pm]: please see the [Auxiliary proc-macro section](compiletest.html#auxiliary-proc-macro) in the compiletest chapter for specifics.
 
@@ -243,18 +243,18 @@ ignoring debuggers.
 
 ### Affecting how tests are built
 
-| Directive           | Explanation                                                                                  | Supported test suites     | Possible values                                                                            |
-|---------------------|----------------------------------------------------------------------------------------------|---------------------------|--------------------------------------------------------------------------------------------|
-| `compile-flags`     | Flags passed to `rustc` when building the test or aux file                                   | All except for `run-make` | Any valid `rustc` flags, e.g. `-Awarnings -Dfoo`. Cannot be `-Cincremental` or `--edition` |
-| `edition`           | The edition used to build the test                                                           | All except for `run-make` | Any valid `--edition` value                                                                |
-| `rustc-env`         | Env var to set when running `rustc`                                                          | All except for `run-make` | `<KEY>=<VALUE>`                                                                            |
-| `unset-rustc-env`   | Env var to unset when running `rustc`                                                        | All except for `run-make` | Any env var name                                                                           |
-| `incremental`       | Proper incremental support for tests outside of incremental test suite                       | `ui`, `crashes`           | N/A                                                                                        |
-| `no-prefer-dynamic` | Don't use `-C prefer-dynamic`, don't build as a dylib via a `--crate-type=dylib` preset flag | `ui`, `crashes`           | N/A                                                                                        |
+| Directive           | Explanation                                                                                  | Supported test suites                      | Possible values                                                                            |
+|---------------------|----------------------------------------------------------------------------------------------|--------------------------------------------|--------------------------------------------------------------------------------------------|
+| `compile-flags`     | Flags passed to `rustc` when building the test or aux file                                   | All except for `run-make`/`run-make-cargo` | Any valid `rustc` flags, e.g. `-Awarnings -Dfoo`. Cannot be `-Cincremental` or `--edition` |
+| `edition`           | The edition used to build the test                                                           | All except for `run-make`/`run-make-cargo` | Any valid `--edition` value                                                                |
+| `rustc-env`         | Env var to set when running `rustc`                                                          | All except for `run-make`/`run-make-cargo` | `<KEY>=<VALUE>`                                                                            |
+| `unset-rustc-env`   | Env var to unset when running `rustc`                                                        | All except for `run-make`/`run-make-cargo` | Any env var name                                                                           |
+| `incremental`       | Proper incremental support for tests outside of incremental test suite                       | `ui`, `crashes`                            | N/A                                                                                        |
+| `no-prefer-dynamic` | Don't use `-C prefer-dynamic`, don't build as a dylib via a `--crate-type=dylib` preset flag | `ui`, `crashes`                            | N/A                                                                                        |
 
 <div class="warning">
 
-Tests (outside of `run-make`) that want to use incremental tests not in the
+Tests (outside of `run-make`/`run-make-cargo`) that want to use incremental tests not in the
 incremental test-suite must not pass `-C incremental` via `compile-flags`, and
 must instead use the `//@ incremental` directive.
 
@@ -264,9 +264,9 @@ Consider writing the test as a proper incremental test instead.
 
 ### Rustdoc
 
-| Directive   | Explanation                                                  | Supported test suites                    | Possible values           |
-|-------------|--------------------------------------------------------------|------------------------------------------|---------------------------|
-| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json`  | Any valid `rustdoc` flags |
+| Directive   | Explanation                                                  | Supported test suites                   | Possible values           |
+|-------------|--------------------------------------------------------------|-----------------------------------------|---------------------------|
+| `doc-flags` | Flags passed to `rustdoc` when building the test or aux file | `rustdoc`, `rustdoc-js`, `rustdoc-json` | Any valid `rustdoc` flags |
 
 <!--
 **FIXME(rustdoc)**: what does `check-test-line-numbers-match` do?

src/tests/misc.md

@@ -24,8 +24,8 @@ In `ui` tests and other test suites that support `//@ rustc-env`, you can specif
 //@ rustc-env:RUSTC_BOOTSTRAP=-1
 ```
 
-For `run-make` tests, `//@ rustc-env` is not supported. You can do something
-like the following for individual `rustc` invocations.
+For `run-make`/`run-make-cargo` tests, `//@ rustc-env` is not supported. You can do
+something like the following for individual `rustc` invocations.
 
 ```rust,ignore
 use run_make_support::rustc;