[#688] Add selective testing tool to the project#692
Open
phongvhd93 wants to merge 4 commits intodevelopfrom
Open
[#688] Add selective testing tool to the project#692phongvhd93 wants to merge 4 commits intodevelopfrom
phongvhd93 wants to merge 4 commits intodevelopfrom
Conversation
📝 WalkthroughWalkthroughThis PR integrates XcodeSelectiveTesting into CI workflows and the project template to enable selective test execution based on changed files in PRs. It refactors scheme generation to use Xcode test plans ( Changes
Sequence Diagram(s)sequenceDiagram
participant PR as Pull Request
participant CI as CI Workflow
participant Mint as Mint / XcodeSelectiveTesting
participant XcodeBuild as Xcode / Build System
participant TestPlan as Test Plan (.xctestplan)
PR->>CI: Trigger on PR creation/update
activate CI
CI->>Mint: Setup Mint tool
activate Mint
Mint-->>CI: Mint ready
deactivate Mint
CI->>Mint: Run XcodeSelectiveTesting (compare origin/base_ref to current)
activate Mint
Mint->>TestPlan: Analyze changed files
Mint->>TestPlan: Modify test targets in .xctestplan
Mint-->>CI: Test plan updated
deactivate Mint
CI->>XcodeBuild: Build and Test (using modified test plan)
activate XcodeBuild
XcodeBuild->>TestPlan: Load .xctestplan
XcodeBuild->>XcodeBuild: Execute selective tests only
XcodeBuild-->>CI: Test results
deactivate XcodeBuild
deactivate CI
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
phongvhd93
force-pushed
the
feature/688-selective-testing
branch
from
098c731 to
ae77fb4
Compare
phongvhd93
changed the title
phongvhd93
requested review from
Shayokh144,
Thieurom,
donmartial,
dquangit,
ducbm051291,
kevinhoangpq,
markgravity,
minhnimble,
nhgia,
nkhanh44,
nmint8m,
patcharapon-j,
roman-nimble,
suho,
thinh2k1310 and
vnntsu
as code owners
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
template/fastlane/Fastfile.swift (1)
169-191: Duplicate guard logic inbuildAndTestLaneandbuildAndTestDevLane.Both lanes repeat the same
testTargetCount() > 0check with the same echo message. Consider extracting into a small helper to keep the two lanes in sync going forward.♻️ Suggested refactor
func buildAndTestLane() { desc("Build and Test project") - guard testTargetCount() > 0 else { - echo(message: "🚨 Nothing to test") - return - } + guard hasTestsToRun() else { return } Test.buildAndTest( environment: .staging, devices: Constant.devices ) } func buildAndTestDevLane() { desc("Build and Test dev project") - guard testTargetCount() > 0 else { - echo(message: "🚨 Nothing to test") - return - } + guard hasTestsToRun() else { return } Test.buildAndTest( environment: .dev, devices: Constant.devices ) } + + private func hasTestsToRun() -> Bool { + guard testTargetCount() > 0 else { + echo(message: "🚨 Nothing to test") + return false + } + return true + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@template/fastlane/Fastfile.swift` around lines 169 - 191, Both buildAndTestLane and buildAndTestDevLane duplicate the guard testTargetCount() > 0 and echo logic; extract that check into a small helper (e.g., ensureTestTargetsExist()) that performs the guard, calls echo(message: "🚨 Nothing to test") and returns a Bool or throws to short-circuit; replace the inline guards in buildAndTestLane and buildAndTestDevLane with a call to this helper before invoking Test.buildAndTest, preserving the same environment and devices arguments.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
@.cicdtemplate/.github/self-hosted-workflows/automatic_pull_request_review.yml:
- Around line 44-48: The workflow calls "mint run" but never installs Mint; fix
by either adding mint to the [tools] section in .mise.toml so the existing
"Install tools with mise" step provides it, or add an explicit "Setup Mint" step
(e.g., using irgaly/setup-mint@v1) before the job that runs mint; update the
self-hosted workflow in automatic_pull_request_review.yml to include one of
these options so the mint command is available when Modify test plan (mint run
...) executes.
In `@template/`{PROJECT_NAME}.xctestplan:
- Around line 14-43: The test plan contains hardcoded "TestTemplate" values that
prevent template substitution; update the four containerPath entries (currently
"container:TestTemplate.xcodeproj") and the test target name "TestTemplateTests"
to use the project placeholder {PROJECT_NAME} (so they resolve to
{PROJECT_NAME}.xcodeproj and {PROJECT_NAME}Tests at generation time) so
SetUpIOSProject.replaceTextInFiles() can correctly replace {PROJECT_NAME} with
projectNameNoSpace/name; ensure you only replace the literal "TestTemplate"
occurrences in the "testTargets" objects (containerPath and name fields) with
the {PROJECT_NAME} placeholder and do not pre-expand other placeholders.
In `@template/fastlane/Fastfile.swift`:
- Around line 280-287: In testTargetCount(in:) the guard unnecessarily uses
optional chaining on json (declared non-optional); change json?["testTargets"]
to json["testTargets"] in the guard so it reads let testTargets =
json["testTargets"] as? [Any], keeping the rest of the guard and return logic
unchanged to avoid the compiler warning in the private func testTargetCount(in
testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int.
In `@template/Package.resolved`:
- Line 2: Revert the spurious change to Package.resolved by restoring the
original "originHash" value (undo the modification of the "originHash" entry) or
remove Package.resolved from this commit so the auto-generated file isn't
changed without a corresponding Package.swift update; target the
Package.resolved file and ensure the "originHash" key matches the
upstream/generated value before committing.
---
Nitpick comments:
In `@template/fastlane/Fastfile.swift`:
- Around line 169-191: Both buildAndTestLane and buildAndTestDevLane duplicate
the guard testTargetCount() > 0 and echo logic; extract that check into a small
helper (e.g., ensureTestTargetsExist()) that performs the guard, calls
echo(message: "🚨 Nothing to test") and returns a Bool or throws to
short-circuit; replace the inline guards in buildAndTestLane and
buildAndTestDevLane with a call to this helper before invoking
Test.buildAndTest, preserving the same environment and devices arguments.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2cf14ad9-e937-41d7-b934-08685d1c6447
📒 Files selected for processing (9)
.cicdtemplate/.github/self-hosted-workflows/automatic_pull_request_review.yml.cicdtemplate/.github/workflows/automatic_pull_request_review.ymlscripts/iOSTemplateMaker/Sources/iOSTemplateMaker/SetUpiOSProject.swifttemplate/Package.resolvedtemplate/Project.swifttemplate/Tuist/ProjectDescriptionHelpers/Scheme+Initializing.swifttemplate/fastlane/Fastfile.swifttemplate/fastlane/Helpers/Test.swifttemplate/{PROJECT_NAME}.xctestplan
Comment on lines
+44
to
+48
| - name: Modify test plan | ||
| run: | | ||
| mint run mikeger/XcodeSelectiveTesting@1c0927e5553a9ddb6b3634d6d13497651a0ef476 {PROJECT_NAME}.xcodeproj \ | ||
| --test-plan {PROJECT_NAME}.xctestplan \ | ||
| --base-branch origin/${{ github.base_ref }} |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check whether mint is declared in mise configuration or mentioned in self-hosted setup
fd -H -t f '\.mise\.toml$' | xargs -I{} sh -c 'echo "--- {} ---"; cat "{}"'
rg -n --hidden 'mint' -g '!*.lock' -g '!node_modules/**'Repository: nimblehq/ios-templates
Length of output: 1537
Add mint to .mise.toml or include an explicit Setup Mint step.
The self-hosted workflow uses mint run at line 46, but unlike the hosted workflow (which adds irgaly/setup-mint@v1), this file has no explicit Mint setup. The Install tools with mise step relies on .mise.toml, but mint is not declared there. This will cause the workflow to fail with command not found when attempting to run the Mint command.
Either add mint to the [tools] section in .mise.toml, or add a Setup Mint step symmetric to the hosted workflow.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In
@.cicdtemplate/.github/self-hosted-workflows/automatic_pull_request_review.yml
around lines 44 - 48, The workflow calls "mint run" but never installs Mint; fix
by either adding mint to the [tools] section in .mise.toml so the existing
"Install tools with mise" step provides it, or add an explicit "Setup Mint" step
(e.g., using irgaly/setup-mint@v1) before the job that runs mint; update the
self-hosted workflow in automatic_pull_request_review.yml to include one of
these options so the mint command is available when Modify test plan (mint run
...) executes.
Comment on lines
+14
to
+43
| "testTargets" : [ | ||
| { | ||
| "target" : { | ||
| "containerPath" : "container:TestTemplate.xcodeproj", | ||
| "identifier" : "PLACEHOLDER", | ||
| "name" : "DomainTests" | ||
| } | ||
| }, | ||
| { | ||
| "target" : { | ||
| "containerPath" : "container:TestTemplate.xcodeproj", | ||
| "identifier" : "PLACEHOLDER", | ||
| "name" : "TestTemplateTests" | ||
| } | ||
| }, | ||
| { | ||
| "target" : { | ||
| "containerPath" : "container:TestTemplate.xcodeproj", | ||
| "identifier" : "PLACEHOLDER", | ||
| "name" : "DataTests" | ||
| } | ||
| }, | ||
| { | ||
| "target" : { | ||
| "containerPath" : "container:TestTemplate.xcodeproj", | ||
| "identifier" : "PLACEHOLDER", | ||
| "name" : "ModelTests" | ||
| } | ||
| } | ||
| ], |
There was a problem hiding this comment.
Hardcoded TestTemplate breaks template substitution.
containerPath values (lines 17, 24, 31, 38) and the main-app test target name TestTemplateTests (line 26) are hardcoded to TestTemplate instead of using the {PROJECT_NAME} placeholder. After SetUpIOSProject.replaceTextInFiles() runs, it replaces {PROJECT_NAME} with projectNameNoSpace, but TestTemplate is not a placeholder and will be left untouched. For any generated project whose name isn't TestTemplate:
container:TestTemplate.xcodeprojwill not resolve to the actual.xcodeproj(which is\(projectNameNoSpace).xcodeproj).TestTemplateTestswill not match the generated app test target\(name)Tests.
As a result, Xcode will refuse to load the test plan and buildAndTestLane will either fail or silently fall through the testTargetCount() > 0 guard (since the JSON is structurally valid, it will actually try to run and fail). Replace the hardcoded string with the project placeholder.
🐛 Proposed fix
{
"target" : {
- "containerPath" : "container:TestTemplate.xcodeproj",
+ "containerPath" : "container:{PROJECT_NAME}.xcodeproj",
"identifier" : "PLACEHOLDER",
"name" : "DomainTests"
}
},
{
"target" : {
- "containerPath" : "container:TestTemplate.xcodeproj",
+ "containerPath" : "container:{PROJECT_NAME}.xcodeproj",
"identifier" : "PLACEHOLDER",
- "name" : "TestTemplateTests"
+ "name" : "{PROJECT_NAME}Tests"
}
},
{
"target" : {
- "containerPath" : "container:TestTemplate.xcodeproj",
+ "containerPath" : "container:{PROJECT_NAME}.xcodeproj",
"identifier" : "PLACEHOLDER",
"name" : "DataTests"
}
},
{
"target" : {
- "containerPath" : "container:TestTemplate.xcodeproj",
+ "containerPath" : "container:{PROJECT_NAME}.xcodeproj",
"identifier" : "PLACEHOLDER",
"name" : "ModelTests"
}
}As per coding guidelines: "Do not replace template placeholders (e.g., {PROJECT_NAME}, {BUNDLE_ID_STAGING}) with concrete values inside the template; preserve them for generation-time substitution".
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| "testTargets" : [ | |
| { | |
| "target" : { | |
| "containerPath" : "container:TestTemplate.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "DomainTests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:TestTemplate.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "TestTemplateTests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:TestTemplate.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "DataTests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:TestTemplate.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "ModelTests" | |
| } | |
| } | |
| ], | |
| "testTargets" : [ | |
| { | |
| "target" : { | |
| "containerPath" : "container:{PROJECT_NAME}.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "DomainTests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:{PROJECT_NAME}.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "{PROJECT_NAME}Tests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:{PROJECT_NAME}.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "DataTests" | |
| } | |
| }, | |
| { | |
| "target" : { | |
| "containerPath" : "container:{PROJECT_NAME}.xcodeproj", | |
| "identifier" : "PLACEHOLDER", | |
| "name" : "ModelTests" | |
| } | |
| } | |
| ], |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@template/`{PROJECT_NAME}.xctestplan around lines 14 - 43, The test plan
contains hardcoded "TestTemplate" values that prevent template substitution;
update the four containerPath entries (currently
"container:TestTemplate.xcodeproj") and the test target name "TestTemplateTests"
to use the project placeholder {PROJECT_NAME} (so they resolve to
{PROJECT_NAME}.xcodeproj and {PROJECT_NAME}Tests at generation time) so
SetUpIOSProject.replaceTextInFiles() can correctly replace {PROJECT_NAME} with
projectNameNoSpace/name; ensure you only replace the literal "TestTemplate"
occurrences in the "testTargets" objects (containerPath and name fields) with
the {PROJECT_NAME} placeholder and do not pre-expand other placeholders.
Comment on lines
+280
to
+287
| private func testTargetCount(in testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int { | ||
| guard let data = FileManager.default.contents(atPath: testPlanPath), | ||
| let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], | ||
| let testTargets = json?["testTargets"] as? [Any] else { | ||
| return 0 | ||
| } | ||
| return testTargets.count | ||
| } |
There was a problem hiding this comment.
Unnecessary optional chaining on json.
After the guard let json = ... unwrap, json is a non-optional [String: Any], so json?["testTargets"] is redundant optional chaining — the compiler will warn. Prefer json["testTargets"].
🛠️ Proposed fix
- private func testTargetCount(in testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int {
- guard let data = FileManager.default.contents(atPath: testPlanPath),
- let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
- let testTargets = json?["testTargets"] as? [Any] else {
- return 0
- }
- return testTargets.count
- }
+ private func testTargetCount(in testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int {
+ guard let data = FileManager.default.contents(atPath: testPlanPath),
+ let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
+ let testTargets = json["testTargets"] as? [Any] else {
+ return 0
+ }
+ return testTargets.count
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| private func testTargetCount(in testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int { | |
| guard let data = FileManager.default.contents(atPath: testPlanPath), | |
| let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], | |
| let testTargets = json?["testTargets"] as? [Any] else { | |
| return 0 | |
| } | |
| return testTargets.count | |
| } | |
| private func testTargetCount(in testPlanPath: String = "\(Constant.projectName).xctestplan") -> Int { | |
| guard let data = FileManager.default.contents(atPath: testPlanPath), | |
| let json = try? JSONSerialization.jsonObject(with: data) as? [String: Any], | |
| let testTargets = json["testTargets"] as? [Any] else { | |
| return 0 | |
| } | |
| return testTargets.count | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@template/fastlane/Fastfile.swift` around lines 280 - 287, In
testTargetCount(in:) the guard unnecessarily uses optional chaining on json
(declared non-optional); change json?["testTargets"] to json["testTargets"] in
the guard so it reads let testTargets = json["testTargets"] as? [Any], keeping
the rest of the guard and return logic unchanged to avoid the compiler warning
in the private func testTargetCount(in testPlanPath: String =
"\(Constant.projectName).xctestplan") -> Int.
| @@ -1,5 +1,5 @@ | |||
| { | |||
| "originHash" : "f20f61ffa814731b5dbc7abb5719681df174b8f6a45d9dbd426c5e8a3ecc538f", | |||
| "originHash" : "38800aafaa37508835c8747531d4b300ffb86052e604eef3d958cf6cff0cbadc", | |||
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check if Package.swift was modified in this PR and show what changed
# Check if Package.swift is in the PR diff
git diff origin/develop...HEAD --name-only | grep -E 'Package\.swift$'
# If found, show the changes
if git diff origin/develop...HEAD --name-only | grep -qE 'Package\.swift$'; then
echo "=== Package.swift was modified. Changes: ==="
git diff origin/develop...HEAD -- '**/Package.swift'
else
echo "=== Package.swift was NOT modified in this PR ==="
fiRepository: nimblehq/ios-templates
Length of output: 220
Revert the Package.resolved change.
Package.swift was not modified in this PR, so the originHash change is spurious. Since this file is auto-generated and its change has no corresponding source modification, revert it to avoid introducing unnecessary noise in the PR.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@template/Package.resolved` at line 2, Revert the spurious change to
Package.resolved by restoring the original "originHash" value (undo the
modification of the "originHash" entry) or remove Package.resolved from this
commit so the auto-generated file isn't changed without a corresponding
Package.swift update; target the Package.resolved file and ensure the
"originHash" key matches the upstream/generated value before committing.
ducbm051291
reviewed
Apr 22, 2026
| "testTargets" : [ | ||
| { | ||
| "target" : { | ||
| "containerPath" : "container:TestTemplate.xcodeproj", |
Contributor
There was a problem hiding this comment.
This test plan is still hardcoded to TestTemplate...please update it
Suggested change
| "containerPath" : "container:TestTemplate.xcodeproj", | |
| "containerPath" : "container:{PROJECT_NAME}.xcodeproj", |
Thieurom
reviewed
Apr 22, 2026
Comment on lines
+28
to
+30
| - name: Setup Mint | ||
| uses: irgaly/setup-mint@8566ee44b3a79d20642d1924db21c8ab0402859f # irgaly/setup-mint@v1 | ||
|
|
There was a problem hiding this comment.
Can we use mise to manage the new CLI tool instead of introducing a new package manager?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened 👀
automatic_pull_request_reviewworkflowtestTargetCountto Fastfile to count the number of test targets in a test plan.buildAndTestLaneInsight 📝
This focuses on GitHub Actions, the remaining services will be updated in another PRs
To save the CI run time, I implemented XCodeSelectiveTesting, which will only execute tests of the affected target. The selective testing is applied to review the pull request workflow.
Since we use test plans in schemes, the file
Scheme+Initializingis refactored to adapt the test plan and remove the old test target approachProof Of Work 📹
All tests are executed: https://github.com/phongvhd93/ios-template-playground/actions/runs/24499279138/job/71601691346
No test is excuted: https://github.com/phongvhd93/ios-template-playground/actions/runs/24498169139/job/71597987206
Summary by CodeRabbit
New Features
Chores