parallel plan and parallel apply no longer works

[nvanheuverzwijn]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Setting parallel_apply and parallel_plan to true no longer work in parallel.

Reproduction Steps

Set these configuration to true in your atlantis.yaml, make sure you have a couple of project and watch how atlantis only run one step at a time instead of having the default 15 project at the same time.

Logs

There aren't any valuable logs here but running a ps aux when atlantis is processing a plan or apply command reveal that only one step is running at a time.

Additional Context

This problem is introduced in this PR: https://github.com/runatlantis/atlantis/pull/6230/changes
Issue: #6185

I beleive this lock shouldn't be acquired on this the workspace but only on workspace/projectname directory, which we already get.

[dosubot]

This is a confirmed regression from PR #6230. The root cause is that Clone() unconditionally acquires an exclusive write lock before checking if the repo is already at the correct commit [1]. Since runSteps() holds a read lock for the entire duration of Terraform execution (init, plan, apply), the RWMutex semantics mean each subsequent goroutine's Clone() write lock is blocked until the previous goroutine's plan/apply finishes — effectively serializing everything regardless of parallel_plan/parallel_apply settings.

You're correct that the lock scope is too broad for the common case. The fix is tracked in PR #6376 [2], which introduces a read-lock fast path: when the clone directory already exists at the correct commit, it verifies this under a shared read lock (allowing concurrent checks) and only falls back to a write lock when an actual clone/reset is needed. This PR is still open and unmerged as of today.

There's also a related deadlock issue (#6408) that occurs with 10+ parallel projects due to inconsistent lock ordering between HasDiverged() and recheckDiverged() — that one was fixed in PR #6409 which has been merged [3].

In the meantime, the only workarounds are to apply PR #6376 as a local patch, or pin to a version before PR #6230 (accepting the original race condition it was meant to fix).

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[rorikuqu07421-pixel]

It appears that recent changes may have affected the functionality of parallel plan and parallel apply features. To troubleshoot, check if any dependencies have been updated or if there are any breaking changes in the latest releases. Additionally, reviewing the logs for error messages during execution may provide specific insight into the underlying issue.