Provider produced inconsistent final plan for source_code_hash

[djakielski]

Description

I had an issue with changes and deployment of lambda function when using auto build. Maybe this is more a provoder bug?

Versions

• Module version: 7.20.1

• Terraform version:

Terraform v1.10.3
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.84.0
+ provider registry.terraform.io/hashicorp/external v2.3.4
+ provider registry.terraform.io/hashicorp/http v3.4.0
+ provider registry.terraform.io/hashicorp/local v2.5.2
+ provider registry.terraform.io/hashicorp/null v3.2.3
+ provider registry.terraform.io/hashicorp/random v3.6.0
+ provider registry.terraform.io/hashicorp/tls v4.0.4
+ provider registry.terraform.io/jianyuan/sentry v0.13.1

• Provider version(s):
  Providers required by configuration:

.
├── provider[registry.terraform.io/hashicorp/random] 3.6.0
├── provider[registry.terraform.io/hashicorp/http] 3.4.0
├── provider[registry.terraform.io/hashicorp/tls] 4.0.4
├── provider[registry.terraform.io/jianyuan/sentry] 0.13.1
├── provider[registry.terraform.io/hashicorp/aws] ~> 5.0

Reproduction Code

module "cdn_function_reisemagazin_rewrite" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "7.20.1"

  function_name                = "cdn-reisemagazin-rewrite"
  description                  = "Rewrite headers"
  handler                      = "dist/main.handler"
  runtime                      = "nodejs22.x"
  trigger_on_package_timestamp = false
  recreate_missing_package     = false

  publish        = true
  lambda_at_edge = true
  source_path = [
    {
      path = "cdn-functions/reisemagazin-rewrite"
      commands = [
        "npm ci",
        "npm run build",
        "npm prune --production",
        ":zip"
      ]
      prefix_in_zip = "dist"
    }
  ]

  providers = {
    aws = aws.us-east-1
  }
  tags = local.holidayguruAllTags
}

Steps to reproduce the behavior:

terraform apply

Expected behavior

Should be deployed.

Actual behavior

Deployment fails. After two retries, it works fine.

Terminal Output Screenshot(s)

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for
│ module.cdn_function_reisemagazin_rewrite.aws_lambda_function.this[0] to include new
│ values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .source_code_hash: was
│ cty.StringVal("pL+RtsVm8jioo3I9uYiR4yPfc0hfbKtBz1zkSqbuMWk="), but now
│ cty.StringVal("Gli5ygJZc6siVv3F42MWnJnROrQcNfTgVgzScmsw2CY=").
│
│ This is a bug in the provider, which should be reported in the provider's own
│ issue tracker.

[mmerickel]

I'm encountering this on a similar example. I ran into it specifically (so far) when I changed from true to false on trigger_on_package_timestamp and recreate_missing_package settings. I haven't tested it enough to know if it was only a one-time mishap when changing the settings but it happened consistently on every environment (like 5 times so far) where I changed the settings - had to re-run apply again and then it went through successfully.

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[mmerickel]

Not stale

[cwong-archy]

having same issue as well. initial apply works fine. but once i modify the index.py script. i get this error

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.backend_bucket_event_notification.module.lambda.aws_lambda_function.this[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .source_code_hash: was
│ cty.StringVal("GdPtjAWNuoWRS6HyfU3p6IWp7r7ElPUKSwO2RYPim/Y="), but now cty.StringVal("iXM0Tq3dgmF/mSdlHY5KWpvwAaDcYwRl2wgz9ZuiiV8=").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

[cwong-archy]

having same issue as well. initial apply works fine. but once i modify the index.py script. i get this error

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for module.backend_bucket_event_notification.module.lambda.aws_lambda_function.this[0] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .source_code_hash: was
│ cty.StringVal("GdPtjAWNuoWRS6HyfU3p6IWp7r7ElPUKSwO2RYPim/Y="), but now cty.StringVal("iXM0Tq3dgmF/mSdlHY5KWpvwAaDcYwRl2wgz9ZuiiV8=").
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

well. i re-ran terraform apply and it worked. ¯_(ツ)_/¯

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[mmerickel]

not stale

[remi-f-artelia]

same issue here

Terraform v1.11.4
on linux_amd64
+ provider registry.terraform.io/hashicorp/archive v2.7.0
+ provider registry.terraform.io/hashicorp/aws v5.97.0
+ provider registry.terraform.io/hashicorp/external v2.3.4
+ provider registry.terraform.io/hashicorp/local v2.5.2
+ provider registry.terraform.io/hashicorp/null v3.2.4

2025-05-02T16:58:51.255Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an invalid plan for module.lambda.aws_lambda_function.this[0], but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .image_uri: planned value cty.StringVal("") for a non-computed attribute
      - .kms_key_arn: planned value cty.StringVal("") for a non-computed attribute
      - .code_signing_config_arn: planned value cty.StringVal("") for a non-computed attribute
      - .skip_destroy: planned value cty.False for a non-computed attribute
      - .logging_config[0].application_log_level: planned value cty.StringVal("") for a non-computed attribute
      - .logging_config[0].system_log_level: planned value cty.StringVal("") for a non-computed attribute
      - .tracing_config: block count in plan (1) disagrees with count in config (0)

2025-05-02T16:58:51.255Z [ERROR] vertex "module.lambda.aws_lambda_function.this[0]" error: Provider produced inconsistent final plan

2025-05-02T16:58:51.257Z [DEBUG] states/remote: state read serial is: 7; serial is: 7
2025-05-02T16:58:51.257Z [DEBUG] states/remote: state read lineage is: 29383748-4a04-6c15-f754-46e34d5823d5; lineage is: 29383748-4a04-6c15-f754-46e34d5823d5

2025-05-02T16:58:51.262Z [INFO]  backend-s3: Uploading remote state: tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate

2025-05-02T16:58:51.263Z [DEBUG] backend-s3: HTTP Request Sent: aws.region=eu-west-3 aws.s3.bucket=<bucket_name> aws.s3.key=<backend_path>/terraform.tfstate rpc.method=PutObject rpc.service=S3 rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_aws.signing_region="" tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate net.peer.name=<bucket_name>.s3.eu-west-3.amazonaws.com http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.11.3 (+https://www.terraform.io/) aws-sdk-go-v2/1.36.0 ua/2.1 os/linux lang/go#1.23.3 md/GOOS#linux md/GOARCH#amd64 api/s3#1.75.2 ft/s3-transfer m/G,Z" http.request.header.x_amz_decoded_content_length=107951 http.request.header.x_amz_server_side_encryption=AES256 http.request.header.x_amz_date=20250502T165851Z http.method=PUT http.request.header.content_encoding=aws-chunked http.request.header.amz_sdk_invocation_id=3ce85f27-2f0d-4467-889c-d3d0dbe346c5 http.request.header.x_amz_content_sha256=STREAMING-UNSIGNED-PAYLOAD-TRAILER http.request.header.content_type=application/json http.request.body="[Redacted: 105.5 KB (108,033 bytes), Type: application/json]" http.request_content_length=108033 http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************DT3X/20250502/eu-west-3/s3/aws4_request, SignedHeaders=accept-encoding;amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-security-token;x-amz-server-side-encryption;x-amz-trailer, Signature=*****" http.request.header.x_amz_security_token="*****" http.request.header.x_amz_sdk_checksum_algorithm=SHA256 http.request.header.accept_encoding=identity http.url=https://<bucket_name>.s3.eu-west-3.amazonaws.com/<backend_path>/terraform.tfstate?x-id=PutObject http.request.header.amz_sdk_request="attempt=1; max=5" http.request.header.x_amz_trailer=x-amz-checksum-sha256

2025-05-02T16:58:51.357Z [DEBUG] backend-s3: HTTP Response Received: aws.region=eu-west-3 aws.s3.bucket=<bucket_name> aws.s3.key=<backend_path>/terraform.tfstate rpc.method=PutObject rpc.service=S3 rpc.system=aws-api tf_aws.sdk=aws-sdk-go-v2 tf_aws.signing_region="" tf_backend.operation=Put tf_backend.req_id=01181a20-b5df-51ed-ebad-cfd59cce01e4 tf_backend.s3.bucket=<bucket_name> tf_backend.s3.path=<backend_path>/terraform.tfstate http.response.header.x_amz_request_id=SP1SH4D0HCN2N8TD http.response.header.etag="\"ee0c4cd70502159840cbaf12924656ad\"" http.response.header.date="Fri, 02 May 2025 16:58:52 GMT" http.response.header.x_amz_server_side_encryption=AES256 http.response.header.x_amz_checksum_sha256="1sN+4agm1SX8dnOk3PXVrQJt8MQfnzaorer58qRWoDg=" http.response.body="" http.duration=93 http.status_code=200 http.response.header.x_amz_version_id=LA595x2bC6kqWSzFpoPSZa50iQQm3TZp http.response.header.x_amz_id_2="ZEwPXPfX9lLf5o6D8XxrNE3KgS9ptRWt3CiFtBVatrlZTpP/UUlaWdoyO8ZiUbp3x9xkJEFze78gLOvYJ4vLaOh1wizQ/J+J" http.response.header.server=AmazonS3 http.response.header.x_amz_checksum_type=FULL_OBJECT

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.lambda.aws_lambda_function.this[0] to
│ include new values learned so far during apply, provider
│ "registry.terraform.io/hashicorp/aws" produced an invalid new value for
│ .source_code_hash: was
│ cty.StringVal("eJtNL22SQynso3BANmWCGaA4B3q7/i/SoxddGABd3js="), but now
│ cty.StringVal("ZZXy+dAo8vXB0H0JrTb6RUdwK5po5pfVTPmWlEtfwvA=").
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.

I have this issue when I'm on a multi stage within my Azure Devops CICD
I do not have the issue locally when I perform an apply directly (apply -auto-approve after doing a plan before)