auth: support AWS ECS default credentials detection

[m-iskw335]

Is your feature request related to a problem? Please describe.

The externalaccount AWS subject token supplier currently supports AWS EC2 (IMDS) out of the box, but does not natively support AWS ECS.

When running on ECS (Fargate), applications cannot obtain AWS credentials from the container metadata endpoint automatically. Instead, if a credential_source.url (pointing to EC2 IMDS) is present in the configuration, the library attempts to connect to it and times out.

Example Configuration causing the issue

Standard configurations generated by gcloud include the EC2 IMDS URL:

{
  "type": "external_account",
  // ...
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
  }
}

On Fargate, this url is unreachable because the EC2 metadata service is not available, even though AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is correctly set by the platform.

Note on ECS EC2 Launch Type:
While this issue highlights Fargate where the application times out, this fix is also required for the ECS EC2 launch type. Without prioritizing ECS credentials, the library might incorrectly fall back to the EC2 instance profile (node role) instead of the intended ECS Task Role, leading to incorrect permissions.

Describe the solution you'd like

To resolve this, I propose extending the AWS subject token supplier to:

1. Detect ECS environment: Check for standard variables AWS_CONTAINER_CREDENTIALS_RELATIVE_URI or AWS_CONTAINER_CREDENTIALS_FULL_URI.

2. Prioritize ECS detection: If ECS environment variables are present, use them before falling back to the credential_source.url defined in the JSON. This ensures ECS workloads don't attempt to access unreachable EC2 endpoints and use the intended Task Role credentials.

3. Handle protocol differences: Unlike EC2 IMDS which typically requires a two-step process (listing roles then fetching credentials), the ECS metadata endpoint returns credentials directly at the path provided by the environment variable. The implementation will handle this simplified flow.

4. Retrieve credentials: Fetch credentials from the container metadata endpoint using standard net/http (no AWS SDK dependency).

5. Keep the public API unchanged: These changes will be internal to the default credential detection logic. No changes will be required in downstream client libraries.

This approach aligns with how EC2 IMDS is currently handled and provides a seamless experience for ECS users.

I have identified auth/credentials/internal/externalaccount/aws_provider.go as the primary location for this implementation.

Describe alternatives you've considered

Currently, applications must implement a custom AwsSecurityCredentialsProvider that wraps the AWS SDK to handle ECS credentials. Native support in the library is preferred to avoid additional dependencies and boilerplate code.

Additional context

Willingness to contribute:
I have investigated the codebase and would be happy to submit a PR to implement this feature.

[quartzmo]

Link to referenced file: https://github.com/googleapis/google-cloud-go/blob/main/auth/credentials/internal/externalaccount/aws_provider.go

[quartzmo]

@m-iskw335 Thank you for opening this issue. I need to confer with the team that originally specified the AWS external account support to understand: 1. Why this implementation did not originally support AWS ECS. 2. If we can now move forward with support for AWS ECS. Once I get a green light for it, I'd be happy to help you contribute your PR.

[m-iskw335]

@quartzmo
Thank you for looking into this. I will wait for your update.