[bug]: Tags that normalize to the same label name causes error

[frh-td]

What did you do

An AWS resource (in this case an EC2 instance) has two metrics with special characters that normalize to the same label name in CloudWatch Exporter.

An EC2 instance with the following two tags:

"foo/bar" = "baz"
"foo:bar" = "baz"

What did you expect to see?

The tags should be deduplicated to produce only one metric label. In the case of the same tag value, this is mostly unproblematic. In the case of different tag values, the best compromise would probably be to use the value of the last one.

What did you see instead? Under which circumstances?

The tags will both normalize to label name tag_foo_bar, causing a metric that looks like this:
aws_resource_info{job="aws_ec2",instance="",arn="arn:aws:ec2:eu-central-1:12345678:instance/i-0123456789",instance_id="i-0123456789",tag_foo_bar="baz",tag_foo_bar="baz"} 1.0

This causes Prometheus to fail to scrape the target with the following error:
Error scraping target: label name "tag_foo_bar" is not unique: invalid sample

Are you currently working around this issue?

No current workaround other than not having tags that normalize to the same name

Additional context

The reason we have these odd "duplicate" tag names is because we've been using tag names with / in them for a while, but since that is not allowed by instance metadata tags we are transitioning to replacing / with :. In a transition period we need to have both defined.

Environment

• Exporter version: 0.16.0
• Operating system & architecture: Linux ip-1-2-3-4 6.8.0-1018-aws #20-Ubuntu SMP Thu Oct 10 18:48:14 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
• Running in containers? n
• Using the official image? y

Exporter configuration file

expand

---
region: eu-central-1
use_get_metric_data: true
metrics:
 - aws_namespace: AWS/EC2
   aws_metric_name: CPUCreditBalance
   aws_dimensions: [InstanceId]
   aws_statistics: [Average]
   set_timestamp: false
   aws_tag_select:
    resource_type_selection: "ec2:instance"
    resource_id_dimension: InstanceId

Logs

N/A