Finalize migration to Go based exporter

[mnaser]

We need to finish the entire migration to a Go based exporter using all the new components and cover every single aspect with tests.

The final goal deliverable to close out of this issue is (ideally) an identical output of the regular OpenStack exporter and this one.

Tadas Sutkaitis (@fitbeard): Since I've been struggling with keeping up on reviews, I think you can push and merge all the work needed on your side, perhaps after Copilot reviews. The key thing as well is to make sure that nothing we have in the code is "poor" queries that wouldn't hit indexes, etc.

[mnaser]

Tadas Sutkaitis (@fitbeard) just checking in, how's this going?

[fitbeard]

Mohammed Naser (@mnaser) preparing code to push. eta ~1h.

[fitbeard]

Audit Report — OpenStack Database Exporter

Date: 2026-02-19

1. Build & Tests Status

• Go version: 1.25.5 (upgraded from 1.24.1)
• Lint: golangci-lint run ./... — 0 issues
• Build: passes (nix develop -c go build ./cmd/openstack-database-exporter)
• Unit tests: all pass (16 packages, 0 failures)
• Integration/e2e tests: all 5 E2E tests pass + per-service integration tests
  • E2E: 11 MariaDB containers, full exporter wired up, 115 metric families validated
  • Per-service: cinder (4), glance, keystone, magnum, manila, neutron (8), octavia, placement
• Live comparison: both exporters tested against the same OpenStack cluster

2. Implemented Services

Service	Status	Collectors	Unit Tests	Integration Tests
Cinder	Done	volumes, snapshots, agents, limits	Yes	Yes
Glance	Done	images	Yes	Yes
Heat	Done	stacks	Yes	Yes
Ironic	Done	baremetal, nodes	Yes	No (no live DB)
Keystone	Done	domains, projects, users, groups, regions	Yes	Yes
Magnum	Done	clusters, masters, nodes, container_infra	Yes	Yes
Manila	Done	shares	Yes	Yes
Neutron	Done	HA routers, floating IPs, networks, ports, routers, security groups, subnets, quotas	Yes (32)	Yes (8)
Nova	Done	servers, agents, flavors, compute nodes, limits	Yes	Yes
Octavia	Done	amphorae, load balancers, pools	Yes	Yes
Placement	Done	resources	Yes	Yes

3. Live Metrics Comparison

Environment: Same OpenStack cluster. DB exporter on :9180 (localhost:13306 via kubectl), original API exporter on :9181 (clouds.yaml).

Metric line count: DB=555, Original=596

Exact Match

Service	Verdict
Cinder (agents, limits, volumes, snapshots)	Match
Glance	Match
Heat	Match
Identity/Keystone	Match
Magnum	Match
Manila	Match
Neutron (28 metrics)	179/179 lines match
Nova (agents, flavors, servers, limits, compute nodes)	Match
Loadbalancer/Octavia (all 7 metrics)	Byte-for-byte identical
Placement (all 4 metrics)	Match

DB Exporter Improvements Over Original

Metric	Detail
Placement allocation_ratio	0.9 vs 0.8999999761581421 — proper float64 precision
Nova server_status flavor_id	Actual UUID vs <nil> from API
Nova limits	Adds domain_id label
Nova compute node metrics	10 extra metrics (current_workload, free_disk_bytes, local_storage, memory, running_vms, vcpus)

Not Implementable from DB

Missing Metric	Reason
cinder_pool_capacity_* (2)	Runtime Ceph/driver data not in DB
neutron_agent_state (13)	Requires API heartbeat liveness check; DB only has raw timestamps

Nova Limits Tenant Discrepancy

Original reports 4 tenants, DB reports 3. internal_cinder tenant is missing because it has no placement allocations. This is correct behavior for the unified limits driver.

4. SQL Security Audit

Safe Practices

• All query files contain SELECT-only operations — zero write operations in production
• All production code uses sqlc-generated parameterized queries — no SQL injection possible
• No fmt.Sprintf or string concatenation for SQL construction
• No db.Exec() in production code (only in test SeedSQL helper)
• DSN parsing uses net/url.Parse() — no injection risk
• Connection pool: max 5 open, 2 idle

Unbounded Queries (Performance Risk)

Query	File	Risk
GetInstances	sql/nova/queries.sql	No LIMIT — millions of instances possible
GetConsumers	sql/placement/queries.sql	No LIMIT + ORDER BY created_at DESC (no index)
GetAllVolumes	sql/cinder/queries.sql	No LIMIT
GetAllProjectQuotas	sql/cinder/queries.sql	No LIMIT
GetResourceMetrics	sql/placement/queries.sql	Complex GROUP BY, no LIMIT
GetAllocationsByProject	sql/placement/queries.sql	Multiple JOINs, no LIMIT
GetComputeNodes	sql/nova/queries.sql	No LIMIT
All nova_api queries	sql/nova_api/queries.sql	None have LIMIT

In large OpenStack deployments (thousands of instances/volumes), these unbounded queries could cause memory pressure and slow scrapes. Consider adding pagination or streaming if deploying at scale.

5. Fixes Applied

PR #54 — Nova

All reviewer issues fixed:

Issue	Fix
Status casing	strings.ToUpper() on vm_state
Wrong flavor ID in server.go	Lookup via nova_api flavors.flavorid
Wrong service ID	Use service.Uuid instead of numeric PK
Wrong flavor ID in flavors.go	Use flavor.Flavorid instead of flavor.ID
COALESCE(SUM(a.used), 0) type	CAST(... AS SIGNED)
Services query includes NULL topic	Added AND topic IS NOT NULL filter

PR #53 — Ironic

All reviewer issues fixed:

Issue	Fix
Double query for _up	Single GetNodeMetrics call, rows passed to CollectFromRows
Collect returns error	CollectFromRows has no return value
Duplicate labelsets (NULL uuid)	Rows with NULL uuid skipped
retired_reason unbounded cardinality	Capped to 128 chars
Unused db field	Removed from both collectors
Schema TINYINT defaults	DEFAULT 0 not DEFAULT '0'
NULL provision_state dropped	WHERE provision_state IS NULL OR provision_state != 'deleted'
Unnecessary ORDER BY	Removed
DownCollector on failure	RegisterCollectors emits ironic_up=0 if DB connection fails

Neutron — New Implementation

7 collectors, 28 metrics, validated 179/179 lines against live original exporter. Key implementation details:

• is_shared uses RBAC access_as_shared (not access_as_external)
• ports_no_ips excludes ip_allocation='none'
• external_network_id resolved via gw_port LEFT JOIN
• IP availability total computed from allocation pool ranges (IPv4/IPv6)
• Quotas use project resolver with per-resource default values

Infrastructure

• Go: upgraded from 1.24.1 to 1.25.5 (flake.nix, go.mod, flake.lock)
• CI: split into lint, unit-test, integration-test, build, image jobs; image gated on lint+test+build
• CI cleanup: removed Rust cargo job, fixed periodic scan image reference
• .gitignore: anchored /openstack-database-exporter to stop matching cmd/ directory
• Lint: fixed errcheck warnings in test utility (db.Close() return values)

6. Remaining Work

• Ironic live validation: No Ironic DB available locally; implementation is based on PR feat: add ironic metrics #53 code + unit tests only.

[fitbeard]

Mohammed Naser (@mnaser) details in a comment above.
PR: #71
E2E: https://github.com/vexxhost/openstack_database_exporter/actions/runs/22190479981/job/64176998671?pr=71

[mnaser]

Tadas Sutkaitis (@fitbeard) neutron_agent_state is pretty important for agent up-ness alerts, in theory, could we not grab the live timestamp - the default service up time value in Neutron and "reimplement" the up time logic?

i think that's what we did for nova agents i believe.

[fitbeard]

Mohammed Naser (@mnaser) ah i see. Added very very similar check based on heartbeat_timestamp. Metrics will set to 0 when timestamp is older then https://docs.openstack.org/neutron/latest/configuration/neutron.html#DEFAULT.agent_down_time (75s).