feat: add metricsgateway proxy to decouple syslog certs from metricsforwarder#1141
Draft
bonzofenix wants to merge 15 commits intomainfrom
Draft
feat: add metricsgateway proxy to decouple syslog certs from metricsforwarder#1141bonzofenix wants to merge 15 commits intomainfrom
bonzofenix wants to merge 15 commits intomainfrom
Conversation
The metricsforwarder previously sent custom metrics to log-cache via syslog (TCP/TLS), which required TLS certificates. This replaces that with a direct gRPC connection to log-cache's Ingress API using the existing go-log-cache/v3 library, removing the TLS cert requirement. - Add LogCacheEmitter using logcache_v1.IngressClient.Send() - Support plain gRPC (default) and optional TLS - Remove SyslogEmitter, SyslogConfig, and all syslog references - Remove syslog-client resource from mta.tpl.yaml and build scripts - Update default config: syslog section replaced with log_cache
The syslog emitter removal left loggregator-agent-release and its transitive dependencies unused. This cleans up go.mod and go.sum.
This reverts commit 946f554.
This reverts commit bbae60a.
…ter" This reverts commit 973f536.
…orwarder Introduce a new metricsgateway app that runs in the system org with syslog TLS certs and receives metrics from metricsforwarder over HTTPS. This eliminates the need for metricsforwarder to hold syslog certs. Auth between components uses CF instance identity (XFCC header) with org_guid validation. - New metricsgateway app (config, syslog emitter, HTTP server, main) - New GatewayEmitter in metricsforwarder sends metrics via HTTPS - XFCC auth space validation now optional (empty string skips check) - MTA deployment updated: syslog-client moved to metricsgateway - Extension file updated with metricsgateway config and org GUID
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
The metricsforwarder was configured to use the gateway URL in the extension file, but the metricsgateway isn't deployed yet in CI. Keep syslog-client on metricsforwarder and don't set the gateway URL until the metricsgateway deployment is verified.
…is deployed" This reverts commit 12a5d72.
The metricsgateway module was not being deployed because it wasn't in the MODULES variable used by cf deploy -m. Also adds security group setup for metricsgateway (syslog port 6067 access).
- Add comment explaining empty dummyCounter.Add() method - Extract /v1/envelopes literal to constant
Remove InsecureSkipVerify by configuring proper TLS on both sides: **Root cause**: Previous code skipped cert validation because gateway server had no TLS cert configured. **Changes**: 1. Metricsgateway now serves with CF instance identity cert 2. Metricsforwarder validates using CF_INSTANCE_CA_CERT 3. Removed InsecureSkipVerify from gateway_emitter.go **Security improvement**: - Previous: No server cert validation (vulnerable to MITM) - Now: Full mTLS with CF instance identity (client + server auth) **Tests added**: - metricsgateway/config/config_test.go: verify TLS config loading - metricsforwarder/forwarder/gateway_emitter_tls_test.go: verify cert validation works All other component communication already used proper TLS validation. This brings metricsgateway to same security standard.
- Fix NewVCAPConfigurationReader call (returns 2 values) - Fix TLS test cert validation (use self-signed cert as CA)
httptest server cert needs PEM encoding, not raw bytes. Fixes x509 'certificate signed by unknown authority' error.
- Check json.Unmarshal errors (errcheck) - Check pem.Encode errors (errcheck) - Check file Close errors (errcheck) - Support CA-only TLS client config (no client cert required) Fixes TLS test failure where client only validates server CA without presenting its own certificate.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
metricsgatewayGo app deployed in the system org that holds syslog TLS certs and proxies metrics to log-cache via syslogmetricsforwarderno longer needs syslog certs — it sends metrics tometricsgatewayover HTTPS using CF instance identity (XFCC) for authorg_guidfrom the XFCC header to ensure only the autoscaler deployment org can send metricsChanges
metricsgateway/app — config, syslog emitter (adapted from original metricsforwarder syslog emitter), HTTP server with XFCC auth middleware, health endpoint, main.go usingstartup.BootstrapGatewayEmitterin metricsforwarder — HTTP client that POSTsCustomMetricJSON to the gateway usingCF_INSTANCE_CERT/CF_INSTANCE_KEYas client TLS certshelpers/auth/xfcc_auth.go— Space/org validation now optional (empty string skips the check), allowing the gateway to validate only org_guidmetricsforwarder/config/config.go— AddedMetricsGatewayConfigandInstanceTLSCertsfields,UsingGateway()method, conditional VCAP loading (gateway certs vs syslog certs)mta.tpl.yaml— Added metricsgateway module and config resource, movedsyslog-clientbinding from metricsforwarder to metricsgatewayscripts/build-extension-file.sh— Added metricsgateway health password,AUTOSCALER_ORG_GUIDresolution viacf org --guid, gateway URL injected into metricsforwarder configTesting
🤖 Generated with Claude Code