From c600fb602e2297a00ccc4e72ea61e8f97e6e7087 Mon Sep 17 00:00:00 2001 From: malpou Date: Fri, 30 May 2025 12:32:58 +0200 Subject: [PATCH 01/29] feat: Add custom placeholder pages for scale-from-zero scenarios Allows HTTPScaledObjects to serve configurable HTML pages while workloads scale up from zero, with support for templates, custom headers, and automatic refresh. Signed-off-by: malpou --- CHANGELOG.md | 1 + .../bases/http.keda.sh_httpscaledobjects.yaml | 42 +++ docs/ref/vX.X.X/http_scaled_object.md | 66 ++++ examples/vX.X.X/httpscaledobject.yaml | 41 +++ interceptor/handler/placeholder.go | 215 +++++++++++++ interceptor/handler/placeholder_test.go | 298 ++++++++++++++++++ interceptor/main.go | 14 +- interceptor/main_test.go | 6 + interceptor/proxy_handlers.go | 30 ++ .../proxy_handlers_integration_test.go | 4 +- interceptor/proxy_handlers_test.go | 20 ++ .../http/v1alpha1/httpscaledobject_types.go | 29 ++ .../http/v1alpha1/zz_generated.deepcopy.go | 27 ++ .../placeholder_pages_test.go | 238 ++++++++++++++ 14 files changed, 1028 insertions(+), 3 deletions(-) create mode 100644 examples/vX.X.X/httpscaledobject.yaml create mode 100644 interceptor/handler/placeholder.go create mode 100644 interceptor/handler/placeholder_test.go create mode 100644 tests/checks/placeholder_pages/placeholder_pages_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 051f529cc..a226dd873 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,7 @@ This changelog keeps track of work items that have been completed and are ready - **General**: Allow using HSO and SO with different names ([#1293](https://github.com/kedacore/http-add-on/issues/1293)) - **General**: Support profiling for KEDA components ([#4789](https://github.com/kedacore/keda/issues/4789)) - **General**: Add possibility to skip TLS verification for upstreams in interceptor ([#1307](https://github.com/kedacore/http-add-on/pull/1307)) +- **General**: Add custom placeholder pages for scale-from-zero scenarios ([#874](https://github.com/kedacore/http-add-on/issues/874)) ### Improvements - **Interceptor**: Support HTTPScaledObject scoped timeout ([#813](https://github.com/kedacore/http-add-on/issues/813)) diff --git a/config/crd/bases/http.keda.sh_httpscaledobjects.yaml b/config/crd/bases/http.keda.sh_httpscaledobjects.yaml index 4af745de3..b9afd1f56 100644 --- a/config/crd/bases/http.keda.sh_httpscaledobjects.yaml +++ b/config/crd/bases/http.keda.sh_httpscaledobjects.yaml @@ -108,6 +108,48 @@ spec: items: type: string type: array + placeholderConfig: + description: (optional) Configuration for placeholder pages during + scale-from-zero + properties: + content: + description: Inline HTML content for placeholder page (takes precedence + over ContentConfigMap) + type: string + contentConfigMap: + description: Path to ConfigMap containing placeholder HTML content + (in same namespace) + type: string + contentConfigMapKey: + default: template.html + description: Key in ConfigMap containing the HTML template + type: string + enabled: + default: false + description: Enable placeholder page when replicas are scaled + to zero + type: boolean + headers: + additionalProperties: + type: string + description: Additional HTTP headers to include with placeholder + response + type: object + refreshInterval: + default: 5 + description: Refresh interval for client-side polling in seconds + format: int32 + maximum: 60 + minimum: 1 + type: integer + statusCode: + default: 503 + description: HTTP status code to return with placeholder page + format: int32 + maximum: 599 + minimum: 100 + type: integer + type: object replicas: description: (optional) Replica information properties: diff --git a/docs/ref/vX.X.X/http_scaled_object.md b/docs/ref/vX.X.X/http_scaled_object.md index 313729f5e..9d32fb107 100644 --- a/docs/ref/vX.X.X/http_scaled_object.md +++ b/docs/ref/vX.X.X/http_scaled_object.md @@ -33,6 +33,23 @@ spec: window: 1m concurrency: targetValue: 100 + placeholderConfig: + enabled: true + refreshInterval: 5 + statusCode: 503 + headers: + X-Service-Status: "warming-up" + content: | + + + + Service Starting + + + +

{{.ServiceName}} is starting...

+ + ``` This document is a narrated reference guide for the `HTTPScaledObject`. @@ -134,3 +151,52 @@ This section enables scaling based on the request concurrency. >Default: 100 This is the target value for the scaling configuration. + +## `placeholderConfig` + +This optional section enables serving placeholder pages when the workload is scaled to zero. When enabled, instead of returning an error while waiting for the workload to scale up, the interceptor will serve a customizable HTML page that refreshes automatically. + +### `enabled` + +>Default: false + +Whether to enable placeholder pages for this HTTPScaledObject. + +### `refreshInterval` + +>Default: 5 + +The interval in seconds at which the placeholder page will refresh. This should be set based on your expected cold start time. + +### `statusCode` + +>Default: 503 + +The HTTP status code to return with the placeholder page. Common values are 503 (Service Unavailable) or 202 (Accepted). + +### `headers` + +>Default: {} + +A map of custom HTTP headers to include in the placeholder response. Useful for adding service-specific headers. + +### `content` + +>Default: Built-in template + +Custom HTML content for the placeholder page. Supports Go template syntax with the following variables: +- `{{.ServiceName}}` - The name of the service from scaleTargetRef +- `{{.Namespace}}` - The namespace of the HTTPScaledObject +- `{{.RefreshInterval}}` - The configured refresh interval +- `{{.RequestID}}` - The X-Request-ID header value if present +- `{{.Timestamp}}` - The current timestamp in RFC3339 format + +### `contentConfigMap` + +The name of a ConfigMap containing the placeholder page template. This is an alternative to inline `content`. + +### `contentConfigMapKey` + +>Default: "template.html" + +The key within the ConfigMap that contains the template content. diff --git a/examples/vX.X.X/httpscaledobject.yaml b/examples/vX.X.X/httpscaledobject.yaml new file mode 100644 index 000000000..fcb2bef7f --- /dev/null +++ b/examples/vX.X.X/httpscaledobject.yaml @@ -0,0 +1,41 @@ +kind: HTTPScaledObject +apiVersion: http.keda.sh/v1alpha1 +metadata: + name: xkcd +spec: + hosts: + - myhost.com + pathPrefixes: + - /test + scaleTargetRef: + name: xkcd + kind: Deployment + apiVersion: apps/v1 + service: xkcd + port: 8080 + replicas: + min: 1 + max: 10 + scaledownPeriod: 300 + scalingMetric: + requestRate: + granularity: 1s + targetValue: 100 + window: 1m + placeholderConfig: + enabled: true + refreshInterval: 5 + statusCode: 503 + headers: + X-Service-Status: "warming-up" + content: | + + + + Service Starting + + + +

{{.ServiceName}} is starting...

+ + \ No newline at end of file diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go new file mode 100644 index 000000000..8bae3234b --- /dev/null +++ b/interceptor/handler/placeholder.go @@ -0,0 +1,215 @@ +package handler + +import ( + "bytes" + "context" + "fmt" + "html/template" + "net/http" + "time" + + "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" + "github.com/kedacore/http-add-on/pkg/routing" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/client-go/kubernetes" +) + +const defaultPlaceholderTemplate = ` + + + Service Starting + + + + + +
+

{{.ServiceName}} is starting up...

+
+

Please wait while we prepare your service.

+

This page will refresh automatically every {{.RefreshInterval}} seconds.

+
+ +` + +// PlaceholderHandler handles serving placeholder pages during scale-from-zero +type PlaceholderHandler struct { + k8sClient kubernetes.Interface + routingTable routing.Table + templateCache map[string]*template.Template + defaultTmpl *template.Template +} + +// PlaceholderData contains data for rendering placeholder templates +type PlaceholderData struct { + ServiceName string + Namespace string + RefreshInterval int32 + RequestID string + Timestamp string +} + +// NewPlaceholderHandler creates a new placeholder handler +func NewPlaceholderHandler(k8sClient kubernetes.Interface, routingTable routing.Table) (*PlaceholderHandler, error) { + defaultTmpl, err := template.New("default").Parse(defaultPlaceholderTemplate) + if err != nil { + return nil, fmt.Errorf("failed to parse default template: %w", err) + } + + return &PlaceholderHandler{ + k8sClient: k8sClient, + routingTable: routingTable, + templateCache: make(map[string]*template.Template), + defaultTmpl: defaultTmpl, + }, nil +} + +// ServePlaceholder serves a placeholder page based on the HTTPScaledObject configuration +func (h *PlaceholderHandler) ServePlaceholder(w http.ResponseWriter, r *http.Request, hso *v1alpha1.HTTPScaledObject) error { + if hso.Spec.PlaceholderConfig == nil || !hso.Spec.PlaceholderConfig.Enabled { + http.Error(w, "Service temporarily unavailable", http.StatusServiceUnavailable) + return nil + } + + config := hso.Spec.PlaceholderConfig + + statusCode := int(config.StatusCode) + if statusCode == 0 { + statusCode = http.StatusServiceUnavailable + } + + for k, v := range config.Headers { + w.Header().Set(k, v) + } + + w.Header().Set("Content-Type", "text/html; charset=utf-8") + w.Header().Set("X-KEDA-HTTP-Placeholder-Served", "true") + w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") + + tmpl, err := h.getTemplate(r.Context(), hso) + if err != nil { + w.WriteHeader(statusCode) + fmt.Fprintf(w, "

%s is starting up...

", + hso.Spec.ScaleTargetRef.Service, config.RefreshInterval) + return nil + } + + data := PlaceholderData{ + ServiceName: hso.Spec.ScaleTargetRef.Service, + Namespace: hso.Namespace, + RefreshInterval: config.RefreshInterval, + RequestID: r.Header.Get("X-Request-ID"), + Timestamp: time.Now().Format(time.RFC3339), + } + + var buf bytes.Buffer + if err := tmpl.Execute(&buf, data); err != nil { + w.WriteHeader(statusCode) + fmt.Fprintf(w, "

%s is starting up...

", + hso.Spec.ScaleTargetRef.Service, config.RefreshInterval) + return nil + } + + w.WriteHeader(statusCode) + _, err = w.Write(buf.Bytes()) + return err +} + +// getTemplate retrieves the template for the given HTTPScaledObject +func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTPScaledObject) (*template.Template, error) { + config := hso.Spec.PlaceholderConfig + + if config.Content != "" { + cacheKey := fmt.Sprintf("%s/%s/inline", hso.Namespace, hso.Name) + if tmpl, ok := h.templateCache[cacheKey]; ok { + return tmpl, nil + } + + tmpl, err := template.New("inline").Parse(config.Content) + if err != nil { + return nil, err + } + h.templateCache[cacheKey] = tmpl + return tmpl, nil + } + + if config.ContentConfigMap != "" { + cacheKey := fmt.Sprintf("%s/%s/cm/%s", hso.Namespace, hso.Name, config.ContentConfigMap) + if tmpl, ok := h.templateCache[cacheKey]; ok { + return tmpl, nil + } + + cm, err := h.k8sClient.CoreV1().ConfigMaps(hso.Namespace).Get(ctx, config.ContentConfigMap, metav1.GetOptions{}) + if err != nil { + return nil, fmt.Errorf("failed to get ConfigMap %s: %w", config.ContentConfigMap, err) + } + + key := config.ContentConfigMapKey + if key == "" { + key = "template.html" + } + + content, ok := cm.Data[key] + if !ok { + return nil, fmt.Errorf("key %s not found in ConfigMap %s", key, config.ContentConfigMap) + } + + tmpl, err := template.New("configmap").Parse(content) + if err != nil { + return nil, err + } + h.templateCache[cacheKey] = tmpl + return tmpl, nil + } + + return h.defaultTmpl, nil +} + +// ClearCache clears the template cache +func (h *PlaceholderHandler) ClearCache() { + h.templateCache = make(map[string]*template.Template) +} diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go new file mode 100644 index 000000000..2612db2c1 --- /dev/null +++ b/interceptor/handler/placeholder_test.go @@ -0,0 +1,298 @@ +package handler + +import ( + "context" + "fmt" + "html/template" + "net/http" + "net/http/httptest" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + v1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/client-go/kubernetes/fake" + + v1alpha1 "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" + "github.com/kedacore/http-add-on/pkg/routing/test" +) + +func TestNewPlaceholderHandler(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + + handler, err := NewPlaceholderHandler(k8sClient, routingTable) + assert.NoError(t, err) + assert.NotNil(t, handler) + assert.NotNil(t, handler.k8sClient) + assert.NotNil(t, handler.routingTable) + assert.NotNil(t, handler.templateCache) + assert.NotNil(t, handler.defaultTmpl) +} + +func TestServePlaceholder_DisabledConfig(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + // Create HTTPScaledObject with disabled placeholder + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: false, + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Contains(t, w.Body.String(), "Service temporarily unavailable") +} + +func TestServePlaceholder_DefaultTemplate(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + // Create HTTPScaledObject with enabled placeholder + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 503, + RefreshInterval: 5, + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Contains(t, w.Body.String(), "test-service is starting up...") + assert.Contains(t, w.Body.String(), "refresh") + assert.Equal(t, "text/html; charset=utf-8", w.Header().Get("Content-Type")) + assert.Equal(t, "true", w.Header().Get("X-KEDA-HTTP-Placeholder-Served")) +} + +func TestServePlaceholder_InlineContent(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + customContent := `

Custom placeholder for {{.ServiceName}}

` + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 202, + RefreshInterval: 3, + Content: customContent, + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, http.StatusAccepted, w.Code) + assert.Contains(t, w.Body.String(), "Custom placeholder for test-service") +} + +func TestServePlaceholder_ConfigMapContent(t *testing.T) { + // Create fake k8s client with a ConfigMap + configMapContent := `

ConfigMap placeholder for {{.ServiceName}}

` + cm := &v1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: "placeholder-cm", + Namespace: "default", + }, + Data: map[string]string{ + "template.html": configMapContent, + }, + } + k8sClient := fake.NewSimpleClientset(cm) + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 503, + RefreshInterval: 5, + ContentConfigMap: "placeholder-cm", + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Contains(t, w.Body.String(), "ConfigMap placeholder for test-service") +} + +func TestServePlaceholder_CustomHeaders(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 503, + RefreshInterval: 5, + Headers: map[string]string{ + "X-Custom-Header": "custom-value", + "X-Service-Name": "test-service", + }, + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, "custom-value", w.Header().Get("X-Custom-Header")) + assert.Equal(t, "test-service", w.Header().Get("X-Service-Name")) +} + +func TestServePlaceholder_InvalidTemplate(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + // Invalid template syntax + invalidContent := `{{.UnknownField` + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test-service", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 503, + RefreshInterval: 5, + Content: invalidContent, + }, + }, + } + + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) // Should not return error, but fall back + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + // Should fall back to simple response + assert.Contains(t, w.Body.String(), "test-service is starting up...") +} + +func TestGetTemplate_Caching(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + customContent := `{{.ServiceName}}` + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Content: customContent, + }, + }, + } + + ctx := context.Background() + + // First call - should parse and cache + tmpl1, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl1) + + // Second call - should return from cache + tmpl2, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl2) + + // Should be the same template instance + assert.Equal(t, fmt.Sprintf("%p", tmpl1), fmt.Sprintf("%p", tmpl2)) +} + +func TestClearCache(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + // Add something to cache + tmpl, _ := template.New("test").Parse("test") + handler.templateCache["test-key"] = tmpl + + // Verify cache has content + assert.Len(t, handler.templateCache, 1) + + // Clear cache + handler.ClearCache() + + // Verify cache is empty + assert.Len(t, handler.templateCache, 0) +} diff --git a/interceptor/main.go b/interceptor/main.go index 238f50ed7..8ab13af20 100644 --- a/interceptor/main.go +++ b/interceptor/main.go @@ -198,7 +198,7 @@ func main() { setupLog.Info("starting the proxy server with TLS enabled", "port", proxyTLSPort) - if err := runProxyServer(ctx, ctrl.Log, queues, waitFunc, routingTable, svcCache, timeoutCfg, proxyTLSPort, proxyTLSEnabled, proxyTLSConfig, tracingCfg); !util.IsIgnoredErr(err) { + if err := runProxyServer(ctx, ctrl.Log, queues, waitFunc, routingTable, svcCache, timeoutCfg, proxyTLSPort, proxyTLSEnabled, proxyTLSConfig, tracingCfg, cl, endpointsCache); !util.IsIgnoredErr(err) { setupLog.Error(err, "tls proxy server failed") return err } @@ -212,7 +212,7 @@ func main() { setupLog.Info("starting the proxy server with TLS disabled", "port", proxyPort) k8sSharedInformerFactory.WaitForCacheSync(ctx.Done()) - if err := runProxyServer(ctx, ctrl.Log, queues, waitFunc, routingTable, svcCache, timeoutCfg, proxyPort, false, nil, tracingCfg); !util.IsIgnoredErr(err) { + if err := runProxyServer(ctx, ctrl.Log, queues, waitFunc, routingTable, svcCache, timeoutCfg, proxyPort, false, nil, tracingCfg, cl, endpointsCache); !util.IsIgnoredErr(err) { setupLog.Error(err, "proxy server failed") return err } @@ -409,6 +409,8 @@ func runProxyServer( tlsEnabled bool, tlsConfig map[string]interface{}, tracingConfig *config.Tracing, + k8sClient kubernetes.Interface, + endpointsCache k8s.EndpointsCache, ) error { dialer := kedanet.NewNetDialer(timeouts.Connect, timeouts.KeepAlive) dialContextFunc := kedanet.DialContextWithRetry(dialer, timeouts.DefaultBackoff()) @@ -436,6 +438,12 @@ func runProxyServer( forwardingTLSCfg.InsecureSkipVerify = tlsCfg.InsecureSkipVerify } + // Create placeholder handler + placeholderHandler, err := handler.NewPlaceholderHandler(k8sClient, routingTable) + if err != nil { + return fmt.Errorf("creating placeholder handler: %w", err) + } + upstreamHandler = newForwardingHandler( logger, dialContextFunc, @@ -443,6 +451,8 @@ func runProxyServer( newForwardingConfigFromTimeouts(timeouts), forwardingTLSCfg, tracingConfig, + placeholderHandler, + endpointsCache, ) upstreamHandler = middleware.NewCountingMiddleware( q, diff --git a/interceptor/main_test.go b/interceptor/main_test.go index adefff29a..8f24f2543 100644 --- a/interceptor/main_test.go +++ b/interceptor/main_test.go @@ -94,6 +94,8 @@ func TestRunProxyServerCountMiddleware(t *testing.T) { false, map[string]interface{}{}, &tracingCfg, + nil, + nil, ) }) // wait for server to start @@ -234,6 +236,8 @@ func TestRunProxyServerWithTLSCountMiddleware(t *testing.T) { true, map[string]interface{}{"certificatePath": "../certs/tls.crt", "keyPath": "../certs/tls.key", "skipVerify": true}, &tracingCfg, + nil, + nil, ) }) @@ -384,6 +388,8 @@ func TestRunProxyServerWithMultipleCertsTLSCountMiddleware(t *testing.T) { true, map[string]interface{}{"certstorePaths": "../certs"}, &tracingCfg, + nil, + nil, ) }) diff --git a/interceptor/proxy_handlers.go b/interceptor/proxy_handlers.go index fe679cd82..1b29fe88a 100644 --- a/interceptor/proxy_handlers.go +++ b/interceptor/proxy_handlers.go @@ -13,6 +13,7 @@ import ( "github.com/kedacore/http-add-on/interceptor/config" "github.com/kedacore/http-add-on/interceptor/handler" + "github.com/kedacore/http-add-on/pkg/k8s" kedanet "github.com/kedacore/http-add-on/pkg/net" "github.com/kedacore/http-add-on/pkg/util" ) @@ -52,6 +53,8 @@ func newForwardingHandler( fwdCfg forwardingConfig, tlsCfg *tls.Config, tracingCfg *config.Tracing, + placeholderHandler *handler.PlaceholderHandler, + endpointsCache k8s.EndpointsCache, ) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { var uh *handler.Upstream @@ -59,6 +62,33 @@ func newForwardingHandler( httpso := util.HTTPSOFromContext(ctx) hasFailover := httpso.Spec.ColdStartTimeoutFailoverRef != nil + // Check if we should serve a placeholder page + if placeholderHandler != nil && httpso.Spec.PlaceholderConfig != nil && httpso.Spec.PlaceholderConfig.Enabled { + endpoints, err := endpointsCache.Get(httpso.GetNamespace(), httpso.Spec.ScaleTargetRef.Service) + if err != nil { + // Error getting endpoints cache - return 503 Service Unavailable + lggr.Error(err, "failed to get endpoints from cache while placeholder is configured", + "namespace", httpso.GetNamespace(), + "service", httpso.Spec.ScaleTargetRef.Service) + w.WriteHeader(http.StatusServiceUnavailable) + if _, writeErr := w.Write([]byte("Service temporarily unavailable - unable to check service status")); writeErr != nil { + lggr.Error(writeErr, "could not write error response to client") + } + return + } + + if workloadActiveEndpoints(endpoints) == 0 { + if placeholderErr := placeholderHandler.ServePlaceholder(w, r, httpso); placeholderErr != nil { + lggr.Error(placeholderErr, "failed to serve placeholder page") + w.WriteHeader(http.StatusBadGateway) + if _, err := w.Write([]byte("error serving placeholder page")); err != nil { + lggr.Error(err, "could not write error response to client") + } + } + return + } + } + conditionWaitTimeout := fwdCfg.waitTimeout roundTripper := &http.Transport{ Proxy: http.ProxyFromEnvironment, diff --git a/interceptor/proxy_handlers_integration_test.go b/interceptor/proxy_handlers_integration_test.go index 77f0df016..1bd67ff45 100644 --- a/interceptor/proxy_handlers_integration_test.go +++ b/interceptor/proxy_handlers_integration_test.go @@ -319,7 +319,9 @@ func newHarness( respHeaderTimeout: time.Second, }, &tls.Config{}, - &config.Tracing{}), + &config.Tracing{}, + nil, + nil), svcCache, false, ) diff --git a/interceptor/proxy_handlers_test.go b/interceptor/proxy_handlers_test.go index 1343730e1..996845362 100644 --- a/interceptor/proxy_handlers_test.go +++ b/interceptor/proxy_handlers_test.go @@ -80,6 +80,8 @@ func TestImmediatelySuccessfulProxy(t *testing.T) { }, &tls.Config{}, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) @@ -132,6 +134,8 @@ func TestImmediatelySuccessfulProxyTLS(t *testing.T) { }, &TestTLSConfig, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) @@ -248,6 +252,8 @@ func TestWaitFailedConnection(t *testing.T) { }, &tls.Config{}, &config.Tracing{}, + nil, + nil, ) stream, err := url.Parse("http://0.0.0.0:0") r.NoError(err) @@ -299,6 +305,8 @@ func TestWaitFailedConnectionTLS(t *testing.T) { }, &TestTLSConfig, &config.Tracing{}, + nil, + nil, ) stream, err := url.Parse("http://0.0.0.0:0") r.NoError(err) @@ -351,6 +359,8 @@ func TestTimesOutOnWaitFunc(t *testing.T) { }, &tls.Config{}, &config.Tracing{}, + nil, + nil, ) stream, err := url.Parse("http://1.1.1.1") r.NoError(err) @@ -424,6 +434,8 @@ func TestTimesOutOnWaitFuncTLS(t *testing.T) { }, &TestTLSConfig, &config.Tracing{}, + nil, + nil, ) stream, err := url.Parse("http://1.1.1.1") r.NoError(err) @@ -508,6 +520,8 @@ func TestWaitsForWaitFunc(t *testing.T) { }, &tls.Config{}, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) @@ -575,6 +589,8 @@ func TestWaitsForWaitFuncTLS(t *testing.T) { }, &TestTLSConfig, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) @@ -646,6 +662,8 @@ func TestWaitHeaderTimeout(t *testing.T) { }, &tls.Config{}, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) @@ -705,6 +723,8 @@ func TestWaitHeaderTimeoutTLS(t *testing.T) { }, &TestTLSConfig, &config.Tracing{}, + nil, + nil, ) const path = "/testfwd" res, req, err := reqAndRes(path) diff --git a/operator/apis/http/v1alpha1/httpscaledobject_types.go b/operator/apis/http/v1alpha1/httpscaledobject_types.go index 93cf0b0ec..62167a940 100644 --- a/operator/apis/http/v1alpha1/httpscaledobject_types.go +++ b/operator/apis/http/v1alpha1/httpscaledobject_types.go @@ -132,6 +132,32 @@ type HTTPScaledObjectTimeoutsSpec struct { ResponseHeader metav1.Duration `json:"responseHeader" description:"How long to wait between when the HTTP request is sent to the backing app and when response headers need to arrive"` } +// PlaceholderConfig defines the configuration for serving placeholder pages during scale-from-zero +type PlaceholderConfig struct { + // Enable placeholder page when replicas are scaled to zero + // +kubebuilder:default=false + // +optional + Enabled bool `json:"enabled" description:"Enable placeholder page when replicas are scaled to zero"` + // Inline content for placeholder page (can be HTML, JSON, plain text, etc.) + // +optional + Content string `json:"content,omitempty" description:"Inline content for placeholder page"` + // HTTP status code to return with placeholder page + // +kubebuilder:default=503 + // +kubebuilder:validation:Minimum=100 + // +kubebuilder:validation:Maximum=599 + // +optional + StatusCode int32 `json:"statusCode,omitempty" description:"HTTP status code to return with placeholder page (Default 503)"` + // Refresh interval for client-side polling in seconds + // +kubebuilder:default=5 + // +kubebuilder:validation:Minimum=1 + // +kubebuilder:validation:Maximum=60 + // +optional + RefreshInterval int32 `json:"refreshInterval,omitempty" description:"Refresh interval for client-side polling in seconds (Default 5)"` + // Additional HTTP headers to include with placeholder response + // +optional + Headers map[string]string `json:"headers,omitempty" description:"Additional HTTP headers to include with placeholder response"` +} + // HTTPScaledObjectSpec defines the desired state of HTTPScaledObject type HTTPScaledObjectSpec struct { // The hosts to route. All requests which the "Host" header @@ -171,6 +197,9 @@ type HTTPScaledObjectSpec struct { // (optional) Timeouts that override the global ones // +optional Timeouts *HTTPScaledObjectTimeoutsSpec `json:"timeouts,omitempty" description:"Timeouts that override the global ones"` + // (optional) Configuration for placeholder pages during scale-from-zero + // +optional + PlaceholderConfig *PlaceholderConfig `json:"placeholderConfig,omitempty" description:"Configuration for placeholder pages during scale-from-zero"` } // HTTPScaledObjectStatus defines the observed state of HTTPScaledObject diff --git a/operator/apis/http/v1alpha1/zz_generated.deepcopy.go b/operator/apis/http/v1alpha1/zz_generated.deepcopy.go index dcc03f42c..d2ab8d03c 100644 --- a/operator/apis/http/v1alpha1/zz_generated.deepcopy.go +++ b/operator/apis/http/v1alpha1/zz_generated.deepcopy.go @@ -196,6 +196,11 @@ func (in *HTTPScaledObjectSpec) DeepCopyInto(out *HTTPScaledObjectSpec) { *out = new(HTTPScaledObjectTimeoutsSpec) **out = **in } + if in.PlaceholderConfig != nil { + in, out := &in.PlaceholderConfig, &out.PlaceholderConfig + *out = new(PlaceholderConfig) + (*in).DeepCopyInto(*out) + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new HTTPScaledObjectSpec. @@ -245,6 +250,28 @@ func (in *HTTPScaledObjectTimeoutsSpec) DeepCopy() *HTTPScaledObjectTimeoutsSpec return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PlaceholderConfig) DeepCopyInto(out *PlaceholderConfig) { + *out = *in + if in.Headers != nil { + in, out := &in.Headers, &out.Headers + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PlaceholderConfig. +func (in *PlaceholderConfig) DeepCopy() *PlaceholderConfig { + if in == nil { + return nil + } + out := new(PlaceholderConfig) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *RateMetricSpec) DeepCopyInto(out *RateMetricSpec) { *out = *in diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go new file mode 100644 index 000000000..d61f87a00 --- /dev/null +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -0,0 +1,238 @@ +//go:build e2e +// +build e2e + +package placeholder_pages + +import ( + "fmt" + "net/http" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + + . "github.com/kedacore/http-add-on/tests/helper" +) + +const ( + testName = "placeholder-pages-test" + testServiceName = testName + testNamespace = testName + "-ns" +) + +type placeholderTemplateData struct { + TestNamespace string + TestName string + TestServiceName string +} + +const placeholderTemplate = ` +apiVersion: v1 +kind: Namespace +metadata: + name: {{.TestNamespace}} +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{.TestName}} + namespace: {{.TestNamespace}} + labels: + app: {{.TestName}} +spec: + replicas: 1 + selector: + matchLabels: + app: {{.TestName}} + template: + metadata: + labels: + app: {{.TestName}} + spec: + containers: + - name: app + image: kennethreitz/httpbin:latest + ports: + - containerPort: 80 + name: http + resources: + requests: + memory: "64Mi" + cpu: "100m" + limits: + memory: "128Mi" + cpu: "200m" +--- +apiVersion: v1 +kind: Service +metadata: + name: {{.TestServiceName}} + namespace: {{.TestNamespace}} + labels: + app: {{.TestName}} +spec: + ports: + - port: 80 + targetPort: 80 + name: http + selector: + app: {{.TestName}} + type: ClusterIP +--- +apiVersion: http.keda.sh/v1alpha1 +kind: HTTPScaledObject +metadata: + name: {{.TestName}} + namespace: {{.TestNamespace}} +spec: + hosts: + - {{.TestName}} + pathPrefixes: + - / + scaleTargetRef: + service: {{.TestServiceName}} + port: 80 + deployment: {{.TestName}} + targetPendingRequests: 1 + scaledownPeriod: 60 + placeholderConfig: + enabled: true + statusCode: 503 + refreshInterval: 5 + headers: + X-Service-Status: "warming-up" +` + +func TestPlaceholderPages(t *testing.T) { + // Create test data + data := placeholderTemplateData{ + TestNamespace: testNamespace, + TestName: testName, + TestServiceName: testServiceName, + } + + // Apply the resources + KubectlApplyWithTemplate(t, data, "placeholder-test", placeholderTemplate) + + // Ensure cleanup + defer func() { + KubectlDeleteWithTemplate(t, data, "placeholder-test", placeholderTemplate) + DeleteNamespace(t, testNamespace) + }() + + assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 1, 240, 3), + "deployment replicas should be 1 after 3 minutes") + + // Wait for the deployment to scale down to zero + time.Sleep(90 * time.Second) + assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 0, 240, 3), + "deployment replicas should be 0 after 3 minutes") + + // Make a request and verify placeholder is served + testPlaceholderResponse(t) + + // Test that placeholder is served immediately on cold start + testImmediatePlaceholderResponse(t) +} + +func testPlaceholderResponse(t *testing.T) { + t.Log("Testing placeholder page response") + + var interceptorIP string + if UseIngressHost { + interceptorIP = ExternalIP + } else { + interceptorService := GetKubernetesServiceEndpoint( + t, + KubeClient, + "keda-http-add-on-interceptor-proxy", + "keda", + ) + interceptorIP = interceptorService.IP + } + + url := fmt.Sprintf("http://%s", interceptorIP) + req, err := http.NewRequest("GET", url, nil) + assert.NoError(t, err) + req.Host = testName + + client := &http.Client{ + Timeout: 10 * time.Second, + CheckRedirect: func(req *http.Request, via []*http.Request) error { + return http.ErrUseLastResponse + }, + } + + resp, err := client.Do(req) + assert.NoError(t, err) + defer resp.Body.Close() + + // Check that we got a 503 status (default for placeholder) + assert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode) + + // Read response body + body := make([]byte, 1024) + n, _ := resp.Body.Read(body) + bodyStr := string(body[:n]) + + // Verify placeholder content + assert.True(t, strings.Contains(bodyStr, testServiceName+" is starting up..."), + "response should contain placeholder message") + assert.True(t, strings.Contains(bodyStr, "refresh"), + "response should contain refresh meta tag") + + // Check custom headers + assert.Equal(t, "true", resp.Header.Get("X-KEDA-HTTP-Placeholder-Served"), + "placeholder served header should be present") + assert.Equal(t, "warming-up", resp.Header.Get("X-Service-Status"), + "custom header should be present") +} + +func testImmediatePlaceholderResponse(t *testing.T) { + t.Log("Testing immediate placeholder page response on cold start") + + var interceptorIP string + if UseIngressHost { + interceptorIP = ExternalIP + } else { + interceptorService := GetKubernetesServiceEndpoint( + t, + KubeClient, + "keda-http-add-on-interceptor-proxy", + "keda", + ) + interceptorIP = interceptorService.IP + } + + url := fmt.Sprintf("http://%s", interceptorIP) + req, err := http.NewRequest("GET", url, nil) + assert.NoError(t, err) + req.Host = testName + + client := &http.Client{ + Timeout: 2 * time.Second, // Short timeout to ensure immediate response + CheckRedirect: func(req *http.Request, via []*http.Request) error { + return http.ErrUseLastResponse + }, + } + + // Measure response time + start := time.Now() + resp, err := client.Do(req) + duration := time.Since(start) + + assert.NoError(t, err) + defer resp.Body.Close() + + // Check that response was immediate (less than 500ms) + assert.Less(t, duration.Milliseconds(), int64(500), + "placeholder should be served immediately, but took %v", duration) + + // Check that we got a 503 status + assert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode) + + // Check placeholder header + assert.Equal(t, "true", resp.Header.Get("X-KEDA-HTTP-Placeholder-Served"), + "placeholder served header should be present") +} From 3edd3deb1f1cd01ef45f9307e03f2cc08e84a277 Mon Sep 17 00:00:00 2001 From: malpou Date: Mon, 2 Jun 2025 19:56:58 +0200 Subject: [PATCH 02/29] remove ClearCache funtion Signed-off-by: malpou --- interceptor/handler/placeholder.go | 5 ----- interceptor/handler/placeholder_test.go | 20 -------------------- 2 files changed, 25 deletions(-) diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go index 8bae3234b..1605944c9 100644 --- a/interceptor/handler/placeholder.go +++ b/interceptor/handler/placeholder.go @@ -208,8 +208,3 @@ func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTP return h.defaultTmpl, nil } - -// ClearCache clears the template cache -func (h *PlaceholderHandler) ClearCache() { - h.templateCache = make(map[string]*template.Template) -} diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go index 2612db2c1..34f04e8df 100644 --- a/interceptor/handler/placeholder_test.go +++ b/interceptor/handler/placeholder_test.go @@ -3,7 +3,6 @@ package handler import ( "context" "fmt" - "html/template" "net/http" "net/http/httptest" "testing" @@ -277,22 +276,3 @@ func TestGetTemplate_Caching(t *testing.T) { // Should be the same template instance assert.Equal(t, fmt.Sprintf("%p", tmpl1), fmt.Sprintf("%p", tmpl2)) } - -func TestClearCache(t *testing.T) { - k8sClient := fake.NewSimpleClientset() - routingTable := test.NewTable() - handler, _ := NewPlaceholderHandler(k8sClient, routingTable) - - // Add something to cache - tmpl, _ := template.New("test").Parse("test") - handler.templateCache["test-key"] = tmpl - - // Verify cache has content - assert.Len(t, handler.templateCache, 1) - - // Clear cache - handler.ClearCache() - - // Verify cache is empty - assert.Len(t, handler.templateCache, 0) -} From c2a6c467c4e7aa35edaac0a1f6e8d1c63c737db2 Mon Sep 17 00:00:00 2001 From: malpou Date: Mon, 2 Jun 2025 20:05:46 +0200 Subject: [PATCH 03/29] use same image for e2e tests Signed-off-by: malpou --- .../placeholder_pages_test.go | 28 +++++++++---------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index d61f87a00..418a4dfcf 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -11,8 +11,6 @@ import ( "time" "github.com/stretchr/testify/assert" - - . "github.com/kedacore/http-add-on/tests/helper" ) const ( @@ -51,18 +49,18 @@ spec: app: {{.TestName}} spec: containers: - - name: app - image: kennethreitz/httpbin:latest - ports: - - containerPort: 80 - name: http - resources: - requests: - memory: "64Mi" - cpu: "100m" - limits: - memory: "128Mi" - cpu: "200m" + - name: {{.TestName}} + image: registry.k8s.io/e2e-test-images/agnhost:2.45 + args: + - netexec + ports: + - name: http + containerPort: 8080 + protocol: TCP + readinessProbe: + httpGet: + path: / + port: http --- apiVersion: v1 kind: Service @@ -74,7 +72,7 @@ metadata: spec: ports: - port: 80 - targetPort: 80 + targetPort: 8080 name: http selector: app: {{.TestName}} From 4f349c2225bc453cd89c4d596dbb32fa5e16637c Mon Sep 17 00:00:00 2001 From: malpou Date: Mon, 2 Jun 2025 20:27:53 +0200 Subject: [PATCH 04/29] adjust tests according to comments Signed-off-by: malpou --- .../placeholder_pages_test.go | 48 +++++++------------ 1 file changed, 17 insertions(+), 31 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 418a4dfcf..94cc66ec1 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -119,13 +119,9 @@ func TestPlaceholderPages(t *testing.T) { DeleteNamespace(t, testNamespace) }() - assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 1, 240, 3), - "deployment replicas should be 1 after 3 minutes") - - // Wait for the deployment to scale down to zero - time.Sleep(90 * time.Second) - assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 0, 240, 3), - "deployment replicas should be 0 after 3 minutes") + // Wait for the deployment to exist with 0 replicas + assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 0, 60, 3), + "deployment should exist with 0 replicas within 1 minute") // Make a request and verify placeholder is served testPlaceholderResponse(t) @@ -137,18 +133,13 @@ func TestPlaceholderPages(t *testing.T) { func testPlaceholderResponse(t *testing.T) { t.Log("Testing placeholder page response") - var interceptorIP string - if UseIngressHost { - interceptorIP = ExternalIP - } else { - interceptorService := GetKubernetesServiceEndpoint( - t, - KubeClient, - "keda-http-add-on-interceptor-proxy", - "keda", - ) - interceptorIP = interceptorService.IP - } + interceptorService := GetKubernetesServiceEndpoint( + t, + KubeClient, + "keda-http-add-on-interceptor-proxy", + "keda", + ) + interceptorIP := interceptorService.IP url := fmt.Sprintf("http://%s", interceptorIP) req, err := http.NewRequest("GET", url, nil) @@ -190,18 +181,13 @@ func testPlaceholderResponse(t *testing.T) { func testImmediatePlaceholderResponse(t *testing.T) { t.Log("Testing immediate placeholder page response on cold start") - var interceptorIP string - if UseIngressHost { - interceptorIP = ExternalIP - } else { - interceptorService := GetKubernetesServiceEndpoint( - t, - KubeClient, - "keda-http-add-on-interceptor-proxy", - "keda", - ) - interceptorIP = interceptorService.IP - } + interceptorService := GetKubernetesServiceEndpoint( + t, + KubeClient, + "keda-http-add-on-interceptor-proxy", + "keda", + ) + interceptorIP := interceptorService.IP url := fmt.Sprintf("http://%s", interceptorIP) req, err := http.NewRequest("GET", url, nil) From 8e7904a38f3bf09a9343ed9bf5bf16163f4e1a8f Mon Sep 17 00:00:00 2001 From: malpou Date: Mon, 2 Jun 2025 21:15:38 +0200 Subject: [PATCH 05/29] add RWMutex to ensure that the template cache is locked when being written to Signed-off-by: malpou --- interceptor/handler/placeholder.go | 50 ++++- interceptor/handler/placeholder_test.go | 273 +++++++++++++++++++++++- 2 files changed, 309 insertions(+), 14 deletions(-) diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go index 1605944c9..b97428cf9 100644 --- a/interceptor/handler/placeholder.go +++ b/interceptor/handler/placeholder.go @@ -6,6 +6,7 @@ import ( "fmt" "html/template" "net/http" + "sync" "time" "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" @@ -76,11 +77,19 @@ const defaultPlaceholderTemplate = ` ` +// cacheEntry stores a template along with resource generation info for cache invalidation +type cacheEntry struct { + template *template.Template + hsoGeneration int64 + configMapVersion string +} + // PlaceholderHandler handles serving placeholder pages during scale-from-zero type PlaceholderHandler struct { k8sClient kubernetes.Interface routingTable routing.Table - templateCache map[string]*template.Template + templateCache map[string]*cacheEntry + cacheMutex sync.RWMutex defaultTmpl *template.Template } @@ -103,7 +112,7 @@ func NewPlaceholderHandler(k8sClient kubernetes.Interface, routingTable routing. return &PlaceholderHandler{ k8sClient: k8sClient, routingTable: routingTable, - templateCache: make(map[string]*template.Template), + templateCache: make(map[string]*cacheEntry), defaultTmpl: defaultTmpl, }, nil } @@ -165,29 +174,45 @@ func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTP if config.Content != "" { cacheKey := fmt.Sprintf("%s/%s/inline", hso.Namespace, hso.Name) - if tmpl, ok := h.templateCache[cacheKey]; ok { - return tmpl, nil + + h.cacheMutex.RLock() + entry, ok := h.templateCache[cacheKey] + if ok && entry.hsoGeneration == hso.Generation { + h.cacheMutex.RUnlock() + return entry.template, nil } + h.cacheMutex.RUnlock() tmpl, err := template.New("inline").Parse(config.Content) if err != nil { return nil, err } - h.templateCache[cacheKey] = tmpl + + h.cacheMutex.Lock() + h.templateCache[cacheKey] = &cacheEntry{ + template: tmpl, + hsoGeneration: hso.Generation, + } + h.cacheMutex.Unlock() return tmpl, nil } if config.ContentConfigMap != "" { cacheKey := fmt.Sprintf("%s/%s/cm/%s", hso.Namespace, hso.Name, config.ContentConfigMap) - if tmpl, ok := h.templateCache[cacheKey]; ok { - return tmpl, nil - } cm, err := h.k8sClient.CoreV1().ConfigMaps(hso.Namespace).Get(ctx, config.ContentConfigMap, metav1.GetOptions{}) if err != nil { return nil, fmt.Errorf("failed to get ConfigMap %s: %w", config.ContentConfigMap, err) } + h.cacheMutex.RLock() + entry, ok := h.templateCache[cacheKey] + if ok && entry.hsoGeneration == hso.Generation && entry.configMapVersion == cm.ResourceVersion { + h.cacheMutex.RUnlock() + return entry.template, nil + } + h.cacheMutex.RUnlock() + key := config.ContentConfigMapKey if key == "" { key = "template.html" @@ -202,7 +227,14 @@ func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTP if err != nil { return nil, err } - h.templateCache[cacheKey] = tmpl + + h.cacheMutex.Lock() + h.templateCache[cacheKey] = &cacheEntry{ + template: tmpl, + hsoGeneration: hso.Generation, + configMapVersion: cm.ResourceVersion, + } + h.cacheMutex.Unlock() return tmpl, nil } diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go index 34f04e8df..84a1795d8 100644 --- a/interceptor/handler/placeholder_test.go +++ b/interceptor/handler/placeholder_test.go @@ -5,6 +5,7 @@ import ( "fmt" "net/http" "net/http/httptest" + "sync" "testing" "github.com/stretchr/testify/assert" @@ -251,8 +252,9 @@ func TestGetTemplate_Caching(t *testing.T) { customContent := `{{.ServiceName}}` hso := &v1alpha1.HTTPScaledObject{ ObjectMeta: metav1.ObjectMeta{ - Name: "test-app", - Namespace: "default", + Name: "test-app", + Namespace: "default", + Generation: 1, }, Spec: v1alpha1.HTTPScaledObjectSpec{ PlaceholderConfig: &v1alpha1.PlaceholderConfig{ @@ -263,16 +265,277 @@ func TestGetTemplate_Caching(t *testing.T) { ctx := context.Background() - // First call - should parse and cache tmpl1, err := handler.getTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl1) - // Second call - should return from cache tmpl2, err := handler.getTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl2) - // Should be the same template instance assert.Equal(t, fmt.Sprintf("%p", tmpl1), fmt.Sprintf("%p", tmpl2)) } + +func TestGetTemplate_CacheInvalidation_Generation(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + customContent1 := `Version 1: {{.ServiceName}}` + customContent2 := `Version 2: {{.ServiceName}}` + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + Generation: 1, + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Content: customContent1, + }, + }, + } + + ctx := context.Background() + + tmpl1, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl1) + + hso.Generation = 2 + hso.Spec.PlaceholderConfig.Content = customContent2 + + tmpl2, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl2) + + assert.NotEqual(t, fmt.Sprintf("%p", tmpl1), fmt.Sprintf("%p", tmpl2)) +} + +func TestGetTemplate_CacheInvalidation_ConfigMapVersion(t *testing.T) { + // Create ConfigMap + cm := &v1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: "placeholder-cm", + Namespace: "default", + ResourceVersion: "1", + }, + Data: map[string]string{ + "template.html": `Version 1: {{.ServiceName}}`, + }, + } + k8sClient := fake.NewSimpleClientset(cm) + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + Generation: 1, + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + ContentConfigMap: "placeholder-cm", + }, + }, + } + + ctx := context.Background() + + tmpl1, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl1) + + cm.ResourceVersion = "2" + cm.Data["template.html"] = `Version 2: {{.ServiceName}}` + _, err = k8sClient.CoreV1().ConfigMaps("default").Update(ctx, cm, metav1.UpdateOptions{}) + require.NoError(t, err) + + tmpl2, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + assert.NotNil(t, tmpl2) +} + +func TestGetTemplate_ConcurrentAccess(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + customContent := `{{.ServiceName}}` + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + Generation: 1, + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Content: customContent, + }, + }, + } + + ctx := context.Background() + + _, err := handler.getTemplate(ctx, hso) + require.NoError(t, err) + + var wg sync.WaitGroup + errors := make(chan error, 100) + templates := make(chan interface{}, 100) + + for i := 0; i < 100; i++ { + wg.Add(1) + go func() { + defer wg.Done() + tmpl, err := handler.getTemplate(ctx, hso) + if err != nil { + errors <- err + } else { + templates <- tmpl + } + }() + } + + wg.Wait() + close(errors) + close(templates) + + var errorCount int + for err := range errors { + t.Errorf("Concurrent access error: %v", err) + errorCount++ + } + assert.Equal(t, 0, errorCount) + + var firstTemplate interface{} + templateCount := 0 + for tmpl := range templates { + templateCount++ + if firstTemplate == nil { + firstTemplate = tmpl + } else { + assert.Equal(t, fmt.Sprintf("%p", firstTemplate), fmt.Sprintf("%p", tmpl), + "All templates should be the same cached instance") + } + } + assert.Equal(t, 100, templateCount) +} + +func TestGetTemplate_ConcurrentFirstAccess(t *testing.T) { + k8sClient := fake.NewSimpleClientset() + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + customContent := `{{.ServiceName}}` + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "test-app", + Namespace: "default", + Generation: 1, + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Content: customContent, + }, + }, + } + + ctx := context.Background() + + var wg sync.WaitGroup + errors := make(chan error, 100) + + for i := 0; i < 100; i++ { + wg.Add(1) + go func() { + defer wg.Done() + _, err := handler.getTemplate(ctx, hso) + if err != nil { + errors <- err + } + }() + } + + wg.Wait() + close(errors) + + for err := range errors { + t.Errorf("Concurrent access error: %v", err) + } + + handler.cacheMutex.RLock() + cacheKey := fmt.Sprintf("%s/%s/inline", hso.Namespace, hso.Name) + entry, ok := handler.templateCache[cacheKey] + handler.cacheMutex.RUnlock() + + assert.True(t, ok, "Cache should have an entry") + assert.NotNil(t, entry, "Cache entry should not be nil") + assert.NotNil(t, entry.template, "Cached template should not be nil") +} + +func TestGetTemplate_ConcurrentCacheUpdates(t *testing.T) { + configMaps := make([]*v1.ConfigMap, 10) + for i := 0; i < 10; i++ { + configMaps[i] = &v1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("placeholder-cm-%d", i), + Namespace: "default", + ResourceVersion: "1", + }, + Data: map[string]string{ + "template.html": fmt.Sprintf(`ConfigMap %d: {{.ServiceName}}`, i), + }, + } + } + + k8sClient := fake.NewSimpleClientset() + for _, cm := range configMaps { + _, err := k8sClient.CoreV1().ConfigMaps("default").Create(context.Background(), cm, metav1.CreateOptions{}) + require.NoError(t, err) + } + + routingTable := test.NewTable() + handler, _ := NewPlaceholderHandler(k8sClient, routingTable) + + ctx := context.Background() + + var wg sync.WaitGroup + errors := make(chan error, 100) + + for i := 0; i < 10; i++ { + for j := 0; j < 10; j++ { + wg.Add(1) + go func(cmIndex, iteration int) { + defer wg.Done() + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("test-app-%d", cmIndex), + Namespace: "default", + Generation: int64(iteration), + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + ContentConfigMap: fmt.Sprintf("placeholder-cm-%d", cmIndex), + }, + }, + } + + _, err := handler.getTemplate(ctx, hso) + if err != nil { + errors <- err + } + }(i, j) + } + } + + wg.Wait() + close(errors) + + for err := range errors { + t.Errorf("Concurrent cache update error: %v", err) + } +} From d15d207c99d8db763f900591aa5d249f801e4f70 Mon Sep 17 00:00:00 2001 From: malpou Date: Mon, 2 Jun 2025 21:16:53 +0200 Subject: [PATCH 06/29] adjust e2e tests away from my local setup and over to something that should be comptatible in the GHA environment Signed-off-by: malpou --- .../placeholder_pages_test.go | 209 ++++++------------ 1 file changed, 62 insertions(+), 147 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 94cc66ec1..6be3a685e 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -5,27 +5,22 @@ package placeholder_pages import ( "fmt" - "net/http" - "strings" "testing" - "time" - "github.com/stretchr/testify/assert" + . "github.com/kedacore/http-add-on/tests/helper" ) const ( - testName = "placeholder-pages-test" - testServiceName = testName - testNamespace = testName + "-ns" + testName = "placeholder-test" + testNamespace = testName + "-ns" ) -type placeholderTemplateData struct { - TestNamespace string - TestName string - TestServiceName string +type templateData struct { + TestNamespace string + TestName string } -const placeholderTemplate = ` +const testTemplate = ` apiVersion: v1 kind: Namespace metadata: @@ -36,8 +31,6 @@ kind: Deployment metadata: name: {{.TestName}} namespace: {{.TestNamespace}} - labels: - app: {{.TestName}} spec: replicas: 1 selector: @@ -49,34 +42,23 @@ spec: app: {{.TestName}} spec: containers: - - name: {{.TestName}} - image: registry.k8s.io/e2e-test-images/agnhost:2.45 - args: - - netexec - ports: - - name: http - containerPort: 8080 - protocol: TCP - readinessProbe: - httpGet: - path: / - port: http + - name: {{.TestName}} + image: registry.k8s.io/e2e-test-images/agnhost:2.45 + args: ["netexec"] + ports: + - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: - name: {{.TestServiceName}} + name: {{.TestName}} namespace: {{.TestNamespace}} - labels: - app: {{.TestName}} spec: ports: - port: 80 targetPort: 8080 - name: http selector: app: {{.TestName}} - type: ClusterIP --- apiVersion: http.keda.sh/v1alpha1 kind: HTTPScaledObject @@ -85,138 +67,71 @@ metadata: namespace: {{.TestNamespace}} spec: hosts: - - {{.TestName}} - pathPrefixes: - - / + - {{.TestName}}.test scaleTargetRef: - service: {{.TestServiceName}} - port: 80 deployment: {{.TestName}} - targetPendingRequests: 1 - scaledownPeriod: 60 + service: {{.TestName}} + port: 80 + replicas: + min: 0 + max: 10 placeholderConfig: enabled: true statusCode: 503 refreshInterval: 5 headers: - X-Service-Status: "warming-up" + X-Test-Header: "test-value" ` func TestPlaceholderPages(t *testing.T) { - // Create test data - data := placeholderTemplateData{ - TestNamespace: testNamespace, - TestName: testName, - TestServiceName: testServiceName, + // Setup + data := templateData{ + TestNamespace: testNamespace, + TestName: testName, } - // Apply the resources - KubectlApplyWithTemplate(t, data, "placeholder-test", placeholderTemplate) - - // Ensure cleanup + KubectlApplyWithTemplate(t, data, "placeholder-test", testTemplate) defer func() { - KubectlDeleteWithTemplate(t, data, "placeholder-test", placeholderTemplate) + KubectlDeleteWithTemplate(t, data, "placeholder-test", testTemplate) DeleteNamespace(t, testNamespace) }() - // Wait for the deployment to exist with 0 replicas - assert.True(t, WaitForDeploymentReplicaReadyCount(t, KubeClient, testName, testNamespace, 0, 60, 3), - "deployment should exist with 0 replicas within 1 minute") - - // Make a request and verify placeholder is served - testPlaceholderResponse(t) - - // Test that placeholder is served immediately on cold start - testImmediatePlaceholderResponse(t) -} - -func testPlaceholderResponse(t *testing.T) { - t.Log("Testing placeholder page response") - - interceptorService := GetKubernetesServiceEndpoint( - t, - KubeClient, - "keda-http-add-on-interceptor-proxy", - "keda", - ) - interceptorIP := interceptorService.IP - - url := fmt.Sprintf("http://%s", interceptorIP) - req, err := http.NewRequest("GET", url, nil) - assert.NoError(t, err) - req.Host = testName - - client := &http.Client{ - Timeout: 10 * time.Second, - CheckRedirect: func(req *http.Request, via []*http.Request) error { - return http.ErrUseLastResponse - }, - } - - resp, err := client.Do(req) - assert.NoError(t, err) - defer resp.Body.Close() - - // Check that we got a 503 status (default for placeholder) - assert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode) + // Wait for deployment to scale to 0 + assert.True(t, + WaitForDeploymentReplicaReadyCount(t, GetKubernetesClient(t), testName, testNamespace, 0, 60, 3), + "deployment should scale to 0") - // Read response body - body := make([]byte, 1024) - n, _ := resp.Body.Read(body) - bodyStr := string(body[:n]) + // Make a request through a pod and check placeholder response + curlCmd := fmt.Sprintf("curl -i -H 'Host: %s.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) - // Verify placeholder content - assert.True(t, strings.Contains(bodyStr, testServiceName+" is starting up..."), - "response should contain placeholder message") - assert.True(t, strings.Contains(bodyStr, "refresh"), - "response should contain refresh meta tag") - - // Check custom headers - assert.Equal(t, "true", resp.Header.Get("X-KEDA-HTTP-Placeholder-Served"), - "placeholder served header should be present") - assert.Equal(t, "warming-up", resp.Header.Get("X-Service-Status"), - "custom header should be present") -} - -func testImmediatePlaceholderResponse(t *testing.T) { - t.Log("Testing immediate placeholder page response on cold start") - - interceptorService := GetKubernetesServiceEndpoint( - t, - KubeClient, - "keda-http-add-on-interceptor-proxy", - "keda", - ) - interceptorIP := interceptorService.IP - - url := fmt.Sprintf("http://%s", interceptorIP) - req, err := http.NewRequest("GET", url, nil) - assert.NoError(t, err) - req.Host = testName - - client := &http.Client{ - Timeout: 2 * time.Second, // Short timeout to ensure immediate response - CheckRedirect: func(req *http.Request, via []*http.Request) error { - return http.ErrUseLastResponse - }, - } - - // Measure response time - start := time.Now() - resp, err := client.Do(req) - duration := time.Since(start) - - assert.NoError(t, err) - defer resp.Body.Close() - - // Check that response was immediate (less than 500ms) - assert.Less(t, duration.Milliseconds(), int64(500), - "placeholder should be served immediately, but took %v", duration) - - // Check that we got a 503 status - assert.Equal(t, http.StatusServiceUnavailable, resp.StatusCode) - - // Check placeholder header - assert.Equal(t, "true", resp.Header.Get("X-KEDA-HTTP-Placeholder-Served"), - "placeholder served header should be present") + // Create and run a curl pod + curlPod := fmt.Sprintf(` +apiVersion: v1 +kind: Pod +metadata: + name: curl-test + namespace: %s +spec: + containers: + - name: curl + image: curlimages/curl + command: ["sh", "-c", "%s && sleep 5"] + restartPolicy: Never +`, testNamespace, curlCmd) + + KubectlApplyWithTemplate(t, data, "curl-pod", curlPod) + defer KubectlDeleteWithTemplate(t, data, "curl-pod", curlPod) + + // Wait and get logs + assert.True(t, + WaitForSuccessfulExecCommandOnSpecificPod(t, "curl-test", testNamespace, "echo done", 60, 3), + "curl should complete") + + logs, _ := KubectlLogs(t, "curl-test", testNamespace, "") + + // Verify placeholder response + assert.Contains(t, logs, "HTTP/1.1 503", "should return 503 status") + assert.Contains(t, logs, "X-KEDA-HTTP-Placeholder-Served: true", "should have placeholder header") + assert.Contains(t, logs, "X-Test-Header: test-value", "should have custom header") + assert.Contains(t, logs, "is starting up", "should have placeholder message") } From 6eb727230129966791475aa3f0ece7e48cbe70ec Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 13:43:41 +0200 Subject: [PATCH 07/29] Update placeholder_pages_test.go Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- .../placeholder_pages_test.go | 86 ++++++++++++------- 1 file changed, 55 insertions(+), 31 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 6be3a685e..882482af3 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -7,6 +7,9 @@ import ( "fmt" "testing" + "github.com/stretchr/testify/assert" + "k8s.io/client-go/kubernetes" + . "github.com/kedacore/http-add-on/tests/helper" ) @@ -32,7 +35,7 @@ metadata: name: {{.TestName}} namespace: {{.TestNamespace}} spec: - replicas: 1 + replicas: 0 selector: matchLabels: app: {{.TestName}} @@ -69,12 +72,13 @@ spec: hosts: - {{.TestName}}.test scaleTargetRef: - deployment: {{.TestName}} + name: {{.TestName}} service: {{.TestName}} port: 80 replicas: min: 0 max: 10 + scaledownPeriod: 10 placeholderConfig: enabled: true statusCode: 503 @@ -84,54 +88,74 @@ spec: ` func TestPlaceholderPages(t *testing.T) { - // Setup + // setup + t.Log("--- setting up ---") + // Create kubernetes resources + kc := GetKubernetesClient(t) data := templateData{ TestNamespace: testNamespace, TestName: testName, } + CreateNamespace(t, kc, testNamespace) + defer DeleteNamespace(t, testNamespace) + KubectlApplyWithTemplate(t, data, "placeholder-test", testTemplate) - defer func() { - KubectlDeleteWithTemplate(t, data, "placeholder-test", testTemplate) - DeleteNamespace(t, testNamespace) - }() + defer KubectlDeleteWithTemplate(t, data, "placeholder-test", testTemplate) + + // Wait for deployment to be at 0 replicas (since min is 0) + assert.True(t, WaitForDeploymentReplicaReadyCount(t, kc, testName, testNamespace, 0, 6, 10), + "deployment should be at 0 replicas") - // Wait for deployment to scale to 0 - assert.True(t, - WaitForDeploymentReplicaReadyCount(t, GetKubernetesClient(t), testName, testNamespace, 0, 60, 3), - "deployment should scale to 0") + // Test placeholder response + testPlaceholderResponse(t, kc) +} - // Make a request through a pod and check placeholder response - curlCmd := fmt.Sprintf("curl -i -H 'Host: %s.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) +func testPlaceholderResponse(t *testing.T, kc *kubernetes.Clientset) { + t.Log("--- testing placeholder response ---") - // Create and run a curl pod - curlPod := fmt.Sprintf(` + // Create a test pod to make requests + clientPod := ` apiVersion: v1 kind: Pod metadata: - name: curl-test - namespace: %s + name: curl-client + namespace: ` + testNamespace + ` spec: containers: - name: curl image: curlimages/curl - command: ["sh", "-c", "%s && sleep 5"] - restartPolicy: Never -`, testNamespace, curlCmd) + command: ["sleep", "3600"] +` + // Create the pod using KubectlApplyWithTemplate + data := templateData{ + TestNamespace: testNamespace, + TestName: testName, + } + KubectlApplyWithTemplate(t, data, "curl-client", clientPod) + defer KubectlDeleteWithTemplate(t, data, "curl-client", clientPod) - KubectlApplyWithTemplate(t, data, "curl-pod", curlPod) - defer KubectlDeleteWithTemplate(t, data, "curl-pod", curlPod) + // Wait for curl pod to be ready + assert.True(t, WaitForPodCountInNamespace(t, kc, testNamespace, 1, 30, 2), + "curl client pod should be ready") - // Wait and get logs - assert.True(t, - WaitForSuccessfulExecCommandOnSpecificPod(t, "curl-test", testNamespace, "echo done", 60, 3), - "curl should complete") + // Give pod time to fully initialize + _, _ = ExecuteCommand("sleep 5") + + // Make request through interceptor + curlCmd := fmt.Sprintf("curl -si -H 'Host: %s.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) + stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) + t.Logf("curl output: %s", stdout) + if stderr != "" { + t.Logf("curl stderr: %s", stderr) + } - logs, _ := KubectlLogs(t, "curl-test", testNamespace, "") + assert.NoError(t, err, "curl command should succeed") // Verify placeholder response - assert.Contains(t, logs, "HTTP/1.1 503", "should return 503 status") - assert.Contains(t, logs, "X-KEDA-HTTP-Placeholder-Served: true", "should have placeholder header") - assert.Contains(t, logs, "X-Test-Header: test-value", "should have custom header") - assert.Contains(t, logs, "is starting up", "should have placeholder message") + assert.Contains(t, stdout, "HTTP/1.1 503", "should return 503 status") + assert.Contains(t, stdout, "X-Keda-Http-Placeholder-Served", "should have placeholder header") + assert.Contains(t, stdout, "X-Test-Header", "should have custom header") + assert.Contains(t, stdout, "test-value", "should have custom header value") + assert.Contains(t, stdout, "is starting up", "should have placeholder message") } From c94489c0d0cf4b20348db1743d7457648c341956 Mon Sep 17 00:00:00 2001 From: malpou Date: Tue, 3 Jun 2025 17:51:00 +0200 Subject: [PATCH 08/29] take on injecting reloading watching js using HEAD requests Signed-off-by: malpou --- interceptor/handler/placeholder.go | 80 ++++- interceptor/handler/placeholder_test.go | 273 ++++++++++++++++++ .../placeholder_pages_test.go | 63 ++++ 3 files changed, 411 insertions(+), 5 deletions(-) diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go index b97428cf9..91ec829c4 100644 --- a/interceptor/handler/placeholder.go +++ b/interceptor/handler/placeholder.go @@ -6,6 +6,7 @@ import ( "fmt" "html/template" "net/http" + "strings" "sync" "time" @@ -15,11 +16,38 @@ import ( "k8s.io/client-go/kubernetes" ) -const defaultPlaceholderTemplate = ` +const placeholderScript = `` + +const defaultPlaceholderTemplateWithoutScript = ` Service Starting - @@ -100,7 +96,6 @@ const defaultPlaceholderTemplateWithoutScript = `

{{.ServiceName}} is starting up...

Please wait while we prepare your service.

-

This page will refresh automatically every {{.RefreshInterval}} seconds.

` From 4a7ea4e88e8fd3646885050288f4a7547a9b3c1c Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:28:24 +0200 Subject: [PATCH 10/29] fix unit tests Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- interceptor/handler/placeholder_test.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go index 3a4720afa..ab61012b9 100644 --- a/interceptor/handler/placeholder_test.go +++ b/interceptor/handler/placeholder_test.go @@ -92,7 +92,8 @@ func TestServePlaceholder_DefaultTemplate(t *testing.T) { assert.NoError(t, err) assert.Equal(t, http.StatusServiceUnavailable, w.Code) assert.Contains(t, w.Body.String(), "test-service is starting up...") - assert.Contains(t, w.Body.String(), "refresh") + assert.Contains(t, w.Body.String(), "checkServiceStatus") + assert.Contains(t, w.Body.String(), "checkInterval = 5 * 1000") assert.Equal(t, "text/html; charset=utf-8", w.Header().Get("Content-Type")) assert.Equal(t, "true", w.Header().Get("X-KEDA-HTTP-Placeholder-Served")) } From 5c36d7b1add8db82904855caeb9ef9f453bf7677 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:29:02 +0200 Subject: [PATCH 11/29] fix e2e tests Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- .../placeholder_pages_test.go | 34 ++++++++----------- 1 file changed, 15 insertions(+), 19 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index fa8aa8fa0..d3924cf5f 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -103,20 +103,6 @@ func TestPlaceholderPages(t *testing.T) { KubectlApplyWithTemplate(t, data, "placeholder-test", testTemplate) defer KubectlDeleteWithTemplate(t, data, "placeholder-test", testTemplate) - // Wait for deployment to be at 0 replicas (since min is 0) - assert.True(t, WaitForDeploymentReplicaReadyCount(t, kc, testName, testNamespace, 0, 6, 10), - "deployment should be at 0 replicas") - - // Test placeholder response - testPlaceholderResponse(t, kc) - - // Test custom placeholder with script injection - testCustomPlaceholderWithScript(t, kc, data) -} - -func testPlaceholderResponse(t *testing.T, kc *kubernetes.Clientset) { - t.Log("--- testing placeholder response ---") - // Create a test pod to make requests clientPod := ` apiVersion: v1 @@ -131,10 +117,6 @@ spec: command: ["sleep", "3600"] ` // Create the pod using KubectlApplyWithTemplate - data := templateData{ - TestNamespace: testNamespace, - TestName: testName, - } KubectlApplyWithTemplate(t, data, "curl-client", clientPod) defer KubectlDeleteWithTemplate(t, data, "curl-client", clientPod) @@ -145,6 +127,20 @@ spec: // Give pod time to fully initialize _, _ = ExecuteCommand("sleep 5") + // Wait for deployment to be at 0 replicas (since min is 0) + assert.True(t, WaitForDeploymentReplicaReadyCount(t, kc, testName, testNamespace, 0, 6, 10), + "deployment should be at 0 replicas") + + // Test placeholder response + testPlaceholderResponse(t, kc) + + // Test custom placeholder with script injection + testCustomPlaceholderWithScript(t, kc, data) +} + +func testPlaceholderResponse(t *testing.T, kc *kubernetes.Clientset) { + t.Log("--- testing placeholder response ---") + // Make request through interceptor curlCmd := fmt.Sprintf("curl -si -H 'Host: %s.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) @@ -191,7 +187,7 @@ spec: content: | -

Custom placeholder for {{"{{"}} .ServiceName {{"}}"}}

+

Custom placeholder for {{ "{{" }}.ServiceName{{ "}}" }}

Please wait...

From 67ed6ddd0b38e53aaaf1c59b59e1e834b2377c73 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:49:21 +0200 Subject: [PATCH 12/29] simplify e2e tests Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- .../placeholder_pages/placeholder_pages_test.go | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index d3924cf5f..079b5712a 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -127,10 +127,6 @@ spec: // Give pod time to fully initialize _, _ = ExecuteCommand("sleep 5") - // Wait for deployment to be at 0 replicas (since min is 0) - assert.True(t, WaitForDeploymentReplicaReadyCount(t, kc, testName, testNamespace, 0, 6, 10), - "deployment should be at 0 replicas") - // Test placeholder response testPlaceholderResponse(t, kc) @@ -187,8 +183,8 @@ spec: content: | -

Custom placeholder for {{ "{{" }}.ServiceName{{ "}}" }}

-

Please wait...

+

Service is starting - custom page

+

This is a test placeholder page.

` @@ -197,7 +193,7 @@ spec: defer KubectlDeleteWithTemplate(t, data, "custom-placeholder", customTemplate) // Give time for the new HTTPScaledObject to be processed - _, _ = ExecuteCommand("sleep 5") + _, _ = ExecuteCommand("sleep 10") // Make request to custom placeholder curlCmd := fmt.Sprintf("curl -s -H 'Host: %s-custom.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) @@ -210,8 +206,8 @@ spec: assert.NoError(t, err, "curl command should succeed") // Verify custom content is there - assert.Contains(t, stdout, "Custom placeholder for", "should have custom placeholder content") - assert.Contains(t, stdout, "Please wait...", "should have custom message") + assert.Contains(t, stdout, "Service is starting - custom page", "should have custom placeholder content") + assert.Contains(t, stdout, "This is a test placeholder page.", "should have custom message") // Verify script was injected assert.Contains(t, stdout, "checkServiceStatus", "should have injected checkServiceStatus function") From 351e6071b890f63eb4fbe0d63f45e0905f5a01ac Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:52:14 +0200 Subject: [PATCH 13/29] remove sleep Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- tests/checks/placeholder_pages/placeholder_pages_test.go | 3 --- 1 file changed, 3 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 079b5712a..698a4c7e0 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -192,9 +192,6 @@ spec: KubectlApplyWithTemplate(t, data, "custom-placeholder", customTemplate) defer KubectlDeleteWithTemplate(t, data, "custom-placeholder", customTemplate) - // Give time for the new HTTPScaledObject to be processed - _, _ = ExecuteCommand("sleep 10") - // Make request to custom placeholder curlCmd := fmt.Sprintf("curl -s -H 'Host: %s-custom.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) From 0584407858dab24669de82c0061c6e6c9a88fbd6 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Tue, 3 Jun 2025 18:55:31 +0200 Subject: [PATCH 14/29] refinement Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- tests/checks/placeholder_pages/placeholder_pages_test.go | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 698a4c7e0..1455f6d48 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -24,11 +24,6 @@ type templateData struct { } const testTemplate = ` -apiVersion: v1 -kind: Namespace -metadata: - name: {{.TestNamespace}} ---- apiVersion: apps/v1 kind: Deployment metadata: @@ -124,8 +119,8 @@ spec: assert.True(t, WaitForPodCountInNamespace(t, kc, testNamespace, 1, 30, 2), "curl client pod should be ready") - // Give pod time to fully initialize - _, _ = ExecuteCommand("sleep 5") + // Give container time to fully initialize + _, _ = ExecuteCommand("sleep 2") // Test placeholder response testPlaceholderResponse(t, kc) From d47117143a19f54e498086b4fce13fc9dd9b7c72 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Wed, 4 Jun 2025 08:54:42 +0200 Subject: [PATCH 15/29] Update interceptor/handler/placeholder.go Co-authored-by: Jorge Turrado Ferrero Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Signed-off-by: malpou --- interceptor/handler/placeholder.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go index c849b5dfa..59a362fa9 100644 --- a/interceptor/handler/placeholder.go +++ b/interceptor/handler/placeholder.go @@ -10,10 +10,11 @@ import ( "sync" "time" + "k8s.io/client-go/kubernetes" + "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" "github.com/kedacore/http-add-on/pkg/routing" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/client-go/kubernetes" ) const placeholderScript = `` - -const defaultPlaceholderTemplateWithoutScript = ` - - - Service Starting - - - - -
-

{{.ServiceName}} is starting up...

-
-

Please wait while we prepare your service.

-
- -` // cacheEntry stores a template along with resource generation info for cache invalidation type cacheEntry struct { @@ -112,7 +28,6 @@ type PlaceholderHandler struct { cacheMutex sync.RWMutex defaultTmpl *template.Template servingCfg *config.Serving - enableScript bool } // PlaceholderData contains data for rendering placeholder templates @@ -126,105 +41,32 @@ type PlaceholderData struct { // NewPlaceholderHandler creates a new placeholder handler func NewPlaceholderHandler(servingCfg *config.Serving) (*PlaceholderHandler, error) { - var defaultTemplate string + var defaultTmpl *template.Template - // Try to load template from configured path + // Try to load template from configured path if provided if servingCfg.PlaceholderDefaultTemplatePath != "" { content, err := os.ReadFile(servingCfg.PlaceholderDefaultTemplatePath) - if err == nil { - defaultTemplate = string(content) - } else { - // Fall back to built-in template if file cannot be read - fmt.Printf("Warning: Could not read placeholder template from %s: %v. Using built-in template.\n", + if err != nil { + fmt.Printf("Warning: Could not read placeholder template from %s: %v. No default template will be used.\n", servingCfg.PlaceholderDefaultTemplatePath, err) - defaultTemplate = defaultPlaceholderTemplateWithoutScript + } else { + defaultTmpl, err = template.New("default").Parse(string(content)) + if err != nil { + fmt.Printf("Warning: Could not parse placeholder template from %s: %v. No default template will be used.\n", + servingCfg.PlaceholderDefaultTemplatePath, err) + defaultTmpl = nil + } } - } else { - defaultTemplate = defaultPlaceholderTemplateWithoutScript - } - - // Inject script if enabled - if servingCfg.PlaceholderEnableScript { - defaultTemplate = injectPlaceholderScript(defaultTemplate) - } - - defaultTmpl, err := template.New("default").Parse(defaultTemplate) - if err != nil { - return nil, fmt.Errorf("failed to parse default template: %w", err) } return &PlaceholderHandler{ templateCache: make(map[string]*cacheEntry), defaultTmpl: defaultTmpl, servingCfg: servingCfg, - enableScript: servingCfg.PlaceholderEnableScript, }, nil } -// injectPlaceholderScript injects the placeholder refresh script into a template -func injectPlaceholderScript(templateContent string) string { - lowerContent := strings.ToLower(templateContent) - - // Look for tag (case-insensitive) - bodyCloseIndex := strings.LastIndex(lowerContent, "") - if bodyCloseIndex != -1 { - // Insert script before - return templateContent[:bodyCloseIndex] + placeholderScript + templateContent[bodyCloseIndex:] - } - - // Look for tag if no body tag - htmlCloseIndex := strings.LastIndex(lowerContent, "") - if htmlCloseIndex != -1 { - // Insert script before - return templateContent[:htmlCloseIndex] + placeholderScript + templateContent[htmlCloseIndex:] - } - // Check if content appears to be HTML (has any HTML tags) - if strings.Contains(templateContent, "<") && strings.Contains(templateContent, ">") { - // It looks like HTML, append the script - return templateContent + placeholderScript - } - - // Don't wrap non-HTML content - return as-is - return templateContent -} - -// detectContentType determines the appropriate content type based on Accept header and content -func detectContentType(acceptHeader string, content string) string { - // Check Accept header for specific content types - if strings.Contains(acceptHeader, "application/json") { - return "application/json" - } - if strings.Contains(acceptHeader, "application/xml") { - return "application/xml" - } - if strings.Contains(acceptHeader, "text/plain") { - return "text/plain" - } - - // Default to HTML for browser requests or when HTML is accepted - if strings.Contains(acceptHeader, "text/html") || strings.Contains(acceptHeader, "*/*") || acceptHeader == "" { - // Check if content looks like HTML - if strings.Contains(content, "<") && strings.Contains(content, ">") { - return "text/html; charset=utf-8" - } - } - - // Try to detect based on content - trimmed := strings.TrimSpace(content) - if (strings.HasPrefix(trimmed, "{") && strings.HasSuffix(trimmed, "}")) || - (strings.HasPrefix(trimmed, "[") && strings.HasSuffix(trimmed, "]")) { - return "application/json" - } - if strings.HasPrefix(trimmed, "<") { - if strings.HasPrefix(trimmed, "")) { - content = injectPlaceholderScript(content) - } - tmpl, err := template.New("inline").Parse(content) + tmpl, err := template.New("inline").Parse(config.Content) if err != nil { h.cacheMutex.Unlock() return nil, err @@ -329,5 +156,9 @@ func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTP return tmpl, nil } - return h.defaultTmpl, nil + if h.defaultTmpl != nil { + return h.defaultTmpl, nil + } + + return nil, fmt.Errorf("no placeholder template configured") } diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go index d83dccbc2..6bfde7957 100644 --- a/interceptor/handler/placeholder_test.go +++ b/interceptor/handler/placeholder_test.go @@ -5,7 +5,6 @@ import ( "fmt" "net/http" "net/http/httptest" - "strings" "sync" "testing" @@ -20,23 +19,17 @@ import ( const testCustomContent = `{{.ServiceName}}` func TestNewPlaceholderHandler(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, err := NewPlaceholderHandler(servingCfg) assert.NoError(t, err) assert.NotNil(t, handler) assert.NotNil(t, handler.servingCfg) assert.NotNil(t, handler.templateCache) - assert.NotNil(t, handler.defaultTmpl) - assert.True(t, handler.enableScript) } func TestServePlaceholder_DisabledConfig(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) // Create HTTPScaledObject with disabled placeholder @@ -65,9 +58,7 @@ func TestServePlaceholder_DisabledConfig(t *testing.T) { } func TestServePlaceholder_DefaultTemplate(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) // Create HTTPScaledObject with enabled placeholder @@ -95,16 +86,12 @@ func TestServePlaceholder_DefaultTemplate(t *testing.T) { assert.NoError(t, err) assert.Equal(t, http.StatusServiceUnavailable, w.Code) assert.Contains(t, w.Body.String(), "test-service is starting up...") - assert.Contains(t, w.Body.String(), "checkServiceStatus") - assert.Contains(t, w.Body.String(), "checkInterval = 5 * 1000") - assert.Equal(t, "text/html; charset=utf-8", w.Header().Get("Content-Type")) + assert.Equal(t, "text/plain; charset=utf-8", w.Header().Get("Content-Type")) assert.Equal(t, "true", w.Header().Get("X-KEDA-HTTP-Placeholder-Served")) } func TestServePlaceholder_InlineContent(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) customContent := `

Custom placeholder for {{.ServiceName}}

` @@ -134,16 +121,10 @@ func TestServePlaceholder_InlineContent(t *testing.T) { assert.NoError(t, err) assert.Equal(t, http.StatusAccepted, w.Code) assert.Contains(t, w.Body.String(), "Custom placeholder for test-service") - - // Verify script was injected - assert.Contains(t, w.Body.String(), "checkServiceStatus") - assert.Contains(t, w.Body.String(), "X-KEDA-HTTP-Placeholder-Served") } func TestServePlaceholder_NonHTMLContent(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) // Test with JSON content @@ -163,6 +144,9 @@ func TestServePlaceholder_NonHTMLContent(t *testing.T) { StatusCode: 503, RefreshInterval: 5, Content: jsonContent, + Headers: map[string]string{ + "Content-Type": "application/json", + }, }, }, } @@ -175,16 +159,11 @@ func TestServePlaceholder_NonHTMLContent(t *testing.T) { assert.NoError(t, err) assert.Equal(t, http.StatusServiceUnavailable, w.Code) assert.Contains(t, w.Body.String(), `"service": "test-service"`) - - // Verify script was NOT injected into JSON - assert.NotContains(t, w.Body.String(), "checkServiceStatus") assert.Equal(t, "application/json", w.Header().Get("Content-Type")) } func TestServePlaceholder_CustomHeaders(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) hso := &v1alpha1.HTTPScaledObject{ @@ -218,9 +197,7 @@ func TestServePlaceholder_CustomHeaders(t *testing.T) { } func TestServePlaceholder_InvalidTemplate(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) // Invalid template syntax @@ -255,9 +232,7 @@ func TestServePlaceholder_InvalidTemplate(t *testing.T) { } func TestGetTemplate_Caching(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) hso := &v1alpha1.HTTPScaledObject{ @@ -287,9 +262,7 @@ func TestGetTemplate_Caching(t *testing.T) { } func TestGetTemplate_CacheInvalidation_Generation(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) customContent1 := `Version 1: {{.ServiceName}}` @@ -374,9 +347,7 @@ func TestGetTemplate_CacheInvalidation_ConfigMapVersion_REMOVED(t *testing.T) { } func TestGetTemplate_ConcurrentAccess(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) hso := &v1alpha1.HTTPScaledObject{ @@ -440,9 +411,7 @@ func TestGetTemplate_ConcurrentAccess(t *testing.T) { } func TestGetTemplate_ConcurrentFirstAccess(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) hso := &v1alpha1.HTTPScaledObject{ @@ -492,9 +461,7 @@ func TestGetTemplate_ConcurrentFirstAccess(t *testing.T) { } func TestGetTemplate_ConcurrentCacheUpdates(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) ctx := context.Background() @@ -537,254 +504,268 @@ func TestGetTemplate_ConcurrentCacheUpdates(t *testing.T) { } } -func TestInjectPlaceholderScript(t *testing.T) { - tests := []struct { - name string - input string - expected string - }{ - { - name: "HTML with body tag", - input: `

Hello

`, - expected: `

Hello

` + placeholderScript + ``, - }, - { - name: "HTML with uppercase BODY tag", - input: `

Hello

`, - expected: `

Hello

` + placeholderScript + ``, - }, - { - name: "HTML without body tag", - input: `
Hello
`, - expected: `
Hello
` + placeholderScript + ``, - }, - { - name: "HTML fragment with angle brackets", - input: `

Just some text

`, - expected: `

Just some text

` + placeholderScript, - }, - { - name: "Empty string", - input: ``, - expected: ``, - }, - { - name: "Multiple body tags (uses last one)", - input: `FirstSecond`, - expected: `FirstSecond` + placeholderScript + ``, - }, - { - name: "HTML with only html close tag", - input: `
Hello
`, - expected: `
Hello
` + placeholderScript + ``, - }, - { - name: "Non-HTML content NOT wrapped anymore", - input: `Just plain text without HTML`, - expected: `Just plain text without HTML`, - }, - { - name: "Partial HTML without closing tags", - input: `
Some content
`, - expected: `
Some content
` + placeholderScript, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - result := injectPlaceholderScript(tt.input) - assert.Equal(t, tt.expected, result) - }) - } -} +// Content-Agnostic Tests - Verify the feature works with any content format -func TestServePlaceholder_InlineContentWithScriptInjection(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } +func TestServePlaceholder_JSONResponse(t *testing.T) { + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) - // Custom content without the script - customContent := `

Custom placeholder for {{.ServiceName}}

` + jsonContent := `{ + "status": "warming_up", + "service": "{{.ServiceName}}", + "namespace": "{{.Namespace}}", + "retry_after_seconds": {{.RefreshInterval}}, + "timestamp": "{{.Timestamp}}" +}` hso := &v1alpha1.HTTPScaledObject{ ObjectMeta: metav1.ObjectMeta{ - Name: "test-app", - Namespace: "default", + Name: "api-service", + Namespace: "production", }, Spec: v1alpha1.HTTPScaledObjectSpec{ ScaleTargetRef: v1alpha1.ScaleTargetRef{ - Service: "test-service", + Service: "api-backend", }, PlaceholderConfig: &v1alpha1.PlaceholderConfig{ Enabled: true, - StatusCode: 503, + StatusCode: 202, RefreshInterval: 10, - Content: customContent, + Content: jsonContent, + Headers: map[string]string{ + "Content-Type": "application/json", + "Retry-After": "10", + }, }, }, } - req := httptest.NewRequest("GET", "http://example.com", nil) + req := httptest.NewRequest("GET", "http://api.example.com/users", nil) w := httptest.NewRecorder() err := handler.ServePlaceholder(w, req, hso) assert.NoError(t, err) - assert.Equal(t, http.StatusServiceUnavailable, w.Code) - - // Check that custom content is there - assert.Contains(t, w.Body.String(), "Custom placeholder for test-service") + assert.Equal(t, http.StatusAccepted, w.Code) + assert.Equal(t, "application/json", w.Header().Get("Content-Type")) + assert.Equal(t, "10", w.Header().Get("Retry-After")) - // Check that script was injected - assert.Contains(t, w.Body.String(), "checkServiceStatus") - assert.Contains(t, w.Body.String(), "X-KEDA-HTTP-Placeholder-Served") - assert.Contains(t, w.Body.String(), "checkInterval = 10 * 1000") + body := w.Body.String() + assert.Contains(t, body, `"service": "api-backend"`) + assert.Contains(t, body, `"namespace": "production"`) + assert.Contains(t, body, `"retry_after_seconds": 10`) + assert.Contains(t, body, `"status": "warming_up"`) } -func TestServePlaceholder_ConfigMapContentWithScriptInjection_REMOVED(t *testing.T) { - t.Skip("ConfigMap support removed per maintainer feedback") - return - // The code below is kept for reference but won't be executed - /* - configMapContent := `

ConfigMap placeholder for {{.ServiceName}}

` - cm := &v1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{ - Name: "placeholder-cm", - Namespace: "default", - }, - Data: map[string]string{ - "template.html": configMapContent, - }, - } - k8sClient := fake.NewSimpleClientset(cm) - routingTable := test.NewTable() - handler, _ := NewPlaceholderHandler(k8sClient, routingTable) +func TestServePlaceholder_XMLResponse(t *testing.T) { + servingCfg := &config.Serving{} + handler, _ := NewPlaceholderHandler(servingCfg) - hso := &v1alpha1.HTTPScaledObject{ - ObjectMeta: metav1.ObjectMeta{ - Name: "test-app", - Namespace: "default", + xmlContent := ` + + unavailable + {{.ServiceName}} + {{.Namespace}} + Service is scaling up + {{.RefreshInterval}} +` + + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: "legacy-service", + Namespace: "default", + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "legacy-backend", }, - Spec: v1alpha1.HTTPScaledObjectSpec{ - ScaleTargetRef: v1alpha1.ScaleTargetRef{ - Service: "test-service", - }, - PlaceholderConfig: &v1alpha1.PlaceholderConfig{ - Enabled: true, - StatusCode: 503, - RefreshInterval: 15, - ContentConfigMap: "placeholder-cm", + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + StatusCode: 503, + RefreshInterval: 5, + Content: xmlContent, + Headers: map[string]string{ + "Content-Type": "application/xml; charset=utf-8", }, }, - } - - req := httptest.NewRequest("GET", "http://example.com", nil) - w := httptest.NewRecorder() + }, + } - err := handler.ServePlaceholder(w, req, hso) - assert.NoError(t, err) - assert.Equal(t, http.StatusServiceUnavailable, w.Code) + req := httptest.NewRequest("GET", "http://legacy.example.com", nil) + w := httptest.NewRecorder() - // Check that custom content is there - assert.Contains(t, w.Body.String(), "ConfigMap placeholder for test-service") + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Equal(t, "application/xml; charset=utf-8", w.Header().Get("Content-Type")) - // Check that script was injected - assert.Contains(t, w.Body.String(), "checkServiceStatus") - assert.Contains(t, w.Body.String(), "X-KEDA-HTTP-Placeholder-Served") - assert.Contains(t, w.Body.String(), "checkInterval = 15 * 1000") - */ + body := w.Body.String() + // Note: html/template escapes XML, so < becomes < + assert.Contains(t, body, "legacy-backend") + assert.Contains(t, body, "default") + assert.Contains(t, body, "5") + assert.Contains(t, body, "xml version") } -func TestServePlaceholder_NoBodyTagScriptInjection(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } +func TestServePlaceholder_PlainTextResponse(t *testing.T) { + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) - // Custom content without body tag - customContent := `
Simple placeholder for {{.ServiceName}}
` + textContent := `{{.ServiceName}} is currently unavailable. + +The service is scaling up to handle your request. +Please retry in {{.RefreshInterval}} seconds. + +Namespace: {{.Namespace}} +Request ID: {{.RequestID}}` hso := &v1alpha1.HTTPScaledObject{ ObjectMeta: metav1.ObjectMeta{ - Name: "test-app", - Namespace: "default", + Name: "simple-service", + Namespace: "apps", }, Spec: v1alpha1.HTTPScaledObjectSpec{ ScaleTargetRef: v1alpha1.ScaleTargetRef{ - Service: "test-service", + Service: "simple-backend", }, PlaceholderConfig: &v1alpha1.PlaceholderConfig{ Enabled: true, StatusCode: 503, - RefreshInterval: 5, - Content: customContent, + RefreshInterval: 3, + Content: textContent, + Headers: map[string]string{ + "Content-Type": "text/plain; charset=utf-8", + }, }, }, } - req := httptest.NewRequest("GET", "http://example.com", nil) + req := httptest.NewRequest("GET", "http://simple.example.com", nil) + req.Header.Set("X-Request-ID", "abc-123-xyz") w := httptest.NewRecorder() err := handler.ServePlaceholder(w, req, hso) assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Equal(t, "text/plain; charset=utf-8", w.Header().Get("Content-Type")) body := w.Body.String() - // Check that custom content is there - assert.Contains(t, body, "Simple placeholder for test-service") - - // Check that script was appended at the end - assert.True(t, strings.HasSuffix(body, "")) - assert.Contains(t, body, "checkServiceStatus") + assert.Contains(t, body, "simple-backend is currently unavailable") + assert.Contains(t, body, "Please retry in 3 seconds") + assert.Contains(t, body, "Namespace: apps") + assert.Contains(t, body, "Request ID: abc-123-xyz") } -func TestServePlaceholder_NonHTMLContentWrapping(t *testing.T) { - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } +func TestServePlaceholder_HTMLWithUserControlledRefresh(t *testing.T) { + servingCfg := &config.Serving{} handler, _ := NewPlaceholderHandler(servingCfg) - // Non-HTML content that should get wrapped - plainTextContent := `Welcome! Your service is starting up. -Please wait a moment...` + htmlContent := ` + + + + + {{.ServiceName}} - Starting Up + + +

{{.ServiceName}} is starting...

+

The service will be ready soon. This page will refresh automatically.

+

Namespace: {{.Namespace}}

+ +` hso := &v1alpha1.HTTPScaledObject{ ObjectMeta: metav1.ObjectMeta{ - Name: "test-app", - Namespace: "default", + Name: "web-app", + Namespace: "frontend", }, Spec: v1alpha1.HTTPScaledObjectSpec{ ScaleTargetRef: v1alpha1.ScaleTargetRef{ - Service: "test-service", + Service: "web-frontend", }, PlaceholderConfig: &v1alpha1.PlaceholderConfig{ Enabled: true, StatusCode: 503, RefreshInterval: 5, - Content: plainTextContent, + Content: htmlContent, + Headers: map[string]string{ + "Content-Type": "text/html; charset=utf-8", + }, }, }, } - req := httptest.NewRequest("GET", "http://example.com", nil) + req := httptest.NewRequest("GET", "http://webapp.example.com", nil) w := httptest.NewRecorder() err := handler.ServePlaceholder(w, req, hso) assert.NoError(t, err) + assert.Equal(t, http.StatusServiceUnavailable, w.Code) + assert.Equal(t, "text/html; charset=utf-8", w.Header().Get("Content-Type")) body := w.Body.String() + assert.Contains(t, body, "") + assert.Contains(t, body, ``) + assert.Contains(t, body, "

web-frontend is starting...

") + assert.Contains(t, body, "Namespace: frontend") + // Verify no automatic script injection + assert.NotContains(t, body, "checkServiceStatus") +} - // Check that content was NOT wrapped in HTML (new behavior) - assert.NotContains(t, body, "") - assert.NotContains(t, body, "") - assert.NotContains(t, body, "") +func TestServePlaceholder_ContentTypeUserControl(t *testing.T) { + servingCfg := &config.Serving{} + handler, _ := NewPlaceholderHandler(servingCfg) - // Check that original content is preserved as-is - assert.Contains(t, body, "Welcome! Your service is starting up.") - assert.Contains(t, body, "Please wait a moment...") + // Test that user-provided Content-Type is respected + // Note: Currently using html/template which auto-escapes, so XML/HTML chars become entities + testCases := []struct { + name string + content string + contentType string + expectedContent string + }{ + { + name: "application/json", + content: `{"service": "{{.ServiceName}}"}`, + contentType: "application/json", + expectedContent: `test`, + }, + { + name: "text/plain", + content: `Service: {{.ServiceName}}`, + contentType: "text/plain", + expectedContent: `Service: test`, + }, + } - // Check that script was NOT injected into plain text - assert.NotContains(t, body, "checkServiceStatus") + for i, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + hso := &v1alpha1.HTTPScaledObject{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("test-app-%d", i), + Namespace: "default", + Generation: int64(i + 1), + }, + Spec: v1alpha1.HTTPScaledObjectSpec{ + ScaleTargetRef: v1alpha1.ScaleTargetRef{ + Service: "test", + }, + PlaceholderConfig: &v1alpha1.PlaceholderConfig{ + Enabled: true, + Content: tc.content, + Headers: map[string]string{ + "Content-Type": tc.contentType, + }, + }, + }, + } - // Check content type is plain text - assert.Equal(t, "text/plain; charset=utf-8", w.Header().Get("Content-Type")) + req := httptest.NewRequest("GET", "http://example.com", nil) + w := httptest.NewRecorder() + + err := handler.ServePlaceholder(w, req, hso) + assert.NoError(t, err) + assert.Equal(t, tc.contentType, w.Header().Get("Content-Type")) + assert.Contains(t, w.Body.String(), tc.expectedContent) + }) + } } diff --git a/interceptor/main_test.go b/interceptor/main_test.go index c74f47a90..95646ac66 100644 --- a/interceptor/main_test.go +++ b/interceptor/main_test.go @@ -81,9 +81,7 @@ func TestRunProxyServerCountMiddleware(t *testing.T) { fmt.Println(err, "Error setting up tracer") } - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} g.Go(func() error { return runProxyServer( ctx, @@ -226,9 +224,7 @@ func TestRunProxyServerWithTLSCountMiddleware(t *testing.T) { return false, nil } tracingCfg := config.Tracing{Enabled: true, Exporter: "otlphttp"} - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} g.Go(func() error { return runProxyServer( @@ -382,9 +378,7 @@ func TestRunProxyServerWithMultipleCertsTLSCountMiddleware(t *testing.T) { } tracingCfg := config.Tracing{Enabled: true, Exporter: "otlphttp"} - servingCfg := &config.Serving{ - PlaceholderEnableScript: true, - } + servingCfg := &config.Serving{} g.Go(func() error { return runProxyServer( diff --git a/operator/apis/http/v1alpha1/httpscaledobject_types.go b/operator/apis/http/v1alpha1/httpscaledobject_types.go index 62167a940..f6dbfe340 100644 --- a/operator/apis/http/v1alpha1/httpscaledobject_types.go +++ b/operator/apis/http/v1alpha1/httpscaledobject_types.go @@ -132,28 +132,32 @@ type HTTPScaledObjectTimeoutsSpec struct { ResponseHeader metav1.Duration `json:"responseHeader" description:"How long to wait between when the HTTP request is sent to the backing app and when response headers need to arrive"` } -// PlaceholderConfig defines the configuration for serving placeholder pages during scale-from-zero +// PlaceholderConfig defines the configuration for serving placeholder responses during scale-from-zero type PlaceholderConfig struct { - // Enable placeholder page when replicas are scaled to zero + // Enable placeholder response when replicas are scaled to zero // +kubebuilder:default=false // +optional - Enabled bool `json:"enabled" description:"Enable placeholder page when replicas are scaled to zero"` - // Inline content for placeholder page (can be HTML, JSON, plain text, etc.) + Enabled bool `json:"enabled" description:"Enable placeholder response when replicas are scaled to zero"` + // Inline content for placeholder response. Supports any format (HTML, JSON, XML, plain text, etc.) + // Content is processed as a Go template with variables: ServiceName, Namespace, RefreshInterval, RequestID, Timestamp + // Content-Type should be set via the Headers field to match your content format // +optional - Content string `json:"content,omitempty" description:"Inline content for placeholder page"` - // HTTP status code to return with placeholder page + Content string `json:"content,omitempty" description:"Inline content for placeholder response (any format supported)"` + // HTTP status code to return with placeholder response // +kubebuilder:default=503 // +kubebuilder:validation:Minimum=100 // +kubebuilder:validation:Maximum=599 // +optional - StatusCode int32 `json:"statusCode,omitempty" description:"HTTP status code to return with placeholder page (Default 503)"` - // Refresh interval for client-side polling in seconds + StatusCode int32 `json:"statusCode,omitempty" description:"HTTP status code to return with placeholder response (Default 503)"` + // RefreshInterval is a template variable available in content (in seconds) + // This is just data passed to the template, not used by the interceptor for automatic refresh // +kubebuilder:default=5 // +kubebuilder:validation:Minimum=1 // +kubebuilder:validation:Maximum=60 // +optional - RefreshInterval int32 `json:"refreshInterval,omitempty" description:"Refresh interval for client-side polling in seconds (Default 5)"` + RefreshInterval int32 `json:"refreshInterval,omitempty" description:"Template variable for refresh interval in seconds (Default 5)"` // Additional HTTP headers to include with placeholder response + // Use this to set Content-Type matching your content format (e.g., 'Content-Type: application/json') // +optional Headers map[string]string `json:"headers,omitempty" description:"Additional HTTP headers to include with placeholder response"` } diff --git a/tests/checks/placeholder_pages/placeholder_pages_test.go b/tests/checks/placeholder_pages/placeholder_pages_test.go index 76404fb2e..7dd12caae 100644 --- a/tests/checks/placeholder_pages/placeholder_pages_test.go +++ b/tests/checks/placeholder_pages/placeholder_pages_test.go @@ -79,11 +79,13 @@ spec: statusCode: 503 refreshInterval: 5 headers: + Content-Type: "text/plain; charset=utf-8" X-Test-Header: "test-value" + content: "{{ "{{" }} .ServiceName {{ "}}" }} is starting up..." ` func TestPlaceholderPages(t *testing.T) { - // setup + // Test content-agnostic placeholder responses (HTML, JSON, plain text) t.Log("--- setting up ---") // Create kubernetes resources kc := GetKubernetesClient(t) @@ -116,25 +118,31 @@ spec: defer KubectlDeleteWithTemplate(t, data, "curl-client", clientPod) // Wait for curl pod to be ready - assert.True(t, WaitForPodCountInNamespace(t, kc, testNamespace, 1, 30, 2), - "curl client pod should be ready") - - // Give container time to fully initialize - _, _ = ExecuteCommand("sleep 2") + assert.True(t, WaitForPodCountInNamespace(t, kc, testNamespace, 1, 60, 2), + "curl client pod should exist") + assert.True(t, WaitForAllPodRunningInNamespace(t, kc, testNamespace, 60, 2), + "curl client pod should be running") // Test placeholder response testPlaceholderResponse(t, kc) - // Test custom placeholder with script injection - testCustomPlaceholderWithScript(t, kc, data) + // Test HTML placeholder with user-controlled content + testHTMLPlaceholder(t, kc, data) + + // Test JSON placeholder for API communication + testJSONPlaceholder(t, kc, data) + + // Test plain text placeholder + testPlainTextPlaceholder(t, kc, data) } func testPlaceholderResponse(t *testing.T, kc *kubernetes.Clientset) { - t.Log("--- testing placeholder response ---") + t.Log("--- testing default placeholder response ---") // Make request through interceptor curlCmd := fmt.Sprintf("curl -si -H 'Host: %s.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) + stdout = RemoveANSI(stdout) t.Logf("curl output: %s", stdout) if stderr != "" { t.Logf("curl stderr: %s", stderr) @@ -147,22 +155,25 @@ func testPlaceholderResponse(t *testing.T, kc *kubernetes.Clientset) { assert.Contains(t, stdout, "X-Keda-Http-Placeholder-Served", "should have placeholder header") assert.Contains(t, stdout, "X-Test-Header", "should have custom header") assert.Contains(t, stdout, "test-value", "should have custom header value") + assert.Contains(t, stdout, "Content-Type: text/plain", "should have correct Content-Type") assert.Contains(t, stdout, "is starting up", "should have placeholder message") + + // Verify NO automatic script injection + assert.NotContains(t, stdout, "checkServiceStatus", "should NOT have automatic script injection") } -func testCustomPlaceholderWithScript(t *testing.T, kc *kubernetes.Clientset, data templateData) { - t.Log("--- testing custom placeholder with script injection ---") +func testHTMLPlaceholder(t *testing.T, kc *kubernetes.Clientset, data templateData) { + t.Log("--- testing HTML placeholder with user-controlled content ---") - // Create a custom HTTPScaledObject with inline content - customTemplate := ` + htmlTemplate := ` apiVersion: http.keda.sh/v1alpha1 kind: HTTPScaledObject metadata: - name: {{.TestName}}-custom + name: {{.TestName}}-html namespace: {{.TestNamespace}} spec: hosts: - - {{.TestName}}-custom.test + - {{.TestName}}-html.test scaleTargetRef: name: {{.TestName}} service: {{.TestName}} @@ -174,35 +185,169 @@ spec: placeholderConfig: enabled: true statusCode: 503 - refreshInterval: 7 + refreshInterval: 5 + headers: + Content-Type: "text/html; charset=utf-8" content: | + - -

Service is starting - custom page

-

This is a test placeholder page.

- + + Service Starting + + + +

{{ "{{" }} .ServiceName {{ "}}" }} is starting - custom HTML

+

This is a user-controlled HTML placeholder.

+ ` - KubectlApplyWithTemplate(t, data, "custom-placeholder", customTemplate) - defer KubectlDeleteWithTemplate(t, data, "custom-placeholder", customTemplate) + KubectlApplyWithTemplate(t, data, "html-placeholder", htmlTemplate) + defer KubectlDeleteWithTemplate(t, data, "html-placeholder", htmlTemplate) - // Make request to custom placeholder - curlCmd := fmt.Sprintf("curl -s -H 'Host: %s-custom.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) + // Make request to HTML placeholder + curlCmd := fmt.Sprintf("curl -si -H 'Host: %s-html.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) - t.Logf("custom placeholder output: %s", stdout) + stdout = RemoveANSI(stdout) + t.Logf("HTML placeholder output: %s", stdout) if stderr != "" { - t.Logf("custom placeholder stderr: %s", stderr) + t.Logf("HTML placeholder stderr: %s", stderr) } assert.NoError(t, err, "curl command should succeed") - // Verify custom content is there - assert.Contains(t, stdout, "Service is starting - custom page", "should have custom placeholder content") - assert.Contains(t, stdout, "This is a test placeholder page.", "should have custom message") + // Verify custom HTML content + assert.Contains(t, stdout, "HTTP/1.1 503", "should return 503 status") + assert.Contains(t, stdout, "Content-Type: text/html", "should have HTML Content-Type") + assert.Contains(t, stdout, "", "should have HTML doctype") + assert.Contains(t, stdout, "is starting - custom HTML", "should have custom HTML content") + assert.Contains(t, stdout, "user-controlled HTML placeholder", "should have custom message") + assert.Contains(t, stdout, ``, "should have user-controlled meta refresh") + + // Verify NO automatic script injection + assert.NotContains(t, stdout, "checkServiceStatus", "should NOT have automatic script injection") +} + +func testJSONPlaceholder(t *testing.T, kc *kubernetes.Clientset, data templateData) { + t.Log("--- testing JSON placeholder for API communication ---") + + jsonTemplate := ` +apiVersion: http.keda.sh/v1alpha1 +kind: HTTPScaledObject +metadata: + name: {{.TestName}}-json + namespace: {{.TestNamespace}} +spec: + hosts: + - {{.TestName}}-json.test + scaleTargetRef: + name: {{.TestName}} + service: {{.TestName}} + port: 80 + replicas: + min: 0 + max: 10 + scaledownPeriod: 10 + placeholderConfig: + enabled: true + statusCode: 202 + refreshInterval: 10 + headers: + Content-Type: "application/json" + Retry-After: "10" + content: | + { + "status": "warming_up", + "service": "{{ "{{" }} .ServiceName {{ "}}" }}", + "namespace": "{{ "{{" }} .Namespace {{ "}}" }}", + "retry_after_seconds": {{ "{{" }} .RefreshInterval {{ "}}" }} + } +` + + KubectlApplyWithTemplate(t, data, "json-placeholder", jsonTemplate) + defer KubectlDeleteWithTemplate(t, data, "json-placeholder", jsonTemplate) + + // Make request to JSON placeholder + curlCmd := fmt.Sprintf("curl -si -H 'Host: %s-json.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) + stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) + stdout = RemoveANSI(stdout) + t.Logf("JSON placeholder output: %s", stdout) + if stderr != "" { + t.Logf("JSON placeholder stderr: %s", stderr) + } + + assert.NoError(t, err, "curl command should succeed") + + // Verify JSON response + assert.Contains(t, stdout, "HTTP/1.1 202", "should return 202 Accepted status") + assert.Contains(t, stdout, "Content-Type: application/json", "should have JSON Content-Type") + assert.Contains(t, stdout, "Retry-After: 10", "should have Retry-After header") + assert.Contains(t, stdout, `"status": "warming_up"`, "should have status field") + assert.Contains(t, stdout, `"service":`, "should have service field") + assert.Contains(t, stdout, `"retry_after_seconds": 10`, "should have retry_after_seconds field") + + // Verify it's valid JSON structure (contains braces) + assert.Contains(t, stdout, "{", "should have JSON opening brace") + assert.Contains(t, stdout, "}", "should have JSON closing brace") +} + +func testPlainTextPlaceholder(t *testing.T, kc *kubernetes.Clientset, data templateData) { + t.Log("--- testing plain text placeholder ---") + + textTemplate := ` +apiVersion: http.keda.sh/v1alpha1 +kind: HTTPScaledObject +metadata: + name: {{.TestName}}-text + namespace: {{.TestNamespace}} +spec: + hosts: + - {{.TestName}}-text.test + scaleTargetRef: + name: {{.TestName}} + service: {{.TestName}} + port: 80 + replicas: + min: 0 + max: 10 + scaledownPeriod: 10 + placeholderConfig: + enabled: true + statusCode: 503 + refreshInterval: 3 + headers: + Content-Type: "text/plain; charset=utf-8" + content: | + {{ "{{" }} .ServiceName {{ "}}" }} is currently unavailable. + + The service is scaling up to handle your request. + Please retry in {{ "{{" }} .RefreshInterval {{ "}}" }} seconds. + + Namespace: {{ "{{" }} .Namespace {{ "}}" }} +` + + KubectlApplyWithTemplate(t, data, "text-placeholder", textTemplate) + defer KubectlDeleteWithTemplate(t, data, "text-placeholder", textTemplate) + + // Make request to plain text placeholder + curlCmd := fmt.Sprintf("curl -si -H 'Host: %s-text.test' http://keda-add-ons-http-interceptor-proxy.keda:8080/", testName) + stdout, stderr, err := ExecCommandOnSpecificPod(t, "curl-client", testNamespace, curlCmd) + stdout = RemoveANSI(stdout) + t.Logf("Plain text placeholder output: %s", stdout) + if stderr != "" { + t.Logf("Plain text placeholder stderr: %s", stderr) + } + + assert.NoError(t, err, "curl command should succeed") + + // Verify plain text response + assert.Contains(t, stdout, "HTTP/1.1 503", "should return 503 status") + assert.Contains(t, stdout, "Content-Type: text/plain", "should have plain text Content-Type") + assert.Contains(t, stdout, "is currently unavailable", "should have unavailable message") + assert.Contains(t, stdout, "Please retry in 3 seconds", "should have retry message with interval") + assert.Contains(t, stdout, "Namespace:", "should have namespace in output") - // Verify script was injected - assert.Contains(t, stdout, "checkServiceStatus", "should have injected checkServiceStatus function") - assert.Contains(t, stdout, "X-KEDA-HTTP-Placeholder-Served", "should have header check in script") - assert.Contains(t, stdout, "checkInterval = 7 * 1000", "should have correct refresh interval in script") + // Verify it's plain text (no HTML tags) + assert.NotContains(t, stdout, "", "should NOT have HTML tags") + assert.NotContains(t, stdout, "", "should NOT have body tag") } diff --git a/tests/helper/helper.go b/tests/helper/helper.go index 27f314ac4..8fdec5f33 100644 --- a/tests/helper/helper.go +++ b/tests/helper/helper.go @@ -126,7 +126,7 @@ func ExecCommandOnSpecificPod(t *testing.T, podName string, namespace string, co Stdin: false, Stdout: true, Stderr: true, - TTY: true, + TTY: false, }, scheme.ParameterCodec) exec, err := remotecommand.NewSPDYExecutor(KubeConfig, "POST", request.URL()) assert.NoErrorf(t, err, "cannot execute command - %s", err) From 06322d27025562e5ed2fc559990749eead63b8f3 Mon Sep 17 00:00:00 2001 From: malpou Date: Sat, 4 Oct 2025 10:44:16 +0200 Subject: [PATCH 22/29] fix: add defensive nil check for placeholderHandler Reorder condition checks to verify placeholder configuration before checking handler to prevent potential panics if placeholder is enabled in spec but handler is nil. Signed-off-by: malpou --- interceptor/proxy_handlers.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/interceptor/proxy_handlers.go b/interceptor/proxy_handlers.go index 8c8a8623b..1867929bb 100644 --- a/interceptor/proxy_handlers.go +++ b/interceptor/proxy_handlers.go @@ -63,7 +63,8 @@ func newForwardingHandler( hasFailover := httpso.Spec.ColdStartTimeoutFailoverRef != nil // Check if we should serve a placeholder page - if placeholderHandler != nil && httpso.Spec.PlaceholderConfig != nil && httpso.Spec.PlaceholderConfig.Enabled { + // Ensure placeholderHandler is not nil to prevent panics even if placeholder is enabled in spec + if httpso.Spec.PlaceholderConfig != nil && httpso.Spec.PlaceholderConfig.Enabled && placeholderHandler != nil { endpoints, err := endpointsCache.Get(httpso.GetNamespace(), httpso.Spec.ScaleTargetRef.Service) if err != nil { // Error getting endpoints cache - return 503 Service Unavailable From d97f58433cebfca06f494295e727bff1e0c84fdd Mon Sep 17 00:00:00 2001 From: malpou Date: Sat, 4 Oct 2025 11:40:14 +0200 Subject: [PATCH 23/29] refactor: improve placeholder handler code quality Improve maintainability and readability of placeholder pages feature: - Add constants for headers and messages to eliminate magic strings - Remove self-explanatory comments that duplicate code intent - Extract helper methods to reduce duplication and improve separation of concerns - Rename getTemplate to resolveTemplate for clarity - Simplify template caching with cleaner double-check locking pattern - Extract placeholder serving logic into focused helper functions No functional changes - pure refactoring for long-term maintainability. Signed-off-by: malpou --- interceptor/handler/placeholder.go | 109 +++++++++--------- interceptor/handler/placeholder_test.go | 21 ++-- interceptor/proxy_handlers.go | 73 +++++++----- .../http/v1alpha1/httpscaledobject_types.go | 12 +- 4 files changed, 108 insertions(+), 107 deletions(-) diff --git a/interceptor/handler/placeholder.go b/interceptor/handler/placeholder.go index f5416e739..33f8c74e9 100644 --- a/interceptor/handler/placeholder.go +++ b/interceptor/handler/placeholder.go @@ -14,15 +14,20 @@ import ( "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" ) +const ( + headerPlaceholderServed = "X-KEDA-HTTP-Placeholder-Served" + headerCacheControl = "Cache-Control" + headerContentType = "Content-Type" + cacheControlValue = "no-cache, no-store, must-revalidate" + fallbackContentType = "text/plain; charset=utf-8" + fallbackMessageFormat = "%s is starting up...\n" +) -// cacheEntry stores a template along with resource generation info for cache invalidation type cacheEntry struct { - template *template.Template - hsoGeneration int64 - configMapVersion string + template *template.Template + hsoGeneration int64 } -// PlaceholderHandler handles serving placeholder pages during scale-from-zero type PlaceholderHandler struct { templateCache map[string]*cacheEntry cacheMutex sync.RWMutex @@ -30,7 +35,6 @@ type PlaceholderHandler struct { servingCfg *config.Serving } -// PlaceholderData contains data for rendering placeholder templates type PlaceholderData struct { ServiceName string Namespace string @@ -39,11 +43,9 @@ type PlaceholderData struct { Timestamp string } -// NewPlaceholderHandler creates a new placeholder handler func NewPlaceholderHandler(servingCfg *config.Serving) (*PlaceholderHandler, error) { var defaultTmpl *template.Template - // Try to load template from configured path if provided if servingCfg.PlaceholderDefaultTemplatePath != "" { content, err := os.ReadFile(servingCfg.PlaceholderDefaultTemplatePath) if err != nil { @@ -66,9 +68,6 @@ func NewPlaceholderHandler(servingCfg *config.Serving) (*PlaceholderHandler, err }, nil } - - -// ServePlaceholder serves a placeholder page based on the HTTPScaledObject configuration func (h *PlaceholderHandler) ServePlaceholder(w http.ResponseWriter, r *http.Request, hso *v1alpha1.HTTPScaledObject) error { if hso.Spec.PlaceholderConfig == nil || !hso.Spec.PlaceholderConfig.Enabled { http.Error(w, "Service temporarily unavailable", http.StatusServiceUnavailable) @@ -82,20 +81,13 @@ func (h *PlaceholderHandler) ServePlaceholder(w http.ResponseWriter, r *http.Req statusCode = http.StatusServiceUnavailable } - // Set custom headers first for k, v := range config.Headers { w.Header().Set(k, v) } - // Get template and render content - tmpl, err := h.getTemplate(r.Context(), hso) + tmpl, err := h.resolveTemplate(r.Context(), hso) if err != nil { - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - w.Header().Set("X-KEDA-HTTP-Placeholder-Served", "true") - w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") - w.WriteHeader(statusCode) - fmt.Fprintf(w, "%s is starting up...\n", hso.Spec.ScaleTargetRef.Service) - return nil + return h.serveFallbackPlaceholder(w, hso.Spec.ScaleTargetRef.Service, statusCode) } data := PlaceholderData{ @@ -108,52 +100,30 @@ func (h *PlaceholderHandler) ServePlaceholder(w http.ResponseWriter, r *http.Req var buf bytes.Buffer if err := tmpl.Execute(&buf, data); err != nil { - w.Header().Set("Content-Type", "text/plain; charset=utf-8") - w.Header().Set("X-KEDA-HTTP-Placeholder-Served", "true") - w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") - w.WriteHeader(statusCode) - fmt.Fprintf(w, "%s is starting up...\n", hso.Spec.ScaleTargetRef.Service) - return nil + return h.serveFallbackPlaceholder(w, hso.Spec.ScaleTargetRef.Service, statusCode) } - content := buf.String() - - // Set standard headers - w.Header().Set("X-KEDA-HTTP-Placeholder-Served", "true") - w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate") + w.Header().Set(headerPlaceholderServed, "true") + w.Header().Set(headerCacheControl, cacheControlValue) + w.WriteHeader(statusCode) + _, err = w.Write(buf.Bytes()) + return err +} +func (h *PlaceholderHandler) serveFallbackPlaceholder(w http.ResponseWriter, serviceName string, statusCode int) error { + w.Header().Set(headerContentType, fallbackContentType) + w.Header().Set(headerPlaceholderServed, "true") + w.Header().Set(headerCacheControl, cacheControlValue) w.WriteHeader(statusCode) - _, err = w.Write([]byte(content)) + _, err := fmt.Fprintf(w, fallbackMessageFormat, serviceName) return err } -// getTemplate retrieves the template for the given HTTPScaledObject -func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTPScaledObject) (*template.Template, error) { +func (h *PlaceholderHandler) resolveTemplate(_ context.Context, hso *v1alpha1.HTTPScaledObject) (*template.Template, error) { config := hso.Spec.PlaceholderConfig if config.Content != "" { - cacheKey := fmt.Sprintf("%s/%s/inline", hso.Namespace, hso.Name) - - h.cacheMutex.RLock() - entry, ok := h.templateCache[cacheKey] - if ok && entry.hsoGeneration == hso.Generation { - h.cacheMutex.RUnlock() - return entry.template, nil - } - h.cacheMutex.RUnlock() - - h.cacheMutex.Lock() - tmpl, err := template.New("inline").Parse(config.Content) - if err != nil { - h.cacheMutex.Unlock() - return nil, err - } - h.templateCache[cacheKey] = &cacheEntry{ - template: tmpl, - hsoGeneration: hso.Generation, - } - h.cacheMutex.Unlock() - return tmpl, nil + return h.getCachedInlineTemplate(hso, config.Content) } if h.defaultTmpl != nil { @@ -162,3 +132,30 @@ func (h *PlaceholderHandler) getTemplate(ctx context.Context, hso *v1alpha1.HTTP return nil, fmt.Errorf("no placeholder template configured") } + +func (h *PlaceholderHandler) getCachedInlineTemplate(hso *v1alpha1.HTTPScaledObject, content string) (*template.Template, error) { + cacheKey := fmt.Sprintf("%s/%s/inline", hso.Namespace, hso.Name) + + h.cacheMutex.RLock() + entry, ok := h.templateCache[cacheKey] + if ok && entry.hsoGeneration == hso.Generation { + h.cacheMutex.RUnlock() + return entry.template, nil + } + h.cacheMutex.RUnlock() + + h.cacheMutex.Lock() + defer h.cacheMutex.Unlock() + + tmpl, err := template.New("inline").Parse(content) + if err != nil { + return nil, err + } + + h.templateCache[cacheKey] = &cacheEntry{ + template: tmpl, + hsoGeneration: hso.Generation, + } + + return tmpl, nil +} diff --git a/interceptor/handler/placeholder_test.go b/interceptor/handler/placeholder_test.go index 6bfde7957..3451a6a59 100644 --- a/interceptor/handler/placeholder_test.go +++ b/interceptor/handler/placeholder_test.go @@ -250,11 +250,11 @@ func TestGetTemplate_Caching(t *testing.T) { ctx := context.Background() - tmpl1, err := handler.getTemplate(ctx, hso) + tmpl1, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl1) - tmpl2, err := handler.getTemplate(ctx, hso) + tmpl2, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl2) @@ -283,14 +283,14 @@ func TestGetTemplate_CacheInvalidation_Generation(t *testing.T) { ctx := context.Background() - tmpl1, err := handler.getTemplate(ctx, hso) + tmpl1, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl1) hso.Generation = 2 hso.Spec.PlaceholderConfig.Content = customContent2 - tmpl2, err := handler.getTemplate(ctx, hso) + tmpl2, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl2) @@ -299,7 +299,6 @@ func TestGetTemplate_CacheInvalidation_Generation(t *testing.T) { func TestGetTemplate_CacheInvalidation_ConfigMapVersion_REMOVED(t *testing.T) { t.Skip("ConfigMap support removed per maintainer feedback") - return // The code below is kept for reference but won't be executed /* cm := &v1.ConfigMap{ @@ -331,7 +330,7 @@ func TestGetTemplate_CacheInvalidation_ConfigMapVersion_REMOVED(t *testing.T) { ctx := context.Background() - tmpl1, err := handler.getTemplate(ctx, hso) + tmpl1, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl1) @@ -340,7 +339,7 @@ func TestGetTemplate_CacheInvalidation_ConfigMapVersion_REMOVED(t *testing.T) { _, err = k8sClient.CoreV1().ConfigMaps("default").Update(ctx, cm, metav1.UpdateOptions{}) require.NoError(t, err) - tmpl2, err := handler.getTemplate(ctx, hso) + tmpl2, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) assert.NotNil(t, tmpl2) */ @@ -365,7 +364,7 @@ func TestGetTemplate_ConcurrentAccess(t *testing.T) { ctx := context.Background() - _, err := handler.getTemplate(ctx, hso) + _, err := handler.resolveTemplate(ctx, hso) require.NoError(t, err) var wg sync.WaitGroup @@ -376,7 +375,7 @@ func TestGetTemplate_ConcurrentAccess(t *testing.T) { wg.Add(1) go func() { defer wg.Done() - tmpl, err := handler.getTemplate(ctx, hso) + tmpl, err := handler.resolveTemplate(ctx, hso) if err != nil { errors <- err } else { @@ -436,7 +435,7 @@ func TestGetTemplate_ConcurrentFirstAccess(t *testing.T) { wg.Add(1) go func() { defer wg.Done() - _, err := handler.getTemplate(ctx, hso) + _, err := handler.resolveTemplate(ctx, hso) if err != nil { errors <- err } @@ -488,7 +487,7 @@ func TestGetTemplate_ConcurrentCacheUpdates(t *testing.T) { }, } - _, err := handler.getTemplate(ctx, hso) + _, err := handler.resolveTemplate(ctx, hso) if err != nil { errors <- err } diff --git a/interceptor/proxy_handlers.go b/interceptor/proxy_handlers.go index 1867929bb..6d4a2bca3 100644 --- a/interceptor/proxy_handlers.go +++ b/interceptor/proxy_handlers.go @@ -13,6 +13,7 @@ import ( "github.com/kedacore/http-add-on/interceptor/config" "github.com/kedacore/http-add-on/interceptor/handler" + "github.com/kedacore/http-add-on/operator/apis/http/v1alpha1" "github.com/kedacore/http-add-on/pkg/k8s" kedanet "github.com/kedacore/http-add-on/pkg/net" "github.com/kedacore/http-add-on/pkg/util" @@ -40,12 +41,6 @@ func newForwardingConfigFromTimeouts(t *config.Timeouts) forwardingConfig { } } -// newForwardingHandler takes in the service URL for the app backend -// and forwards incoming requests to it. Note that it isn't multitenant. -// It's intended to be deployed and scaled alongside the application itself. -// -// fwdSvcURL must have a valid scheme in it. The best way to do this is -// creating a URL with url.Parse("https://...") func newForwardingHandler( lggr logr.Logger, dialCtxFunc kedanet.DialContextFunc, @@ -62,30 +57,8 @@ func newForwardingHandler( httpso := util.HTTPSOFromContext(ctx) hasFailover := httpso.Spec.ColdStartTimeoutFailoverRef != nil - // Check if we should serve a placeholder page - // Ensure placeholderHandler is not nil to prevent panics even if placeholder is enabled in spec - if httpso.Spec.PlaceholderConfig != nil && httpso.Spec.PlaceholderConfig.Enabled && placeholderHandler != nil { - endpoints, err := endpointsCache.Get(httpso.GetNamespace(), httpso.Spec.ScaleTargetRef.Service) - if err != nil { - // Error getting endpoints cache - return 503 Service Unavailable - lggr.Error(err, "failed to get endpoints from cache while placeholder is configured", - "namespace", httpso.GetNamespace(), - "service", httpso.Spec.ScaleTargetRef.Service) - w.WriteHeader(http.StatusServiceUnavailable) - if _, writeErr := w.Write([]byte("Service temporarily unavailable - unable to check service status")); writeErr != nil { - lggr.Error(writeErr, "could not write error response to client") - } - return - } - - if workloadActiveEndpoints(endpoints) == 0 { - if placeholderErr := placeholderHandler.ServePlaceholder(w, r, httpso); placeholderErr != nil { - lggr.Error(placeholderErr, "failed to serve placeholder page") - w.WriteHeader(http.StatusBadGateway) - if _, err := w.Write([]byte("error serving placeholder page")); err != nil { - lggr.Error(err, "could not write error response to client") - } - } + if shouldServePlaceholder(httpso, placeholderHandler) { + if err := servePlaceholderIfNoEndpoints(lggr, w, r, httpso, placeholderHandler, endpointsCache); err != nil { return } } @@ -145,3 +118,43 @@ func newForwardingHandler( uh.ServeHTTP(w, r) }) } + +func shouldServePlaceholder(httpso *v1alpha1.HTTPScaledObject, placeholderHandler *handler.PlaceholderHandler) bool { + return httpso.Spec.PlaceholderConfig != nil && + httpso.Spec.PlaceholderConfig.Enabled && + placeholderHandler != nil +} + +func servePlaceholderIfNoEndpoints( + lggr logr.Logger, + w http.ResponseWriter, + r *http.Request, + httpso *v1alpha1.HTTPScaledObject, + placeholderHandler *handler.PlaceholderHandler, + endpointsCache k8s.EndpointsCache, +) error { + endpoints, err := endpointsCache.Get(httpso.GetNamespace(), httpso.Spec.ScaleTargetRef.Service) + if err != nil { + lggr.Error(err, "failed to get endpoints from cache while placeholder is configured", + "namespace", httpso.GetNamespace(), + "service", httpso.Spec.ScaleTargetRef.Service) + w.WriteHeader(http.StatusServiceUnavailable) + if _, writeErr := w.Write([]byte("Service temporarily unavailable - unable to check service status")); writeErr != nil { + lggr.Error(writeErr, "could not write error response to client") + } + return err + } + + if workloadActiveEndpoints(endpoints) == 0 { + if placeholderErr := placeholderHandler.ServePlaceholder(w, r, httpso); placeholderErr != nil { + lggr.Error(placeholderErr, "failed to serve placeholder page") + w.WriteHeader(http.StatusBadGateway) + if _, err := w.Write([]byte("error serving placeholder page")); err != nil { + lggr.Error(err, "could not write error response to client") + } + } + return fmt.Errorf("placeholder served") + } + + return nil +} diff --git a/operator/apis/http/v1alpha1/httpscaledobject_types.go b/operator/apis/http/v1alpha1/httpscaledobject_types.go index f6dbfe340..ec5c51748 100644 --- a/operator/apis/http/v1alpha1/httpscaledobject_types.go +++ b/operator/apis/http/v1alpha1/httpscaledobject_types.go @@ -132,32 +132,24 @@ type HTTPScaledObjectTimeoutsSpec struct { ResponseHeader metav1.Duration `json:"responseHeader" description:"How long to wait between when the HTTP request is sent to the backing app and when response headers need to arrive"` } -// PlaceholderConfig defines the configuration for serving placeholder responses during scale-from-zero type PlaceholderConfig struct { - // Enable placeholder response when replicas are scaled to zero // +kubebuilder:default=false // +optional Enabled bool `json:"enabled" description:"Enable placeholder response when replicas are scaled to zero"` - // Inline content for placeholder response. Supports any format (HTML, JSON, XML, plain text, etc.) - // Content is processed as a Go template with variables: ServiceName, Namespace, RefreshInterval, RequestID, Timestamp - // Content-Type should be set via the Headers field to match your content format + // Supports Go template variables: ServiceName, Namespace, RefreshInterval, RequestID, Timestamp // +optional Content string `json:"content,omitempty" description:"Inline content for placeholder response (any format supported)"` - // HTTP status code to return with placeholder response // +kubebuilder:default=503 // +kubebuilder:validation:Minimum=100 // +kubebuilder:validation:Maximum=599 // +optional StatusCode int32 `json:"statusCode,omitempty" description:"HTTP status code to return with placeholder response (Default 503)"` - // RefreshInterval is a template variable available in content (in seconds) - // This is just data passed to the template, not used by the interceptor for automatic refresh + // Template variable only - not used by interceptor for automatic refresh // +kubebuilder:default=5 // +kubebuilder:validation:Minimum=1 // +kubebuilder:validation:Maximum=60 // +optional RefreshInterval int32 `json:"refreshInterval,omitempty" description:"Template variable for refresh interval in seconds (Default 5)"` - // Additional HTTP headers to include with placeholder response - // Use this to set Content-Type matching your content format (e.g., 'Content-Type: application/json') // +optional Headers map[string]string `json:"headers,omitempty" description:"Additional HTTP headers to include with placeholder response"` } From 995060fb3b065f7d1e53e47ea03cb79599db63c9 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:15:42 +0200 Subject: [PATCH 24/29] Update kustomization.yaml Remove left over stuff from testing --- config/interceptor/kustomization.yaml | 30 +-------------------------- 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/config/interceptor/kustomization.yaml b/config/interceptor/kustomization.yaml index 3f783b7d2..7afebff32 100644 --- a/config/interceptor/kustomization.yaml +++ b/config/interceptor/kustomization.yaml @@ -16,32 +16,4 @@ labels: - includeSelectors: true includeTemplates: true pairs: - app.kubernetes.io/instance: interceptor -images: -- name: ghcr.io/kedacore/http-add-on-interceptor - newName: ghcr.io/kedacore/http-add-on-interceptor - newTag: main -patches: -- path: e2e-test/otel/deployment.yaml - target: - group: apps - kind: Deployment - name: interceptor - version: v1 -- path: e2e-test/otel/scaledobject.yaml - target: - group: keda.sh - kind: ScaledObject - name: interceptor - version: v1alpha1 -- path: e2e-test/tls/deployment.yaml - target: - group: apps - kind: Deployment - name: interceptor - version: v1 -- path: e2e-test/tls/proxy.service.yaml - target: - kind: Service - name: interceptor-proxy - version: v1 + app.kubernetes.io/instance: interceptor \ No newline at end of file From e0469b7243a50abc46e0417bdd0d124cf8d40c2d Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:24:15 +0200 Subject: [PATCH 25/29] Update kustomization.yaml Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> --- config/scaler/kustomization.yaml | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/config/scaler/kustomization.yaml b/config/scaler/kustomization.yaml index bdb0bb65a..f8ee5d06e 100644 --- a/config/scaler/kustomization.yaml +++ b/config/scaler/kustomization.yaml @@ -11,14 +11,3 @@ labels: includeTemplates: true pairs: app.kubernetes.io/instance: external-scaler -images: -- name: ghcr.io/kedacore/http-add-on-scaler - newName: ghcr.io/kedacore/http-add-on-scaler - newTag: main -patches: -- path: e2e-test/otel/deployment.yaml - target: - group: apps - kind: Deployment - name: scaler - version: v1 From f3b28a9cb07d16a1a0848d2ae2423ce409bbde79 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:24:38 +0200 Subject: [PATCH 26/29] Update kustomization.yaml Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> --- config/operator/kustomization.yaml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/config/operator/kustomization.yaml b/config/operator/kustomization.yaml index 7f09262a2..08e075759 100644 --- a/config/operator/kustomization.yaml +++ b/config/operator/kustomization.yaml @@ -10,7 +10,3 @@ labels: includeTemplates: true pairs: app.kubernetes.io/instance: operator -images: -- name: ghcr.io/kedacore/http-add-on-operator - newName: ghcr.io/kedacore/http-add-on-operator - newTag: main From 43ffaabd446465a44c05f34542b77b9b4cd92d2f Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:25:38 +0200 Subject: [PATCH 27/29] Update proxy_handlers.go Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> --- interceptor/proxy_handlers.go | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/interceptor/proxy_handlers.go b/interceptor/proxy_handlers.go index 6d4a2bca3..146fccdae 100644 --- a/interceptor/proxy_handlers.go +++ b/interceptor/proxy_handlers.go @@ -41,6 +41,12 @@ func newForwardingConfigFromTimeouts(t *config.Timeouts) forwardingConfig { } } +// newForwardingHandler takes in the service URL for the app backend +// and forwards incoming requests to it. Note that it isn't multitenant. +// It's intended to be deployed and scaled alongside the application itself. +// +// fwdSvcURL must have a valid scheme in it. The best way to do this is +// creating a URL with url.Parse("https://...") func newForwardingHandler( lggr logr.Logger, dialCtxFunc kedanet.DialContextFunc, From f8f26ce7704af459d57b35b0172820c98f052ff8 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:26:31 +0200 Subject: [PATCH 28/29] Update helper.go Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> --- tests/helper/helper.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/helper/helper.go b/tests/helper/helper.go index 8fdec5f33..27f314ac4 100644 --- a/tests/helper/helper.go +++ b/tests/helper/helper.go @@ -126,7 +126,7 @@ func ExecCommandOnSpecificPod(t *testing.T, podName string, namespace string, co Stdin: false, Stdout: true, Stderr: true, - TTY: false, + TTY: true, }, scheme.ParameterCodec) exec, err := remotecommand.NewSPDYExecutor(KubeConfig, "POST", request.URL()) assert.NoErrorf(t, err, "cannot execute command - %s", err) From f77f07fd61136724f9bacaaa81eb0ea7dd71b532 Mon Sep 17 00:00:00 2001 From: Malthe Poulsen <30603252+malpou@users.noreply.github.com> Date: Sat, 4 Oct 2025 18:26:58 +0200 Subject: [PATCH 29/29] Update CHANGELOG.md Signed-off-by: Malthe Poulsen <30603252+malpou@users.noreply.github.com> --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a226dd873..1a46dfe27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,7 +31,7 @@ This changelog keeps track of work items that have been completed and are ready - **General**: Allow using HSO and SO with different names ([#1293](https://github.com/kedacore/http-add-on/issues/1293)) - **General**: Support profiling for KEDA components ([#4789](https://github.com/kedacore/keda/issues/4789)) - **General**: Add possibility to skip TLS verification for upstreams in interceptor ([#1307](https://github.com/kedacore/http-add-on/pull/1307)) -- **General**: Add custom placeholder pages for scale-from-zero scenarios ([#874](https://github.com/kedacore/http-add-on/issues/874)) +- **General**: Add custom placeholder responses for scale-from-zero scenarios ([#874](https://github.com/kedacore/http-add-on/issues/874)) ### Improvements - **Interceptor**: Support HTTPScaledObject scoped timeout ([#813](https://github.com/kedacore/http-add-on/issues/813))