Why does Workload Identity Federation for using Vertex Backend only applicable when HTTPClient is not used

[somakdutta]

Hello,

I am attempting to connect to Vertex AI Backend using the go-genai library for using gemini models.

Additionally i would like to use WIF for authentication. My question is :
based on the code here why is WIF / or automatic credential look up only respected when httpclient is not set up in the clientConfig?

What if we would like to use the WIF and a wrapped httpclient , lets say with specific error codes being set as retry options?

The error message received when httpclient option was provided was

Error 401, Message: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project., Status: UNAUTHENTICATED, Details: [map[@type:type.googleapis.com/google.rpc.ErrorInfo domain:googleapis.com metadata:map[method:google.cloud.aiplatform.v1beta1.PredictionService.GenerateContent service:aiplatform.googleapis.com] reason:CREDENTIALS_MISSING]] 

Regards

[Sivasankaran25]

Hi @somakdutta Workload Identity Federation works with Gemini on Vertex AI only when the client allows Google libraries to manage authentication via Application Default Credentials. Passing a custom http.Client bypasses the Workload Identity Federation flow, causing missing OAuth tokens. To fix this, use the default client or wrap its transport so authentication via Application Default Credentials still works (official docs).

[somakdutta]

Hi @somakdutta Workload Identity Federation works with Gemini on Vertex AI only when the client allows Google libraries to manage authentication via Application Default Credentials. Passing a custom http.Client bypasses the Workload Identity Federation flow, causing missing OAuth tokens. To fix this, use the default client or wrap its transport so authentication via Application Default Credentials still works (official docs).

Hi @Sivasankaran25 Thank you for looking into this. I did get this to work and connect to vertex AI using WIF using the client config code as given below

&genai.ClientConfig{
			Project:  c.Project,
			Location: c.Location,
			Backend:  genai.BackendVertexAI,
		}

However i would like to have the option to provide my own http client, eg say with the below retry options.

&ehttp.RetryParameters{
			RetryCodes: map[string][]int{
				"*": {http.StatusInternalServerError, http.StatusTooManyRequests, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout, http.StatusTooEarly},
			},
		}

You mentioned something about wrapping the transport? i am not sure i am seeing these options in the current go-genai library. Would appreciate your help here. Is there a sample example you could point me to pls.

[Sivasankaran25]

@somakdutta Thank you for your response. I am using a custom HTTP client via HttpClient or HttpOptions in ClientConfig. Please try this way, and if it doesn’t work, share a code snippet.

[somakdutta]

@somakdutta Thank you for your response. I am using a custom HTTP client via HttpClient or HttpOptions in ClientConfig. Please try this way, and if it doesn’t work, share a code snippet.

I am using something as simple as the below code :
But it could be any http client because -
The gen ai library appears to only perform vertex WIF when httpclient is nil as show here

package main

import (
	"context"
	"net/http"

	"github.com/hashicorp/go-retryablehttp"
	"google.golang.org/genai"
)

type RetryConfig struct {
	RetryCodes map[string][]int
}

type retryChecker struct {
	codes map[string][]int
}

func (r *retryChecker) shouldRetry(ctx context.Context, resp *http.Response, err error) (bool, error) {

	if ctx.Err() != nil {
		return false, ctx.Err()
	}

	// for sample example, hardwiring true here
	return true, nil
}

func NewRetryClient(config *RetryConfig) *retryablehttp.Client {

	retryClient := retryablehttp.NewClient()
	retrier := &retryChecker{
		codes: config.RetryCodes,
	}
	retryClient.CheckRetry = retrier.shouldRetry
	retryClient.HTTPClient = &http.Client{}
	return retryClient
}

func main() {

	client := NewRetryClient(&RetryConfig{
		RetryCodes: map[string][]int{
			"*": {http.StatusInternalServerError,
				http.StatusTooManyRequests,
				http.StatusBadGateway,
				http.StatusServiceUnavailable},
		},
	})
	genAiClient, _ := genai.NewClient(ctx, &genai.ClientConfig{
		Project:    "...",
		Location:   "...",
		Backend:    genai.BackendVertexAI,
		HTTPClient: client,
	})

}

[Sivasankaran25]

@somakdutta Thank you for your response. I am using a custom HTTP client via HttpClient or HttpOptions in ClientConfig. Please try this way, and if it doesn’t work, share a code snippet.

I am using something as simple as the below code : But it could be any http client because - The gen ai library appears to only perform vertex WIF when httpclient is nil as show here

package main

import (
	"context"
	"net/http"

	"github.com/hashicorp/go-retryablehttp"
	"google.golang.org/genai"
)

type RetryConfig struct {
	RetryCodes map[string][]int
}

type retryChecker struct {
	codes map[string][]int
}

func (r *retryChecker) shouldRetry(ctx context.Context, resp *http.Response, err error) (bool, error) {

	if ctx.Err() != nil {
		return false, ctx.Err()
	}

	// for sample example, hardwiring true here
	return true, nil
}

func NewRetryClient(config *RetryConfig) *retryablehttp.Client {

	retryClient := retryablehttp.NewClient()
	retrier := &retryChecker{
		codes: config.RetryCodes,
	}
	retryClient.CheckRetry = retrier.shouldRetry
	retryClient.HTTPClient = &http.Client{}
	return retryClient
}

func main() {

	client := NewRetryClient(&RetryConfig{
		RetryCodes: map[string][]int{
			"*": {http.StatusInternalServerError,
				http.StatusTooManyRequests,
				http.StatusBadGateway,
				http.StatusServiceUnavailable},
		},
	})
	genAiClient, _ := genai.NewClient(ctx, &genai.ClientConfig{
		Project:    "...",
		Location:   "...",
		Backend:    genai.BackendVertexAI,
		HTTPClient: client,
	})

}

Thank you for sharing your code. I will reproduce this issue and then find the solution for this

[somakdutta]

hi @Sivasankaran25 , do let me know if you had any luck reproducing the issue?

[Sivasankaran25]

hi @Sivasankaran25 , do let me know if you had any luck reproducing the issue?
Yes, I was able to reproduce the issue, so I am escalating it and we will work to resolve it as soon as possible.

[somakdutta]

hi @Sivasankaran25 , do let me know if you had any luck reproducing the issue?
Yes, I was able to reproduce the issue, so I am escalating it and we will work to resolve it as soon as possible.

hi @Sivasankaran25 , wanted to find out if there is a time line that we can expect for the fix?

[Sivasankaran25]

wanted to find out if there is a time line that we can expect for the fix?

Thank you for bringing this to our attention. We truly appreciate you flagging this issue, we will file a bug internally.

[somakdutta]

I think we can close this issue. Leaving the solution below for future reference.

It is possible to use retryable httpclient when using vertex ai using the mechanism below ( and it still connects via WIF)

cc := &genai.ClientConfig{
			Project:    ...,
			Location:   ....,
			Backend:    genai.BackendVertexAI,
			HTTPClient: httpClient , //( any instance of retryable http),
		}
if err := cc.UseDefaultCredentials(); err != nil {
			return nil, err
		}