xds bootstrap: enable using JWT Call Credentials (part 2 for A97)

[dimpavloff]

Part two for grpc/proposal#492 (A97), following #8431 .

What this PR does is:

• update internal/xds/bootstrap with support for loading multiple PerRPCCallCredentials specifed in a new call_creds field in the boostrap file as per A97
• adjust xds/internal/xdsclient/clientimpl.goto use the call credentials when constructing the client
• update xds/bootstrap to register the jwtcreds call credentials and make them available if GRPC_EXPERIMENTAL_XDS_BOOTSTRAP_CALL_CREDS is enabled

I have added DialOptionsWithCallCredsForTransport because, even though current and future call credentials are likely to all expect secure transport, I thought it would be safer to check of insecure transport just in case. If you prefer, I can just update DialOptions to use all call credentials regardless of the transport.

Relates to istio/istio#53532

RELEASE NOTES:

• xds: add support for loading a JWT from file and use it as Call Credentials (A97). To enable this feature, set the environment variable GRPC_EXPERIMENTAL_XDS_BOOTSTRAP_CALL_CREDS to true (case insensitive).

[dimpavloff]

Thanks for the review and for the fixes @easwars ! I've addressed the latest comments.

[arjan-bal]

Mostly looks good, left some minor comments.

[arjan-bal]

Is there a benefit to allowing the returned PerRPCCredentials to be nil?

It seems that if a credential's configuration required a no-op, the implementer could simply return a dedicated no-op implementation to achieve the same result.

My concern is that allowing nil forces every consumer of the API to add nil checks or risk a panic.

[dimpavloff]

I don't think there's any benefit to allowing them to be nil, I'm just being defensive here. It's not something that can be enforced with the current interface (Build(config json.RawMessage) (credentials.PerRPCCredentials, func(), error) ) other than through convention.

I could document it in the interface if you want

[arjan-bal]

gRPC Go utilizes global registries for builders in many places, and the library does not expect these builders to return nil. When a builder returns nil, it is considered an illegal state and can result in a panic. Some examples:

• Channel Creds builder
• Name Resolver builder
• Load Balancing policy builder. Here, the nil is considered illegal, but it's handled by returning an error. There are other places where nil load balancers are not handled.

In my opinion, it is okay to omit the nil check here and document in the Build method's comments that it must not return nil. We could also return an error if we don't want gRPC to panic. If nil PerRPCCredentials are silently ignored, developers might begin to rely on this implicit nil check, forcing every future call site of Build to also handle nil values for consistency.

Also note that checking nil values for interfaces can be error-prone. Consider the following code:

package main

import "fmt"

type Element interface{}

type SomeImplementation struct{}

func f() Element {
	return nil
}

func g() Element {
	var result *SomeImplementation
	return result
}

func main() {
	fmt.Printf("f() != nil: %v, f(): %v\n", f() == nil, f()) // Prints: false, %!p(<nil>)
	fmt.Printf("g() != nil: %v, g(): %v\n", g() != nil, g()) // Prints: true, <nil>
}

Go playground: https://go.dev/play/p/hbc9cnbihBk
This document has an explanation: https://go.dev/doc/faq#nil_error

[dimpavloff]

Thanks @arjan-bal . Could you please take another look?

[dimpavloff]

I don't think there's any benefit to allowing them to be nil, I'm just being defensive here. It's not something that can be enforced with the current interface (Build(config json.RawMessage) (credentials.PerRPCCredentials, func(), error) ) other than through convention.

I could document it in the interface if you want