KEP-5339: add additional cluster-specific auth info field to the cluster profile object

keps/sig-multicluster/5339-clusterprofile-plugin-credentials/README.md

@@ -226,7 +226,8 @@ controller, applications or consumers without requiring changes. It also cannot
 is not sensitive.
 
 The definition is as follows:
-```
+
+```golang
 type CredentialProviders struct {
   // +listType=map
   // +listMapKey=name
@@ -239,7 +240,7 @@ type CredentialsType string
 // CredentialsConfig gives more details on data that is necessary to reach out the cluster for this kind of Credentials
 type CredentialsConfig struct {
   Name string // name of the provider type
-  Cluster *Cluster // Configuration to reach the cluster (endpoints, proxy, etc) // See following section for details.
+  Cluster *Cluster // Configuration to reach the cluster (endpoints, proxy, etc) // See the sections below for details.
 }
 ```
 
@@ -292,6 +293,12 @@ In this structure, not all fields would apply, such as:
 
 * `CertificateAuthority`, which points to a file (and a ClusterProfile doesn't have a filesystem)
 
+And there are fields that require special attention:
+
+* `Extensions`, which holds additional, usually cluster-specific information, that might help authenticate with the cluster.
+
+For more information about these fields and how they are handled, see the section below:
+
 #### Passing plugin configuration via extensions
 
 Some credential providers require cluster-specific, non-secret parameters (for example, a `clusterName`) in order to obtain credentials. To standardize how this information is conveyed from a `ClusterProfile` to a plugin, the library follows the existing convention defined by the client authentication API:
@@ -312,9 +319,61 @@ extensions:
     clusterName: spoke-1
 ```
 
+In practice, however, there exist certain scenarios where setting the reserved `client.authentication.k8s.io/exec` extension
+to pass cluster-specific data might not be appropriate: libraries such as `client/go` will eventually save the extension data
+(along with other information, including the CA bundles for a cluster) to an environment variable, `KUBERNETES_EXEC_INFO`,
+which exec plugins can read; however:
+
+* some exec plugins might be expecting inputs from CLI arguments or plugin-specific environment variables directly; they
+might not read the `KUBERNETES_EXEC_INFO` environment variable at all, or might make only limited use of the environment
+variable.
+* it might not be proper to set the `KUBERNETES_EXEC_INFO` environment variable in the target environment: for example,
+`KUBERNETES_EXEC_INFO` includes CA bundles for a cluster and its size might exceed length limitations in the environment.
+* the `client.authentication.k8s.io/exec` extension keeps the data in the free form, `runtime.RawExtension`; before it is 
+saved to the `KUBERNETES_EXEC_INFO` environment variable, `client/go` will pass it to the `ExecConfig.Config` struct first,
+which accepts only `runtime.Object` data. This brings about possible marshalling/unmarshalling complications, which could be
+difficult to handle gracefully for the community-provided library proposed in this KEP.
+
+To address the deficiencies above, we further propose that:
+
+* this KEP reserves a name in the extensions, `multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-args`, which holds
+additional CLI arguments that would be supplied to the exec plugin when the ClusterProfile API and community-provided
+library are used for authentication.
+
+  If an extension under this name is present, the community-provided library will extract the data, and append the
+  additional arguments to the `ExecConfig` struct (specifically the `ExecConfig.Args` field) that will be used to
+  prepare the `rest.Config` output. The arguments will then be used to invoke the exec plugin.
+
+  The additional arguments shall be saved as a string array in the YAML format.
+
+  For simplicity reasons, the community-provided library will not perform any de-duplication on the CLI arguments
+  after the additional arguments are appended.
+
+* this KEP reserves another name in the extensions, `multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-envs`, which
+holds additional environment variables that would be supplied upon calling the exec plugin when the ClusterProfile API
+and community-provided library are used for authentication.
+
+  If an extension under this name is present, the community-provided library will extract the data, and add the additional
+  variables to the `ExecConfig` struct (specifically the `ExecConfig.Env` field) that will be used to prepare the
+  `rest.Config` output. The variables will then be set when invoking the exec plugin.
+
+  The additional environment variables shall be represented as a string map in the YAML format.
+
+  The community-provided library will de-duplicate the list of environment variables when adding the additional variables;
+  if two entries are present under the same name, the one from the extension will prevail.
+
+###### Security concerns
+
+With the addition of newly reserved extensions, understandably there might be situations where users might want to block
+additional CLI arguments or environment variables from being set due to security reasons. To resolve this, the KEP proposes
+that the community-provided library implementation must allow users to specify whether additional CLI arguments or environment
+variables can be set by a `ClusterProfile` object. By default the reserved extensions should be ignored.
+
+See the [Configuring plugins in the controller](#configuring-plugins-in-the-controller) section for more information.
+
 #### ClusterProfile Example
 
-Example of a GKE ClusterProfile, which would map to a plugin providing credentials of type `google`:
+Below is an example of a GKE ClusterProfile, which would map to a plugin providing credentials of type `google`:
 
 ```
 apiVersion: multicluster.x-k8s.io/v1alpha1
@@ -339,30 +398,81 @@ status:
       server: https://connectgateway.googleapis.com/v1/projects/123456789/locations/us-central1/gkeMemberships/my-cluster-1
 ```
 
-Example of a SecretReader ClusterProfile using the `extensions` convention to pass `clusterName` to the plugin:
+Below are some examples that feature the use of extensions in ClusterProfiles:
+
+* This example uses the reserved `client.authentication.k8s.io/exec` extension to pass cluster names to a plugin of the
+secret reader type:
+
+  ```yaml
+  apiVersion: multicluster.x-k8s.io/v1alpha1
+  kind: ClusterProfile
+  metadata:
+    name: my-cluster-1
+  spec:
+    displayName: my-cluster-1
+    clusterManager:
+      name: inhouse-manager
+  status:
+    credentialProviders:
+    - name: secretreader
+      cluster:
+        server: https://<spoke-server>
+        certificate-authority-data: <BASE64_CA>
+        extensions:
+        - name: client.authentication.k8s.io/exec
+          extension:
+            clusterName: spoke-1
+  ```
+
+* This example uses the "multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-args" extension to pass additional
+CLI arguments (`-audience https://my-on-prem-k8s.example.dev`) to the exec plugin when the `spire-agent` credential
+provider is used, as the cluster's authentication solution is expecting tokens with this specific audience for
+security reasons.
+
+  ```
+  apiVersion: multicluster.x-k8s.io/v1alpha1
+  kind: ClusterProfile
+  metadata:
+    name: my-on-prem-cluster
+  spec: ...
+  status:
+    ...
+    credentialProviders:
+    - name: spire-agent
+      cluster:
+        server: https://my-on-prem-k8s.example.dev
+        ...
+        extensions:
+        - name: "multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-args"
+          extension:
+          - "-audience"
+          - "https://my-on-prem-k8s.example.dev"
+  ```
+
+* This example uses the "multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-envs" extension to pass
+additional environment variables `CLIENT_ID` and `TENANT_ID` to the exec plugin when the `kubelogin` credential
+provider is used; these entries can help the exec plugin exchange for cluster-specific access tokens.
 
 ```
 apiVersion: multicluster.x-k8s.io/v1alpha1
 kind: ClusterProfile
 metadata:
-  name: my-cluster-1
-spec:
-  displayName: my-cluster-1
-  clusterManager:
-    name: inhouse-manager
+ name: my-aks-cluster
+spec: ...
 status:
+  ...
   credentialProviders:
-  - name: secretreader
+  - name: kubelogin
     cluster:
-      server: https://<spoke-server>
-      certificate-authority-data: <BASE64_CA>
+      server: https://braveion-abcxyz.hcp.eastus2.azmk8s.io
+      ...
       extensions:
-      - name: client.authentication.k8s.io/exec
+      - name: "multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-envs"
         extension:
-          clusterName: spoke-1
+          "CLIENT_ID": "my-client-id"
+          "TENANT_ID": "my-tenant-id"
 ```
 
-
 ### Configuring plugins in the controller
 
 Plugins are selected by a string which represents the type of credentials that is expected by the cluster, for example, "google" for GKE Clusters.
@@ -380,12 +490,19 @@ type Provider struct {
   CredentialsType string
   ExecutablePath string
   args []string
+  AllowClusterProfileSourcedCLIArgs bool
+  AllowClusterProfileSourcedEnvVars bool
 }
 ```
 
 Given the plugin is executed directly by the controller, it may expect to have access to the same environment as the controller itself, inclusive of envvars, filesystem and network.
 It is expected that the identity of the plugin is the same as the controller itself.
 
+The `AllowClusterProfileSourcedCLIArgs` and `AllowClusterProfileSourcedEnvVars` flags control whether the library will process
+`multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-args` and `multicluster.x-k8s.io/clusterprofiles/auth/exec/additional-envs`
+extensions, as described earlier. If set to false, additional CLI arguments and/or environment variables cannot be set
+from the ClusterProfile side.
+
 ### Plugin Examples
 
 As an example, we provide pseudocode for plugins that could easily be implemented with the protocol. They are ultrasimplified