Tenant Operator Installation Guide

This guide outlines the steps to install the Tenant Operator on a Kubernetes cluster.

Images:

All necessary images you can find in this repo

Prerequisites

• kubectl: Ensure you have kubectl installed and configured to interact with your Kubernetes cluster.
• Helm: Ensure you have Helm installed.
• Kubernetes Cluster Access: Verify you have the necessary permissions to deploy resources to your Kubernetes cluster.
• Placeholder Values: Be prepared to replace placeholder values (such as registry URLs or secrets) with your actual environment-specific data.

Installation Steps

1. Install Cert-Manager

Cert-Manager is required for managing certificates within the cluster.

• Apply Cert-Manager CRDs:

  kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.8.0/cert-manager.crds.yaml

• Add the Jetstack Helm Repository and Update:

  helm repo add jetstack https://charts.jetstack.io
  helm repo update

• Install Cert-Manager in the 'cert-manager' Namespace:

  helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.8.0

2. Create a PriorityClass

A PriorityClass can be used to ensure critical pods are scheduled with higher priority.

• Save the following YAML as hpe-critical-priorityclass.yaml:

  apiVersion: scheduling.k8s.io/v1
  kind: PriorityClass
  metadata:
    name: hpe-critical
  value: 1000000
  globalDefault: false
  description: "Priority class for critical HPE pods"

• Apply the PriorityClass:

  kubectl apply -f hpe-critical-priorityclass.yaml

3. Add Registry Certificate on Nodes

Install the registry certificate on all cluster nodes according to your internal security procedures.

Note: The exact steps for this will vary depending on your specific environment. Refer to your internal documentation for guidance.

4. Install Tenant Operator

• Install the Tenant Operator using Helm:

  helm install tenant-operator tenant-operator-chart/ -n test-tenant --create-namespace -f tenant-operator-chart/values.yaml

5. Generate External Secrets

Run the external secret generation script (gen-external-secrets.sh) on a cluster node. This script is assumed to be located in your environment.

6. Apply External Secrets and Additional Configurations

a. Apply External Secrets

• Create the hpe-externalclusterinfo Namespace:

  kubectl create namespace hpe-externalclusterinfo

• Apply the External Secrets YAML:

  kubectl apply -f /tmp/mapr-external-secrets.yaml

b. Create the hpe-secure Namespace and LDAP ConfigMap

• Create the hpe-secure namespace if it does not already exist:

  kubectl create namespace hpe-secure

• Save the following as ldapclient-cm.yaml (replace with your actual LDAP configuration):

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: ldapclient-cm
    namespace: hpe-secure
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: "{}"
  data:
    ldap.conf: |
      BASE http://example.com
      URI test
      TLS_CACERTDIR /etc/openldap/certs
      TLS_REQCERT  allow
      SASL_NOCANON  on

• Apply the ConfigMap:

  kubectl apply -f ldapclient-cm.yaml

c. Create an ImagePull Secret (in the tenant namespace)

• Save the following as imagepull-secret.yaml (replace "........" with your base64-encoded .dockerconfigjson):

  apiVersion: v1
  kind: Secret
  metadata:
    name: imagepull
    namespace: sampletenant
    labels:
      hpe.com/cluster: none
      hpe.com/component: imagepull
      hpe.com/namespacetype: Tenant
      hpe.com/tenant: sampletenant
      hpe.com/version: 7.0.0
  data:
    .dockerconfigjson: "........"
  type: kubernetes.io/dockerconfigjson

• Apply the secret:

  kubectl apply -f imagepull-secret.yaml

d. Create a Cluster Role for Persistent Volume Operations

• Save the following as hpe-pvcreate-clusterrole.yaml:

  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: hpe-pvcreate
  rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["create", "delete", "get", "list", "watch"]

• Apply the ClusterRole:

  kubectl apply -f hpe-pvcreate-clusterrole.yaml

7. Deploy the Tenant

Deploy the tenant using the correct container images, EEP cluster name, and repository references.

• Refer to the example configuration:

  kubectl apply -f tenant-crs/external-full.yaml

8. Update Role Bindings

Modify the roles (hpe-sampletenant-role and hpe-sampletenant-terminalrole) to include the necessary permissions for managing persistent volumes, secrets, pods, and ConfigMaps as required by your tenant workloads.

9. Run Ticket Creator and Launch Spark Shell

a. Run the Ticket Creator Script

• Navigate to the tenant CLI and execute the bundled to the pod script:

  ./ticketcreator.sh

b. Launch Spark Shell (or any other executable)

• Execute the Spark Shell command, replacing placeholder values as needed:

  /opt/mapr/spark/spark-3.5.1/bin/spark-shell \
    --master k8s://https://<kubernetes controller host>:6443 \
    --conf spark.executor.instances=2 \
    --conf spark.mapr.user.secret=<generated secret name> \
    --conf spark.kubernetes.container.image=<spark image> \
    --conf spark.kubernetes.namespace=<tenant namespace> \
    --conf spark.kubernetes.container.image.pullPolicy=Always \
    --conf spark.mapr.cluster.configMap=cluster-cm \
    --conf spark.authenticate=false \
    --conf spark.authenticate.enableSaslEncryption=false \
    --conf spark.kubernetes.authenticate.driver.serviceAccountName=hpe-<tenant namespace>

  Important: Replace the following placeholders:

  • <kubernetes controller host>: The hostname or IP address of your Kubernetes controller.
  • <generated secret name>: The name of the secret generated by the ticketcreator.sh script.
  • <spark image>: The full name of the Spark container image you intend to use.
  • <tenant namespace>: The namespace where your tenant is deployed (e.g., sampletenant).