samba client pod is not coming up after `k apply -f client-test-pod.yaml`

[mwaykole]

steps :

1. create a minikue deployment
2. clone the repo and make deploy
3. $ k config set-context --current --namespace=samba-operator-system
4. $ # k get pods
   NAME READY STATUS RESTARTS AGE
   samba-operator-controller-manager-844d976b7b-nlgqb 2/2 Running 0

---

[a@dhcp47-98 files]$ k apply -f client-test-pod.yaml
pod/smbclient created

[a@dhcp47]$ k get pods
NAME READY STATUS RESTARTS AGE
samba-ad-server-86b7dd9856-shvxp 1/1 Running 0 46m
samba-operator-controller-manager-844d976b7b-nlgqb 2/2 Running 0 19h
smbclient 0/1 ContainerCreating 0 30m -->> pod is not comming up

---

events:

35m Normal Scheduled pod/smbclient Successfully assigned samba-operator-system/smbclient to minikube
34m Warning FailedMount pod/smbclient MountVolume.SetUp failed for volume "data" : configmap "sample-data1" not found
27m Warning FailedMount pod/smbclient MountVolume.SetUp failed for volume "data" : configmap "sample-data1" not found
25m Warning FailedMount pod/smbclient Unable to attach or mount volumes: unmounted volumes=[kube-api-access-45s5r data], unattached volumes=[kube-api-access-45s5r data]: timed out waiting for the condition
4m36s Normal Scheduled pod/smbclient Successfully assigned samba-operator-system/smbclient to minikube
2m28s Warning FailedMount pod/smbclient MountVolume.SetUp failed for volume "data" : configmap "sample-data1" not found
2m33s Warning FailedMount pod/smbclient Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[data kube-api-access-88s6g]: timed out waiting for the condition
87s Normal Scheduled pod/smbclient Successfully assigned samba-operator-system/smbclient to minikube
24s Warning FailedMount pod/smbclient MountVolume.SetUp failed for volume "data" : configmap "sample-data1" not found
16s Warning FailedMount pod/smbclient Unable to attach or mount volumes: unmounted volumes=[data kube-api-access-88s6g], unattached volumes=[data kube-api-access-88s6g]: timed out waiting for the condition

---

Hi plz, let me know if I missed any steps ,

[phlogistonjohn]

Hello there. I'm happy to see someone new trying out samba operator!

The yaml manifest client-test-pod.yaml depends on the config map sample-data1. This config map is currently kept at tests/files/data1.yaml. Create the configmap using something like kubectl apply -f tests/files/data1.yaml and then the client test pod should start running.

[mwaykole]

Hi Thanks the above problem is solved , but now

---

samba-ad-server-86b7dd9856-h8dh5                     1/1     Running    0          27m
samba-operator-controller-manager-844d976b7b-hb94m   2/2     Running    0          2d19h
smbclient                                            1/1     Running    0          2d19h
tshare1-7dbf77d95f-vgd4f                             1/1     Running    0          68m
tshare2-85464f98c4-njldh                             0/4     Init:0/2   0          4m41s

---

tshare2-85464f98c4-njldh <<<<-- is not coming up ? in events also not seen any errors

[mwaykole]

Events:
Type Reason Age From Message

---

Normal Scheduled 2m5s default-scheduler Successfully assigned samba-operator-system/tshare2-8bf9f566d-k2gw6 to minikube
Warning FailedMount 61s (x8 over 2m5s) kubelet MountVolume.SetUp failed for volume "join-data-0" : references non-existent secret key: join2.json
Warning FailedMount 2s kubelet Unable to attach or mount volumes: unmounted volumes=[join-data-0], unattached volumes=[join-data-0 samba-wb-sockets-dir tshare2-pvc-smb svcwatch samba-container-config samba-state-dir kube-api-access-fsdfm]: timed out waiting for the condition

[phlogistonjohn]

If you're using the same YAML files from tests/files, much like the client pod, these have dependencies. In the case of our SmbShares, these are based on data files defined in the SmbSecurityConfig. Take a look at the file smbsecurityconfig1.yaml for tshare1 and smbsecurityconfig2.yaml for tshare2. Both refer to a secret. These secrets must be loaded in the same namespace as the pod.

For tshare2 this is kept in the yaml file joinsecret1.yaml. Use kubectl to load that file and see if that resolves your problem.

[mwaykole]

I am trying to use something like

---
apiVersion: v1
kind: Secret
metadata:
  namespace: samba-operator-system
  name: join1
type: Opaque
stringData:
  # Change the value below to match the username and password for a user that
  # can join systems your test AD Domain
  join.json: |
    {"username": "admin", "password": "admin"}
  # join2.json: |
  #   {"username": "Administrator", "password": "admin"}
---
apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbSecurityConfig
metadata:
  name: adsec1
spec:
  mode: active-directory
  realm: domain1.sink.test
  joinSources:
  - userJoin:
      secret: join1
      key: join1.json
  dns:
    register: cluster-ip

is it correct?

[phlogistonjohn]

Looks mostly OK, but I do see that the secret resource specifies a namespace but the SmbSecurityConfig does not. So depending on what your kubectl command line is the resources may or may not be created in the same namespace. It's not wrong - but I can't say if you'll get the expected result.

Also, what password you use depends on your AD if you're using the same AD server container that the tests do, the password is different. But it's totally OK if want to change the password. So again, I can't say if it's right or wrong. But it is different.

[mrrajan]

@phlogistonjohn I am facing the same issue as @milindw96 mentioned. I am using the files from /test/files. Standalone pods are up and running without issues. But the pods with active-directory mode on smbsecurityconfig stuck at init state.
Please find the list of resources created for "tshare2" below:

Please clarify any prerequisite needs to be done before running the samba-operator for AD.

[phlogistonjohn]

Hi @rravi6121 I suspect that the issue is similar too, the AD domain member instances has more dependencies that the standalone and more opportunities to get stuck. :-)

Can you please run kubectl describe pod <podname>?

PS. It's generally better to copy and paste text from your terminal over a screenshot image. When you paste you can surround the text in three backticks (```) on their own line before and after your paste to make it clear what you pasted. An example:

Hello world!

[mrrajan]

Thanks @phlogistonjohn, Please find the output of "kubectl describe pod" below:

Name:         tshare2-5b78b7764-5rwxc
Namespace:    samba-operator-system
Priority:     0
Node:         minikube/<<ipaddress>>
Start Time:   Thu, 22 Jul 2021 10:38:30 +0530
Labels:       app=samba
              app.kubernetes.io/component=smbd
              app.kubernetes.io/instance=samba-tshare2
              app.kubernetes.io/managed-by=samba-operator
              app.kubernetes.io/name=samba
              app.kubernetes.io/part-of=samba
              pod-template-hash=5b78b7764
              samba-operator.samba.org/service=tshare2
Annotations:  kubectl.kubernetes.io/default-container: samba
              kubectl.kubernetes.io/default-logs-container: samba
Status:       Pending
IP:           10.244.0.5
IPs:
  IP:           10.244.0.5
Controlled By:  ReplicaSet/tshare2-5b78b7764
Init Containers:
  init:
    Container ID:  cri-o://82b5401b6a519a95fd2db99344256950cf8779b0ebc4dededa25128a705cc80a
    Image:         quay.io/samba.org/samba-server:latest
    Image ID:      quay.io/samba.org/samba-server@sha256:4ef1fd9f02b6e5cef4d32c4b4388dde1411fcab651eb2bb1b2fa33423d8c0bd3
    Port:          <none>
    Host Port:     <none>
    Args:
      init
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 23 Jul 2021 09:39:52 +0530
      Finished:     Fri, 23 Jul 2021 09:39:52 +0530
    Ready:          True
    Restart Count:  0
    Environment:
      SAMBA_CONTAINER_ID:  tshare2
      SAMBACC_CONFIG:      /etc/container-config/config.json
    Mounts:
      /etc/container-config from samba-container-config (rw)
      /var/lib/samba from samba-state-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
  must-join:
    Container ID:  cri-o://ae28980851302223f03597962c8e8c339e05568aa02611b29bc28f762d67380c
    Image:         quay.io/samba.org/samba-server:latest
    Image ID:      quay.io/samba.org/samba-server@sha256:4ef1fd9f02b6e5cef4d32c4b4388dde1411fcab651eb2bb1b2fa33423d8c0bd3
    Port:          <none>
    Host Port:     <none>
    Args:
      must-join
    State:          Running
      Started:      Fri, 23 Jul 2021 09:40:08 +0530
    Ready:          False
    Restart Count:  0
    Environment:
      SAMBA_CONTAINER_ID:  tshare2
      SAMBACC_CONFIG:      /etc/container-config/config.json
      SAMBACC_JOIN_FILES:  /var/tmp/join/0/join.json:/var/tmp/join/1/join.json
    Mounts:
      /etc/container-config from samba-container-config (rw)
      /var/lib/samba from samba-state-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
      /var/tmp/join/0 from join-data-0 (rw)
      /var/tmp/join/1 from join-data-1 (rw)
Containers:
  samba:
    Container ID:  
    Image:         quay.io/samba.org/samba-server:latest
    Image ID:      
    Port:          445/TCP
    Host Port:     0/TCP
    Args:
      run
      smbd
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       tcp-socket :445 delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SAMBA_CONTAINER_ID:  tshare2
      SAMBACC_CONFIG:      /etc/container-config/config.json
    Mounts:
      /etc/container-config from samba-container-config (rw)
      /mnt/aa5ccaa6-0445-4e89-a0ae-108ca118a769 from tshare2-pvc-smb (rw)
      /run/samba/winbindd from samba-wb-sockets-dir (rw)
      /var/lib/samba from samba-state-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
  wb:
    Container ID:  
    Image:         quay.io/samba.org/samba-server:latest
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Args:
      run
      winbindd
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Liveness:       exec [samba-container check winbind] delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      SAMBA_CONTAINER_ID:  tshare2
      SAMBACC_CONFIG:      /etc/container-config/config.json
    Mounts:
      /etc/container-config from samba-container-config (rw)
      /run/samba/winbindd from samba-wb-sockets-dir (rw)
      /var/lib/samba from samba-state-dir (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
  dns-register:
    Container ID:  
    Image:         quay.io/samba.org/samba-server:latest
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Args:
      dns-register
      --watch
      --target=internal
      /var/lib/svcwatch/status.json
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Environment:
      SAMBA_CONTAINER_ID:  tshare2
      SAMBACC_CONFIG:      /etc/container-config/config.json
    Mounts:
      /etc/container-config from samba-container-config (rw)
      /run/samba/winbindd from samba-wb-sockets-dir (rw)
      /var/lib/samba from samba-state-dir (rw)
      /var/lib/svcwatch from svcwatch (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
  svc-watch:
    Container ID:   
    Image:          quay.io/samba.org/svcwatch:latest
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Environment:
      DESTINATION_PATH:     /var/lib/svcwatch/status.json
      SERVICE_LABEL_KEY:    samba-operator.samba.org/service
      SERVICE_LABEL_VALUE:   (v1:metadata.labels['samba-operator.samba.org/service'])
      SERVICE_NAMESPACE:    samba-operator-system (v1:metadata.namespace)
    Mounts:
      /var/lib/svcwatch from svcwatch (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-plb9l (ro)
Conditions:
  Type              Status
  Initialized       False 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  samba-container-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      samba-container-config
    Optional:  false
  samba-state-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  tshare2-pvc-smb:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  tshare2-pvc
    ReadOnly:   false
  samba-wb-sockets-dir:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  join-data-0:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  join1
    Optional:    false
  join-data-1:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  join1
    Optional:    false
  svcwatch:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  <unset>
  default-token-plb9l:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-plb9l
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason   Age    From     Message
  ----    ------   ----   ----     -------
  Normal  Pulling  6m36s  kubelet  Pulling image "quay.io/samba.org/samba-server:latest"
  Normal  Pulled   6m32s  kubelet  Successfully pulled image "quay.io/samba.org/samba-server:latest" in 4.612177804s
  Normal  Created  6m31s  kubelet  Created container init
  Normal  Started  6m31s  kubelet  Started container init
  Normal  Pulling  6m31s  kubelet  Pulling image "quay.io/samba.org/samba-server:latest"
  Normal  Pulled   6m15s  kubelet  Successfully pulled image "quay.io/samba.org/samba-server:latest" in 15.324456624s
  Normal  Created  6m15s  kubelet  Created container must-join
  Normal  Started  6m15s  kubelet  Started container must-join

[phlogistonjohn]

Thanks. Based on that output I suspect that "must-join" is unable to join the pod to active directory. The best thing we can do next is to see why must-join is not proceeding. That's usually because the join information doesn't match AD or AD is unreachable.

Run kubectl logs tshare2-5b78b7764-5rwxc -c must-join to get the logs from the must-join container. Then if you don't have any sensitive data in it, share your join secret with kubectl get secret join1 -o yaml. And lastly it would help if you described how you set up AD - is it just the AD pod yaml from our tests/files directory? And if so, did you deploy it using the script in ./tests/test-deploy-ad-server.sh and/ or did you configure the coredns config for your cluster?

[mrrajan]

Thanks @phlogistonjohn I have deployed AD pod using yaml with "kubectl apply -f" command and that might be the reason for the issue.

I tried deploying with shell script ./tests/test-deploy-ad-server.sh and the pod is up and running without issues

NAME                                                     READY   STATUS    RESTARTS   AGE
pod/samba-ad-server-86b7dd9856-7rwc5                     1/1     Running   0          45m
pod/samba-operator-controller-manager-844d976b7b-cx4qj   2/2     Running   0          59m
pod/smbclient                                            1/1     Running   0          55m
pod/tshare1-7f47b774b4-d4plg                             1/1     Running   0          33m
pod/tshare2-76f95db4d8-xlt42                             4/4     Running   0          31m

Steps followed:

1. Install and Start minikube. Check the status and make sure minikube running without issues
2. Git clone samba-operator repository and navigate into it.
3. Run "make deploy" from the "samba-operator" folder
4. Run "kubectl config set-context --current --namespace=samba-operator-system" to change the minikube namespace to "samba-operator-system"
5. Run "kubectl apply -f" command to deploy yaml files with kind Secret, ConfigMap, SmbCommonConfig and SmbSecurityConfig from "./test/files" folder. Make sure Secrets and ConfigMaps deployed before SmbSecurityConfig
6. Run shell script "./tests/test-deploy-ad-server.sh" to deploy samba-ad-server
7. Deploy Smbshare by running "kubectl apply -f ./tests/tshare2.yaml"

[mrrajan]

Hi @phlogistonjohn ,
I am trying to connect one of my windows AD server with SINK. But must-join step on the smbshare pod fails with the below error message and smbshare pod stuck at init status

Enter Administrator's password:
Failed to join domain: failed to find DC for domain WINDOWS2K16 - The object was not found.
Enter Administrator's password:
Failed to join domain: failed to find DC for domain WINDOWS2K16 - The object was not found.
ERROR: failed 2 join attempts
  - failed to run ['net', 'ads', 'join', '--no-dns-updates', '-U', 'Administrator']
  - failed to run ['net', 'ads', 'join', '--no-dns-updates', '-U', 'Administrator']

I am trying to manually deploy the pods with the configuration yaml files and not using the shell script on tests folder.

Please find the yaml files below:

joinsecret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: join1
type: Opaque
stringData:
  # Change the value below to match the username and password for a user that
  # can join systems your test AD Domain
  join.json: |
    {"username": "Administrator", "password": "P4ssw0rd"}
  join2.json: |
    {"username": "Administrator", "password": "Passw0rd"}
  joinad.json: |
    {"username": "Administrator", "password": "redhat@1"}

smbsecurityconfig.yaml

--
apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbSecurityConfig
metadata:
  name: adsec2
spec:
  mode: active-directory
  realm: Window2k16.DOMAIN.sinktest
  joinSources:
  - userJoin:
      secret: join1
  - userJoin:
      secret: join1
      key: joinad.json
  dns:
    register: cluster-ip

samba-ad-server-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: samba-ad-server
  labels:
    app: samba-ad
spec:
  replicas: 1
  selector:
    matchLabels:
      app: samba-ad
  template:
    metadata:
      labels:
        app: samba-ad
    spec:
      hostname: w2K16
      containers:
      - name: samba
        image: quay.io/samba.org/samba-ad-server:latest
        securityContext:
          capabilities:
            add: ["SYS_ADMIN"]
        ports:
        - containerPort: 53
          name: dns
        - containerPort: 135
          name: epm
          protocol: TCP
        - containerPort: 137
          name: netbios-ns
          protocol: UDP
        - containerPort: 138
          name: netbios-dgram
          protocol: UDP
        - containerPort: 139
          name: netbios-session
          protocol: TCP
        - containerPort: 389
          name: ldap
        - containerPort: 445
          name: smb
          protocol: TCP
        - containerPort: 464
          name: kerberos
        - containerPort: 636
          name: ldaps
          protocol: TCP
        - containerPort: 3268
          name: gc
          protocol: TCP
        - containerPort: 3269
          name: gc-ssl
          protocol: TCP

smbshare.yaml

---
apiVersion: samba-operator.samba.org/v1alpha1
kind: SmbShare
metadata:
  name: tsharead
spec:
  shareName: "My Kingdom"
  readOnly: false
  securityConfig: adsec2
  storage:
    pvc:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

coredns-snippet.template

   domain1.sink.test:53 {
        errors
        cache 30
        forward . AD_SERVER_IP
    }
    Window2K16.DOMAIN.sinktest:53 {
        errors
        cache 30
        forward . AD_SERVER_IP
    }

/etc/hosts/ file inside samba-ad-server pod

127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
fe00::0	ip6-mcastprefix
fe00::1	ip6-allnodes
fe00::2	ip6-allrouters
172.17.0.5	dc1
##.##.##.###	w2k16

Windows server manager

Please let me know whether I have missed anything and help me on this.

[phlogistonjohn]

Hi, I'm happy to help, but it feels like a lot of what in in this issue is general setup and debugging help rather than a defect/bug. Do you mind if I first convert this issue to a github discussion?

[mrrajan]

Sure @phlogistonjohn , It wont be a problem.