Not able to submit : skein.exceptions.DriverError: Failed to submit application, exception: null

[ZelinLiu2016]

Hello, thanks for providing such a useful library.
Unfortunately I was stuck in the issue when I tried to run a very simple example.
`
import dask_yarn
import skein
import os
os.environ['PATH'] = '/usr/lib/hadoop-3.3.1/bin' + ':' + os.environ['PATH']

client = skein.Client()
spec = skein.ApplicationSpec.from_yaml(
"""
name: hello_world
queue: default

master:
resources:
vcores: 1
memory: 512 MiB
script: |
sleep 60
echo "Hello World!"
""")
app = client.submit_and_connect(spec)
`

But we got error message

21/07/14 14:32:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
21/07/14 14:32:16 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
21/07/14 14:32:17 INFO skein.Driver: Driver started, listening on 39496
21/07/14 14:32:17 INFO conf.Configuration: resource-types.xml not found
21/07/14 14:32:17 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
21/07/14 14:32:17 INFO skein.Driver: Uploading application resources to file:/home/marliu/.skein/application_1626234411488_0042
Traceback (most recent call last):
File "/home/marliu/workspace/dask_yarn_test/test.py", line 33, in
app = client.submit_and_connect(spec)
File "/home/marliu/dask_env/lib/python3.6/site-packages/skein/core.py", line 530, in submit_and_connect
app_id = self.submit(spec)
File "/home/marliu/dask_env/lib/python3.6/site-packages/skein/core.py", line 509, in submit
resp = self._call('submit', spec.to_protobuf())
File "/home/marliu/dask_env/lib/python3.6/site-packages/skein/core.py", line 289, in _call
raise self._server_error(exc.details())
skein.exceptions.DriverError: Failed to submit application, exception:
null

Thanks for looking into it
Version Info
- Python version: python3.6
- Hadoop version, 3.1.1
- Skein version 0.8.1

[ZelinLiu2016]

Further more, I stepped into it and found the error in
skein/core.py
line 273: return getattr(self._stub, method)(req, timeout=timeout)

[jainpriyansh786]

Hi @ZelinLiu2016 ,
I am also getting the same error , Did you find any workaround for it.
Version Info:

• Python version: python3.7
• Hadoop version, 3.2.1
• Skein version 0.8.1

I think its related to the hadoop/yarn version , I have a different emr cluster with hadoop 2.8.5 , it seems to work on it.

[jainpriyansh786]

Update its working now.
Resolved my issue by changing the requested memory for the application container . I was asking for larger memory container which was not available in the yarn cluster.