horovod in powerai 1.6.2, error occurs with hvd.init()

[wacoder]

Hi,
Try to install Horovod with PyTorch based on powerai image following official tutorial, error occurs when try to hvd.init(). Is there any material on how to set up DDL_OPTIONS to use ddl? Thanks

• code

import horovod.torch as hvd
hvd.init()
[2020-06-11 03:01:22.297540: W horovod/common/ops/ddl_operations.cc:105] DDL backend has been deprecated. Please, start using the NCCL backend by building Horovod with 'HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_GPU_BROADCAST=NCCL'.
terminate called after throwing an instance of 'std::logic_error'
what(): DDL_OPTIONS env variable needs to be set to use DDL.
Aborted (core dumped)`

---

I also tried to set up env variable for DDL_OPTIONS with export DDL_OPTIONS=DDL it produce another error

`

import horovod.torch as hvd
hvd.init()
[2020-06-11 03:14:33.814715: W horovod/common/ops/ddl_operations.cc:105] DDL backend has been deprecated. Please, start using the NCCL backend by building Horovod with 'HOROVOD_GPU_ALLREDUCE=NCCL HOROVOD_GPU_BROADCAST=NCCL'.
E 03:14:33.814 634 652 DDL:86 ] ill-formated option :[DDL]
terminate called after throwing an instance of 'std::system_error'
what(): Resource deadlock avoided
Aborted (core dumped)
`

[bnemanich]

You will need to launch the job using the ddlrun tool. This will setup the appropriate environment variables. For more information on ddlrun, see https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.2/navigation/wmlce_ddlrun.html.

[wacoder]

@bnemanich thanks very much. Does that mean I can use horovod in script, and use ddlrun to run?