ERROR: need at least one array to concatenate

[antonfj]

Hi,
I'm trying to peel an outlier source but at the moment it fails at the step smooth_amps_normalized with the following error

2018-01-26 18:38:12 ERROR node.lofar.ap.dias.ie.python_plugin: need at least one array to concatenate
2018-01-26 18:38:12 INFO node.lofar.ap.dias.ie.python_plugin: Total time 26.5053s; user time: 14.6480s; system time: 26.1160s
2018-01-26 18:38:12 DEBUG node.lofar.ap.dias.ie.python_plugin: Start time was 1516991866.0641s; end time was 1516991892.5695s
2018-01-26 18:38:13 DEBUG outlierpeel_3c123.executable_args:
2018-01-26 18:38:13 WARNING outlierpeel_3c123.executable_args:
2018-01-26 18:38:13 INFO outlierpeel_3c123.executable_args: Subprocess completed with exit status 1: /bin/sh -c python /usr/lib/python2.7/dist-packages/lofarpipe/recipes/nodes/python_plugin.py 0 127.0.1.1 40198
2018-01-26 18:38:13 ERROR outlierpeel_3c123.executable_args: Remote process python /usr/lib/python2.7/dist-packages/lofarpipe/recipes/nodes/python_plugin.py ['/mnt/big-data/antonfj/DGTau/factor/results/outlierpeel/3c123/L192022_SB000_uv.dppp_1239ECF9Dt_115MHz.pre-cal_chunk0_1239ECF9Dt_8g.merge_amp_parmdbs', '/home/antonfj/.local/lib/python2.7/site-packages/FACTOR-1.3-py2.7.egg/factor/scripts/smooth_amps_spline.py', ['/mnt/big-data/antonfj/DGTau/factor/results/outlierpeel/3c123/L192022_SB000_uv.dppp_1239ECF9Dt_115MHz.pre-cal_chunk0_1239ECF9Dt_8g.merge_amp_parmdbs', '/mnt/big-data/antonfj/DGTau/factor/results/outlierpeel/3c123/L192022_SB000_uv.dppp_1239ECF9Dt_115MHz.pre-cal_chunk0_1239ECF9Dt_8g.smooth_amps_normalized'], {'normalize': 'True'}, '/mnt/big-data/antonfj/DGTau/factor/results/outlierpeel/3c123', False, {'args_format_option_argument': '=', 'args_format_option': '-', 'args_format': 'gnu', 'args_formatlongoption': '--', 'args_format_argument': ''}, {'LOFARDATAROOT': '/opt/lofar/data', 'OMP_NUM_THREADS': '8', 'PYTHONPATH': '/usr/lib/python2.7/dist-packages', 'LOFARROOT': '/usr', 'PATH': '/usr/sbin:/usr/bin:/home/antonfj/bin:/home/antonfj/.local/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/games', 'LD_LIBRARY_PATH': '/usr/lib:/mnt/big-data/antonfj/LOFAR_software/lib'}] failed on localhost
2018-01-26 18:38:13 DEBUG outlierpeel_3c123.executable_args: compute.dispatch results job 0: job_duration: 27.3976299763, returncode: 1
2018-01-26 18:38:13 DEBUG outlierpeel_3c123.executable_args: Adding node_logging_information
2018-01-26 18:38:13 ERROR outlierpeel_3c123.executable_args: A job has failed with returncode 1 and error_tolerance is not set. Bailing out!
2018-01-26 18:38:13 WARNING outlierpeel_3c123.executable_args: Note: recipe outputs are not complete
2018-01-26 18:38:13 WARNING outlierpeel_3c123.executable_args: recipe executable_args completed with errors
2018-01-26 18:38:13 WARNING outlierpeel_3c123: smooth_amps_spline reports failure (using executable_args recipe)
2018-01-26 18:38:13 ERROR outlierpeel_3c123: *******************************************
2018-01-26 18:38:13 ERROR outlierpeel_3c123: Failed pipeline run: 3c123
2018-01-26 18:38:13 ERROR outlierpeel_3c123: Detailed exception information:
2018-01-26 18:38:13 ERROR outlierpeel_3c123: <class 'lofarpipe.support.lofarexceptions.PipelineRecipeFailed'>
2018-01-26 18:38:13 ERROR outlierpeel_3c123: smooth_amps_spline failed
2018-01-26 18:38:13 ERROR outlierpeel_3c123: *******************************************
2018-01-26 18:38:13 ERROR outlierpeel_3c123: LOFAR Pipeline finished unsuccesfully.
2018-01-26 18:38:13 WARNING outlierpeel_3c123: recipe outlierpeel_3c123 completed with errors

[antonfj]

I also tried skipping peeling the outlier source and tried processing the facet patches.
For the first facet patch, facetselfcal and facetsub ran successfully.
However, for the second facet patch, facetselfcal failed at the step smooth_amp2 and gave the same error as for outlierpeel:

2018-01-31 15:44:26 ERROR node.lofar.ap.dias.ie.python_plugin: need at least one array to concatenate
2018-01-31 15:44:26 INFO node.lofar.ap.dias.ie.python_plugin: Total time 38.1528s; user time: 28.0400s; system time: 35.6160s
2018-01-31 15:44:26 DEBUG node.lofar.ap.dias.ie.python_plugin: Start time was 1517413427.9354s; end time was 1517413466.0883s
2018-01-31 15:44:27 DEBUG facetselfcal_facet_patch_259.executable_args:
2018-01-31 15:44:27 WARNING facetselfcal_facet_patch_259.executable_args:
2018-01-31 15:44:27 INFO facetselfcal_facet_patch_259.executable_args: Subprocess completed with exit status 1: /bin/sh -c python /usr/lib/python2.7/dist-packages/lofarpipe/recipes/nodes/python_plugin.py 0 127.0.1.1 58478
2018-01-31 15:44:27 ERROR facetselfcal_facet_patch_259.executable_args: Remote process python /usr/lib/python2.7/dist-packages/lofarpipe/recipes/nodes/python_plugin.py ['/mnt/big-data/antonfj/DGTau/factor/results/facetselfcal/facet_patch_259/L192022_SB000_uv.dppp_1239ECF9Dt_117MHz.pre-cal_chunk0_1239ECF9Dt_0g.merge_amp_parmdbs2', '/home/antonfj/.local/lib/python2.7/site-packages/FACTOR-1.3-py2.7.egg/factor/scripts/smooth_amps_spline.py', ['/mnt/big-data/antonfj/DGTau/factor/results/facetselfcal/facet_patch_259/L192022_SB000_uv.dppp_1239ECF9Dt_117MHz.pre-cal_chunk0_1239ECF9Dt_0g.merge_amp_parmdbs2', '/mnt/big-data/antonfj/DGTau/factor/results/facetselfcal/facet_patch_259/L192022_SB000_uv.dppp_1239ECF9Dt_117MHz.pre-cal_chunk0_1239ECF9Dt_0g.smooth_amp2'], {}, '/mnt/big-data/antonfj/DGTau/factor/results/facetselfcal/facet_patch_259', False, {'args_format_option_argument': '=', 'args_format_option': '-', 'args_format': 'gnu', 'args_formatlongoption': '--', 'args_format_argument': ''}, {'LOFARDATAROOT': '/opt/lofar/data', 'OMP_NUM_THREADS': '8', 'PYTHONPATH': '/usr/lib/python2.7/dist-packages', 'LOFARROOT': '/usr', 'PATH': '/usr/sbin:/usr/bin:/home/antonfj/bin:/home/antonfj/.local/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/games', 'LD_LIBRARY_PATH': '/usr/lib:/mnt/big-data/antonfj/LOFAR_software/lib'}] failed on localhost
2018-01-31 15:44:27 DEBUG facetselfcal_facet_patch_259.executable_args: compute.dispatch results job 0: job_duration: 39.3189930916, returncode: 1
2018-01-31 15:44:28 DEBUG facetselfcal_facet_patch_259.executable_args: Adding node_logging_information
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259.executable_args: A job has failed with returncode 1 and error_tolerance is not set. Bailing out!
2018-01-31 15:44:28 WARNING facetselfcal_facet_patch_259.executable_args: Note: recipe outputs are not complete
2018-01-31 15:44:28 WARNING facetselfcal_facet_patch_259.executable_args: recipe executable_args completed with errors
2018-01-31 15:44:28 WARNING facetselfcal_facet_patch_259: smooth_amps_spline reports failure (using executable_args recipe)
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: *******************************************
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: Failed pipeline run: facet_patch_259
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: Detailed exception information:
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: <class 'lofarpipe.support.lofarexceptions.PipelineRecipeFailed'>
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: smooth_amps_spline failed
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: *******************************************
2018-01-31 15:44:28 ERROR facetselfcal_facet_patch_259: LOFAR Pipeline finished unsuccesfully.
2018-01-31 15:44:28 WARNING facetselfcal_facet_patch_259: recipe facetselfcal_facet_patch_259 completed with errors