Source interactive tool

[Marie59]

Hi Claudia ! I am testing the notebook locally through a docker image with a jupyterlab and I noticed some small issues that need fixing.

3. Download the index files

Run the next cell to download the index files:

user=getpass.getuser()
Local_dir='Users'+user+'/INPUT_DATA/data/index_files'
if not os.path.exists(Local_dir):
    os.makedirs(Local_dir)
    
getIndexFiles(CMEMS_user,CMEMS_password,Local_dir,dataset)

Here the function getIndexFiles needs only 3 arguments.

info = getIndexFilesInfo(CMEMS_user, CMEMS_password, dataset, Local_dir, targeted_bbox)

Here getIndexFilesInfo needs only 4 arguments.

Maybe there's a wrong copy paste. I did not continue further down the notebook maybe this error is present afterwards also.

I suggest that concerning source interactive tool we exchange here. Does it seem okay for you ?

[cfrat74]

Dear Marie
since I'm testing the notebook both on my local machine and on my PC I got the old version of the download_CMEMS.py. I will replace it on the drive so you can access to it.

[Marie59]

Hello @cfrat74, I have the tool up and running locally ! I am going further in your notebook to make sure everything is ok. I have an error message in de part 2 of "Observations module" when running

SOURCE.obs_postpro.insitu_tac_pre_processing.insitu_tac_pre_processing(InDir,InFields,WorkDir,PreDir,sel_QF,pr_mode,str_date,end_date,Reg,True)

The error is the following

`---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[19], line 1
----> 1 SOURCE.obs_postpro.insitu_tac_pre_processing.insitu_tac_pre_processing(InDir,InFields,WorkDir,PreDir,sel_QF,pr_mode,str_date,end_date,Reg,True)

File /opt/conda/lib/python3.10/site-packages/SOURCE/obs_postpro/insitu_tac_pre_processing.py:297, in insitu_tac_pre_processing(in_dir, in_fields_standard_name_str, work_dir, out_dir, valid_qc_values, update_mode, first_date_str, last_date_str, region_boundaries_str, med_sea_masking, in_instrument_types_str, names_file, verbose)
    294 else:
    295     in_instrument_types_list = None
--> 297 file_list = [in_dir + '/' + file for file in os.listdir(in_dir) if file.endswith('.nc')]
    298 if not file_list:
    299     time.sleep(sleep_time)

FileNotFoundError: [Errno 2] No such file or directory: '/Users/root/INPUT_DATA/OBSERVATION/MO'

Any ideas on how to fix that ?
Thanks !

[cfrat74]

Hello @Marie59 did you create MO directory where observations must be downloaded ?

[Marie59]

I tried that but then it tells me that Error. No processable files in input directory. . But, shouldn't the folder be automatically created when running the notebook ? It would be easier I think.

[cfrat74]

Yes in the downloading section (9) of the SOURCE INPUT DATA it has created. Check if in this directory you have effectively downloaded some files...

[Marie59]

okay I think I found the issue for that I will do the changes on the notebook. Now I am getting a new error (still at the same point)

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[48], line 1
----> 1 SOURCE.obs_postpro.insitu_tac_pre_processing.insitu_tac_pre_processing(InDir,InFields,WorkDir,PreDir,sel_QF,pr_mode,str_date,end_date,Reg,True)

File /opt/conda/lib/python3.10/site-packages/SOURCE/obs_postpro/insitu_tac_pre_processing.py:329, in insitu_tac_pre_processing(in_dir, in_fields_standard_name_str, work_dir, out_dir, valid_qc_values, update_mode, first_date_str, last_date_str, region_boundaries_str, med_sea_masking, in_instrument_types_str, names_file, verbose)
    326     time.sleep(sleep_time)
    327     print(' -------------------------')
--> 329 url_data = open(os.path.dirname(__file__) + '/url_countries.csv', 'rb')
    330 url_data = \
    331     pd.read_csv(url_data, na_filter=False, dtype=object, quotechar='"', delimiter=',').values
    332 if url_data.ndim == 1:

FileNotFoundError: [Errno 2] No such file or directory: '/opt/conda/lib/python3.10/site-packages/SOURCE/obs_postpro/url_countries.csv'

I will try to find the issue here but if you have any idea @cfrat74 don't hesitate.