analysis-ashutosh/journal.txt at master · MIDOSS/analysis-ashutosh · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
19 Feb 2019

    Replicate Shihan's run using files from nextcloud
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    Purpose: To determine of the model was chnaged since the intial run

        Download files from Nextcloud, upload to cedar
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            Create directory (/home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate)
            Write .yaml file pointing to all relevant input files
            Edit file paths in Atmosphere.dat, Hydrodynamic.dat

        Compile Mohid
        ^^^^^^^^^^^^^
            salloc --time=0:10:0 --cpus-per-task=1 --mem-per-cpu=1024m --account=def-allen
            cd $PROJECT/$USER/MIDOSS/MIDOSS-MOHID/Solutions/linux
            ./compile_mohid.sh -mb1 -mb2 -mw

            Delete all of the compiled objects, libraries, and executables:
            ./compile_mohid --clean
            so that next build is clean

        Run Mohid
        ^^^^^^^^^
            make run directory as set in yaml file ($SCRATCH/MIDOSS/runs/ashu_testing/)
            cd /home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate
            mohid run nextcloud_replicate.yaml $PROJECT/$USER/MIDOSS/results/ashu/nextcloud_replicate

        Outcome
        ^^^^^^^
            crashed with multiple error messages. will investigate.

20 Feb 2019

    Continue developing scripts to generate hdf5 input files for MOHID
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        Added w ocean velocity to input file generator function
        !! need to add date validation on input
        !! need to create local file for lat lon data for HRDPS and WW3 grids

    Replicate Shihan's run using files from nextcloud
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    Purpose: To determine of the model was chnaged since the intial run

        Download files from Nextcloud, upload to cedar
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            Create directory (/home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate)
            Write .yaml file pointing to all relevant input files
            Edit file paths in Atmosphere.dat, Hydrodynamic.dat

        Compile Mohid
        ^^^^^^^^^^^^^
            salloc --time=0:10:0 --cpus-per-task=1 --mem-per-cpu=1024m --account=def-allen
            cd $PROJECT/$USER/MIDOSS/MIDOSS-MOHID/Solutions/linux
            ./compile_mohid.sh -mb1 -mb2 -mw

            Delete all of the compiled objects, libraries, and executables:
            ./compile_mohid --clean
            so that next build is clean

        Run Mohid
        ^^^^^^^^^
            make run directory as set in yaml file ($SCRATCH/MIDOSS/runs/ashu_testing/)
            cd /home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate
            mohid run nextcloud_replicate.yaml $PROJECT/$USER/MIDOSS/results/ashu/nextcloud_replicate
            Job ID: 17103921

        Outcome
        ^^^^^^^
            crashed again. stderr output:
                ReadInitialImposedSolution  - ModuleHydrodynamic - ERR170
                Usage: hdf5-to-netcdf4 [OPTIONS] HDF5_FILE NETCDF4_FILE
                Try "hdf5-to-netcdf4 --help" for help.

                Error: Invalid value for "HDF5_FILE": Path "/scratch/abhudia/MIDOSS/runs/ashu_testing/nextcloud_replicate_2019-02-20T171143.680870-0800/res/Lagrangian_nextcloud_replicate.hdf5" does not exist.


21 Feb 2019

    Continue developing script to generate hdf5 input files for MOHID
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

        added salinity and temperature
        refactored script to generate file paths and produce hdf5 files in separate functions. added additional validation.
        added date validation on input

    Testing Script on Salish
    ^^^^^^^^^^^^^^^^^^^^^^^^

        SalishSeacast output directory /results2/MIDOSS/forcing/SalishSeaCast/ashu_testing/hdf5/01oct18-07oct18/ created
        HRDPS output directory /results2/MIDOSS/forcing/SalishSeaCast/ashu_testing/hrdps/01oct18-07oct18/ created
        WW3 output directory /results2/MIDOSS/forcing/SalishSeaCast/ashu_testing/ww3/01oct18-07oct18/ created

        Crashed on WW3. Output:
        Creating WW3 parameters file ... N/A% (0 of 7) |                                                          |ETA:  --:--:--Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "make-hdf5.py", line 1013, in init
            run_choice()
        File "make-hdf5.py", line 949, in run_choice
            run(runs)
        File "make-hdf5.py", line 48, in f
            rv      = func(*args, *kwargs)
        File "make-hdf5.py", line 972, in run
            create_ww3_hdf5(*ww3)
        File "make-hdf5.py", line 48, in f
            rv      = func(*args, *kwargs)
        File "make-hdf5.py", line 790, in create_ww3_hdf5
            mean_wave = np.expand_dims(griddata(points, mean_wave_array[0].ravel(), xi, method='cubic'),0)
        File "/home/abhudia/anaconda3/lib/python3.6/site-packages/scipy/interpolate/ndgriddata.py", line 206, in griddata
            raise ValueError("invalid number of dimensions in xi")
        ValueError: invalid number of dimensions in xi
        >>> mean_wave_array
        Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        NameError: name 'mean_wave_array' is not defined

22 Feb 2019
^^^^^^^^^^^
        Replicate Shihan's run using files from nextcloud
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
             Purpose: To determine of the model was chnaged since the intial run

        Download files from Nextcloud, upload to cedar
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            Create directory (/home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate)
            Write .yaml file pointing to all relevant input files
            Edit file paths in Atmosphere.dat, Hydrodynamic.dat

        Compile Mohid
        ^^^^^^^^^^^^^
            salloc --time=0:10:0 --cpus-per-task=1 --mem-per-cpu=1024m --account=def-allen
            cd $PROJECT/$USER/MIDOSS/MIDOSS-MOHID/Solutions/linux
            ./compile_mohid.sh -mb1 -mb2 -mw

            Delete all of the compiled objects, libraries, and executables:
            ./compile_mohid --clean
            so that next build is clean

        Run Mohid
        ^^^^^^^^^
            make run directory as set in yaml file ($SCRATCH/MIDOSS/runs/ashu_testing/)
            cd /home/abhudia/project/abhudia/MIDOSS/MIDOSS-MOHID-config/ashu_testing/nextcloud_replicate
            mohid run nextcloud_replicate.yaml $PROJECT/$USER/MIDOSS/results/ashu/nextcloud_replicate
            Job ID: 17187901

        Crashed again. This time the oil was spilled on land point.

        Second attempt:
        Used shihan's output file to dfigure out where the oil was spilt and got the lat lon.

25 Feb 2019
^^^^^^^^^^^
        Run MOHID
        ^^^^^^^^^
            in config:
            mohid run nextcloud_replicate.yaml $PROJECT/$USER/MIDOSS/results/ashu/nextcloud_replicate --no-submit
            in scratch:
            ln -s /project/6001313/dlatorne/MIDOSS/MIDOSS-MOHID/Solutions/linux/bin/MohidWater.exe
            made chnage to MOHID.sh
            sbatch MOHID.sh