monte carlo: get_length

[RachaelDMueller]

General Monte Carlo overview plan is here

Function: get_length
Input: month, vessel_type, lat, lon
Select: Use vessel type and month to select shapefile (made by Cam) from set of ~108 shapefiles (12 months x 9 vessel types). Use lat/lon with circle or bounding box to select a set of vessel traffic line segments. Each shapefile will include the following attributes:

• MMSI (vessel ID)
• Vessel type (atb, barge, tanker, cargo, cruise, ferry, small passenger, fishing, other)
• Timestamp
• Vessel length

For the barge shapefiles: Add oil cargo vs. non-oil cargo
R4: Randomly select ship track from those within a bounding box, weighted by vessel time exposure. Use information from shapefile to define characteristics of that ship track.
Decide whether spill type is fuel or cargo if vessel type is atb, tanker, or oil barge. Use this decision on spill type to select from either tank or non-tank oil capacity and oil type.
Output: vessel length (m), origin/destination for tank traffic only oil type

[RachaelDMueller]

First draft is done. Development notebook is here and an un-tested python script is here.

A couple discussion points:

• @campower, the length value I got for my test case is 16 m. Am I using the right attribute in defining vessel length from “.LENGTH”? If so...we need to consider what to do with a 16 m cargo ship. I suggest we assume that this is an erroneous length entry but instead of throwing out the ship track we just set a min length for all vessel types. It’s also possible that I did something wrong (me? never!), so I’ll double check that possibility on my end too.
• I use the first lat/lon pair from the line segment set in order to grab poly lines within a certain radius, which I have set to 0.5 km. Any reason why I need to consider using the second lat/lon pair from the set? (hint: say ’no’)

[RachaelDMueller]

@doug, I have updated the monte_carlo notebook and python script. The script works, but these are issues that I see as needing to still be addressed:

• Consider whether we want to set min/max vessel lengths for different vessels (motivated by a result of a 16 m long cargo vessel) OR if we just want to set min/max spill volume capacities. My current plan is to do the latter and keep track of how many upper/lower bounds are imposed
• Error check for case of no ship tracks within 0.5 km radius,
• Efficiency upgrade for lines 31-32. I currently read in all lat and lon values to find values w/in a certain distance, but this method is time consuming! The script takes ~45 s. on my mac to run.

Also, we only have 1 cargo shapefile to test with right now. @douglatornell, where shall I/we put the shape files when Cam has finished them? (ETA Monday)

[douglatornell]

@RachaelDMueller How big are the cargo shapefiles? How many of them will there be?

[RachaelDMueller]

That’s a good question @douglatornell. The shapefile itself is 26.1 MB; however, my understanding of a shapefile is that it requires information in the complimentary files and the suite of these files is 203.7 MB. We aren’t using projection information, but I will still need to test whether a stand-alone shapefile can be opened without error with geopandas.

As for quantity, we will have 108 shapefiles (1 file for 12 months and per 9 vessel types)

[douglatornell]

Okay @RachaelDMueller , those size number definitely rule out a repo that we push to GitHub!

So, let's use /data/MIDOSS/. You can create whatever directory structure makes sense there - maybe starting with something like /data/MIDOSS/cargo-shapefiles/.

[RachaelDMueller]

Indeed! If these files took on a shape, it would be a big one!

I tested whether all files are necessary by placing the .shp file into a separate folder and running my script from that folder. The first error asked for the .shx file, which I transferred before re-running. The error that shows up now is that there is no length attribute. I have no idea which file 6 remaining files has the length attribute. If size is a problem then I can hash this problem out and see which files I can throw-away; however, we may still want to keep all the files together so that these shapefiles can be imported to Arc and other GIS platforms.

I don’t see a problem with 203.7 * 108 = 22 GB on /data, so I say we go the route of not spending time hashing out what’s needed and just keeping all information. Yes?

@campower, 22 GB will be a problem if we use our shared google drive to as the share mechanism. Do you have an ftp or similar option for sharing? Perhaps NextCloud?

[RachaelDMueller]

@campower, I can help with NextCloud if needed.

[campower]

We do have an ftp! I'll check in with Ryan regarding whether this is the best option for sharing. Who will require access?

[RachaelDMueller]

Just me, @campower. I’ll ftp directly to /data/MIDOSS/. It’s easy for @douglatornell and @SusanEAllen to access files here. @douglatornell, do you prefer a different folder for each vessel type or just one “shapefile” folder?

[douglatornell]

Whatever folder structure makes sense for get_length() is fine with me, @RachaelDMueller .

[campower]

@RachaelDMueller FYI the maximum length of tankers in the study area is 287 meters.

[RachaelDMueller]

Fantastic! Thanks, @campower! This tells me that there are Suez-Max size tankers used for U.S. oil traffic. These have a bigger capacity then Aframax. I’ll set the max spill volume size to 159,000 m3 to reflect Suez-Max maximum cargo size.