2.11.0

docs/pages/api-reference.md

@@ -255,6 +255,37 @@ def load_ggg2020_vmr(filepath: str) -> pl.DataFrame
 Load the Atmospheric profile from a GGG2020 vmr file.
 
 
+## `tum_esm_utils.dataframes`
+
+Dataframe-related utility functions.
+
+Implements: `fill_df_time_gaps_with_nans`
+
+This requires you to install this utils library with the optional `polars` dependency:
+
+```bash
+pip install "tum_esm_utils[polars]"
+## `or`
+pdm add "tum_esm_utils[polars]"
+```
+
+
+##### `fill_df_time_gaps_with_nans`
+
+```python
+def fill_df_time_gaps_with_nans(df: pl.DataFrame, time_col: str,
+                                max_gap_seconds: int) -> pl.DataFrame
+```
+
+Fill time gaps in a dataframe with NaN rows. This is very useful for plotting dataframes where time gaps should be visible.
+
+**Arguments**:
+
+- `df` - The input dataframe.
+- `time_col` - The name of the time column.
+- `max_gap_seconds` - The maximum gap in seconds to fill with NaN rows.
+
+
 ## `tum_esm_utils.datastructures`
 
 Datastructures not in the standard library.
@@ -812,6 +843,180 @@ can lead to floating point errors, i.e. `1 % 0.1 == 0.09999999999999998`.
 Using `math.fmod` also does not seem to work correctly with floats.
 
 
+## `tum_esm_utils.netcdf`
+
+A thin wrapper over the netCDF4 library to make working with NetCDF files easier.
+
+Implements: `NetCDFFile`, `remove_elements_from_netcdf_file`, `compress_netcdf_file`.
+
+This requires you to install this utils library with the optional `netcdf` dependencies:
+
+```bash
+pip install "tum_esm_utils[netcdf]"
+## `or`
+pdm add "tum_esm_utils[netcdf]"
+```
+
+
+### `NetCDFFile` Objects
+
+```python
+class NetCDFFile()
+```
+
+
+##### `__init__`
+
+```python
+def __init__(filepath: str,
+             parallel: bool = False,
+             diskless: bool = True,
+             mode: Literal["w", "a", "r"] = "r") -> None
+```
+
+A simple wrapper around netCDF4.Dataset to make the interaction with NetCDF files easier.
+
+If writing to a new file, it will first write to the filepath+ ".tmp" and  rename it to the final
+filepath when closing the file. This ensures that the final filepath will only exist if the file
+was written completely. In append mode, the filepath is not changes.
+
+
+##### `create_dimension`
+
+```python
+def create_dimension(name: str, size: int) -> None
+```
+
+Create a new dimension in the NetCDF file.
+
+**Raises**:
+
+- `ValueError` - If the dimension already exists
+- `RuntimeError` - If the NetCDF file is not opened in write mode.
+
+
+##### `create_variable`
+
+```python
+def create_variable(name: str,
+                    dimensions: tuple[nc.Dimension | str, ...],
+                    units: str,
+                    long_name: Optional[str] = None,
+                    description: Optional[str] = None,
+                    fill_value: Optional[float | int] = None,
+                    chunk_dimensions: list[str] = [],
+                    datatype: Literal["f4", "f8", "i4", "i8"] = "f4",
+                    zlib: bool = True,
+                    compression_level: int = 2) -> None
+```
+
+Create a new variable in the NetCDF file.
+
+**Raises**:
+
+- `ValueError` - If the variable already exists or if a dimension is not found.
+- `RuntimeError` - If the NetCDF file is not opened in write mode.
+
+
+##### `import_dimension`
+
+```python
+def import_dimension(dimension: nc.Dimension,
+                     new_name: Optional[str] = None) -> None
+```
+
+Import a dimension from another NetCDF file.
+
+**Raises**:
+
+- `ValueError` - If the dimension already exists.
+- `RuntimeError` - If the NetCDF file is not opened in write mode.
+
+
+##### `import_variable`
+
+```python
+def import_variable(variable: "nc.Variable[Any]",
+                    new_name: Optional[str] = None,
+                    zlib: bool = True,
+                    compression_level: int = 2) -> None
+```
+
+Import a variable from another NetCDF file.
+
+**Raises**:
+
+- `ValueError` - If the variable already exists.
+- `RuntimeError` - If the NetCDF file is not opened in write mode.
+
+
+##### `add_attribute`
+
+```python
+def add_attribute(key: str, value: str, allow_overwrite: bool = False) -> None
+```
+
+Add a global attribute to the NetCDF file.
+
+**Raises**:
+
+- `ValueError` - If the attribute already exists and `allow_overwrite` is False.
+- `RuntimeError` - If the NetCDF file is not opened in write mode.
+
+
+##### `close`
+
+```python
+def close() -> None
+```
+
+Close the NetCDF file, possibly renaming the temporary file to the final filepath.
+
+
+##### `__getitem__`
+
+```python
+def __getitem__(key: str) -> "nc.Variable[Any]"
+```
+
+Get a variable from the NetCDF file.
+
+
+##### `remove_elements_from_netcdf_file`
+
+```python
+def remove_elements_from_netcdf_file(source_filepath: str,
+                                     destination_filepath: str,
+                                     variables_to_remove: list[str] = [],
+                                     dimensions_to_remove: list[str] = [],
+                                     attributes_to_remove: list[str] = [],
+                                     compression_level: int = 2) -> None
+```
+
+Create a new NetCDF file by copying an existing one, but removing specified variables, dimensions, and attributes. This is useful because NetCDF4 does not support removing elements from an existing file.
+
+**Raises**:
+
+- `FileNotFoundError` - If the source file does not exist.
+- `FileExistsError` - If the destination file already exists.
+
+
+##### `compress_netcdf_file`
+
+```python
+def compress_netcdf_file(source_filepath: str,
+                         destination_filepath: str,
+                         compression_level: int = 2) -> None
+```
+
+Compress an existing NetCDF file by creating a new one with the specified compression level. This is useful because some NetCDF4 files given to you might not be (very well) compressed.
+
+**Raises**:
+
+- `FileNotFoundError` - If the source file does not exist.
+- `FileExistsError` - If the destination file already exists.
+
+
 ## `tum_esm_utils.opus`
 
 Functions for interacting with OPUS files.
@@ -1376,6 +1581,41 @@ terminated forcefully after the given timeout (in seconds).
   The list of terminated PIDs.
 
 
+## `tum_esm_utils.rebinning`
+
+Functions to rebin binned data poins
+
+Implements: `rebin_1d`, `rebin_2d`.
+
+This requires you to install this utils library with the optional `modeling` dependency:
+
+```bash
+pip install "tum_esm_utils[modeling]"
+## `or`
+pdm add "tum_esm_utils[modeling]"
+```
+
+
+##### `rebin_1d`
+
+```python
+def rebin_1d(arr: np.ndarray[Any, Any],
+             new_bin_count: int) -> np.ndarray[Any, Any]
+```
+
+Rebins a 1D array to a new number of bins.
+
+
+##### `rebin_2d`
+
+```python
+def rebin_2d(arr: np.ndarray[Any, Any], new_x_bins: int,
+             new_y_bins: int) -> np.ndarray[Any, Any]
+```
+
+Rebins a 2D array to new number of bins in x and y dimensions.
+
+
 ## `tum_esm_utils.shell`
 
 Implements custom logging functionality, because the
@@ -2112,6 +2352,61 @@ with timed_section("my_section"):
 ```
 
 
+##### `datetime_to_julian_day_number`
+
+```python
+def datetime_to_julian_day_number(
+        dt: datetime.datetime, variant: Literal["JDN", "MJD",
+                                                "MJD2K"]) -> float
+```
+
+Convert a datetime to a Julian Day Number (JDN) or MJD/MJD2K.
+
+The Julian Day Number (JDN) is the continuous count of days since the beginning
+of the Julian Period on January 1, 4713 BC. THe modified variant MJD starts
+counting from November 17, 1858 at 00:00:00 UTC, and MJD2K starts counting
+from January 1, 2000 at 00:00:00 UTC.
+
+**Arguments**:
+
+- `dt` - The datetime to convert.
+- `variant` - The variant of the Julian Day Number ("JDN", "MJD", "MJD2K").
+
+
+**Returns**:
+
+  The Julian Day Number as a float.
+
+
+##### `julian_day_number_to_datetime`
+
+```python
+def julian_day_number_to_datetime(
+        jdn: float, variant: Literal["JDN", "MJD",
+                                     "MJD2K"]) -> datetime.datetime
+```
+
+Convert a Julian Day Number (JDN) or MJD/MJD2K to a datetime.
+
+The Julian Day Number (JDN) is the continuous count of days since the beginning
+of the Julian Period on January 1, 4713 BC. THe modified variant MJD starts
+counting from November 17, 1858 at 00:00:00 UTC, and MJD2K starts counting
+from January 1, 2000 at 00:00:00 UTC.
+
+This function was validated against
+https://ssd.jpl.nasa.gov/tools/jdc/#/cd
+
+**Arguments**:
+
+- `jdn` - The Julian Day Number to convert.
+- `variant` - The variant of the Julian Day Number ("JDN", "MJD", "MJD2K").
+
+
+**Returns**:
+
+  The corresponding datetime.
+
+
 ## `tum_esm_utils.validators`
 
 Implements validator utils for use with pydantic models.

docs/scripts/sync-docs.py

@@ -25,16 +25,19 @@
     "column.astronomy",
     "column.averaging_kernel",
     "column.ncep_profiles",
+    "dataframes",
     "datastructures",
     "decorators",
     "em27",
     "files",
     "mathematics",
+    "netcdf",
     "opus",
     "opus.file_interface",
     "opus.http_interface",
     "plotting",
     "processes",
+    "rebinning",
     "shell",
     "sqlitelock",
     "system",