Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
4078d8c
Implement ERA5PressureDataset, ERA5HourlyPressureLevels, ERA5MonthlyP…
ewquon Mar 3, 2026
54a2a0c
Add ERA5*PressureLevels exports
ewquon Mar 3, 2026
8fede16
Add InverseGravity for geopotential height calc
ewquon Mar 3, 2026
13e4fab
Add ERA5 pressure-levels downloader
ewquon Mar 3, 2026
afdd227
Add CDS tests, verified 95/95 pass
ewquon Mar 3, 2026
2cb7141
Merge branch 'main' into eq/era5_pressure_levels
ewquon Mar 3, 2026
c09f6da
Fix imports
ewquon Mar 3, 2026
9c48937
Add day, hour to date_str for filenaming
ewquon Mar 3, 2026
aec39c8
Allow simultaneous download of multiple variables
ewquon Mar 3, 2026
03efebc
Aggregate hourly downloads into API requests per calendar day
ewquon Mar 3, 2026
f1de9e5
Cleanup file naming
ewquon Mar 4, 2026
916614e
Cleanup, clarify
ewquon Mar 4, 2026
8f42dee
Allow one or more variables to be downloaded, at one or more datetime…
ewquon Mar 5, 2026
3cac1f8
Add cleanup flag for multi-datetime downloads
ewquon Mar 5, 2026
c56c6fc
Fix silly Claude vertical-level flip; no inpainting needed for ERA5
ewquon Mar 9, 2026
e552ccb
Fix handling of ERA5 latitude grid
ewquon Mar 9, 2026
fce1404
Rename original "ERA5*" metadata to "ERA5SingleLevels*"
ewquon Mar 9, 2026
4643ffa
Fix ERA5 grid creation
ewquon Mar 10, 2026
9089c99
Fix bbox handling
ewquon Mar 10, 2026
cf2b8e0
Use standard units for pressure_levels
ewquon Mar 10, 2026
4139a6e
No special treatment of lower z_interface, allow z < 0
ewquon Mar 10, 2026
c45200a
Fix defaults
ewquon Mar 11, 2026
18cba4c
Implement mean_geopotential_z_interfaces; make mutable to allow z update
ewquon Mar 11, 2026
ae0f0b1
Organize ERA5 code into single-level and pressure-levels source files
ewquon Mar 11, 2026
c5c2e2f
Use consistent ERA5 naming
ewquon Mar 11, 2026
77d33f3
Automatic geopotential download to calculate mean geopotential heights
ewquon Mar 11, 2026
5aa675b
Streamline automatic download
ewquon Mar 11, 2026
79dcb17
Refactor; add examples
ewquon Mar 12, 2026
3b44aba
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Mar 17, 2026
577b3c4
Address a very very very minor point
ewquon Mar 17, 2026
f4afe09
Retrieve var name from correct dict
ewquon Mar 17, 2026
c28db4f
Add additional surface variables
ewquon Mar 17, 2026
74cb622
Improved readability
ewquon Mar 17, 2026
ad135e3
Follow Oceananigans convention
ewquon Mar 17, 2026
b6baf64
Aesthetics
ewquon Mar 19, 2026
a0206b2
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Mar 19, 2026
7cc877e
Handle download of instantaneous and accumulated/mean quantities
ewquon Mar 23, 2026
567101b
Remove redundant duplicate date when requesting ERA5 single-level hou…
ewquon Mar 24, 2026
48a6b3e
Add mean evaporation rate aka "time-mean moisture flux" [kg m⁻² s⁻¹]
ewquon Mar 24, 2026
9878169
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Mar 24, 2026
0eb2c78
Most robustly handle situations with multiple data types in one request
ewquon Mar 25, 2026
7b7d1c7
Better handle geopotential height variability
ewquon Mar 25, 2026
ae169b9
Allow use of pre-downloaded ERA5 data without depending on CDSAPI
ewquon Mar 26, 2026
4afc729
Fix bug introduced by prev commit; add additional era5 vars
ewquon Mar 28, 2026
d21e0be
Merge branch 'main' of https://github.com/NumericalEarth/NumericalEar…
ewquon Apr 20, 2026
0960d35
Minor cleanup
ewquon Apr 21, 2026
d6dc0ca
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Apr 28, 2026
50cc57d
Fix ERA5 Column grid test to use renamed type
ewquon Apr 29, 2026
294db53
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Apr 29, 2026
bb099f0
Simplify pressure_field to a reduced column Field
ewquon Apr 29, 2026
575b52d
Temporarily define hPa
ewquon Apr 29, 2026
a597c60
Cleanup CDS download test
ewquon Apr 29, 2026
be9657c
Share CDS test fixtures across testsets to avoid redundant downloads
ewquon Apr 30, 2026
6e6e84c
Support Column regions for ERA5 CDS downloads
ewquon Apr 30, 2026
c7ad02d
Add network-free tests for CDSAPIExt dispatch and NetCDF helpers
ewquon Apr 30, 2026
f70a99c
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Apr 30, 2026
5eacbc7
Cleanup
ewquon Apr 30, 2026
eeb699a
Add unit-level tests for ERA5 single-level helpers
ewquon Apr 30, 2026
445f493
Add tests for calendar-day grouping and skip_existing short-circuit
ewquon Apr 30, 2026
e8a77a3
Merge branch 'main' of github.com:NumericalEarth/NumericalEarth.jl in…
ewquon Apr 30, 2026
30b9560
Add unit tests for pure helpers in metadata_field, ERA5_pressure_leve…
ewquon Apr 30, 2026
bdc98d7
Apply suggestion from @glwagner
ewquon Apr 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ Breeze = "0.4"
CDSAPI = "2.2.1"
CFTime = "0.1, 0.2"
ClimaSeaIce = "0.4.4, 0.5"
CubedSphere = "0.3.4"
CondaPkg = "0.2.33"
CopernicusMarine = "0.1.1"
CubedSphere = "0.3.4"
DataDeps = "0.7"
Dates = "<0.0.1, 1"
Distances = "0.10"
Expand Down
120 changes: 120 additions & 0 deletions examples/ERA5_bounded_pressure_level_data.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Download 3-D snapshots of the atmospheric state from ERA5, e.g., to be used with FieldTimeSeries
#
# ## Install dependencies
#
# ```julia
# using Pkg
# pkg"add NumericalEarth CDSAPI"
# ```
#
# You also need CDS API credentials in `~/.cdsapirc`.
# See <https://cds.climate.copernicus.eu/how-to-api> for setup instructions.
#
# See <https://cds.climate.copernicus.eu/datasets/reanalysis-era5-pressure-levels?tab=download> for
# more information about pressure-level data

using NumericalEarth
using NumericalEarth.DataWrangling: Metadata, BoundingBox, download_dataset
using NumericalEarth.DataWrangling.ERA5
using CDSAPI
using Dates
using Oceananigans

using Statistics
using CairoMakie

selected_levels = filter(≥(250hPa), ERA5_all_pressure_levels) # select all levels below 250 hPa
dataset = ERA5HourlyPressureLevels(selected_levels)

dates = DateTime(2004, 12, 16):Hour(1):DateTime(2005, 01, 09) # van Zanten et al 2011 study period

# Rauber et al 2007, Fig 1: focus region
region = BoundingBox(latitude=(17, 18.5), longitude=(-62.5, -61))

# OPTIONAL: download all variables at once (fewer CDS API requests)
variables = [:geopotential, # for calculating zlevels as the mean geopotential height
:eastward_velocity,
:northward_velocity,
:temperature,
:specific_humidity]
download_dataset(variables, dataset, dates; region)

# ## Create time series

T_meta = Metadata(:temperature; dataset, dates, region)
q_meta = Metadata(:specific_humidity; dataset, dates, region)
u_meta = Metadata(:eastward_velocity; dataset, dates, region)
v_meta = Metadata(:northward_velocity; dataset, dates, region)

T_series = FieldTimeSeries(T_meta; time_indices_in_memory=2)
q_series = FieldTimeSeries(q_meta; time_indices_in_memory=2)
u_series = FieldTimeSeries(u_meta; time_indices_in_memory=2)
v_series = FieldTimeSeries(v_meta; time_indices_in_memory=2)

# ## Vertical profiles (mean ± spread)

# Height coordinate (m) from the grid
z = znodes(T_series[1])
Nz = length(z)
Nt = length(T_series.times)

# Pressure at each level (hPa), ordered bottom-to-top (k=1 ⇒ highest pressure)
p_levels = sort(selected_levels, rev=true) ./ hPa # Pa → hPa

# Horizontal-mean profile for each snapshot → (Nz, Nt) arrays
function horizontal_mean_profiles(series)
profiles = zeros(Nz, Nt)
for n in 1:Nt
data = interior(series[n], :, :, :)
profiles[:, n] = mean(data, dims=(1, 2))
end
return profiles
end
Comment thread
ewquon marked this conversation as resolved.

T_profiles = horizontal_mean_profiles(T_series)
q_profiles = horizontal_mean_profiles(q_series)
u_profiles = horizontal_mean_profiles(u_series)
v_profiles = horizontal_mean_profiles(v_series)

# Convert T → potential temperature: θ = T (p₀/p)^(R/cₚ)
Rₐ_over_cₚ = 0.286
θ_profiles = T_profiles .* (1000 ./ p_levels) .^ Rₐ_over_cₚ

# Convert specific humidity from kg/kg → g/kg
q_profiles .*= 1000

# ## Plot

fig = Figure(size=(900, 500), fontsize=12)

ax_θ = Axis(fig[1, 1], xlabel="θ [K]", ylabel="Height [m]")
ax_q = Axis(fig[1, 2], xlabel="qᵥ [g kg⁻¹]", ylabel="Height [m]")
ax_u = Axis(fig[1, 3], xlabel="u [m s⁻¹]", ylabel="Height [m]")
ax_v = Axis(fig[1, 4], xlabel="v [m s⁻¹]", ylabel="Height [m]")

for (ax, profiles) in [(ax_θ, θ_profiles),
(ax_q, q_profiles),
(ax_u, u_profiles),
(ax_v, v_profiles)]
μ = vec(mean(profiles, dims=2))
#lo = vec(minimum(profiles, dims=2))
#hi = vec(maximum(profiles, dims=2))
lo = [quantile(r, 0.25) for r in eachrow(profiles)]
hi = [quantile(r, 0.75) for r in eachrow(profiles)]
band!(ax, z, lo, hi; direction=:y, color=(:gray, 0.4))
lines!(ax, μ, z; color=:black, linewidth=2)
end

xlims!(ax_θ, 293, 317)
xlims!(ax_q, 0, 20.4)
xlims!(ax_u, -10, 0)
xlims!(ax_v, -10, 0)

linkyaxes!(ax_θ, ax_q, ax_u, ax_v)
ylims!(ax_θ, 0, 4000)

hideydecorations!(ax_q)
hideydecorations!(ax_u)
hideydecorations!(ax_v)

save("ERA5_pressure_level_profiles.png", fig; px_per_unit=4)
92 changes: 92 additions & 0 deletions examples/ERA5_bounded_surface_data.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Download 3-D snapshots of the atmospheric state from ERA5, e.g., to be used with FieldTimeSeries
#
# ## Install dependencies
#
# ```julia
# using Pkg
# pkg"add NumericalEarth CDSAPI"
# ```
#
# You also need CDS API credentials in `~/.cdsapirc`.
# See <https://cds.climate.copernicus.eu/how-to-api> for setup instructions.
#
# See <https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels?tab=download> for
# more information about single-level data

using NumericalEarth.DataWrangling: Metadata, BoundingBox, download_dataset
using NumericalEarth.DataWrangling.ERA5
using CDSAPI
using Dates
using Oceananigans

using Statistics: mean
using CairoMakie
#precip_cmap = :rain
precip_cmap = cgrad([:indigo, :darkblue, :blue, :deepskyblue, :cyan,
:palegreen, :green, :yellow, :orange, :red, :pink])


dataset = ERA5HourlySingleLevel()

#dates = DateTime(2004, 12, 16):Hour(1):DateTime(2005, 01, 09) # van Zanten et al 2011 study period
dates = DateTime(2004, 12, 16):Hour(1):DateTime(2004, 12, 23) # shorter time range for demo

# Rauber et al 2007, Fig 1: precip map
region = BoundingBox(latitude=(-25, 35), longitude=(-110, 30))

# OPTIONAL: download all variables at once (fewer CDS API requests)
# variables = [:total_precipitation,
# :sea_surface_temperature,
# :temperature, # at 2 m
# :dewpoint_temperature, # at 2 m
# :surface_pressure]
# download_dataset(variables, dataset, dates; region)

# ## Create time series

precip_meta = Metadata(:total_precipitation; dataset, dates, region)
precip_series = FieldTimeSeries(precip_meta)

# ## Plotting

Nt = length(precip_series.times)
λ, φ, _ = nodes(precip_series[1])

# ERA5 total_precipitation is in metres per hour; convert to mm/day
to_mm_day = 1000 * 24

# ### Time-averaged precipitation map

precip_avg = mean(interior(precip_series[n], :, :, 1) for n in 1:Nt) .* to_mm_day

fig1 = Figure(size=(900, 400))
ax1 = Axis(fig1[1, 1],
title = "Mean precipitation (2004-12-16 to 2005-01-08)",
xlabel = "Longitude (°)",
ylabel = "Latitude (°)")
ax1.xticks = -90:30:30
hm = heatmap!(ax1, λ, φ, precip_avg; colormap=precip_cmap, colorrange=(0, 12))
Colorbar(fig1[1, 2], hm, label="Precipitation (mm/day)")
save("ERA5_mean_precipitation.png", fig1)

# ### Precipitation animation

fig2 = Figure(size=(900, 400))

n = Observable(1)
precip_n = @lift interior(precip_series[$n], :, :, 1) .* to_mm_day
anim_title = @lift "Precipitation (mm/day), " * string(first(dates) + Second(round(Int, precip_series.times[$n])))

ax2 = Axis(fig2[1, 1],
title = anim_title,
xlabel = "Longitude (°)",
ylabel = "Latitude (°)")
ax2.xticks = -90:30:30

hm2 = heatmap!(ax2, λ, φ, precip_n; colormap=precip_cmap, colorrange=(0, 12))
Colorbar(fig2[1, 2], hm2, label="Precipitation (mm/day)")

record(fig2, "ERA5_precipitation.mp4", 1:Nt; framerate=12) do nn
n[] = nn
end

6 changes: 3 additions & 3 deletions examples/ERA5_winds_and_stokes_drift.jl
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

using NumericalEarth
using NumericalEarth.DataWrangling: Metadatum
using NumericalEarth.DataWrangling.ERA5: ERA5Hourly
using NumericalEarth.DataWrangling.ERA5: ERA5HourlySingleLevel
using CDSAPI

using Oceananigans
Expand All @@ -29,7 +29,7 @@ using Dates
# while ocean wave variables (Stokes drift) live on a 0.5° grid (720×361).
# We define metadata for each variable at a single date.

dataset = ERA5Hourly()
dataset = ERA5HourlySingleLevel()
date = DateTime(2020, 1, 15, 12) # January 15, 2020 at 12:00 UTC

u_stokes_meta = Metadatum(:eastward_stokes_drift; dataset, date)
Expand All @@ -42,7 +42,7 @@ v_wind_meta = Metadatum(:northward_velocity; dataset, date)
# We build a single `LatitudeLongitudeGrid` and use `set!` to download
# and interpolate all four variables onto it.

grid = LatitudeLongitudeGrid(size = (1440, 721, 1),
grid = LatitudeLongitudeGrid(size = (1440, 720, 1),
longitude = (0, 360),
latitude = (-90, 90),
z = (0, 1))
Expand Down
Loading
Loading