Add Conditional Value at Risk (CVAR) metric#100
Add Conditional Value at Risk (CVAR) metric#100abdelrahman-ayad wants to merge 27 commits intomainfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #100 +/- ##
==========================================
+ Coverage 83.13% 83.57% +0.43%
==========================================
Files 45 45
Lines 2325 2466 +141
==========================================
+ Hits 1933 2061 +128
- Misses 392 405 +13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…capacity calculations
There was a problem hiding this comment.
Pull request overview
Adds a Conditional Value at Risk (CVAR) (and normalized NCVAR) reliability metric for tail-risk assessment of shortfalls, including capacity-shortfall variants, and updates simulation recording + tests accordingly.
Changes:
- Introduces
CVAR/NCVARmetric types (plus display/validation) and computes them forShortfallResultandShortfallSamplesResult. - Extends result recording/finalization to retain per-sample unserved energy and max capacity shortfall needed for CVAR.
- Adds/updates unit tests and reference values (including system test data) to cover CVAR/NCVAR behavior.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| PRASCore.jl/src/Results/metrics.jl | Adds CVAR/NCVAR metric structs and show methods; adjusts MeanEstimate for singleton samples. |
| PRASCore.jl/src/Results/Shortfall.jl | Stores per-sample unserved energy and adds CVAR/NCVAR computations for ShortfallResult. |
| PRASCore.jl/src/Results/ShortfallSamples.jl | Adds max capacity shortfall storage and CVAR/NCVAR computations for ShortfallSamplesResult. |
| PRASCore.jl/src/Results/Results.jl | Exports CVAR/NCVAR and adds broadcast convenience methods. |
| PRASCore.jl/src/Simulations/recording.jl | Records per-sample UE totals into the shortfall accumulator. |
| PRASCore.jl/src/Systems/TestData.jl | Adds expected CVAR reference values for simulation tests. |
| PRASCore.jl/test/Results/metrics.jl | Adds tests for CVAR/NCVAR formatting and domain checks. |
| PRASCore.jl/test/Results/shortfall.jl | Adds correctness tests for CVAR/NCVAR on results (overall/region/period). |
| PRASCore.jl/test/Simulations/runtests.jl | Adds simulation-level assertions for CVAR values. |
| PRASCore.jl/test/dummydata.jl | Adds dummy alpha/sample vectors for new result fields. |
| PRAS.jl/test/runtests.jl | Expands integration tests to validate CVAR/NCVAR return types for both result kinds. |
Comments suppressed due to low confidence (1)
PRASCore.jl/test/Results/shortfall.jl:43
- Typo in variable name:
cap_shortfalis missing anl(should becap_shortfall) to match intent and improve readability.
cap_shortfal = vec(reshape(result.capacity_shortfall_mean, 1, :))
var = quantile(cap_shortfal, alpha)
tail_losses = cap_shortfal[cap_shortfal .>= var]
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This PR adds the conditional value at risk (CVAR) metric to assess tail risk shortfalls. CVAR measures the expected value observations above a pre-determined threshold$\alpha$ (Nth percentile).
The current CVAR implementation measures two risk metrics: total unserved energy and capacity shortfalls, as follow:
ShortfallandShortfallSamples. To calculate the values from theShortfall, an additional vector of total unserved energy for each sample is stored before being removed from memory.CVARmetric reports the unserved energy by default (1), and it has capacity cvar fields for (2) if calculated from theShortfallSamplesor (3) if calculated from theShortfallAttached are
@benchmarktesting forShortfallandShortfallSamplescomparison after implementing the CVAR metrics.Main branch: