Skip to content

Add Conditional Value at Risk (CVAR) metric#100

Open
abdelrahman-ayad wants to merge 27 commits intomainfrom
aa/CVAR
Open

Add Conditional Value at Risk (CVAR) metric#100
abdelrahman-ayad wants to merge 27 commits intomainfrom
aa/CVAR

Conversation

@abdelrahman-ayad
Copy link
Collaborator

@abdelrahman-ayad abdelrahman-ayad commented Feb 2, 2026

This PR adds the conditional value at risk (CVAR) metric to assess tail risk shortfalls. CVAR measures the expected value observations above a pre-determined threshold $\alpha$ (Nth percentile).
The current CVAR implementation measures two risk metrics: total unserved energy and capacity shortfalls, as follow:

  1. CVAR of unserved energy: Mean of the worst ($1-\alpha$) unserved energy across all samples, calculated on the Shortfall and ShortfallSamples. To calculate the values from the Shortfall, an additional vector of total unserved energy for each sample is stored before being removed from memory.
  2. CVAR of capacity shortfall samples: Mean of the worst ($1-\alpha$) maximum capacity shortfall in each sample, calculated on the `ShortfallSamples.
  3. CVAR of capacity shortfall: Mean of the worst ($1-\alpha$) capacity shortfall periods, calculated on the `Shortfall.
  • Note: The CVAR metric reports the unserved energy by default (1), and it has capacity cvar fields for (2) if calculated from the ShortfallSamples or (3) if calculated from the Shortfall
Screenshot 2026-02-23 at 12 52 37 PM
  • Attached are @benchmark testing for Shortfall and ShortfallSamples comparison after implementing the CVAR metrics.

  • Main branch:

main_branch
  • CVAR branch:
CVAR_branch

@abdelrahman-ayad abdelrahman-ayad self-assigned this Feb 2, 2026
@abdelrahman-ayad abdelrahman-ayad added the enhancement New feature or request label Feb 2, 2026
@codecov-commenter
Copy link

codecov-commenter commented Feb 2, 2026

Codecov Report

❌ Patch coverage is 90.84507% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.57%. Comparing base (fbde94e) to head (8de1ba0).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
PRASCore.jl/src/Results/Results.jl 0.00% 5 Missing ⚠️
PRASCore.jl/src/Results/Shortfall.jl 92.45% 4 Missing ⚠️
PRASCore.jl/src/Results/ShortfallSamples.jl 93.84% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #100      +/-   ##
==========================================
+ Coverage   83.13%   83.57%   +0.43%     
==========================================
  Files          45       45              
  Lines        2325     2466     +141     
==========================================
+ Hits         1933     2061     +128     
- Misses        392      405      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@abdelrahman-ayad abdelrahman-ayad marked this pull request as draft February 23, 2026 19:59
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Conditional Value at Risk (CVAR) (and normalized NCVAR) reliability metric for tail-risk assessment of shortfalls, including capacity-shortfall variants, and updates simulation recording + tests accordingly.

Changes:

  • Introduces CVAR / NCVAR metric types (plus display/validation) and computes them for ShortfallResult and ShortfallSamplesResult.
  • Extends result recording/finalization to retain per-sample unserved energy and max capacity shortfall needed for CVAR.
  • Adds/updates unit tests and reference values (including system test data) to cover CVAR/NCVAR behavior.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
PRASCore.jl/src/Results/metrics.jl Adds CVAR/NCVAR metric structs and show methods; adjusts MeanEstimate for singleton samples.
PRASCore.jl/src/Results/Shortfall.jl Stores per-sample unserved energy and adds CVAR/NCVAR computations for ShortfallResult.
PRASCore.jl/src/Results/ShortfallSamples.jl Adds max capacity shortfall storage and CVAR/NCVAR computations for ShortfallSamplesResult.
PRASCore.jl/src/Results/Results.jl Exports CVAR/NCVAR and adds broadcast convenience methods.
PRASCore.jl/src/Simulations/recording.jl Records per-sample UE totals into the shortfall accumulator.
PRASCore.jl/src/Systems/TestData.jl Adds expected CVAR reference values for simulation tests.
PRASCore.jl/test/Results/metrics.jl Adds tests for CVAR/NCVAR formatting and domain checks.
PRASCore.jl/test/Results/shortfall.jl Adds correctness tests for CVAR/NCVAR on results (overall/region/period).
PRASCore.jl/test/Simulations/runtests.jl Adds simulation-level assertions for CVAR values.
PRASCore.jl/test/dummydata.jl Adds dummy alpha/sample vectors for new result fields.
PRAS.jl/test/runtests.jl Expands integration tests to validate CVAR/NCVAR return types for both result kinds.
Comments suppressed due to low confidence (1)

PRASCore.jl/test/Results/shortfall.jl:43

  • Typo in variable name: cap_shortfal is missing an l (should be cap_shortfall) to match intent and improve readability.
    cap_shortfal = vec(reshape(result.capacity_shortfall_mean, 1, :))
    var = quantile(cap_shortfal, alpha)
    tail_losses = cap_shortfal[cap_shortfal .>= var]

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@NatLabRockies NatLabRockies deleted a comment from Copilot AI Feb 25, 2026
@abdelrahman-ayad abdelrahman-ayad marked this pull request as ready for review February 25, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants