[ENH] Add Samformer model for PTF v2 from DSIPTS

[PranavBhatP]

Reference Issues/PRs

Closes #1940

[codecov]

Codecov Report

❌ Patch coverage is 84.61538% with 26 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@3821c0b). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...ytorch_forecasting/layers/_normalization/_revin.py	71.73%	13 Missing ⚠️
...orch_forecasting/models/samformer/_samformer_v2.py	83.54%	13 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1952   +/-   ##
=======================================
  Coverage        ?   87.19%           
=======================================
  Files           ?      148           
  Lines           ?     9065           
  Branches        ?        0           
=======================================
  Hits            ?     7904           
  Misses          ?     1161           
  Partials        ?        0           

Flag	Coverage Δ
cpu	87.19% <84.61%> (?)
pytest	87.19% <84.61%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fkiraly]

FYI @PranavBhatP, working on a check_estimator utility that should help with getting tests to pass: #1954

It seems like this PR is also not up to date with main?

[PranavBhatP]

I will be pushing changes in a while.

[PranavBhatP]

@fkiraly could this be assigned to me? I don't think it will be visible on the kanban board...

[agobbifbk]

In the original implementation there is this switch:

if hasattr(nn.functional, 'scaled_dot_product_attention'):
            att_score = nn.functional.scaled_dot_product_attention(queries, keys, values) # (n, D, L)
        else:
            att_score = scaled_dot_product_attention(queries, keys, values) # (n, D, L)

in your implementation you are leveraging on nn.functional.scaled_dot_product_attention, can you check if the error persists if you use the scaled_dot_product_attention there was in the original implementation?

If the error persists, we can have a look together!

[PranavBhatP]

works now! i am using the custom implementation.

[fkiraly]

In the original implementation there is this switch:

what is this switch for, @agobbifbk? Is this to handle different versions of nn.functional?

[fkiraly]

@fkiraly could this be assigned to me? I don't think it will be visible on the kanban board...

apologies, it was assigned to the wrong Pranav. GitHub autocompletes when you start typing @, so if you have very similar GH names it is easy to select unintentionally.

[agobbifbk]

In the original implementation there is this switch:

what is this switch for, @agobbifbk? Is this to handle different versions of nn.functional?

The function was introduced recently, and probably it is bugged for some version (or some platform). The authors put a switch to be sure that it works in different versions of PT.

[fkiraly]

lgtm