query_wind_and_solar_forecast() no longer works as expected

[bantu]

It appear that when more than 1000 entries are queried, the API returns NaN values.

[glasid]

Observed similar issues when querying generation data per production type.

Did some quick analysis yday night. I believe ENTSOE fucked it up. When querying generation data per production type most psrTypes seem to be capped at 1000 entries in the xml file while for some others the xml file contains the full data set.

Will check with ENTSOE.

[BerriJ]

This is related to #478 (comment)

[glasid]

Not sure if this is exactly the same issue.

For Actual Production per Generation Type they publish different amount of data for each psrType in the same xml file which does not make any sense.

[runyao-yu]

the wind onshore is always missing after 12. each month

[BerriJ]

the wind onshore is always missing after 12. each month

True, I'll report this to ENTSOE.
Here is a curl querry that demonstrates this:

curl --location "https://web-api.tp.entsoe.eu/api?documentType=A69&processType=A01&PsrType=B19&in_Domain=10YNL----------L&periodStart=202511010000&periodEnd=202512010000&offset=100&securityToken=${ENTSOE_API}"

We only get the first 1000 rows. And this is 1000/96 ~ 10.4 Days for quarter hourly data.

[glasid]

The journey continues. Despite ENTSOE's claim, the problem isn't solved. Things are getting worse.

The latest changes have resulted in a duplicate XML block for psrType B01, with the TimeSeries data missing from the first one. All other psrTypes are completely missing.

Back to the drawing board, ENTSOE!

[glasid]

Still waiting for final confirmation from ENTSOE, but it looks like the issue with the malformed XML is fixed.

@bantu Can you test if this is also the case for query_wind_and_solar_forecast(), please?