Issue seeing past page 10 when 100+ results are found

[hjohns]

Current Behavior

When reaching the last page (page 10) of a list of results with 100+ items, you can no longer press next page to see page 11 onwards, however you can see the next page of results if you enter ?page=11 in the URL.

Discussed with @recalcitrantsupplant, mentioned the API could adapt the number of results found as you continue to page through the list. E.g. when you get to page 8, the count returned could be 120+ results found, allow you to page past page 10. Hopefully fixable via PrezAPI changes only.

Expected Behavior

Be able to page past page 10 of 100+ results.

Steps To Reproduce

• navigate to the last page where there are 100+ results (page 10)
• you can no longer hit next page to see past page 10

Example link: https://demo.dev.kurrawong.ai/catalogs/ns33:icsm-vocabs/collections/csdm:wa-locality/items?page=10

Anything else?

No response

[recalcitrantsupplant]

can test with this https://github.com/RDFLib/prez/tree/david/adjust-listing-count-with-paging

NB there is some logic to this in order to prevent large count queries, for performance reasons.
There is a count limit setting, by default 100.
If you are below the configured page limit (limit * page) then the settings limit is used.
If you are above the configured page limit (e.g. settings limit=100, user requests limit=50, page=3, = 150 then this is the limit.
The limit + 1 is used in a count query. If the limit + 1 is hit (e.g. limit 151 returns 151 results) then ">150" is returned. If any number of focus nodes less than the limit is returned, that number is returned.

[recalcitrantsupplant]

The above is already somewhat complicated for performance reasons, and I don't think we should add more logic to it as users are not expected to sit there paging through data as a primary means of using the system. Unless anyone has a way to simplify it :)

[lalewis1]

was it historically causing issues to get the full count?

Just wondering how expensive a full count could be.

[recalcitrantsupplant]

Yes it has historically caused timeouts on listing pages. I would encourage you to test it with different datasets! It depends on the dataset size and backend. Fuseki tends to be worse than GraphDB. Keep in mind even if it's 1-2 seconds for the count, that's an extra 1-2 seconds someone has to wait every time they view a listing.

If you've tested for a particular dataset/database and found the count queries all run fast, you can just increase the count limit beyond the max count of items, so you will always get the actual count, not the ">x".