MANPAGER=vimpager man ls
# type '/list' to start search, than 'n' a couple of times
I think it's because the pager tries to be too smart about placing the cursor, instead of putting it exactly on the search occurrence (or at least the exact line). In my case, the cursor starts off about 5 lines from the bottom (which is not a search occurrence), and stays there no matter how many times I press n. If I scroll manually (down arrow), the search starts moving again.
In any case, placing the cursor anywhere else but on the search occurrence makes searching frequent patterns kind of difficult, as there is no visual feedback when pressing n.