I was using Serply to retrieve Google Scholar paper details.
papers_per_page = 21
search_count = req_paper_count//papers_per_page
retrieved_paper_count = 0
for i in range(search_count):
results = serply.scholar(query, start=retrieved_paper_count, num=papers_per_page)
retrieved_paper_count += len(results["articles"])
print(results)
Here, when papers_per_page = 20, the request fails, but if papers_per_page = 21, it retrieves 20 paper details.