[sankaku] update pagination end condition (fixes #265)
Pagination over popular listings (`date:...+order:popular") never terminates, not even on the site itself, and at some point returns the same results over and over again.
This commit is contained in:
@@ -225,6 +225,11 @@ class SankakuTagExtractor(SankakuExtractor):
|
||||
|
||||
next_qs = text.extract(page, 'next-page-url="/?', '"', pos)[0]
|
||||
next_id = text.parse_query(next_qs).get("next")
|
||||
|
||||
# stop if the same "next" parameter occurs twice in a row (#265)
|
||||
if "next" in params and params["next"] == next_id:
|
||||
return
|
||||
|
||||
params["next"] = next_id or (text.parse_int(ids[-1]) - 1)
|
||||
params["page"] = "2"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user