Commit Graph

21 Commits

Author SHA1 Message Date
Mike Fährmann
9ebd29fcc1 update cloudflare bypass (wip)
This commit adds support for the two new JS expressions embedded in the
overall challenge code.

It does compute the correct 'js_answer' value, but the HTTP request to
/cdn-cgi/l/chk_jschl to get the 'cf_clearance' cookie always results in
a 403 response with a CAPTCHA inside (hence 'wip')

All steps to make this HTTP request indistinguishable from a regular web
browser (which passes the test) show no effect. This includes:
- using the exact same HTTP headers as a web browser
- follow query argument order
- different wait times
2019-04-01 15:14:59 +02:00
Mike Fährmann
5476404a5c update and fix Cloudflare bypass 2019-03-25 22:53:36 +01:00
Mike Fährmann
f612284d24 cache cfclearance cookies 2019-03-14 16:14:29 +01:00
Mike Fährmann
6dae6bee37 automatically detect and bypass cloudflare challenge pages
TODO: cache and re-apply cfclearance cookies
2019-03-10 15:31:33 +01:00
Mike Fährmann
0328a04a65 [cloudflare] don't output the whole challenge page
thanks to the embedded animated gifs this is just a bit too much
2018-12-30 13:09:23 +01:00
Mike Fährmann
95392554ee use text.urljoin() 2018-04-26 17:00:26 +02:00
Mike Fährmann
ff643793bd improve and document cloudflare bypass code 2018-04-19 21:32:10 +02:00
Mike Fährmann
d11fcf4804 smaller changes and fixes
- fix the cloudflare challenge result if the last decimal places
  are zero (JS`s toFixed() removes trailing zeroes)
- fix downloading of kissmanga chapter-pages hosted on blogspot
  (accessing blogspot with "kissmanga.com" as referrer yields a 401)
- disable certificate validation for 'mangahere' tests
- update flickr test result
2018-04-06 15:30:09 +02:00
Mike Fährmann
f6c95dccf9 [cloudflare] fix bypass procedure
Cloudflare challenges, at least for kissmanga and readcomiconline,
now use slightly different Javascript expressions.

Instead of a single value per expression, they now have a numerator
and a denominator of a fractional value, which in the end gets
truncated to 10 decimal places.
2018-04-05 20:28:04 +02:00
Mike Fährmann
67d4462d26 [batoto] rudimentary Cloudflare bypass 2018-01-11 18:49:19 +01:00
Mike Fährmann
cf79a47b59 update unit tests 2017-04-08 11:02:32 +02:00
Mike Fährmann
cdecd0b37b add notification when solving cloudflare challenge 2017-03-25 12:23:30 +01:00
Mike Fährmann
4f123b8513 code adjustments according to pep8 2017-01-30 19:40:15 +01:00
Mike Fährmann
40dbea7ed2 rewrite parts of the cloudflare bypass system 2016-12-16 13:28:36 +01:00
Mike Fährmann
9e3788175e implement decorator for cloudflare bypass
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
2016-11-20 18:05:49 +01:00
Mike Fährmann
b634ace39e [readcomiconline] add comic-issue and comic extractor 2016-11-14 18:29:45 +01:00
Mike Fährmann
3c94d85409 fix cloudflare bypass 2016-07-12 12:03:25 +02:00
Mike Fährmann
8accf4b790 use cache-decorator for cloudflare bypass 2016-03-07 16:34:42 +01:00
Mike Fährmann
e78937a580 cache cookies from cloudflare-bypass 2015-11-27 21:32:07 +01:00
Mike Fährmann
079330785a document and fix smaller bugs 2015-11-07 13:06:23 +01:00
Mike Fährmann
822527a2ce module to bypass cloudflare protection 2015-11-07 02:30:08 +01:00