Commit Graph

42 Commits

Author SHA1 Message Date
Mike Fährmann
3cbf15a107 [8chan] don't make network requests in '_init()' (#6387) 2024-10-27 19:36:49 +01:00
Mike Fährmann
d7e34e1dc3 [8chan] automatically determine TOS cookie name (#6318) 2024-10-14 14:25:09 +02:00
Mike Fährmann
93265db9b3 [8chan] support '/last/' thread URLs (#6318) 2024-10-14 14:25:09 +02:00
Mike Fährmann
51ceb33b6c [8chan] also set TOS cookie for the previous day 2024-10-10 18:19:37 +02:00
Mike Fährmann
0389575e0f [8chan] dynamically generate 'TOS' cookie name 2024-09-28 11:04:27 +02:00
Mike Fährmann
fe190be589 [8chan] update 'TOS' cookie name 2024-09-19 20:50:58 +02:00
Mike Fährmann
2c7a0c3ca8 add alternatives for deprecated utc datetime functions 2024-09-19 20:47:05 +02:00
Mike Fährmann
a051e1c955 directly pass exception instances as 'exc_info' logger argument 2024-09-19 14:50:08 +02:00
Mike Fährmann
a595949840 [8chan] update 'TOS' cookie name (#5868) 2024-07-20 02:15:29 +02:00
Mike Fährmann
5b6b5dd81b [8chan] fix downloaded files by sending 'TOS' cookie (#5578) 2024-05-11 15:34:21 +02:00
Mike Fährmann
3ecb512722 send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
a453335a9f remove test results in extractor modules
and add generic example URLs
2023-09-11 16:30:55 +02:00
Mike Fährmann
a383eca7f6 decouple extractor initialization
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba consistent cookie-related names
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
2023-07-22 01:20:50 +02:00
Mike Fährmann
cd931e1139 update extractor test results 2022-12-08 18:58:29 +01:00
Mike Fährmann
1696f68a68 [8chan] add 'thread' and 'board' extractors (#2938) 2022-10-11 10:47:19 +02:00
Mike Fährmann
557e2c018b [8chan] remove module 2019-11-02 20:06:47 +01:00
Mike Fährmann
6284731107 simplify extractor constants
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
3bdfc15be1 [pinterest] don't crash on pins without image info 2018-11-14 11:46:14 +01:00
Mike Fährmann
0ba93650e0 [8chan] replace unit test URL
the other thread is no longer accessible
2018-07-14 09:53:16 +02:00
Mike Fährmann
2ef3c35c98 smaller textual changes
- swapped doc for deviantart.mature and .original
- updated gallery-dl.conf
- "transferred" -> "delegated"
2017-10-09 23:23:19 +02:00
Mike Fährmann
94e10f249a code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
00074a71d7 several changes to make travis build work
- fixed html.unescape not being available on Python3.3
- removed inconsistent test result
- added username/password pairs for authenticating extractors
2017-01-10 13:41:00 +01:00
Mike Fährmann
468c9aeeec make tests more reliable 2017-01-05 00:29:05 +01:00
Mike Fährmann
4c55275305 update tests 2016-12-12 14:17:15 +01:00
Mike Fährmann
0211ec4114 update some tests 2016-12-08 00:24:23 +01:00
Mike Fährmann
a33b13c631 [8chan] fix file urls 2016-10-15 23:06:54 +02:00
Mike Fährmann
f7c993d952 set 'extension' keyword whenever possible 2016-10-07 15:11:58 +02:00
Mike Fährmann
c20a3b3a01 [*chan] code cleanup 2016-10-03 08:23:40 +02:00
Mike Fährmann
56d810c896 update keyword hashes for tests 2016-09-25 17:28:46 +02:00
Mike Fährmann
49a05c32ed add missing tests 2016-09-19 16:15:27 +02:00
Mike Fährmann
d7e168799d consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
2dfe97dd00 [chan] update to new format 2015-11-21 03:13:06 +01:00
Mike Fährmann
686a4ddd3e [8chan] fix file-urls 2015-11-05 00:18:36 +01:00
Mike Fährmann
3c13548f29 rewrite extractors to use config-module 2015-10-05 15:51:08 +02:00
Mike Fährmann
d7e0d81bdd [8chan] use api 2015-09-07 17:16:50 +02:00
Mike Fährmann
30b9db43a0 [8chan] combine patterns 2015-04-09 17:28:14 +02:00
Mike Fährmann
f31c05f522 [8chan] update to new extractor interface 2015-04-08 02:26:13 +02:00
Mike Fährmann
ac002cc4b3 [8chan] fix image urls 2015-02-03 00:22:39 +01:00
Mike Fährmann
0f6cc964cc [8chan] use new domainname for thread-urls 2015-01-18 23:45:40 +01:00
Mike Fährmann
1244c933a1 [8chan] fix image urls 2014-11-26 17:38:50 +01:00
Mike Fährmann
deef91eddc initial commit 2014-10-12 21:56:44 +02:00