github/gallery-dl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,304 Commits 1 Branch 0 Tags
a050fe0d94de7e9caac0216b53fa682d9b45b8a9
Commit Graph

23 Commits

Author SHA1 Message Date
Mike Fährmann
71d3143c35 fix bug in test_extractors.py 
pattern matching tests would succeed
if there is exactly one match
but for the wrong extractor
 2023-01-08 15:35:05 +01:00
Mike Fährmann
36ac2197db [ytdl] add extractor for sites supported by youtube-dl 
(#1680, #878)

Can be used by prefixing any URL with 'ytdl:',
or by setting 'extractor,ytdl.enabled' to 'true'.
 2021-07-10 20:55:47 +02:00
Mike Fährmann
36bf76fa44 update 'oauth:mastodon:<instance>' code 2021-01-28 02:20:12 +01:00
Mike Fährmann
350b1afe1c speed up _list_classes() after iterating over all modules once 2020-10-26 22:18:15 +01:00
Mike Fährmann
3918b69677 remove 'extractor.blacklist' context manager 2020-09-11 13:17:35 +02:00
Mike Fährmann
5df8f2959b insert local directory into PYTHONPATH when running tests 2020-05-02 01:15:50 +02:00
Mike Fährmann
406449b0d6 ensure keys for mastodon instances are available during tests 
Calls to config.clear() from other tests are removing the API
credentials set when importing mastodon.py for the first time.
 2020-04-08 21:56:14 +02:00
Mike Fährmann
3b50c4f49d add tests for "Extractors" in oauth.py (#670) 2020-04-07 20:26:12 +02:00
Mike Fährmann
04bd0472de add tests for Extractor.wait() 2020-04-07 20:24:56 +02:00
Mike Fährmann
2a3bd4e3c7 rename extractor classes starting with a digit 2019-11-02 20:42:09 +01:00
Mike Fährmann
64786363be [4chan] simplify 
- remove 'chan.py'
- slight adjustments to directory and filenames
 2019-11-02 20:11:21 +01:00
Mike Fährmann
6277a739e4 [35photo] add user-, genre-, and image-extractors (#162) 2019-03-18 01:11:30 +01:00
Mike Fährmann
4b1880fa5e propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
9a9cd32461 implement alternative constructor for extractors 2019-02-09 14:42:25 +01:00
Mike Fährmann
abbd45d0f4 update handling of extractor URL patterns 
When loading extractor classes during 'extractor.find(…)', their
'pattern' attribute will be replaced with a compiled version of itself.
 2019-02-08 20:08:16 +01:00
Mike Fährmann
6284731107 simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
bc0951d974 allow for simplified test data structures 
Instead of a strict list of (URL, RESULTS)-tuples, extractor result
tests can now be a single (URL, RESULTS)-tuple, if it's just one test,
and "only matching" tests can now be a simple string.
 2019-02-06 17:24:44 +01:00
Mike Fährmann
347398f692 fix various tests 2019-02-04 14:40:21 +01:00
Mike Fährmann
b8fed34548 add generalized extractors for Mastodon instances (#144) 
Extractors for Mastodon instances can now be dynamically generated,
based on the instance names in the 'extractor.mastodon.*' config path.

Example:
{
    "extractor": {
        "mastodon": {
            "pawoo.net": { ... },
            "mastodon.xyz": { ... },
            "tabletop.social": { ... },
            ...
        }
    }
}

Each entry requires an 'access-token' value, which can be generated with
'gallery-dl oauth:mastodon:<instance URL>'.
An 'access-token' (as well as a 'client-id' and 'client-secret') for
pawoo.net is always available, but can be overwritten as necessary.
 2019-01-19 14:28:59 +01:00
Mike Fährmann
c9e6ccbd7c [test:extractor] small fixes and improvements 2018-08-15 21:49:33 +02:00
Mike Fährmann
10cc59f3b5 fix extractor names 2018-04-18 18:12:57 +02:00
Mike Fährmann
b1325d4d2c fix extractor docstrings 2018-04-18 18:03:43 +02:00
Mike Fährmann
dd314279fb [test] add unit tests for extractor module functions 2018-03-25 11:49:42 +02:00