add general 'blacklist' and 'whitelist' options (#492, #844)

2020-09-10 22:54:10 +02:00
parent abda352a5b
commit c78aa17506
2 changed files with 40 additions and 0 deletions
--- a/docs/configuration.rst
+++ b/docs/configuration.rst
@@ -363,6 +363,20 @@ Description Transfer an extractor's (sub)category values to all child
 =========== =====


+extractor.*.blacklist & .whitelist
+----------------------------------
+=========== =====
+Type        ``list`` of ``strings``
+Default     ``["oauth", "recursive", "test"]`` + current extractor category
+Description A list of extractor categories to ignore (or allow)
+            when spawning child extractors for unknown URLs,
+            e.g. from ``reddit`` or ``plurk``.
+
+            Note: Any ``blacklist`` setting will automatically include
+            ``"oauth"``, ``"recursive"``, and ``"test"``.
+=========== =====
+
+
 extractor.*.archive
 -------------------
 =========== =====