Commit Graph

3 Commits

Author SHA1 Message Date
Mike Fährmann
99b72130ee [reddit] enable recursion (#15)
reddit extractors now recursively visit other submissions/posts
linked to in the initial set of submissions.
This behaviour can be configured via the 'extractor.reddit.recursion'
key in the configuration file or by `-o recursion=<value>`.

Example:
{"extractor": {
  "reddit": {
   "recursion": <value>
}}}

Possible values:
* -1 - infinite recursion (don't do this)
*  0 - recursion is disabled (default)
*  1 and higher - maximum recursion level
2017-05-26 17:01:27 +02:00
Mike Fährmann
e425243b1e [reddit] some small fixes
- filter or complete some URLs
- remove the 'nofollow:' scheme before printing URLs
- (#15)
2017-05-23 11:48:00 +02:00
Mike Fährmann
a22892f494 [reddit] add subreddit- and submission-extractor
- these extractors scan submissions and their comments for
  (external) URLs and defer them to other extractors
- (#15)
2017-05-23 09:38:50 +02:00