22 Commits (8b40429e9552edf5bf484fe2c70368c310d3c6cf)

Author SHA1 Message Date
arkiver 23bfe8b12c Version 20200730.01. Support /user/ post better (like /r/). 4 years ago
arkiver 450d4e0413 Version 20200728.01. Ignore non-reddit URLs. Fix extraction of tokens for morecomments. 4 years ago
arkiver 9a6417ecbc Version 20200727.03. Fix handling video URLs without extension. 4 years ago
arkiver 911c675e74 Version 20200727.02. Set TRACKER_ID to reddittest. 4 years ago
arkiver 147c6416ed Version 20200727.01. Use trackerproxy for dictionaries. Ignore irc: URLs. 4 years ago
arkiver 910687b053 Version 20200726.06. Fix project name for ZSTD dictionary request. 4 years ago
arkiver 496c018eef Version 20200726.05. Add cookies to access some quarantines subreddits. 4 years ago
arkiver 3d5e7e17f9 Version 20200726.04. Use reddittest tracker for size estimate. 4 years ago
arkiver 23fec56409 Version 20200726.03. Support galleries and comments. 4 years ago
arkiver 2f6a602313 Version 20200726.01. Fully support new and old design for posts. 4 years ago
arkiver 56571306dd Use default upload concurrent of 2. 4 years ago
arkiver 40063adcaf Use wget-at with ZSTD. 4 years ago
Arkiver2 831f79f0d9 Do not import warcio. Update version to 20200102.03. 5 years ago
Arkiver2 cf3f6c7af9 Skip URL on status code 204. Update version to 20200102.02. 5 years ago
Arkiver2 ac65b0a818 Update version to 20200102.01. 5 years ago
Arkiver2 4cf7bd18f0 Version 20190729.01; do not get page requisites from outlinks; do not pip install warcio. 5 years ago
Arkiver2 8902255c76 Version 20190405.01; support www.reddit.com; support videos; support outlinks 6 years ago
Arkiver2 9d1ea0c688 rewrite 6 years ago
Arkiver2 11aef69a32 pipeline.py: cookies! 9 years ago
Arkiver2 9f531c900f pipeline.py: use redd.it to discover comments 9 years ago
Arkiver2 61dd537f15 Update pipeline.py 9 years ago
Arkiver2 ff1bb532c6 pipeline.py 9 years ago