emijrp
|
f94a04562f
|
fixed for enwiki which uses several chunks for big files;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@168 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
5254085eb4
|
wikipedia dumps downloader;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@167 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
502d3a3d22
|
more wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@166 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
778a8ad7ae
|
adding support to download images on old mediawikis; regexp4;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@165 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
5b7674edb7
|
params in main() to call it from external scripts (using import dumpgenerator; dumpgenerator.main(params=params))
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@161 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
e4d0de09d6
|
params in main() to call it from external scripts (using import dumpgenerator; dumpgenerator.main(params=params))
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@160 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
852d6658ad
|
more wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@159 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
da6c4d1468
|
print error output when splitters error occurs; 5000 -> 500 in image list parser
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@158 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
c5b6b8a866
|
md5.new->md5; new uploaded wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@157 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
11420962f5
|
print
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@156 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
2acd8415c7
|
removing dupes;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@155 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
835158b79b
|
more wikis; md5 or hashlib
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@154 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
59fe050558
|
more wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@153 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
13 years ago |
emijrp
|
c7b511ee38
|
more wikis; new content spliter;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@152 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
9a20327939
|
undoHTMLEntities for titles when scrapped from Allpages; protocol http https;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@150 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
af64c6b6c5
|
http or https
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@149 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
5cd7be9987
|
removing old code in wikiadownloader;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@148 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
e9f379b888
|
fixing print;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@147 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
1b7fa4c994
|
help
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@146 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
b26a86cdd5
|
verbose false for imagedump;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@144 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
896393016d
|
getting date from index.json (wikia);
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@143 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
523ae5c983
|
months;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@142 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
4e19191cc2
|
wikia downloader;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@141 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
17235cffd9
|
better comments;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@140 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
6e45398878
|
when full history fails, retrieve only the last version; various server errors handled
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@139 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
e4b233cc37
|
print verbose; seconds numbers to variables;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@138 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
7b85e243b0
|
:
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@137 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
3c49a30764
|
fixing issue #12 and issue #13;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@136 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
3388c7e83b
|
print
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@135 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
101cd62e3d
|
print
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@134 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
46303d780d
|
fixing issue #11;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@133 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
1f91b4c63e
|
fixing issue #11;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@132 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
220896ccac
|
more wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@131 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
8bd90b4cde
|
more wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@127 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
e13b4a6428
|
removing http/https for file prefixes;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@126 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
0f3f7d51f5
|
more wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@124 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
d0cd9aa646
|
more wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@123 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
79f961aef5
|
more wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@122 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
2c0d8b97b0
|
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@121 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
2bac6cf3f0
|
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@120 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
d6c1a773a0
|
removing some old #fix comments; moving TODO to Issues section at Google Code;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@119 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
88cbc0e871
|
removing some old #fix comments;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@118 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
23c9a06a31
|
removing some old #fix comments;removing thread option not implemented and better not to be done;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@117 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
05d1fb97c4
|
sorting titles; fixing issue #9;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@116 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
06b6f79989
|
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@113 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
bc36eb0f85
|
uploaded wikis;
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@110 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
e1cf136a97
|
wikia list, > 200000 wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@104 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
d4bc4cba9a
|
readme for wikilists
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@103 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
24ea30f29f
|
scribblewiki
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@101 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |
emijrp
|
eb6f7c2639
|
adding some more empty lists to do
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@100 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
|
14 years ago |