ac4c93c12a
Issue 85: more cross-platform shebang on all scripts
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@962 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
79f912db86
Add wikis from Pavlo lists not verified as uploaded
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@961 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
d0f658a890
Remove old lists, add list of wikis already (re)downloaded from Pavlo and mediawikis_2013 lists
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@960 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
7eba594a2c
Use api.php URLs where we found them to be working, step 2
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@959 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
92f77ab085
Use api.php URLs where we found them to be working
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@958 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
395408e5a6
Also neoseeker and sourceforge
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@956 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
3fde888394
Remove dome wikifarms wikis and a couple duplicates
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@955 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
04e55c6622
Remove a hundred index.php redundant with api.php URLs
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@954 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
18a7f42086
Update with last raw list and checkalive.py
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@953 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
cbd0905cba
Add 2k more URLs from another crawl
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@950 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
4a5c91d471
Typo in filename
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@947 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
eb60580e91
New URLs from Incola
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@946 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
53236811d9
Also reupload the dump when verified missing
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@945 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
bab70e31c0
Use temporary name for history archive too to avoid conflicts
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@944 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
a1c89623a4
Another intermediate update with results from one more run
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@943 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
403dc213ef
Issue 71: English-only match for an older case
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@942 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
a3fe96b782
Revamp uploader.py for wider usage: move issues to tracker; add options --help, --prune-directories, --prune-wikidump, --admin
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@940 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
b74d6f79ce
Reduce requests for existing items and remove whitespace: tested with wiki-smackdownneoseekercom_w
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@939 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
54f9798be0
Mark staging 7z files as .tmp to avoid uploading them by mistake
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@938 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
2bb3fd7a50
Add neoseeker.com, from mutante's wikistats farm list
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@937 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
a358ffbfe0
Issue 88: Escaped a bit too much, some HTML we really need
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@936 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
c95465ac14
Nice to see curl progress, but only for actual upload
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@935 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
0ef0a5b229
Actually update last-updated-date
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@934 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
16ec333e7d
Adding rewrite code so others can build on top of it
...
This is a partial rewrite of the dumpgenerator.py, and
is largely incomplete. I am no longer working on this
rewrite, so I am releasing it for others to build upon
it and work towards releasing DumpGenerator 2.0.
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@933 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
43074360b7
Experience shows 30 seconds is a more realistic timeout
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@932 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
991b96fbe2
Mark dump uploaded only if confirmed by curl exit code
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@931 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
8cf60a3285
Re-updated pavlo list with 30 s timeout
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@930 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
e2956e60a0
Replace some index.php with api.php where available
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@929 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
57da61aac6
a minor syntax-error fix in checkalive.pl
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@928 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
8da3f15b35
Some manual filtering
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@927 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
84a2f9d6dc
Add index.php discovery and other fixes, update checked lists consequently; bad input makes it spit ugly errors but it keeps going
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@926 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
eba5f2d54e
More gamepedia from their homepage
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@925 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
51ee9e9847
added a user-agent and another search string
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@924 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
009682a037
Sync API check needle with checkalive.pl, </api> is unreliable
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@923 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
ac8ed21b0c
Add Terraria, 270 wikis might be missing but where is the list?
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@922 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
01b177bcf2
Update raw list with scraper run by odder
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@921 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
0ac7e477f5
Re-formatting of readme-checkalive.txt
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@920 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
fc9207291a
readme for checkalive.pl
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@919 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
d9247aa0ba
Lots of changes - improved error handling, progress reporting and other minor changes
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@918 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
0ede45b7cf
Special:BadTitle works only in English wikis
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@917 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
034866a32e
Handle permissions-errors for wikis requiring login or whatever
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@916 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
2bdf8da30c
Add orain and gamepedia lists, might have mistakes
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@915 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
6c69d9800f
Followup, delay needs config; should be BC
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@914 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
7e412d1f17
update with export of http://www.shoutwiki.com/wiki/Category:Flat_list_of_all_wikis
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@911 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
2f9985ac6f
instructions to compile LaTeX paper;
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@909 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
a88797717c
first draft of paper;
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@908 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
95c2228f36
creating directory for paper
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@905 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
6912c8bc71
creating directory for research, papers, etc
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@904 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
55185467e1
Add delay to all checking and listing functions, crappy hosts die on them
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@902 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
fb87cd9951
This is the first publicly available version of this Perl script.
...
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@901 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
nemobis
Hydriz
scottdb56
emijrp