Archives/wikiteam
2
0
Fork 0
mirror of https://github.com/WikiTeam/wikiteam synced 2024-11-18 21:27:45 +00:00
Code Issues Projects Releases Wiki Activity
525 Commits 3 Branches 0 Tags 56 MiB
5261811fa4
Commit Graph

525 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
nemobis
b8c90df787 Upload separately some already checked with the script 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@891 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 18:38:19 +00:00
nemobis
500e8ef350 Remove some more obvious duplicates including trailing slash 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@890 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 17:24:53 +00:00
nemobis
31ea06ff86 Remove also wiki/[A-Z].+$ 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@889 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 17:17:42 +00:00
nemobis
664fa18ea3 Remove sourceforge wikis and URLs with parameters to index.php 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@888 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 17:13:46 +00:00
nemobis
3624c02852 Remove Wikimedia Foundation wikis, other 'wikimedia' URLs cleanup 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@887 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 17:04:29 +00:00
nemobis
ccfc95e9f4 Issue 59: Add first dirty list of possible MediaWiki sitesFirst passes of the script, now going on with all TLDs.Just sorted and cleaned of biggest noises like mailing lists, github and stackoverflow. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@886 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2014-01-03 16:57:37 +00:00
nemobis
002a8d6702 Add first list of sourceforge wikis 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@885 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-12-12 13:36:33 +00:00
Hydriz
d5cf4fe592 Deleting things that have no usecase 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@884 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-23 13:05:38 +00:00
nemobis
7c4cb64859 Revert r874: breaks backwards compatibility, no usecase provided 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@883 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-23 12:19:46 +00:00
nemobis
6efe406ea5 Followup r877, first check most common conditions for shortcut performance 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@882 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-17 11:18:42 +00:00
nemobis
b940293136 Add more wikkii wikis from mutante's wikistats 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@881 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-17 11:10:36 +00:00
Hydriz
0dd9c5f299 Follow up r874, forgot to uncomment accesskey and secretkey after testing 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@879 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-16 14:29:34 +00:00
Hydriz
611d13f8c5 Follow up r877, check the number of revision tags 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@878 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-16 14:01:24 +00:00
Hydriz
64bd837cab (Issue 34) XML integry check inside the code 
This *really* fixes the issue and asks the user whether or not to regenerate a dump.


git-svn-id: https://wikiteam.googlecode.com/svn/trunk@877 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-16 13:39:49 +00:00
Hydriz
5bed9cb28e Adding MediaWiki Export XML Schema Descriptions to track changes and add support accordingly 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@876 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-16 05:44:46 +00:00
Hydriz
8680a55e24 Fix identifier name so that it can be more human-readable and backward compatible 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@874 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-15 07:49:06 +00:00
Hydriz
79047a3ded (Issue 71) Use a better check for private wikis 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@873 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-15 07:43:37 +00:00
Hydriz
a0726eed04 Rename revnum.txt to revnum.json and fix downloadurl from GitHub. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@871 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-11 14:28:29 +00:00
Hydriz
e14fb0fe1b Adding revnum.txt, a file used by dumpgenerator.py version 2.0. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@870 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-11 08:12:48 +00:00
nemobis
26873ad495 Fix typo, make domain2prefix quiet again 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@869 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-11 07:33:25 +00:00
nemobis
626118cfab Let's call it 0.2 then, a bump to 1 would require announcements etc. We're not there yet (API support etc.). 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@867 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-10 09:17:06 +00:00
Hydriz
df1e7efafd Change version of dumpgenerator.py to 1.1. Using 0.1 is rather confusing. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@866 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-10 02:47:18 +00:00
nemobis
e1d4de3179 Uncomment appended index.php for guess in most configurations 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@864 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-08 22:20:21 +00:00
nemobis
5d34d9512a Needs to be non-matching group 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@863 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-08 22:01:36 +00:00
nemobis
82ba173739 Issue 22: allimages now uses aicontinue, not aifrom 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@862 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-08 21:57:04 +00:00
nemobis
c6546ff935 Issue 71: Don't try to download private wikis, first workaround 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@861 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-07 22:59:28 +00:00
nemobis
776038666f Issue 72: revert r857, just define everything in launcher.py 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@860 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-07 12:24:50 +00:00
nemobis
6113fa3340 Add delay to getPageTitlesScraper 
We must be nice here too or naughty hosts fail badly, for instance wikkii.com gave

urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Moved Temporarily



git-svn-id: https://wikiteam.googlecode.com/svn/trunk@859 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-06 08:21:57 +00:00
nemobis
9e1b13e173 Correct --help: format is --delay=5, not --delay:5 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@858 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-06 08:13:07 +00:00
nemobis
6430ac5f47 Check for the existence of the array in domain2prefix instead; uploader.py failed on python 2.6 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@857 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-06 07:49:12 +00:00
nemobis
69b00964c8 Add a comment on delay 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@856 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-05 18:24:08 +00:00
nemobis
ef7d527e86 Add some advice about editthis.info for usage via launcher.py 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@855 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-05 18:11:23 +00:00
nemobis
4820339d10 Fix r842, patch by balrog; Schbirid reported python error in CleanHTML 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@854 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-05 17:10:44 +00:00
nemobis
577e8034e6 ahem syntax 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@853 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-02 10:09:36 +00:00
nemobis
6966ba5918 subprocess.check_output is not supported in python 2.6, let's not dieYes, some are still on python 2.6; even 3.2.46-grbfs-kapsi it seems. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@852 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-02 09:50:49 +00:00
nemobis
7c94815e2c Issue 68: Use GET, not POST, to download images; some harm and no? good 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@851 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-11-01 23:07:27 +00:00
nemobis
0eb4913f89 Requires python 2.7 or more (for subprocess.check_output) 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@850 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-28 18:39:50 +00:00
nemobis
8ef08bbe28 Issue 67: launcher.py fails to 7z completed dumps (also get rid of some os.system) 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@849 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-28 10:20:38 +00:00
nemobis
d289c563b1 Link to https://code.google.com/p/wikiteam/wiki/NewTutorial#Download_a_list_of_wikis 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@848 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-16 18:08:45 +00:00
nemobis
ad5d468824 Add some documentation; would need a proper --help 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@846 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-16 17:57:02 +00:00
nemobis
e1b34b7f6b Fix whitespace 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@844 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-14 19:10:10 +00:00
nemobis
57e226c049 Use urllib2 and set user agent in some more places; some webhosts block urllib. 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@842 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-14 09:27:30 +00:00
Hydriz
414fb1988f Add a little more singular/plural support. 
Along with this commit:
* Changed original %s to %d for number of edits, since it is recognised as an integer.
* Directly defined the number of edits to be 1 when the if condition is true, to optimise performance.


git-svn-id: https://wikiteam.googlecode.com/svn/trunk@841 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-13 09:35:48 +00:00
nemobis
13ebee4c28 Issue 60: Add authentication cookie support, patch by Fredrik Roubert 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@840 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-09 20:11:01 +00:00
nemobis
b34f01646a Use getUserAgent in one more place, urllib is blocked by some 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@839 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-10-09 19:48:02 +00:00
nemobis
1a8e393c11 Google, not Gmail account 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@838 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-09-19 10:11:43 +00:00
nemobis
0150b93dcd Issue #64: Improve speed by some orders of magnitude 
Patch by Betacommand, many thanks.

git-svn-id: https://wikiteam.googlecode.com/svn/trunk@837 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-09-09 17:27:50 +00:00
Hydriz
8b4480be64 Adding plural support for number of edits saved for a page 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@831 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-08-24 06:10:04 +00:00
nemobis
5e826c272d Use recursive option for zip 
The * does not scale, I get errors from sh that the arguments list is too long.

git-svn-id: https://wikiteam.googlecode.com/svn/trunk@830 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-08-22 20:53:46 +00:00
nemobis
ed8d174d5a Issue #61: some skins hide that stuff, use meta tag generator 
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@829 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
 2013-08-17 07:34:11 +00:00