Commit Graph

122 Commits (5d416da9134ce994662cde8539b1f9fc5466fe7d)

Author SHA1 Message Date
emijrp fea6ab3b86 more 8 years ago
emijrp 01ccacd138 first version of wikispaces spider 8 years ago
Alexia E. Smith cb766de5ff Update gamepedia.com wikis.
This is current as of 2016-04-07 and is correct at 1,120 wikis.
9 years ago
emijrp dde7eb90ba wiki.wiki info 9 years ago
emijrp 8048b92029 adding wiki.wiki wikifarm list 9 years ago
emijrp e30cd44384 new wikifarm list of wikis 9 years ago
emijrp d44db951c2 update date 9 years ago
emijrp 64c30f2b50 updating neoseeker list and sorting, +1 new wiki 9 years ago
Southparkfan ebffb99f48 Add Miraheze wiki farm 9 years ago
Hydriz Scholz 1550d3755d Update orain.org wiki list 9 years ago
Federico Leva a1921f0919 Update list of wikia.com unarchived wikis
The list of unarchived wikis was compared to the list of wikis that we
managed to download with dumpgenerator.py:
https://archive.org/details/wikia_dump_20141219
To allow the comparison, the naming format was aligned to the format
used by dumpgenerator.py for 7z files.
10 years ago
Federico Leva ce6fbfee55 Use curl --fail instead and other fixes; add list
Now tested and used to produce the list of some 300k Wikia wikis
which don't yet have a public dump. Will soon be archived.
10 years ago
Federico Leva 7471900e56 It's easier if the list has the actual domains 10 years ago
Federico Leva 8bd3373960 Add wikia.py, to list Wikia wikis we'll dump ourselves 10 years ago
Federico Leva 8cf4d4e6ea Add 30k domains from another crawler
11011 were found alive by checkalive.py (though there could be more
if one checks more subdomains and subdirectories), some thousands
more by checklive.pl (but mostly or all false positives).

Of the alive ones, about 6245 were new to WikiApiary!
https://wikiapiary.com/wiki/Category:Oct_2014_Import
10 years ago
Federico Leva 7e0071ae7f Add some UseModWiki-looking domains 10 years ago
nemobis 6b11cef9dc A few thousands more doku.php URLs from own scraping 10 years ago
Southparkfan 8ca9eb8757 Update date of Orain wikilist 10 years ago
Southparkfan 2e2fe9b818 Update list of Orain wikis 10 years ago
nemobis 23a60fa850 MediaWiki CamelCase 10 years ago
nemobis 31112b3a80 checkalive.py: more checks before accessing stuff 10 years ago
nemobis 225c3eb478 A thousand more doku.php URLs from search 10 years ago
nemobis 3fc7dcb5de Add some more doku.php URLs 10 years ago
PiRSquared17 56c2177106 Add (incomplete) list of dokuwikis 10 years ago
PiRSquared17 03ddde3702 Move wiki lists to mediawiki subdirectory 10 years ago
etesp 1309e89d45 Added information 10 years ago
etesp dab53ea491 Added WikiApiary tropicalwiki api urls 10 years ago
Federico Leva c1e6c0ead3 Merge remote-tracking branch 'upstream/master' 10 years ago
Federico Leva 86c65fc9be Issue 161: add shodan export 10 years ago
Emilio J. Rodríguez-Posada 91cb7bef0c adding info file for wiki-site list 10 years ago
Emilio J. Rodríguez-Posada 603f1aefad adding spider for wiki-site 10 years ago
Emilio J. Rodríguez-Posada 767123e89d updating wiki-site.com list 10 years ago
Emilio J. Rodríguez-Posada 636c6a91df adding spider for neoseeker, updating list, adding info file 10 years ago
Emilio J. Rodríguez-Posada 514d5fea0e removing unused modules 10 years ago
Emilio J. Rodríguez-Posada a3e69666fe adding spider for orain wikifarm, updating list too 10 years ago
Emilio J. Rodríguez-Posada 374cf83c54 adding info file for orain.org wikifarm list 10 years ago
Emilio J. Rodríguez-Posada 7d00cfa0de adding list info file for tropicalwikis 10 years ago
Emilio J. Rodríguez-Posada ecd539f1ae adding scribblewiki list info file 10 years ago
Emilio J. Rodríguez-Posada e95d8ba6e1 sort list 10 years ago
Emilio J. Rodríguez-Posada c420d4d843 adding spider for wikkii, updating the list (10 diff wikis, 2 new, 8 dead), adding info for list 10 years ago
Emilio J. Rodríguez-Posada c7fc194f0d add info file for wikkii.com list 10 years ago
Emilio J. Rodríguez-Posada 29a64507c2 last update date 10 years ago
Emilio J. Rodríguez-Posada d90127e9cc adding details to shoutwiki info 10 years ago
Emilio J. Rodríguez-Posada 90c442a5b7 updating shoutwiki list and uploading basic spider 10 years ago
Emilio J. Rodríguez-Posada 75e2234c2c adding details to referata list 10 years ago
Emilio J. Rodríguez-Posada db9bcb68ca adding license to referata-spider.py 10 years ago
Emilio J. Rodríguez-Posada eaec1afa83 adding info about shoutwiki list 10 years ago
Emilio J. Rodríguez-Posada 1befbabb02 updating info for referata list 10 years ago
Emilio J. Rodríguez-Posada 7a6ef18339 add more wikis to referata list; uploading basic referata-spider.py 10 years ago
Emilio J. Rodríguez-Posada d97c46afd1 Merge branch 'master' of https://github.com/WikiTeam/wikiteam 10 years ago