Commit Graph

221 Commits (4bc43a1c0fdb59a29ce446ce2255b66011381bec)

Author SHA1 Message Date
Emilio J. Rodríguez-Posada 4bc43a1c0f improved help messages 10 years ago
Emilio J. Rodríguez-Posada 51806f5a3d fixed #160; improved args parsing and --help; improved API/Index estimate from URL; 10 years ago
Emilio J. Rodríguez-Posada dd7df0cc01 Merge branch 'master' of https://github.com/WikiTeam/wikiteam 10 years ago
Emilio J. Rodríguez-Posada f3b388fc79 a first approach to auto-detect API/Index.php using URL to the Main_Page 10 years ago
Erkan Yilmaz 44b80ceb88 fix link for tutorial 10 years ago
balr0g 8485a5004d Pass session 10 years ago
balr0g fd6ea19b4b config['api'] is set but empty; properly handle this 10 years ago
nemobis 1ff96238eb Denote as alpha until revamp is tested
Per emijrp who asked not to run dumps with this, at https://github.com/WikiTeam/wikiteam/issues/104#issuecomment-48039143
Currently proposed things to fix or check: https://github.com/WikiTeam/wikiteam/issues?milestone=1&state=open
10 years ago
Emilio J. Rodríguez-Posada 89e3c3e462 standarize getImage* functions names 10 years ago
Emilio J. Rodríguez-Posada aaa1822759 improving image list downloader 10 years ago
Emilio J. Rodríguez-Posada 88c9468c0e improving image list downloader 10 years ago
balr0g 3929e4eb9c Cleanups and error fixes suggested by flake8 (pep8 + pyflakes) 10 years ago
Emilio J. Rodríguez-Posada c07b527e5d adding session to getWikiEngine() 10 years ago
Emilio J. Rodríguez-Posada 30c153ce1f chg: using 'with open' for files 10 years ago
balr0g 9aa3c4a0e1 Removed all traces of urllib except for encode/decode; more bugs fixed. 10 years ago
balr0g c8e11a949b Initial port to Requests 10 years ago
Emilio J. Rodríguez-Posada 9553e3550c adding wiki engine detector 10 years ago
Emilio J. Rodríguez-Posada eb97cf1adf version 0.2.2 and tiny bits in --help 10 years ago
balr0g 50b011f90d Initial port to argparse 10 years ago
Emilio J. Rodríguez-Posada 568deef081 adding comments for clarification 10 years ago
Emilio J. Rodríguez-Posada d4eed1f738 fixing #127 and #134 , now works with APIs that returns 'name' field for images and those that don't do it (in this case we unquote over ascii); also fixing bug that re-download image list when it was completed previously 10 years ago
Emilio J. Rodríguez-Posada 005de23c1d adding gzip to siteinfo downloader 10 years ago
Emilio J. Rodríguez-Posada d79ea64d41 fixing issue #97 pretty siteinfo json saving, indenting 4 chars 10 years ago
Emilio J. Rodríguez-Posada 3854a344fe Merge branch 'master' of https://github.com/WikiTeam/wikiteam 10 years ago
Emilio J. Rodríguez-Posada 1c1f0dbb86 replacing XML with JSON in image downloading 10 years ago
balr0g 481323c7f7 Don't try to download sites with disabled API 10 years ago
nemobis 1933db8a94 Merge pull request #124 from balr0g/scraper-unicode-title-fix
Fix scraper for sites with Unicode titles
10 years ago
balr0g 62be069026 Fix scraper for sites with Unicode titles 10 years ago
nemobis 62d961fa97 Fix typo, unused variable spotted by balrog 10 years ago
nemobis 95bc2dec38 Link GitHub issue tracker 10 years ago
balr0g d60e560571 Add Content-Encoding: gzip support 10 years ago
Emilio J. Rodríguez-Posada 5261811fa4 only if api exists 10 years ago
Emilio J. Rodríguez-Posada 610764619a add saveSiteInfo() to download meta=siteinfo data from API to a file 10 years ago
Emilio J. Rodríguez-Posada d395433513 comments and newlines 10 years ago
Emilio J. Rodríguez-Posada 5eff4bd072 comments and tabs 10 years ago
Emilio J. Rodríguez-Posada 0b0c40f5da adding more user-agents, but keeps the first as default by now 10 years ago
Emilio J. Rodríguez-Posada 81468c4a7c using JSON to retrieve namespaces via API 10 years ago
Emilio J. Rodríguez-Posada 703eb9011b improving checkAPI() using JSON properly loaded 10 years ago
Emilio J. Rodríguez-Posada 44d3fe1e36 Merge pull request #117 from nemobis/bug/48
Issue 48: Check that API actually works
10 years ago
Emilio J. Rodríguez-Posada fc80556d8a merging... 10 years ago
Emilio J. Rodríguez-Posada f474deb71f now we use JSON properly in getPageTitlesAPI(), instead of XML; fixing some wrong prints, now support utf-8 10 years ago
Federico Leva 997276110c Issue 46: dumpgenerator should follow redirects
Patch by @balr0g from libsonic (GPLv3+).
10 years ago
Federico Leva a8e1575879 Issue 48: Check that API actually works 10 years ago
Emilio J. Rodríguez-Posada c9aa165504 fixing header with the new year, info and documentation link 10 years ago
nemobis ac4c93c12a Issue 85: more cross-platform shebang on all scripts
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@962 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
10 years ago
nemobis 403dc213ef Issue 71: English-only match for an older case
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@942 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
11 years ago
nemobis 0ede45b7cf Special:BadTitle works only in English wikis
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@917 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
11 years ago
nemobis 034866a32e Handle permissions-errors for wikis requiring login or whatever
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@916 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
11 years ago
nemobis 6c69d9800f Followup, delay needs config; should be BC
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@914 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
11 years ago
nemobis 55185467e1 Add delay to all checking and listing functions, crappy hosts die on them
git-svn-id: https://wikiteam.googlecode.com/svn/trunk@902 31edc4fc-5e31-b4c4-d58b-c8bc928bcb95
11 years ago