Emilio J. Rodríguez-Posada
edb08753d2
Merge branch 'master' of https://github.com/WikiTeam/wikiteam
10 years ago
Emilio J. Rodríguez-Posada
971404ac4c
Merge pull request #155 from balr0g/api-or-index-fix
...
Fix --index
10 years ago
balr0g
fd6ea19b4b
config['api'] is set but empty; properly handle this
10 years ago
nemobis
1ff96238eb
Denote as alpha until revamp is tested
...
Per emijrp who asked not to run dumps with this, at https://github.com/WikiTeam/wikiteam/issues/104#issuecomment-48039143
Currently proposed things to fix or check: https://github.com/WikiTeam/wikiteam/issues?milestone=1&state=open
10 years ago
nemobis
37a4f4fe26
Remember requirements
...
Per https://github.com/WikiTeam/wikiteam/issues/104#issuecomment-48052554
10 years ago
Emilio J. Rodríguez-Posada
91cb7bef0c
adding info file for wiki-site list
10 years ago
Emilio J. Rodríguez-Posada
603f1aefad
adding spider for wiki-site
10 years ago
Emilio J. Rodríguez-Posada
767123e89d
updating wiki-site.com list
10 years ago
Emilio J. Rodríguez-Posada
28643cd5fc
adding more tests: titles
10 years ago
Emilio J. Rodríguez-Posada
89e3c3e462
standarize getImage* functions names
10 years ago
Emilio J. Rodríguez-Posada
78f04a4bcd
adding more tests for image lists
10 years ago
Emilio J. Rodríguez-Posada
aaa1822759
improving image list downloader
10 years ago
Emilio J. Rodríguez-Posada
88c9468c0e
improving image list downloader
10 years ago
Emilio J. Rodríguez-Posada
53802bd8b8
Merge pull request #151 from balr0g/flake8-cleanup
...
Cleanups and error fixes suggested by flake8 (pep8 + pyflakes)
10 years ago
balr0g
3929e4eb9c
Cleanups and error fixes suggested by flake8 (pep8 + pyflakes)
10 years ago
Hydriz Scholz
421df6d4bb
Spliting rewrite files into a separate branch
10 years ago
Emilio J. Rodríguez-Posada
c07b527e5d
adding session to getWikiEngine()
10 years ago
Emilio J. Rodríguez-Posada
30c153ce1f
chg: using 'with open' for files
10 years ago
Emilio J. Rodríguez-Posada
10d269deb0
Merge branch 'balr0g-requests'
10 years ago
balr0g
1391cf893b
Update requirements file
10 years ago
balr0g
9aa3c4a0e1
Removed all traces of urllib except for encode/decode; more bugs fixed.
10 years ago
balr0g
c8e11a949b
Initial port to Requests
10 years ago
Emilio J. Rodríguez-Posada
9553e3550c
adding wiki engine detector
10 years ago
Emilio J. Rodríguez-Posada
eb97cf1adf
version 0.2.2 and tiny bits in --help
10 years ago
Emilio J. Rodríguez-Posada
fc2ed3ed0d
Merge pull request #138 from balr0g/argparse
...
Initial port to argparse
10 years ago
Emilio J. Rodríguez-Posada
636c6a91df
adding spider for neoseeker, updating list, adding info file
10 years ago
Emilio J. Rodríguez-Posada
514d5fea0e
removing unused modules
10 years ago
Emilio J. Rodríguez-Posada
a3e69666fe
adding spider for orain wikifarm, updating list too
10 years ago
Emilio J. Rodríguez-Posada
374cf83c54
adding info file for orain.org wikifarm list
10 years ago
balr0g
32fded6e8a
Add requirements file
10 years ago
Emilio J. Rodríguez-Posada
7c76b77842
revamping image list tests
10 years ago
Emilio J. Rodríguez-Posada
3a8fa1f0cc
adding test for getImageFilenamesURL(), the image scraper
10 years ago
Emilio J. Rodríguez-Posada
5f0ca74d87
adding simple test for delay(), 0 and 3 seconds
10 years ago
balr0g
50b011f90d
Initial port to argparse
10 years ago
Antoine Musso
8a8b87225b
Easily run tests in a virtualenv with tox and nose
...
tox is a nice wrapper around virtualenv. It setup a new env that makes
sure you have a clean install without any module beside the standard
one. If you need more modules, simply add them to 'deps =' and tox will
install them from pypi.
nose makes testing easier, it find any files that looks like a test and
run them for you.
The envs are created in /.tox so ignore that directory entirely.
10 years ago
Emilio J. Rodríguez-Posada
09be8b5f22
prints
10 years ago
Emilio J. Rodríguez-Posada
3da6bc222f
ignoring .pyc files
10 years ago
Emilio J. Rodríguez-Posada
d04c0e5bce
first test
10 years ago
Emilio J. Rodríguez-Posada
568deef081
adding comments for clarification
10 years ago
Emilio J. Rodríguez-Posada
d4eed1f738
fixing #127 and #134 , now works with APIs that returns 'name' field for images and those that don't do it (in this case we unquote over ascii); also fixing bug that re-download image list when it was completed previously
10 years ago
Emilio J. Rodríguez-Posada
005de23c1d
adding gzip to siteinfo downloader
10 years ago
Emilio J. Rodríguez-Posada
d79ea64d41
fixing issue #97 pretty siteinfo json saving, indenting 4 chars
10 years ago
Emilio J. Rodríguez-Posada
3854a344fe
Merge branch 'master' of https://github.com/WikiTeam/wikiteam
10 years ago
Emilio J. Rodríguez-Posada
1c1f0dbb86
replacing XML with JSON in image downloading
10 years ago
Emilio J. Rodríguez-Posada
6bbdf42ed0
Merge pull request #125 from balr0g/check-for-disabled-api
...
Don't try to download sites with disabled API
10 years ago
balr0g
481323c7f7
Don't try to download sites with disabled API
10 years ago
Emilio J. Rodríguez-Posada
084ccc6456
Merge branch 'master' of https://github.com/WikiTeam/wikiteam
10 years ago
nemobis
1933db8a94
Merge pull request #124 from balr0g/scraper-unicode-title-fix
...
Fix scraper for sites with Unicode titles
10 years ago
balr0g
62be069026
Fix scraper for sites with Unicode titles
10 years ago
nemobis
dbfc1b598a
Ask Wikimedia Commons reseed
10 years ago