> **Note:** the `\` and line breaks in the examples below are for legibility in this documentation. `dumpgenerator` can also be run with the arguments in a single line and separated by a single space each.
> **Note:** the `\` and line breaks in the examples below are for legibility in this documentation. Run `dumpgenerator` with the arguments in a single line and a single space between.
### Downloading a wiki with complete XML history and images
If you only want the XML histories, just use `--xml`. For only the images, just `--images`. For only the current version of every page, `--xml --curonly`.
@ -240,11 +232,7 @@ If you only want the XML histories, just use `--xml`. For only the images, just
In the above example, `--path` is only necessary if the download path is not the default.
@ -285,22 +273,23 @@ For the positional parameter `listfile`, `uploader` expects a path to a file tha
`uploader` will search a configurable directory for files with the names generated by `launcher` and upload any that it finds to an Internet Archive item. The item will be created if it does not already exist.
Named arguments:
* `-pd` /`--prune_directories`: After uploading, remove the raw directory generated by `launcher`
* `-pw` /`--prune_wikidump`: After uploading, remove the `wikidump.7z` file generated by `launcher`
* `-c` /`--collection`: Assign the Internet Archive items to the specified collection
* `-a` /`--admin`: Used only if you are an admin of the WikiTeam collection on the Internet Archive
* `-wd` /`--wikidump_dir`: The directory to search for dumps. Defaults to `.`.
* `-u` /`--update`: Update the metadata on an existing Internet Archive item
* `-kf` /`--keysfile`: Path to a file containing Internet Archive API keys. Should contain two lines: the access key, then the secret key. Defaults to `./keys.txt`.
* `-lf` /`--logfile`: Where to store a log of uploaded files (to reduce duplicate work). Defaults to `uploader-X.txt`, where `X` is the final part of the `listfile` path.
Named arguments (short and long versions):
* `-pd`,`--prune_directories`: After uploading, remove the raw directory generated by `launcher`
* `-pw`,`--prune_wikidump`: After uploading, remove the `wikidump.7z` file generated by `launcher`
* `-c`,`--collection`: Assign the Internet Archive items to the specified collection
* `-a`,`--admin`: Used only if you are an admin of the WikiTeam collection on the Internet Archive
* `-wd`,`--wikidump_dir`: The directory to search for dumps. Defaults to `.`.
* `-u`,`--update`: Update the metadata on an existing Internet Archive item
* `-kf`,`--keysfile`: Path to a file containing Internet Archive API keys. Should contain two lines: the access key, then the secret key. Defaults to `./keys.txt`.
* `-lf`,`--logfile`: Where to store a log of uploaded files (to reduce duplicate work). Defaults to `uploader-X.txt`, where `X` is the final part of the `listfile` path.
## Checking dump integrity
If you want to check the XML dump integrity, type this into your command line to count title, page and revision XML tags:
You should see something similar to this (not the actual numbers) - the first three numbers should be the same and the last two should be the same as each other:
@ -315,10 +304,10 @@ You should see something similar to this (not the actual numbers) - the first th
If your first three numbers or your last two numbers are different, then, your XML dump is corrupt (it contains one or more unfinished ```</page>``` or ```</revision>```). This is not common in small wikis, but large or very large wikis may fail at this due to truncated XML pages while exporting and merging. The solution is to remove the XML dump and re-download, a bit boring, and it can fail again...
## WikiTeam Team
## Contributors
**WikiTeam** is the [Archive Team](http://www.archiveteam.org) [[GitHub](https://github.com/ArchiveTeam)] subcommittee on wikis.
It was founded and originally developed by [Emilio J. Rodríguez-Posada](https://github.com/emijrp), a Wikipedia veteran editor and amateur archivist. Thanks to people who have helped, especially to: [Federico Leva](https://github.com/nemobis), [Alex Buie](https://github.com/ab2525), [Scott Boyd](http://www.sdboyd56.com), [Hydriz](https://github.com/Hydriz), Platonides, Ian McEwen, [Mike Dupont](https://github.com/h4ck3rm1k3), [balr0g](https://github.com/balr0g) and [PiRSquared17](https://github.com/PiRSquared17).
The Python 3 initiative is currently being led by [Elsie Hupp](https://github.com/elsiehupp), with contributions from [Victor Gambier](https://github.com/vgambier), [Thomas Karcher](https://github.com/t-karcher), and [Janet Cobb](https://github.com/randomnetcat).
**MediaWiki Scraper**
The Python 3 initiative is currently being led by [Elsie Hupp](https://github.com/elsiehupp), with contributions from [Victor Gambier](https://github.com/vgambier), [Thomas Karcher](https://github.com/t-karcher), [Janet Cobb](https://github.com/randomnetcat), [yzqzss](https://github.com/yzqzss), [NyaMisty](https://github.com/NyaMisty) and [Rob Kam](https://github.com/robkam)