**MediaWiki Scraper can archive wikis from the largest to the tiniest.**
`wikiteam3` is an ongoing project to port the legacy [`wikiteam`](https://github.com/WikiTeam/wikiteam) toolset to Python 3 and PyPI to make it more accessible for today's archivers.
`MediaWiki Scraper` is an ongoing project to port the legacy [`wikiteam`](https://github.com/WikiTeam/wikiteam) toolset to Python 3 and PyPI to make it more accessible for today's archivers.
Most of the focus has been on the core `dumpgenerator` tool, but Python 3 versions of the other `wikiteam` tools may be added over time.
@ -16,14 +16,14 @@ MediaWiki Scraper is a set of tools for archiving wikis. The main general-purpos
`MediaWiki Scraper` has been tested on Linux, macOS, Windows and Android. If you are connecting to Linux or macOS via `ssh`, you can continue using the `bash` or `zsh` command prompt in the same terminal, but if you are starting in a desktop environment and don't already have a preferred Terminal environment you can try one of the following.
> **NOTE:** You may need to update and pre-install dependencies in order for `wikiteam3` to work properly. Shell commands for these dependencies appear below each item in the list. (Also note that while installing and running `wikiteam3` itself should not require administrative priviliges, installing dependencies usually will.)
> **NOTE:** You may need to update and pre-install dependencies in order for `MediaWiki Scraper` to work properly. Shell commands for these dependencies appear below each item in the list. (Also note that while installing and running `MediaWiki Scraper` itself should not require administrative priviliges, installing dependencies usually will.)
* On desktop Linux you can use the default terminal application such as [Konsole](https://konsole.kde.org/) or [GNOME Terminal](https://help.gnome.org/users/gnome-terminal/stable/).
<details>
<summary>Linux Dependencies</summary>
While most Linux distributions will have Python 3 preinstalled, if you are cloning `wikiteam3` rather than downloading it directly you may need to install `git`.
While most Linux distributions will have Python 3 preinstalled, if you are cloning `MediaWiki Scraper` rather than downloading it directly you may need to install `git`.
On Debian, Ubuntu, and the like:
@ -40,7 +40,7 @@ MediaWiki Scraper is a set of tools for archiving wikis. The main general-purpos
<details>
<summary>macOS Dependencies</summary>
While macOS will have Python 3 preinstalled, if you are cloning `wikiteam3` rather than downloading it directly and you are using an older versions of macOS, you may need to install `git`.
While macOS will have Python 3 preinstalled, if you are cloning `MediaWiki Scraper` rather than downloading it directly and you are using an older versions of macOS, you may need to install `git`.
If `git` is not preinstalled, however, macOS will prompt you to install it the first time you run the command. Therefore, to check whether you have `git` installed or to install `git`, simply run `git` (with no arguments) in Terminal:
@ -63,9 +63,9 @@ MediaWiki Scraper is a set of tools for archiving wikis. The main general-purpos
> When installing [Python 3.8](https://www.python.org/downloads/release/python-380/) (from python.org), be sure to check "Add Python to PATH" so that installed Python scripts are accessible from any location. If for some reason installed Python scripts, e.g. `pip`, are not available from any location, you can add Python to the `PATH` environment variable using the instructions [here](https://datatofish.com/add-python-to-windows-path/).
>
> And while doing so should not be necessary if you follow the instructions further down and install `wikiteam3` using `pip`, if you'd prefer that Windows store installed Python scripts somewhere other than the default Python folder under `%appdata%`, you can also add your preferred alternative path such as `C:\Program Files\Python3\Scripts\` or a subfolder of `My Documents`. (You will need to restart any terminal sessions in order for this to take effect.)
> And while doing so should not be necessary if you follow the instructions further down and install `MediaWiki Scraper` using `pip`, if you'd prefer that Windows store installed Python scripts somewhere other than the default Python folder under `%appdata%`, you can also add your preferred alternative path such as `C:\Program Files\Python3\Scripts\` or a subfolder of `My Documents`. (You will need to restart any terminal sessions in order for this to take effect.)
Whenever you'd like to run a Bash session, you can open a Bash terminal prompt from any folder in Windows Explorer by right-clicking and choosing the option from the context menu. (For some purposes you may wish to run Bash as an administrator.) This way you can open a Bash prompt and clone the `wikiteam3` repository in one location, and subsequently or later open another Bash prompt and run `wikiteam3` to dump a wiki wherever else you'd like without having to browse to the directory manually using Bash.
Whenever you'd like to run a Bash session, you can open a Bash terminal prompt from any folder in Windows Explorer by right-clicking and choosing the option from the context menu. (For some purposes you may wish to run Bash as an administrator.) This way you can open a Bash prompt and clone the `MediaWiki Scraper` repository in one location, and subsequently or later open another Bash prompt and run `MediaWiki Scraper` to dump a wiki wherever else you'd like without having to browse to the directory manually using Bash.
</details>
@ -102,13 +102,13 @@ There are two versions of these instructions:
* If you just want to use a version that mostly works
* If you want to follow my progress and help me test my latest commit
> If you run into a problem with the version that mostly works, you can [open an Issue](https://github.com/elsiehupp/wikiteam3/issues/new/choose). Be sure to include the following:
> If you run into a problem with the version that mostly works, you can [open an Issue](https://github.com/mediawiki-client-tools/mediawiki-scraper/issues/new/choose). Be sure to include the following:
>
> 1. The operating system you're using
> 2. What command you ran that didn't work
> 3. What output was printed to your terminal
### 1. Downloading and installing `wikiteam3`
### 1. Downloading and installing `MediaWiki Scraper`
In whatever folder you use for cloned repositories:
### 5. Manually build and install `MediaWiki Scraper`
If you'd like to manually build and install `wikiteam3` from a cloned or downloaded copy of this repository, run the following commands from the downloaded base directory:
If you'd like to manually build and install `MediaWiki Scraper` from a cloned or downloaded copy of this repository, run the following commands from the downloaded base directory:
Rob Kam
1 year ago
committed by
GitHub