@ -12,12 +12,91 @@ Most of the focus has been on the core `dumpgenerator` tool, but Python 3 versio
The main general-purpose module of `wikiteam3` is `dumpgenerator`, which can download XML dumps of MediaWiki sites that can then be parsed or redeployed elsewhere.
## Python Environment
`wikiteam3` requires [Python 3.8](https://www.python.org/downloads/release/python-380/) or later (less than 4.0), but you may be able to get it run with earlier versions of Python 3. On recent versions of Linux and macOS Python 3.8 should come preinstalled, but on Windows you will need to install it from [python.org](https://www.python.org/downloads/release/python-380/).
`wikiteam3` has been tested on Linux, macOS, Windows and Android. If you are connecting to Linux or macOS via `ssh`, you can continue using the `bash` or `zsh` command prompt in the same terminal, but if you are starting in a desktop environment and don't already have a preferred Terminal environment you can try one of the following.
> **NOTE:** You may need to update and pre-install dependencies in order for `wikiteam3` to work properly. Shell commands for these dependencies appear below each item in the list. (Also note that while installing and running `wikiteam3` itself should not require administrative priviliges, installing dependencies usually will.)
* On desktop Linux you can use the default terminal application such as [Konsole](https://konsole.kde.org/) or [GNOME Terminal](https://help.gnome.org/users/gnome-terminal/stable/).
<details>
<summary>Linux Dependencies</summary>
While most Linux distributions will have Python 3 preinstalled, if you are cloning `wikiteam3` rather than downloading it directly you may need to install `git`.
(On Fedora, Arch, etc., use `dnf`, `pacman`, etc., instead.)
</details>
* On macOS you can use the built-in application [Terminal](https://support.apple.com/guide/terminal), which is found in `Applications/Utilities`.
<details>
<summary>macOS Dependencies</summary>
While macOS will have Python 3 preinstalled, if you are cloning `wikiteam3` rather than downloading it directly and you are using an older versions of macOS, you may need to install `git`.
If `git` is not preinstalled, however, macOS will prompt you to install it the first time you run the command. Therefore, to check whether you have `git` installed or to install `git`, simply run `git` (with no arguments) in Terminal:
```bash
git
```
If `git` is already installed, it will print its usage instructions. If `git` is not preinstalled, the command will pop up a window asking if you want to install Apple's command line developer tools, and clicking "Install" in the popup window will install `git`.
</details>
* On Windows 10 or Windows 11 you can use [Windows Terminal](https://aka.ms/terminal).
<details>
<summary>Windows Dependencies</summary>
If you are already using the [Windows Subsystem for Linux](https://learn.microsoft.com/en-us/windows/wsl/about), you can follow the Linux instructions above. If you don't want to install a full WSL distribution, [Git for Windows](https://gitforwindows.org/) provides Bash emulation, so you can use it as a more lightweight option instead.
> When installing [Python 3.8](https://www.python.org/downloads/release/python-380/) (from python.org), be sure to check "Add Python to PATH" so that installed Python scripts are accessible from any location. If for some reason installed Python scripts, e.g. `pip`, are not available from any location, you can add Python to the `PATH` environment variable using the instructions [here](https://datatofish.com/add-python-to-windows-path/).
>
> And while doing so should not be necessary if you follow the instructions further down and install `wikiteam3` using `pip`, if you'd prefer that Windows store installed Python scripts somewhere other than the default Python folder under `%appdata%`, you can also add your preferred alternative path such as `C:\Program Files\Python3\Scripts\` or a subfolder of `My Documents`. (You will need to restart any terminal sessions in order for this to take effect.)
Whenever you'd like to run a Bash session, you can open a Bash terminal prompt from any folder in Windows Explorer by right-clicking and choosing the option from the context menu. (For some purposes you may wish to run Bash as an administrator.) This way you can open a Bash prompt and clone the `wikiteam3` repository in one location, and subsequently or later open another Bash prompt and run `wikiteam3` to dump a wiki wherever else you'd like without having to browse to the directory manually using Bash.
</details>
* On Android you can use [Termux](https://termux.dev).
> **Note:** iSH may automatically quit if your iOS device goes to sleep, and it may lose its status if you switch to another app. You can disable auto-sleep while iSH is running by clicking the gear icon and toggling "Disable Screen Dimming". (You may wish to connect your device to a charger while running iSH.)
</details>
## Using `dumpgenerator`
The Python 3 port of the `dumpgenerator` module of `wikiteam3` is largely functional and can be installed from a downloaded or cloned copy of this repository.
> `wikiteam3` requires [Python 3.8](https://www.python.org/downloads/release/python-380/) or later (less than 4.0), but you may be able to get it run with earlier versions of Python 3.
There are two versions of these instructions:
* If you just want to use a version that mostly works
@ -31,37 +110,58 @@ There are two versions of these instructions:
### If you just want to use a version that mostly works
#### 1. Download and install
#### 1. Downloading and installing `wikiteam3`
In whatever folder you use for cloned repositories:
#### 2. Run `dumpgenerator` for whatever purpose you need
```bash
cd wikiteam3
```
```bash
$ dumpgenerator [args]
git checkout --track origin/python3
```
#### 3. To uninstall the package and delete the cloned repository when you're done
```bash
pip install --force-reinstall dist/*.whl
```
#### 2. Running `dumpgenerator` for whatever purpose you need
```bash
dumpgenerator [args]
```
#### 3. Uninstalling the package and deleting the cloned repository when you're done
```shell
$ pip uninstall wikiteam3
$ rm -r [cloned_wikiteam3_folder]
pip uninstall wikiteam3
```
```bash
rm -r [cloned_wikiteam3_folder]
```
If you'd like to manually build and install `wikiteam3` from a cloned or downloaded copy of this repository, run the following commands from the downloaded base directory:
If you'd like to manually install `wikiteam3` from a cloned or downloaded copy of this repository, run the following commands from the downloaded base directory:
> **Note:** Re-run the following steps each time to reinstall each time the `wikiteam3` branch is updated.
```shell
$ git pull
$ poetry update && poetry install && poetry build
$ pip install --force-reinstall dist/*.whl
git pull
```
```bash
poetry update && poetry install && poetry build
```
```bash
pip install --force-reinstall dist/*.whl
```
#### 4. Then, from anywhere, you should be able to run
```shell
$ dumpgenerator [args]
dumpgenerator [args]
```
> To run the test suite, run:
>
> ```bash
> $ test-dumpgenerator
> test-dumpgenerator
> ```
#### 5. To uninstall the package and delete the cloned repository when you're done
#### 5. Uninstalling the package and deleting the cloned repository when you're done
```shell
$ pip uninstall wikiteam3
$ rm -r [cloned_wikiteam3_folder]
pip uninstall wikiteam3
```
```bash
rm -r [cloned_wikiteam3_folder]
```
### Using `dumpgenerator`
### Using `dumpgenerator` (once installed)
After installing `wikiteam3` using `pip` you should be able to use the `dumpgenerator` command from any local directory.
@ -139,37 +251,57 @@ For basic usage, you can run `dumpgenerator` in the directory where you'd like t
For a brief summary of the `dumpgenerator` command-line options:
```bash
$ dumpgenerator --help
dumpgenerator --help
```
Several examples follow:
Several examples follow.
#### To download a wiki with omplete XML histories and images:
> **Note:** the `\` and line breaks in the examples below are for legibility in this documentation. `dumpgenerator` can also be run with the arguments in a single line and separated by a single space each.
#### Downloading a wiki with complete XML history and images
If you only want the XML histories, just use `--xml`. For only the images, just `--images`. For only the current version of every page, `--xml --current`.
Elsie Hupp
2 years ago
committed by
GitHub