2023-06-01 00:54:05 +00:00
|
|
|
# MediaWikiDump
|
|
|
|
|
|
|
|
|
|
>[MediaWiki XML Dumps](https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps) contain the content of a wiki
|
|
|
|
|
> (wiki pages with all their revisions), without the site-related data. A XML dump does not create a full backup
|
|
|
|
|
> of the wiki database, the dump does not contain user accounts, images, edit logs, etc.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## Installation and Setup
|
|
|
|
|
|
|
|
|
|
We need to install several python packages.
|
|
|
|
|
|
|
|
|
|
The `mediawiki-utilities` supports XML schema 0.11 in unmerged branches.
|
|
|
|
|
```bash
|
|
|
|
|
pip install -qU git+https://github.com/mediawiki-utilities/python-mwtypes@updates_schema_0.11
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The `mediawiki-utilities mwxml` has a bug, fix PR pending.
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
pip install -qU git+https://github.com/gdedrouas/python-mwxml@xml_format_0.11
|
|
|
|
|
pip install -qU mwparserfromhell
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Document Loader
|
|
|
|
|
|
2023-06-16 18:52:56 +00:00
|
|
|
See a [usage example](/docs/modules/data_connection/document_loaders/integrations/mediawikidump.html).
|
2023-06-01 00:54:05 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
from langchain.document_loaders import MWDumpLoader
|
|
|
|
|
```
|
