Commit Graph

10 Commits (731e34299d128f9352fd76e603c960c1f0628ed9)

Author SHA1 Message Date
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection.
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
5 years ago
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages"
This reverts commit 4d1770398a.
6 years ago
Marc Abonce Seguin 5568f24d6c [fix] check language aliases when setting search language 6 years ago
Léo Bourrel 0a37f90990 Fix wikidata tests with updated path to media 6 years ago
Marc Abonce Seguin 9687786226 update unit tests for google news and wikidata 7 years ago
Marc Abonce Seguin 772c048d01 refactor engine's search language handling
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
7 years ago
marc 4d1770398a remove 'all' option from search languages 7 years ago
Adam Tauber 52e615dede [enh] py3 compatibility 7 years ago
marc ad58b14be7 [fix] merge infoboxes based on weight
also minor changes in attributes and images from wikidata
8 years ago
marc a0a1284998 wikidata refactor and more attributes (see issue #560) 8 years ago