* Add a new socketutil module with a few helper functions that allow us to:
* Always use a sane User-Agent (previously, only Wikipedia did so)
* Set timeouts in an almost sane manner. Doing it explicitly prevents an interaction with KOSync that does crazy stuff I don't even want to try to understand.
* Unified said timeouts based on the request's intended usage (except for Wikipedia, which already had meaningful timeout values).
* Stopped using LuaSec directly, LuaSocket defers to LuaSec sanely on its own. Everything now transparently supports HTTPS without code duplication.
This change to the parser in `cloudstorage.lua` adds support for the [Apache WebDAV module][1]
It was manually tested using the [bytemark/webdav][2] docker container.
I developed this in Windows, with a docker container that hosted an extracted AppImage and a VNC server that I viewed using a VNC client.
I will write up my work flow at a later point.
Changes have not been tested against other webdav servers (what was this originally tested against?). Please could someone test against other webdav servers?
I also noticed a logic inversion error where we were looking for a slash at the end of the URL and if it exists, then we explicitly set `has_trailing_slash=false` - so I fixed it to set to `true`. I had to do this so that we weren't visiting the URL without a trailing slash - apache sends back a 301 redirect with a `location` header with a trailing slash, if you don't put a trailing slash.
As a side note, I think we should consider replacing this regexp pattern matching parser with the [XML parser in the newsreader plugin[3]
[1]: https://httpd.apache.org/docs/2.4/mod/mod_dav.html
[2]: https://github.com/BytemarkHosting/docker-webdav
[3]: https://github.com/koreader/koreader/blob/master/plugins/newsdownloader.koplugin/lib/xml.lua
Option to show all files in cloudstorage also not supported by KOReader. We can also download any file.
To show all files in CS we need to enable option Show unsupported files in file manager (introducing in #5129)
Close: #5006
Addition of WebDav to the CloudStorage. The functionality is the same as with Dropbox and FTP, it is possible to browse the files on the server and download a copy.
Tested with:
NextCloud
HubZilla
