2
0
mirror of https://github.com/koreader/koreader synced 2024-11-20 03:25:34 +00:00
Commit Graph

10 Commits

Author SHA1 Message Date
roygbyte
a4763a613f
[chore] Newsdownloader: Fix more luacheck warnings (#8220) 2021-09-16 09:10:03 +02:00
NiLuJe
2f9db25969
Unify LuaSocket usage (#7405)
* Add a new socketutil module with a few helper functions that allow us to:
  * Always use a sane User-Agent (previously, only Wikipedia did so)
  * Set timeouts in an almost sane manner. Doing it explicitly prevents an interaction with KOSync that does crazy stuff I don't even want to try to understand.
* Unified said timeouts based on the request's intended usage (except for Wikipedia, which already had meaningful timeout values).
* Stopped using LuaSec directly, LuaSocket defers to LuaSec sanely on its own. Everything now transparently supports HTTPS without code duplication.
2021-03-15 01:25:10 +01:00
Mikolaj "lich" H
b741fcebac
[NewsDownloader] Added an HTML filter through a CSS selector (#6228)
Fixes #6185.
2020-06-04 22:58:14 +02:00
Frans de Jonge
a2dcfe9aec
[doc] Tag @todo, @fixme and @warning (#5244)
This commit standardizes the various todos around the code a bit in a manner recognized by LDoc.

Besides drawing more attention by being displayed in the developer docs, they're also extractable with LDoc on the command line:

```sh
ldoc --tags todo,fixme *.lua
```

However, whether that particular usage offers any advantage over other search tools is questionable at best.

* and some random beautification
2019-08-23 19:53:53 +02:00
Tom Hall
1e9074ca6a NewsDownloader: Allow including images when using text from feed (#5040)
Split the html page download out of createEpub, so that createFromDescription
can pass in its own html for epub packaging (with optional image download).
Still pass in a url, so that relative urls can be made absolute.
2019-05-26 13:46:25 +02:00
Tom Hall
38f3bfdde7 NewsDownloader: Improve info shown on UI during processing (#5030)
Pass strings into functions, so that they can add their own info to the
UI without overwriting the information shown by the calling function.
2019-05-15 21:31:46 +02:00
Sergey Avseyev
f6e561923c newsdownloader: rebuild URL on redirect to bare path (#5012)
The server might redirect to only path, in this case the scheme and host
should be taken from original URL
2019-05-08 12:41:04 +02:00
Tom Hall
8e23d2a937 NewsDownloader: Process articles sequentially
Lift the Trapper:wrap call out of the individual article processing code,
so that articles are processed one by one, in order to:
* Avoid concurrent progress updates fighting over the UI dialog
* Avoid trying to download many images at the same time
2019-04-30 21:03:48 +02:00
Tom Hall
cad4d25d24 NewsDownloader: Add flag 'include_images' to feed config
Allow the user to specify whether to download images for each individual
feed specified in feed_config.lua. Default to false to stay closest to
existing behaviour.
2019-04-30 21:03:48 +02:00
Tom Hall
1588d4c64a NewsDownloader: Download images and output EPUBs
Initial commit of new NewsDownloader which downloads images as well as text
and packs it all into an epub.

Based on Wikipedia "Download as EPUB" code.
2019-04-30 21:03:48 +02:00