2
0
mirror of https://github.com/koreader/koreader synced 2024-11-13 19:11:25 +00:00
Commit Graph

56 Commits

Author SHA1 Message Date
Scarlett
8a04dc9852
NewsDownloader: new option to allow EPUB volumization (#8263)
When this feature is enabled on a feed and that feed is
synced, all new feed entries will be collected into a new
single EPUB file. This is achieved by implementing a feed
history feature (downloaded feeds are added as M5D hashes
to a LuaSettings file), and by introducing additional
methods into epubdownloader.lua that allow for multiple
HTML documents to be added into single EPUB file.
2022-01-21 19:23:21 +01:00
Frans de Jonge
a21b5be64a
[chore] NewsDownloader: various minor string fixes (#8268)
Follow-up to <https://github.com/koreader/koreader/pull/8267>.
2021-09-26 12:44:43 +02:00
Frans de Jonge
e865fb9114
[chore] NewsDownloader: remove stray space (#8267) 2021-09-26 12:30:17 +02:00
roygbyte
a4763a613f
[chore] Newsdownloader: Fix more luacheck warnings (#8220) 2021-09-16 09:10:03 +02:00
roygbyte
0af6500c88
[chore] Newsdownloader: Fix warnings from luacheck (#8214)
* Fix warnings from luacheck

* Fix punctuation and casing in comments.
2021-09-14 07:42:32 +02:00
roygbyte
b0ceab5b20
[plugin] NewsDownloader: fix XML, better error messages, change default feed, and more (#8145)
* Fix XML, introduce some better error messages, etc.

See roygbyte/newsdownloader.koplugin for more info

* Fix feed attribute saving snafu; Change menu labels; etc.

Also:
- Change default feed
- Change "remove" label to "delete"
- Hide menu after feed sync
2021-09-13 21:04:06 +02:00
NiLuJe
2f9db25969
Unify LuaSocket usage (#7405)
* Add a new socketutil module with a few helper functions that allow us to:
  * Always use a sane User-Agent (previously, only Wikipedia did so)
  * Set timeouts in an almost sane manner. Doing it explicitly prevents an interaction with KOSync that does crazy stuff I don't even want to try to understand.
* Unified said timeouts based on the request's intended usage (except for Wikipedia, which already had meaningful timeout values).
* Stopped using LuaSec directly, LuaSocket defers to LuaSec sanely on its own. Everything now transparently supports HTTPS without code duplication.
2021-03-15 01:25:10 +01:00
NiLuJe
bf6c0cdd6c
LuaSettings: Add a method to initialize a setting properly (#7371)
* LuaSettings/DocSettings: Updated readSetting API to allow proper initialization to default.
Use it to initialize tables, e.g., fixing corner-cases in readerFooter that could prevent settings from being saved.
(Fixes an issue reported on Gitter).
* LuaSettings/DocSettings: Add simpler API than the the flip* ones to toggle boolean settings.
* Update LuaSettings/DocSettigns usage throughout the codebase to use the dedicated boolean methods wher appropriate, and clean up some of the more mind-bending uses.
* FileChooser: Implement an extended default exclusion list (fix #2360)
* ScreenSaver: Refactor to avoid the pile of kludges this was threatening to become. Code should be easier to follow and use, and fallbacks now behave as expected (fix #4418).
2021-03-06 22:44:18 +01:00
Frans de Jonge
1bd4636a03
Standardize directory/folder to folder (#7328)
Closes <https://github.com/koreader/koreader/issues/7157>.
2021-02-22 18:44:16 +01:00
NiLuJe
37a01100b7
Various Wi-Fi QoL improvements (#6424)
* Revamped most actions that require an internet connection to a new/fixed backend that allows forwarding the initial action and running it automatically once connected. (i.e., it'll allow you to set "Action when Wi-Fi is off" to "turn_on", and whatch stuff connect and do what you wanted automatically without having to re-click anywhere instead of showing you a Wi-Fi prompt and then not doing anything without any other feedback).
* Speaking of, fixed the "turn_on" beforeWifi action to, well, actually work. It's no longer marked as experimental.
* Consistently use "Wi-Fi" everywhere.
* On Kobo/Cervantes/Sony, implemented a "Kill Wi-Fi connection when inactive" system that will automatically disconnect from Wi-Fi after sustained *network* inactivity (i.e., you can keep reading, it'll eventually turn off on its own). This should be smart and flexible enough not to murder Wi-Fi while you need it, while still not keeping it uselessly on and murdering your battery.
(i.e., enable that + turn Wi-Fi on when off and enjoy never having to bother about Wi-Fi ever again).
* Made sending `NetworkConnected` / `NetworkDisconnected` events consistent (they were only being sent... sometimes, which made relying on 'em somewhat problematic).
* restoreWifiAsync is now only run when really needed (i.e., we no longer stomp on an existing working connection just for the hell of it).
* We no longer attempt to kill a bogus non-existent Wi-Fi connection when going to suspend, we only do it when it's actually needed.
* Every method of enabling Wi-Fi will now properly tear down Wi-Fi on failure, instead of leaving it in an undefined state.
* Fixed an issue in the fancy crash screen on Kobo/reMarkable that could sometime lead to the log excerpt being missing.
* Worked-around a number of sneaky issues related to low-level Wi-Fi/DHCP/DNS handling on Kobo (see the lengthy comments [below](https://github.com/koreader/koreader/pull/6424#issuecomment-663881059) for details). Fix #6421 
Incidentally, this should also fix the inconsistencies experienced re: Wi-Fi behavior in Nickel when toggling between KOReader and Nickel (use NM/KFMon, and run a current FW for best results).
* For developers, this involves various cleanups around NetworkMgr and NetworkListener. Documentation is in-line, above the concerned functions.
2020-07-27 03:39:06 +02:00
Mikolaj "lich" H
b741fcebac
[NewsDownloader] Added an HTML filter through a CSS selector (#6228)
Fixes #6185.
2020-06-04 22:58:14 +02:00
mwoz123
0e2b430796
Fix/exit dir (#6163) 2020-05-19 18:21:58 +02:00
mwoz123
904a61009d
plugins/NewsDownloader: add never download images (#6003)
Closes #5924.
2020-03-27 14:21:20 +01:00
poire-z
0599c440cc [RTL UI] Bidi-wrap filenames, paths, urls, metadata
bidi.lua:
- Revert "Alias everything to Bidi.nowrap() when in LTR UI,
  as using LTR isolates seems uneeded when already LTR" (part
  of a628714f) which was a wrong assumption: we need proper
  wrappers for all things paths. Enhance some of these wrappers.
- Fix GetText RTL wrapping which was losing empty lines and
  trailing \n.

- Wrap all paths, directories, filenames in the code with
  these wrappers.
- Wrap all book metadata (title, authors...) with BD.auto(),
  as it helps fixing some edge cases (like open/close quotation
  marks which are not considered as bracket types by FriBiDi).
  (Needed some minor logic changes in CoverBrowser.)

- Tweak hyphenation menu text
- Update forgotten SortWidget for UI mirroring
- KoptConfig: update "justification" index for RTL re-ordering,
  following the recent addition of the page_gap_height option.
2020-01-04 01:34:46 +01:00
poire-z
d6d49a64a7 [RTL UI] use auto or LTR text direction in some specific cases
Allow TextBoxWidget new text direction/lang parameters to be
set on upper widgets, and propagate them all the way to it
(ScrollTextWidget, InputText, InputDialog, TextViewer).

Use specific non-default ones in some specific cases:
- Force LTR text direction when showing HTML and CSS, and
  configuration files (in some plugins).
- Use Wikipedia server language and text direction when
  showing an article.
- Use auto with Dictionary results, as we don't know the
  dictionary language, and they may contain mixed content.
- Force LTR when showing some paths (still needs more of them)

TextEditor plugin: add 2 new options "Auto paragraph direction"
and "Force paragraph direction LTR".

Footnotes popup: grab HTML direction, and forward it
to MuPDF for proper display.
2019-12-08 15:10:51 +01:00
Frans de Jonge
a2dcfe9aec
[doc] Tag @todo, @fixme and @warning (#5244)
This commit standardizes the various todos around the code a bit in a manner recognized by LDoc.

Besides drawing more attention by being displayed in the developer docs, they're also extractable with LDoc on the command line:

```sh
ldoc --tags todo,fixme *.lua
```

However, whether that particular usage offers any advantage over other search tools is questionable at best.

* and some random beautification
2019-08-23 19:53:53 +02:00
Frans de Jonge
2f335e2e6e
[fix, plugins] NewsDownloader: close document on go to folder (#5064)
Same as https://github.com/koreader/koreader/pull/5063. Also see https://github.com/koreader/koreader/issues/5060#issuecomment-499416617.
2019-06-09 15:40:49 +02:00
Tom Hall
e213a7665d NewsDownloader: Fix createFromDescription for atom feeds (#5041)
Atom feed content is found under the xml tag, 'content', not 'context'.
The content field is a actually a table, with the text/html at [1].
2019-05-26 16:22:50 +02:00
Tom Hall
1e9074ca6a NewsDownloader: Allow including images when using text from feed (#5040)
Split the html page download out of createEpub, so that createFromDescription
can pass in its own html for epub packaging (with optional image download).
Still pass in a url, so that relative urls can be made absolute.
2019-05-26 13:46:25 +02:00
Tom Hall
38f3bfdde7 NewsDownloader: Improve info shown on UI during processing (#5030)
Pass strings into functions, so that they can add their own info to the
UI without overwriting the information shown by the calling function.
2019-05-15 21:31:46 +02:00
Frans de Jonge
8815cbe07a
[fix, chore] Abstract filename logic in util.getSafeFilename() (#5026)
Fixes https://github.com/koreader/koreader/issues/5025

The OPDS browser was doing some fancier stuff in a way that should be abstracted away in util (because it applies anywhere files will be saved):

eace8d25c1/frontend/ui/widget/opdsbrowser.lua (L482-L491)
2019-05-14 19:10:41 +02:00
Sergey Avseyev
f6e561923c newsdownloader: rebuild URL on redirect to bare path (#5012)
The server might redirect to only path, in this case the scheme and host
should be taken from original URL
2019-05-08 12:41:04 +02:00
Tom Hall
0636ab7c79 NewsDownloader: Skip re-downloading files which already exist (#5008)
Fixes #5005

Results from re-running "Download news" should be as before, except when the
article content has changed between runs, but the title has not and an updated
date has not been given in the feed. User can force it to re-download an
article by manually deleting the file then running "Download news".
2019-05-08 08:05:47 +02:00
Tom Hall
9b41adb2f1 NewsDownloader: Fix UI when downloading multiple feeds (#4987)
Wrap the whole of loadConfigAndProcessFeeds with Trapper:wrap so that feeds are downloaded sequentially, otherwise they become interleaved and the UI progress updates get confusing. Use Trapper:info in place of manually constructed InfoMessages.
2019-05-01 00:37:03 +02:00
Tom Hall
8e23d2a937 NewsDownloader: Process articles sequentially
Lift the Trapper:wrap call out of the individual article processing code,
so that articles are processed one by one, in order to:
* Avoid concurrent progress updates fighting over the UI dialog
* Avoid trying to download many images at the same time
2019-04-30 21:03:48 +02:00
Tom Hall
cad4d25d24 NewsDownloader: Add flag 'include_images' to feed config
Allow the user to specify whether to download images for each individual
feed specified in feed_config.lua. Default to false to stay closest to
existing behaviour.
2019-04-30 21:03:48 +02:00
Tom Hall
1588d4c64a NewsDownloader: Download images and output EPUBs
Initial commit of new NewsDownloader which downloads images as well as text
and packs it all into an epub.

Based on Wikipedia "Download as EPUB" code.
2019-04-30 21:03:48 +02:00
sonix-github
4d15058773 NewsDownloader: fix RSS titles decoding (#4502) 2019-01-24 23:04:49 +01:00
Robert
a706fde2d7 Cleanup downloadmgr titles (#4222) 2018-09-13 13:20:09 +02:00
poire-z
850be52177
Keep some menus open when Tap or Hold (#4189)
TouchMenu: added options to menu items with the following defaults:
    keep_menu_open = false
    hold_keep_menu_open = true
So, default for Tap callback is to close menu, and for Hold callback
to keep menu open.
In both cases, provide the TouchMenu instance as the 1st argument to
the callback functions (instead of a refresh_menu_func I added in #3941)
so the callback can do more things, like closing, refreshing,
changing menu items text and re-ordering...

ReaderZooming: show symbol for default (like it was done for
ReaderFont, ReaderHyphenation...)
TextEditor plugin: update the previously opened files list in real
time, so the menu can be kept open and used as the TextEditor main
interface.
SSH plugin: keep menu open and update the Start/Stop state in real time
ReadTimer plugin: tried to do what feels right (but I don't use it)

Also remove forgotten cp in the move/paste file code
2018-09-04 23:55:58 +02:00
Robert
5344e0b672 [Fix] Don't load disabled plugins (#4169) 2018-08-18 20:41:37 +02:00
Robert
4428ecb422 Plugin manager (#4159)
Also adds descriptions to all plugins.
2018-08-17 20:54:11 +02:00
mwoz123
3fcd7344e4 News Downloader: feed config editor (#4126) 2018-08-07 21:03:43 +02:00
NiLuJe
5871132c25
UI Behavior tweaks (#3983)
* Switch all initial highlights to "fast" update

i.e., everything that does an invert
Plus a few other things that refresh small UI elements onTap
Re #3130

* Tweak refreshtype for a number of widgets:
  * Fix iconbutton dimen
  * Make touchmenu flash on close & initial menu popup. Full-screen on close.
  * Use flashing updates when opening/closing dictionary popup. Full-screen on close.
  * Switch FileManager to partial.
    It's mostly text, and we want flash promotion there.
  * Make configdialog & menu flash on exit
  * Make FLWidget flash on close
  * virtualkeyboard: flash on layout change & popup.
  * Potentially not that great workaround to ensure we actually see the
highlights in the FM's chevrons
  * Flash when closing BookStatus Widget
  * Optimize away a quirk of the dual "fast" update in touchmenu

* Promote updates to flashing slightly more agressively.

* Document what each refreshtype actually does.

With a few guidelines on their optimal usecases.

* Switch remaining scheduleIn(0.0) to nextTick()

* Tighter scheduling timers

Shaving a hundred ms off UI callbacks...

* Cache FFI C Library namespace

* Ask MuPDF to convert pixmaps to BGR on Kobo

Fix #3949

* Mention koxtoolchain in the README

re #3972

* Kindle: Handle *all* fonts via EXT_FONT_DIR instead of bind mounts insanity

* Make black flashes in UI elements user-configurable

(All or nothing).

* Jot down some random KOA2 sysfs path
2018-06-02 12:10:55 -04:00
onde2rock
dad7f209fa [CI] Add a check to detect tab in lua files (#3919) 2018-04-29 15:15:11 +02:00
mwoz123
a6b824ed90 NewsDownloader: add date time to filename (#3863)
* Download all files from ftp server(Not only epub). Show failed download number

* add date and time to filename

* optimalization - decrese max redirect number

* remove new external lib from luacheck

* add new lib licence info
2018-04-12 16:05:50 +02:00
poire-z
c5b547f6b3 Fix some minor plugin errors on exit (#3739) 2018-03-11 15:05:30 +01:00
mwoz123
c63d31f8d5 NewsDownloader: Message with details when invalid configuration. (#3658) 2018-02-03 20:35:07 +01:00
mwoz123
016c17e616 NewsDownloader: better exception handling and basic redirects (#3652) 2018-01-31 23:37:19 +01:00
mwoz123
d0f9b53eb3 NewsDownloader externalize download engine (#3618) 2018-01-31 12:39:04 +01:00
Frans de Jonge
7d7280a1dd
[fix] NewsDownloader: non-existent logger.error to logger.err (#3643) 2018-01-30 17:00:38 +01:00
mwoz123
1d0a5b4aa3 [feat] Add NetworkManager:beforeWifiAction() to enable optional auto-connect (#3482) 2018-01-17 18:16:11 +01:00
mwoz123
6d1e185555 NewsDownloader: Number items in processing InfoMessage (#3467) 2017-11-05 22:04:40 +01:00
mwoz123
456e31a358 NewsDownloader: Compatibility for users with previous configuration files (#3445) 2017-10-29 22:23:08 +01:00
mwoz123
c2bed639e3 NewsDownloader: wifi off prompt after download (#3395) 2017-10-28 17:53:37 +02:00
mwoz123
424593ca7c NewsDownloader: Remove NewsDownloader files from history (#3424) 2017-10-27 23:00:49 +02:00
mwoz123
97a72e2dd4 NewsDownloader: use feed.description as news context instead download full web page (#3426)
Fixes #3425.
2017-10-26 22:44:03 +02:00
mwoz123
23ab33c0b9 NewsDownloader promptWifiOn (#3388) 2017-10-22 07:11:23 +00:00
mwoz123
e7da93da89 Adapt ND to changes in code where 0 means no limit (#3258)
+ reorganize config, improved comments
2017-09-24 18:30:45 +02:00
Frans de Jonge
16dd5170b8 Newsdownloader: add HTTPS support and fixes
* Fixes #3041 (HTTPS)
* Fixes unlimited (limit = 0) was broken
* Fixes titles and links with attributes
2017-08-11 21:02:21 +02:00