* Move natural sorting algo to a dedicated sort module to avoid code duplication
* Use a slightly more accurate algorithm, and speed it up by caching intermediary strings
* Calibre: Use natural sorting in metadata search (fix#10009)
Basically:
* Use `extend` for class definitions
* Use `new` for object instantiations
That includes some minor code cleanups along the way:
* Updated `Widget`'s docs to make the semantics clearer.
* Removed `should_restrict_JIT` (it's been dead code since https://github.com/koreader/android-luajit-launcher/pull/283)
* Minor refactoring of LuaSettings/LuaData/LuaDefaults/DocSettings to behave (mostly, they are instantiated via `open` instead of `new`) like everything else and handle inheritance properly (i.e., DocSettings is now a proper LuaSettings subclass).
* Default to `WidgetContainer` instead of `InputContainer` for stuff that doesn't actually setup key/gesture events.
* Ditto for explicit `*Listener` only classes, make sure they're based on `EventListener` instead of something uselessly fancier.
* Unless absolutely necessary, do not store references in class objects, ever; only values. Instead, always store references in instances, to avoid both sneaky inheritance issues, and sneaky GC pinning of stale references.
* ReaderUI: Fix one such issue with its `active_widgets` array, with critical implications, as it essentially pinned *all* of ReaderUI's modules, including their reference to the `Document` instance (i.e., that was a big-ass leak).
* Terminal: Make sure the shell is killed on plugin teardown.
* InputText: Fix Home/End/Del physical keys to behave sensibly.
* InputContainer/WidgetContainer: If necessary, compute self.dimen at paintTo time (previously, only InputContainers did, which might have had something to do with random widgets unconcerned about input using it as a baseclass instead of WidgetContainer...).
* OverlapGroup: Compute self.dimen at *init* time, because for some reason it needs to do that, but do it directly in OverlapGroup instead of going through a weird WidgetContainer method that it was the sole user of.
* ReaderCropping: Under no circumstances should a Document instance member (here, self.bbox) risk being `nil`ed!
* Kobo: Minor code cleanups.
* Calibre plugin - Series/Tag browser updated to show back button.
* Handling page number
* Few updates to search logic:
- Moved "find books" option from CalibreSearch:find(option)
into CalibreSearch:browse(option) function. This way all search options
are handled in one place only.
- Menu is created only once and it is in CalibreSearch:browse(option)
function. This is where it is also invoked with UIManager:show(w).
Switching between different menu content (tags/series/books) is
done using CalibreSearch:switchResults function which will internally
call Menu:switchItemTable. Previously menu was being instantiated
in 2 places depending on are we searching books or series/tags
- Return arrow navigation: Border around Menu is gone to give space
for back arrow. Navigation seems to be working fine so far but will
give it some time to test in case I find any other bugs
`Manual` could mean several things, most notably some kind of documentation and something you do by hand.
The added context is partially to aid translators and partially to preclude potential collisions in the future.
Cf. <https://github.com/koreader/koreader-translations/pull/138>.
FocusManager: fix round x use y layout
FocusManager: add tab and shift tab focus navigation support
FocusManager: handle Press key by default
FocusManager: make sure selected in instance level
FocusManager: add hold event support
FocusManager: Half move instead of edge move
FocusManager: add keymap override support
FocusManager: refocusWidget will delegate to parent FocusManager
Focusmanager: refocusWidget can execute on next tick
inputtext: can move out of focus on back
inputtext: fix cannot exit for non-touch device
inputtext: fix cannot input text with kindle dx physical keyboard
fontlightwidget: add non-touch support
datetimewidget: add non-touch support
datetimewidget: fix set date failed in kindle DX, fix datetimewidget month range to 1~23 by default
datetimewidget: make hour max value to 23
multiinputdialog: add non-touch support
checkbox: focusable and focus style
virtualkeyboard: no need to press two back to unfocus inputtext
virtualkeyboard: collect FocusManager event key names to let VirtualKeyboard disable them
openwithdialog: add non-touch support
inputdialog: can close via back button
enable all InputDialog and MultiInputDialog can be close by back
keyboardlayoutdialog: non-touch support
readertoc: non touch device can expand/collapse in toc
bookstatuswidget: non touch support
keyvaluepage: non-touch support
calendarview: non-touch support
* Ensure that going from one to the other tears down the former and
its plugins before instantiating the latter and its plugins.
UIManager: Unify Event sending & broadcasting
* Make the two behave the same way (walk the widget stack from top to
bottom), and properly handle the window stack shrinking shrinking
*and* growing.
Previously, broadcasting happened bottom-to-top and didn't really
handle the list shrinking/growing, while sending only handled the list
shrinking by a single element, and hopefully that element being the one
the event was just sent to.
These two items combined allowed us to optimize suboptimal
refresh behavior with Menu and other Menu classes when
opening/closing a document.
e.g., the "opening document" Notification is now properly regional,
and the "open last doc" option no longer flashes like a crazy person
anymore.
Plugins: Allow optimizing Menu refresh with custom menus, too.
Requires moving Menu's close_callback *after* onMenuSelect, which, eh,
probably makes sense, and is probably harmless in the grand scheme of
things.
The object was never re-assigned, so closing a smaller menu (e.g.,
Calibre metadata search) made the underlying one (e.g., CoverBrowser's
ListMenu) inherit the smaller dimensions...
Instead of creating the object in the Class constructor, create it in the
instance constructor (i.e., :init).
Similar cleanups in other Menu* related classes.
* Wireless: Optimize memory usage in StreamMessageQueue (use an array of string ropes, that we only concatenate once). Allowed to relax the throttling, making transfers that much faster.
* Persist: Add a "zstd" codec, that uses the "luajit" codec, but compressed via zstd. Since both of those are very fast, it pretty much trounces everything in terms of speed and size ;).
* Persist: Implemented a "writes_to_file" framework, much like the existing "reads_from_file" one. And use it in the zstd codec to avoid useless temporary string interning.
* Metadata: Switch to the zstd codec.
* CalibreMetadata: Get rid of the now useless NULL-hunt: here, this was basically looking for `rapidjson.null` to replace them with... `rapidjson.null` :?. IIRC, that's a remnant of a quirk of the previous JSON parser (possibly even the previous, *previous* JSON parser ^^).
* CalibreSearch: Update the actually relevant NULL-hunt to make it explicit: replace JSON NULLs with Lua nils, instead of relying on an implementation detail of Lua-RapidJSON, because that detail just changed data type ;).
* UIManager: Make sure tasks scheduled during the final ZMQ callback are honored. e.g., the Calibre "Disconnect" handler. This happened to mostly work purely by chance before the event loop rework.
* Calibre: Restore a proper receiveCallback handler after receiving a book, in order not to break the "Disconnect" handler's state (and, well, get a working Disconnect handler, period ^^).
* Calibre: Unbreak metadata cache when it's initialized by a search (regression since #7159).
* Calibre: Handle UTC <-> local time conversions when checking the cache's timestamp against the Calibre metadata timestamp.
* Bump base (Unbreak CRe on Android, update RapidJSON)
* Change 'Find a file' to 'File search' for consistency
There is 'File search' in the Gesture manager already.
There is 'Fulltext search' in the readermenu.
Some help text added.
* ReaderDictionary: Port delay computations to TimeVal
* ReaderHighlight: Port delay computations to TimeVal
* ReaderView: Port delay computations to TimeVal
* Android: Reset gesture detection state on APP_CMD_TERM_WINDOW.
This prevents potentially being stuck in bogus gesture states when switching apps.
* GestureDetector:
* Port delay computations to TimeVal
* Fixed delay computations to handle time warps (large and negative deltas).
* Simplified timed callback handling to invalidate timers much earlier, preventing accumulating useless timers that no longer have any chance of ever detecting a gesture.
* Fixed state clearing to handle the actual effective slots, instead of hard-coding slot 0 & slot 1.
* Simplified timed callback handling in general, and added support for a timerfd backend for better performance and accuracy.
* The improved timed callback handling allows us to detect and honor (as much as possible) the three possible clock sources usable by Linux evdev events.
The only case where synthetic timestamps are used (and that only to handle timed callbacks) is limited to non-timerfd platforms where input events use
a clock source that is *NOT* MONOTONIC.
AFAICT, that's pretty much... PocketBook, and that's it?
* Input:
* Use the <linux/input.h> FFI module instead of re-declaring every constant
* Fixed (verbose) debug logging of input events to actually translate said constants properly.
* Completely reset gesture detection state on suspend. This should prevent bogus gesture detection on resume.
* Refactored the waitEvent loop to make it easier to comprehend (hopefully) and much more efficient.
Of specific note, it no longer does a crazy select spam every 100µs, instead computing and relying on sane timeouts,
as afforded by switching the UI event/input loop to the MONOTONIC time base, and the refactored timed callbacks in GestureDetector.
* reMarkable: Stopped enforcing synthetic timestamps on input events, as it should no longer be necessary.
* TimeVal:
* Refactored and simplified, especially as far as metamethods are concerned (based on <bsd/sys/time.h>).
* Added a host of new methods to query the various POSIX clock sources, and made :now default to MONOTONIC.
* Removed the debug guard in __sub, as time going backwards can be a perfectly normal occurrence.
* New methods:
* Clock sources: :realtime, :monotonic, :monotonic_coarse, :realtime_coarse, :boottime
* Utility: :tonumber, :tousecs, :tomsecs, :fromnumber, :isPositive, :isZero
* UIManager:
* Ported event loop & scheduling to TimeVal, and switched to the MONOTONIC time base.
This ensures reliable and consistent scheduling, as time is ensured never to go backwards.
* Added a :getTime() method, that returns a cached TimeVal:now(), updated at the top of every UI frame.
It's used throughout the codebase to cadge a syscall in circumstances where we are guaranteed that a syscall would return a mostly identical value,
because very few time has passed.
The only code left that does live syscalls does it because it's actually necessary for accuracy,
and the only code left that does that in a REALTIME time base is code that *actually* deals with calendar time (e.g., Statistics).
* DictQuickLookup: Port delay computations to TimeVal
* FootNoteWidget: Port delay computations to TimeVal
* HTMLBoxWidget: Port delay computations to TimeVal
* Notification: Port delay computations to TimeVal
* TextBoxWidget: Port delay computations to TimeVal
* AutoSuspend: Port to TimeVal
* AutoTurn:
* Fix it so that settings are actually honored.
* Port to TimeVal
* BackgroundRunner: Port to TimeVal
* Calibre: Port benchmarking code to TimeVal
* BookInfoManager: Removed unnecessary yield in the metadata extraction subprocess now that subprocesses get scheduled properly.
* All in all, these changes reduced the CPU cost of a single tap by a factor of ten (!), and got rid of an insane amount of weird poll/wakeup cycles that must have been hell on CPU schedulers and batteries..
* LuaSettings/DocSettings: Updated readSetting API to allow proper initialization to default.
Use it to initialize tables, e.g., fixing corner-cases in readerFooter that could prevent settings from being saved.
(Fixes an issue reported on Gitter).
* LuaSettings/DocSettings: Add simpler API than the the flip* ones to toggle boolean settings.
* Update LuaSettings/DocSettigns usage throughout the codebase to use the dedicated boolean methods wher appropriate, and clean up some of the more mind-bending uses.
* FileChooser: Implement an extended default exclusion list (fix#2360)
* ScreenSaver: Refactor to avoid the pile of kludges this was threatening to become. Code should be easier to follow and use, and fallbacks now behave as expected (fix#4418).
* Menu/KeyValuePage/ReaderGoTo: Unify the dialogs. (Generally, "Enter page number" as title, and "Go to page" as OK button).
* Allow *tapping* on pagination buttons, too. Added spacers around the text to accommodate for that.
* Disable input handlers when <= 1 pages, while still printing the label in black.
* Always display both the label and the chevrons, even on single page content. (Menu being an exception, because it can handle showing no content at all, in which case we hide the chevrons).
* KVP: Tweak the pagination buttons layout in order to have consistent centering, regardless of whether the return arrow is enabled or not. (Also, match Menu's layout, more or less).
* Menu: Minor layout tweaks to follow the KVP tweaks above. Fixes, among possibly other things, buttons in (non-FM) "List" menus overlapping the final entry (e.g., OPDS), and popout menus with a border being misaligned (e.g., Calibre, Find a file).
* CalendarView: Minor layout tweaks to follow the KVP tweaks. Ensures the pagination buttons are laid out in the same way as everywhere else (they used to be a wee bit higher).
* Revamped most actions that require an internet connection to a new/fixed backend that allows forwarding the initial action and running it automatically once connected. (i.e., it'll allow you to set "Action when Wi-Fi is off" to "turn_on", and whatch stuff connect and do what you wanted automatically without having to re-click anywhere instead of showing you a Wi-Fi prompt and then not doing anything without any other feedback).
* Speaking of, fixed the "turn_on" beforeWifi action to, well, actually work. It's no longer marked as experimental.
* Consistently use "Wi-Fi" everywhere.
* On Kobo/Cervantes/Sony, implemented a "Kill Wi-Fi connection when inactive" system that will automatically disconnect from Wi-Fi after sustained *network* inactivity (i.e., you can keep reading, it'll eventually turn off on its own). This should be smart and flexible enough not to murder Wi-Fi while you need it, while still not keeping it uselessly on and murdering your battery.
(i.e., enable that + turn Wi-Fi on when off and enjoy never having to bother about Wi-Fi ever again).
* Made sending `NetworkConnected` / `NetworkDisconnected` events consistent (they were only being sent... sometimes, which made relying on 'em somewhat problematic).
* restoreWifiAsync is now only run when really needed (i.e., we no longer stomp on an existing working connection just for the hell of it).
* We no longer attempt to kill a bogus non-existent Wi-Fi connection when going to suspend, we only do it when it's actually needed.
* Every method of enabling Wi-Fi will now properly tear down Wi-Fi on failure, instead of leaving it in an undefined state.
* Fixed an issue in the fancy crash screen on Kobo/reMarkable that could sometime lead to the log excerpt being missing.
* Worked-around a number of sneaky issues related to low-level Wi-Fi/DHCP/DNS handling on Kobo (see the lengthy comments [below](https://github.com/koreader/koreader/pull/6424#issuecomment-663881059) for details). Fix#6421
Incidentally, this should also fix the inconsistencies experienced re: Wi-Fi behavior in Nickel when toggling between KOReader and Nickel (use NM/KFMon, and run a current FW for best results).
* For developers, this involves various cleanups around NetworkMgr and NetworkListener. Documentation is in-line, above the concerned functions.
joins calibre metadata search and calibre wireless connections into a single plugin
search metadata changes:
- search directly into calibre metadata files.
- search can be performed on more than one library (configurable from a menu)
- device scans now find all calibre libraries under a given root
- search options can be configured from a menu. (case sensitive, find by title, author and path)
- removed legacy global variables.
- *option* to search from the reader
- *option* to generate a cache of books for faster searches.
calibre wireless connection changes:
- keep track of books in a library (includes prunning books from calibre metadata if the file was deleted locally)
- remove files on device from calibre
- support password protected connections
- FM integration: if we're in the inbox dir it will be updated each time a book is added or deleted.
- disconnect when requested by calibre, available on newer calibre versions (+4.17)
- remove unused opcodes.
- better report of client name, version and device id
- free disk space checks for all calibre versions
- bump supported extensions to match what KOReader can handle. Users can override this with their own list of extensions (or from calibre, by configuring the wireless device).
