* Don't run multiple collectSubprocesses tasks in parallel.
UIManager:scheduleIn doesn't return anything, using that as a boolean trap made no sense.
* Make the in-place removal of collected pids use a slightly more common idiom
(table.remove in a reverse iteration of said array).
cores
* Only keep a single core online most of the time.
* Device: Add an enableCPUCores method to allow controlling the amount of
online CPU cores.
* Move the initial core onlining setup to Kobo:init, instead of the startup script.
* Enable two CPU cores while hinting new (e.g., cache miss) pages in PDF land.
* Enable two CPU cores while processing book metadata.
* Drive-by fix to isolate the DocCache pressure check to KoptInterface
and actually apply it when it matters most (e.g., k2pdfopt stuff).
* ReaderDictionary: Port delay computations to TimeVal
* ReaderHighlight: Port delay computations to TimeVal
* ReaderView: Port delay computations to TimeVal
* Android: Reset gesture detection state on APP_CMD_TERM_WINDOW.
This prevents potentially being stuck in bogus gesture states when switching apps.
* GestureDetector:
* Port delay computations to TimeVal
* Fixed delay computations to handle time warps (large and negative deltas).
* Simplified timed callback handling to invalidate timers much earlier, preventing accumulating useless timers that no longer have any chance of ever detecting a gesture.
* Fixed state clearing to handle the actual effective slots, instead of hard-coding slot 0 & slot 1.
* Simplified timed callback handling in general, and added support for a timerfd backend for better performance and accuracy.
* The improved timed callback handling allows us to detect and honor (as much as possible) the three possible clock sources usable by Linux evdev events.
The only case where synthetic timestamps are used (and that only to handle timed callbacks) is limited to non-timerfd platforms where input events use
a clock source that is *NOT* MONOTONIC.
AFAICT, that's pretty much... PocketBook, and that's it?
* Input:
* Use the <linux/input.h> FFI module instead of re-declaring every constant
* Fixed (verbose) debug logging of input events to actually translate said constants properly.
* Completely reset gesture detection state on suspend. This should prevent bogus gesture detection on resume.
* Refactored the waitEvent loop to make it easier to comprehend (hopefully) and much more efficient.
Of specific note, it no longer does a crazy select spam every 100µs, instead computing and relying on sane timeouts,
as afforded by switching the UI event/input loop to the MONOTONIC time base, and the refactored timed callbacks in GestureDetector.
* reMarkable: Stopped enforcing synthetic timestamps on input events, as it should no longer be necessary.
* TimeVal:
* Refactored and simplified, especially as far as metamethods are concerned (based on <bsd/sys/time.h>).
* Added a host of new methods to query the various POSIX clock sources, and made :now default to MONOTONIC.
* Removed the debug guard in __sub, as time going backwards can be a perfectly normal occurrence.
* New methods:
* Clock sources: :realtime, :monotonic, :monotonic_coarse, :realtime_coarse, :boottime
* Utility: :tonumber, :tousecs, :tomsecs, :fromnumber, :isPositive, :isZero
* UIManager:
* Ported event loop & scheduling to TimeVal, and switched to the MONOTONIC time base.
This ensures reliable and consistent scheduling, as time is ensured never to go backwards.
* Added a :getTime() method, that returns a cached TimeVal:now(), updated at the top of every UI frame.
It's used throughout the codebase to cadge a syscall in circumstances where we are guaranteed that a syscall would return a mostly identical value,
because very few time has passed.
The only code left that does live syscalls does it because it's actually necessary for accuracy,
and the only code left that does that in a REALTIME time base is code that *actually* deals with calendar time (e.g., Statistics).
* DictQuickLookup: Port delay computations to TimeVal
* FootNoteWidget: Port delay computations to TimeVal
* HTMLBoxWidget: Port delay computations to TimeVal
* Notification: Port delay computations to TimeVal
* TextBoxWidget: Port delay computations to TimeVal
* AutoSuspend: Port to TimeVal
* AutoTurn:
* Fix it so that settings are actually honored.
* Port to TimeVal
* BackgroundRunner: Port to TimeVal
* Calibre: Port benchmarking code to TimeVal
* BookInfoManager: Removed unnecessary yield in the metadata extraction subprocess now that subprocesses get scheduled properly.
* All in all, these changes reduced the CPU cost of a single tap by a factor of ten (!), and got rid of an insane amount of weird poll/wakeup cycles that must have been hell on CPU schedulers and batteries..
Avoid opening two connections in //, by feeding the current connection
to saveSetting, like we were already doing for loadSettings...
It somehow worked just fine on Kobo/Emu, but blew up in fun and
interesting ways on Kindle, probably because of the Fuse proxy.
It was still pretty dumb nonetheless.
* Avoid the Lua string copies when compressing/uncompressing the bb
thumbs
* Delay the statement reset so that the pointer returned by the BLOB
query is actually usable ;).
* Re-use a ZSTD decompression context
* Store date & mtime for successfully extracted documents
c.f., 10477501f9 (r44914917)
* Switch to user_version pragma to handle DB schema versioning
* Update file size/date on *each* extraction attempt.
@poirez-z raised a good point, this ought to be useful to deem a file
re-parseable after an update.
* Remove xutil
Functionality has been moved to ffi/zlib & ffi/zstd
* Preserve settings when migrating the BookInfo DB
* Warn via InfoMessage that the DB was updated
* Only store the series name in series, and move the index in series_index
(Column names chosen to match Calibre's).
* Handle the new series_index column in BookInfo consumers
* This allows us to get rid of the code that stripped empty decimals
from series index in *most* places, since it's now a real Lua
number, and the string formatting library does that magic on its
own.
* stride is now a size_t
On some platforms, that's 64 bits, which means it's no longer
automatically converted to a Lua number to avoid precision loss.
Do that ourselves, because lua-serialize doesn't know how to handle an
uint64_t cdata ;).
* Update the data collection format & handler to make it much less tortuous
* Update the pagecount & resync the stats on document layout changes
* Update the database schema to allow doing most queries against a SQL view that rescales the collected data to be accurate regardless of document layout (thanks to @marek-g for the SQL magic ;)).
* Add a "reset stats for current book" entry in the list of reset options, one that won't horribly break stats in said book ;).
* Fixed a couple of resource (SQL connection) leaks (in ReaderStatistics:getCurrentBookStats & ReaderStatistics:getCurrentBookStats).
* Flush stats to the DB on periodical metadata saves.
* Minor cosmetic tweaks to the code
* File search: Don't traverse hidden folders if we're not showing them
Re https://www.mobileread.com/forums/showpost.php?p=3949194&postcount=21
* Ignore macOS resource forks, too.
* Apply the same logic to the BookInfo directory walker
* And never ever show resource forks in the FM, either.
bidi.lua:
- Revert "Alias everything to Bidi.nowrap() when in LTR UI,
as using LTR isolates seems uneeded when already LTR" (part
of a628714f) which was a wrong assumption: we need proper
wrappers for all things paths. Enhance some of these wrappers.
- Fix GetText RTL wrapping which was losing empty lines and
trailing \n.
- Wrap all paths, directories, filenames in the code with
these wrappers.
- Wrap all book metadata (title, authors...) with BD.auto(),
as it helps fixing some edge cases (like open/close quotation
marks which are not considered as bracket types by FriBiDi).
(Needed some minor logic changes in CoverBrowser.)
- Tweak hyphenation menu text
- Update forgotten SortWidget for UI mirroring
- KoptConfig: update "justification" index for RTL re-ordering,
following the recent addition of the page_gap_height option.
Thanks to <https://github.com/koreader/koreader/pull/5237> we can now extract the knowledge currently embedded in Transifex and put it directly in our source. This positively affects <https://github.com/koreader/koreader/issues/3754>.
Translation instructions and knowledge that comes out of localization-related questions should be preserved in the source, because Transifex is too ephemeral. For example, the links from <https://github.com/koreader/koreader/pull/2290> are no longer accessible. Even when they are, it's quite useful to have this information around while dealing with the code as well, and I also hope it'll be informative to contributors who seldom visit Transifex.
This commit also makes a few minor changes to obviate the need for comments where possible.
It was doing a full document load to get the cover. It now does
the faster "only metadata" load.
Also move the trick of setting a default font in CreDocument, so
that all callers of document:loadDocument(false) benefit from it.
Prevent crash when no cover image is available (even if the cache
says it has one, the file may have been updated and doesn't have
it anymore).
Avoids a use-after-free in mupdf.scaleBlitBuffer
Fixkoreader/koreader-base#821
Thanks for the hint, @poire-z ;).
* As @poire-z suggested, the original unscaled bb should probably be free'd.
* Enable HW dithering on supported devices (Clara HD, Forma; Oasis 2, PW4)
* FileManager and co. (where appropriate, i.e., when covers are shown)
* Book Status
* Reader, where appropriate:
* CRe: on pages whith image content (for over 7.5% of the screen area, should hopefully leave stuff like bullet points or small scene breaks alone).
* Other engines: on user-request (in the gear tab of the bottom menu), via the new "Dithering" knob (will only appear on supported devices).
* ScreenSaver
* ImageViewer
* Minimize repaints when flash_ui is enabled (by, almost everywhere, only repainting the flashing element, and not the toplevel window which hosts it).
(The first pass of this involved fixing a few Button instances whose show_parent was wrong, in particular, chevrons in the FM & TopMenu).
* Hunted down a few redundant repaints (unneeded setDirty("all") calls),
either by switching the widget to nil when only a refresh was needed, and not a repaint,
or by passing the appropritate widget to setDirty.
(Note to self: Enable *verbose* debugging to catch broken setDirty calls via its post guard).
There were also a few instances of 'em right behind a widget close.
* Don't repaint the underlying widget when initially showing TopMenu & ConfigDialog.
We unfortunately do need to do it when switching tabs, because of their variable heights.
* On Kobo, disabled the extra and completely useless full refresh before suspend/reboot/poweroff, as well as on resume. No more double refreshes!
* Fix another debug guard in Kobo sysfs_light
* Switch ImageWidget & ImageViewer mostly to "ui" updates, which will be better suited to image content pretty much everywhere, REAGL or not.
PS: (Almost 💯 commits! :D)
Revert 9971eb85 and make multi-lines strings more readable.
(Multiline translatable strings extraction has been fixed
in koreader-misc tralua_xgettext.py.)
Some problem somewhere with [[...]] strings starting with a
leading newline. Should probably be allowed, but for now
fix the few such cases to allow them being translated.
* Only switch to ondemand when we actually can, and when it's better than the current governor...
This potentially leaves Mk.5 in the lurch, but there's no perfect solution there :/.
* Switch to UI for the unmark event of navigation hints markers
Might help on non-REAGL devices, and doesn't hurt there (re #3983).
* Switch SQLite DBs to WAL
Sounds nice in theory, behaves fine in practice.
It was enabled by default, but it's mostly only needed for CJK users.
Furthermore, when floating punctuation is enabled, some rendering
issues exist (text right alignment, variable margins...) that
only CJK developpers could really fix. So, best to disable it and
avoid these rendering issues for most users.
Also: fix CoverBrowser crash when "Delete cache database"
followed by "Prune cache of removed books".
The Dogear icon is 20x20 pixels and was never scaled where used. Now:
- The bookmark icon (top right of screen) is scaled to 1/32th of the screen
width (previously, it was 1/30th on a 600px wide emulator, 1/53th on a GloHD).
On CreDocument, furthermore decrease its size if needed depending on the
selected margins so it never overwrite the text.
- CoverBrowser list view: scale it to the available room under
the "N % of P page" text, so it does not cover "page".
- CoverBrowser mosaic view: scale it to 1/16th of the cover rectangle, which
should prevent if from overwritting the text thanks to a max text width of
7/8 of the cover rectangle.
Also for CoverBrowser: don't index metadata for unsupported document
(which could happen when browsing files with PathChooser) and show
full filename for such documents.
Also: ImageWidget: small fix in case we use both scale_factor and
scale_for_dpi.
New module RenderImage (alongside existing RenderText) to provides
image rendering and scaling facilities.
Uses MuPDF, but tries first giflib on GIF.
Allows for getting all the frames from an animated GIF.
This adds a button to the Tap Plus menu, that allows
extracting metadata and cover images for books in
current directory. Info about the process and questions are
initially shown and asked, and the process can be aborted at
any moment.
bump crengine:
Allow for quicker loading when interested in metadata only.
Allow for more than 65535 different attribute values.
Increase gamma values range.
Remove the subprocess management functions from xutil.lua, as they were
moved into base/ffi/util.lua, and use them from there.
Also use the cre_storage_size_factor setting when processing credocuments,
to avoid CRE WARNING while indexing too.
Fix dealing with MuPDF document opening failures (previously, these
were not noticed and indexing was retried each time).
cre:loadDocument() may fail in recognizing the document format, and
koreader would previously keep calling other methods on it, which would
make crengine segfaults. We now check loadDocument success at the
various places it is called, and try to deal the best way we can when it fails.
We previously disabled cache, but that may cause excessive
memory usage with big books. We now use a temporary
cache directory, that we clean when no more needed.
So an image file can have a cover in coverbrowser's display.
It also allows for an alternative viewer (ImageViewer widget)
when holding on file / View full size cover.
* CoverBrowser plugin: alt views for File Browser and History
* Added Prune cache and Compact cache menu actions
* Support for book descriptions, and settings stored in db
