* Use a dedicated cache hash for partial tiles from panel-zoom
* Never dump them to disk, as it confuses DocCache's crappy heuristics that rewinds the cache to skip over the hinted page to try to dump the on-screen page to disk.
* Apply the zoom factor in the exact same way as any other page rect (i.e., floor coordinates, ceil dimensions), and make sure said rect is actually a Geom so it doesn't break the cache hash, which relies on Geom's custom tostring method for rects. Said scaling method *also* belongs to the Geom class anyway.
* Handle such pre-scaled rects properly in renderPage, so as not to apply the zoom factor to the full page, which would attempt to create a gigantic buffer.
* And now that the rect is rendered properly in an appropriately-sized buffer, use the rendered tile as-is, no need to blit it to another (potentially way too large because of the above issue) blank BB.
* The zoom factor is now computed for a scale to best-fit (honoring `imageviewer_rotate_auto_for_best_fit`), ensuring the best efficiency (ImageViewer won't have to re-scale).
* Cache: Reduce the maximum item size to 50% of the cache, instead of 75%.
* Warn about the legacy ReaderRotation module, as it turned out to be horribly broken. The whole machinery (which is spread over *a lot* of various codepaths) is left as-is, peppered with notes & fixmes hinting at the problem. Thankfully, that's not how we actually handle rotation, so it was probably hardly ever used (which possibly explains why nobody ever noticed it breaking, and that nugget possibly dates back to the inception of the kpv -> ko refactor!). (#12309)
Basically:
* Use `extend` for class definitions
* Use `new` for object instantiations
That includes some minor code cleanups along the way:
* Updated `Widget`'s docs to make the semantics clearer.
* Removed `should_restrict_JIT` (it's been dead code since https://github.com/koreader/android-luajit-launcher/pull/283)
* Minor refactoring of LuaSettings/LuaData/LuaDefaults/DocSettings to behave (mostly, they are instantiated via `open` instead of `new`) like everything else and handle inheritance properly (i.e., DocSettings is now a proper LuaSettings subclass).
* Default to `WidgetContainer` instead of `InputContainer` for stuff that doesn't actually setup key/gesture events.
* Ditto for explicit `*Listener` only classes, make sure they're based on `EventListener` instead of something uselessly fancier.
* Unless absolutely necessary, do not store references in class objects, ever; only values. Instead, always store references in instances, to avoid both sneaky inheritance issues, and sneaky GC pinning of stale references.
* ReaderUI: Fix one such issue with its `active_widgets` array, with critical implications, as it essentially pinned *all* of ReaderUI's modules, including their reference to the `Document` instance (i.e., that was a big-ass leak).
* Terminal: Make sure the shell is killed on plugin teardown.
* InputText: Fix Home/End/Del physical keys to behave sensibly.
* InputContainer/WidgetContainer: If necessary, compute self.dimen at paintTo time (previously, only InputContainers did, which might have had something to do with random widgets unconcerned about input using it as a baseclass instead of WidgetContainer...).
* OverlapGroup: Compute self.dimen at *init* time, because for some reason it needs to do that, but do it directly in OverlapGroup instead of going through a weird WidgetContainer method that it was the sole user of.
* ReaderCropping: Under no circumstances should a Document instance member (here, self.bbox) risk being `nil`ed!
* Kobo: Minor code cleanups.
Should avoid egregious values that would potentially alloc insanely large buffers (and likely fail to do so).
In the process, tweak the scale_factor computations when zooming so as to produce slightly less annoying behavior.
* Android: Make sure sdcv can find the STL
* DocCache: Be less greedy when serializing to disk, and only do that for the *current* document ;).
* CanvasContext: Explicitly document API quirks.
* Fontlist: Switch the on-disk Persist format to zstd (it's ever so slightly faster).
* Bump base for https://github.com/koreader/koreader-base/pull/1515 (fix#9506)
* Allow doing away with CacheItem
Now that we have working FFI finalizers on BBs, it's mostly useless overhead.
We only keep it for DocCache, because it's slightly larger, and memory pressure might put us in a do or die situation where waiting for the GC might mean an OOM kill.
* Expose's LRU slot-only mode
And use it for CatalogCache, which doesn't care about storage space
* Make GlyphCache slots only (storage space is insignificant here, it was
always going to be evicted by running out of slots).
* More informative warning when we chop the cache in half
Ought to be faster than our naive array-based approach.
Especially for the glyph cache, which has a solid amount of elements,
and is mostly cache hits.
(There are few things worse for performance in Lua than
table.remove @ !tail and table.insert @ !tail, which this was full of :/).
DocCache: New module that's now an actual Cache instance instead of a
weird hack. Replaces "Cache" (the instance) as used across Document &
co.
Only Cache instance with on-disk persistence.
ImageCache: Update to new Cache.
GlyphCache: Update to new Cache.
Also, actually free glyph bbs on eviction.
* Minor updates to the min & max cache sizes (16 & 64MB). Mostly to satisfy my power-of-two OCD.
* Purge broken on-disk cache files
* Optimize free RAM computations
* Start dropping LRU items when running low on memory before pre-rendring (hinting) pages in non-reflowable documents.
* Make serialize dump the most recently *displayed* page, as the actual MRU item is the most recently *hinted* page, not the current one.
* Use more accurate item size estimations across the whole codebase.
TileCacheItem:
* Drop lua-serialize in favor of Persist.
KoptInterface:
* Drop lua-serialize in favor of Persist.
* Make KOPTContext caching actually work by ensuring its hash is stable.
There have been a couple of these this month, and keeping stuff that should only ever run once piling up in their respective module was getting ugly, especially when it's usually simple stuff (settings, files).
So, move everything to a dedicated module, run by reader.lua on startup, and that will actually only do things once, when necessary.
* CalibreMetadata: Get rid of the now useless NULL-hunt: here, this was basically looking for `rapidjson.null` to replace them with... `rapidjson.null` :?. IIRC, that's a remnant of a quirk of the previous JSON parser (possibly even the previous, *previous* JSON parser ^^).
* CalibreSearch: Update the actually relevant NULL-hunt to make it explicit: replace JSON NULLs with Lua nils, instead of relying on an implementation detail of Lua-RapidJSON, because that detail just changed data type ;).
* UIManager: Make sure tasks scheduled during the final ZMQ callback are honored. e.g., the Calibre "Disconnect" handler. This happened to mostly work purely by chance before the event loop rework.
* Calibre: Restore a proper receiveCallback handler after receiving a book, in order not to break the "Disconnect" handler's state (and, well, get a working Disconnect handler, period ^^).
* Calibre: Unbreak metadata cache when it's initialized by a search (regression since #7159).
* Calibre: Handle UTC <-> local time conversions when checking the cache's timestamp against the Calibre metadata timestamp.
* Bump base (Unbreak CRe on Android, update RapidJSON)
Cache's state tracking
Re #7510
Legacy cache has to be deleted in Cache,
*before* the first getDiskCache call.
Reword the "Clear caches" Dev menu entry & callback:
We now have an actual "Restart" action (on most, if not all, platforms).
And we clear the whole cache folder, which may host other things (e.g.,
fontlist)
* Modernize jit syntax
It's built-in, no need for a require anymore.
* Flag Android should_restrict_JIT again
It's mildly helpful at putting off the inevitable.
(very, very, very, very mildly).
* Bump android-luajit-launcher
https://github.com/koreader/android-luajit-launcher/pull/281
If the most recently used global cache item refers to the same
page, it would be overwritten. This will happen when closing a
paged (PDF/DjVu) document on the same page you opened it, as well
as when opening and closing EPUBs with crengine because it uses
its own cache.
Fixes#3846.
to make koreader on Android more stable
and with these opt params:
```
require("jit.opt").start("sizemcode=64","maxmcode=64", "hotloop=10000")
```
The strategy here is that we only use precious mcode memory (jitting)
on deep loops like the several blitting methods in blitbuffer.lua and
the pixel-copying methods in mupdf.lua. So that a small amount of mcode
memory (64KB) allocated when koreader is launched in the android.lua
is enough for the program and it won't need to jit other parts of lua
code and thus won't allocate mcode memory any more which by our
observation will be harder and harder as we run koreader.
Currently only tested on Ubuntu-touch emulator with framework
ubuntu-sdk-14.10 for armhf.
The ubuntu-touch port is binary compatible with the Kobo port
major changes in this PR are:
1. rename the emulator device to sdl device since both the emulator
and the ubuntu-touch target use libsdl to handle input/output.
2. ubuntu-touch app has no write access to the installation dir so
all write-outs should be in a seperate dir definded in `datastorage`.
the cache would behave badly when the same item was insert()ed twice:
it would add the size twice to memory consumption, but would never substract
it twice when purging the (actually single) object from cache. So the cache
would seem to fill up while in fact it wasn't.
In reflowing scroll mode with 2 pages hinting, 4 full page blitbuffers
and koptcontexts should stay well in cache in the most demanding cases,
with two pages shown on screen and two pages rendered in background.
Since blitbuffer size is halved the size of page, we need cache size
to be 6 times an average reflowed page size.
For Kobo Aura HD which has a resolution of 1440×1080, a reflowed page
could become 1080×4800. So 30MB of cache is demanded for this case.
This PR implements dynamic cache size allocating according to size of
system free memory. By default it will use 20 percent of free RAM with
a clip specified by DGLOBAL_CACHE_SIZE_MINIMUM and
DGLOBAL_CACHE_SIZE_MAXIMUM which are 10MB and 30MB respectively by default.
