bump crengine: support for EPUB3 nav toc and page maps
Includes:
- Fix lvRect:isRectInside(rc) with 0-width or 0-height rect
- TOC: parse EPUB3 nav toc, fallback to spine when no toc
- Parse and cache various hardcopy page list maps
- epub.css: hide EPUB3 <span epub:type="pagebreak"> content
cre.cpp: add a few PageMap helper functions.
Adds ReaderPageMap which will add a new menu (under TOC and
Bookmarks) that will allow:
- to list source page numbers (like a TOC)
- to show visible page labels in the right margin
- to use these source page numbers in the footer, the TOC,
the GoTo and SkimTo widgets, and to use the source page
number in the standard bookmark and highlight initial text.
Solve the issue when a font without bold and italic variants
is used as the default font (e.g. FreeSerif), and the style
tweak "Ignore publisher font-family" is used (which uses
a trick to cancel any font-family by requesting a font
named "NoSuchFont"):
When text is italic or bold, any of the registered fonts
which have a real italic or bold variant would win over our
default font, as the best substitute to NoSuchFont-Italic.
This gives our default font a bit more bias so it can
win in its scoring against the other fonts, and be rendered
as fake italic or fake bold - which will ensure consistent
font and line height.
(A bit hacky, but no alternative solution found.)
Enable new rendering feature COMPLETE_INCOMPLETE_TABLES
on all enhanced rendering mode, but have it disabled for
earlier cre_dom_version.
Also increase default_cre_storage_size_factor from 20 to 40.
* Allow running Shell/Python scripts from the FM
* Show an InfoMessage before/after running the script
Since we're blocking the UI ;).
* Allow running scripts from the favorites menu, too.
Also reset cache when calling :enableInternalHistory(), as it solves
screen orientation change unit tests failures (although there is no
issue in regular use of the app).
Optimise usage of some of CreDocument methods by caching
their results, either globally, or per page/pos for those
whose result may depend on current page number or y-pos.
This helps a lot on cre books with a lot of highlights.
Includes:
- New option to tune word spacing: space width scale percent
- Text: look for hyphenation in more words if needed
- CSS: fix "hyphens:none" should override inherited "hyphens:auto"
- getHtml(): grab dir= and lang= attributes from upper nodes
Replace our Word Gap/Space condensing toggle/setting with
a new Word Spacing toggle/setting, made of 2 values:
- 1st number scales the normal width of spaces in all font
(100% uses the font space width untouched)
- 2nd number applies after the 1st has been applied, and
tells how much these spaces can additionally be condensed
to make more text fit on a line.
Includes:
- (Upstream) Unified cache for chars and glyph's indexes
- (Upstream) Glyph caching: optional use of hash table instead of a linked list
- (Upstream) Adds docx and fb3 support
- Update hardcoded elements list and stylesheets
- DocX: build a HTML DOM instead of a FB2 DOM
- Text: allow wrap after '/' and '-'
- Text: with Harfbuzz, split measurement on text node change
- Clear fonts on load and re-rendering
Includes:
- Adds tinyNodeCollection::getStatistics()
- epubfmt.cpp: cleanup indentation and small optimisation
- Add more progress callback events
- Optimisation: avoid re-init for EPUBs with embedded fonts
- Fix: don't draw border when border-color: transparent
- CSS: ignore 'inherit' inside font-family
Show a small progress meter at top left of screen when
crengine is loading, re-rendering, and saving cache file
(after a 2 second delay to not be bothered on small books).
Also add a wrapper for the last remaining bit of code that
was accessing _document directly (:getPageOffsetX()).
This commit standardizes the various todos around the code a bit in a manner recognized by LDoc.
Besides drawing more attention by being displayed in the developer docs, they're also extractable with LDoc on the command line:
```sh
ldoc --tags todo,fixme *.lua
```
However, whether that particular usage offers any advantage over other search tools is questionable at best.
* and some random beautification
Bump crengine: background image and other fixes:
- View HTML: option to show text unicode codepoints
- Fix attribute parsing: decode &-encoded chars
- Text: allow wrap after more unicode spaces and hyphen
- Fix: no left hanging when hanging punctuation disabled
- lvtextfm.cpp: more comments, some formatting cleanup
- Optimize background image drawing
- CSS: fix background-image file path resolution
Bump luasec to 0.8.1
Bump harfbuzz to 2.6.0
cre.cpp:
Adds setBackgroundImage() proxy function to crengine
facility to set background textures (this is unrelated
to the background-image CSS fixes).
No frontend code (yet) to select such an image.
View HTML: adds a 3rd view (extended debug view), showing the
unicode codepoint of each char and crengine rendereing methods.
Adds "Render mode" toggle with 4 modes, to enable some or all
of crengine's new enhanced block rendering features.
Use "legacy" rendering for previously opened books to not mess
with bookmarks, and "web" (all of the new features) for new books.
Also make available the added "HTML5" stylesheet to the Style menu.
- Removed "Swipe to follow first link on page" menu item and
handling code, as it feels not really as practical as
"Swipe to follow nearest link".
- Removed recently added "External link action", as we can
just always present a popup with the url and the available
actions.
- Generic handling of these actions in onGoToExternalLink(),
so they are proposed on the Wikipedia lookup popup too.
- Allow external link on PDF documents (previously, only
internal links were handled).
- Have "Ignore external links on tap" available on all
document types.
- Added "Ignore external links on swipe" (default to true,
the current behaviour).
- Added multiswipe gesture "Follow nearest internal link"
(the existing "Follow nearest link" now follows the
nearest external or internal link)
- ButtonDialogTitle: added an option to look a bit more
alike ConfirmBoxes.
- Footnote popups: fix link unhighlight when tap on external link.
Have the code pre-invert some images when in night mode,
so they get inverted back to normal by night mode.
This will allow images to be displayed normally in
the following contexts:
- images in credocument pages
- long-press on images in credocument viewed in ImageViewer
- cover thumbnails in File browser and History
- full size covers when viewed in ImageViewer
- images in Wikipedia lookup results
It was doing a full document load to get the cover. It now does
the faster "only metadata" load.
Also move the trick of setting a default font in CreDocument, so
that all callers of document:loadDocument(false) benefit from it.
Prevent crash when no cover image is available (even if the cache
says it has one, the file may have been updated and doesn't have
it anymore).
Similar to what's been added for 1 page view, but just turn
one page instead of switching to scroll mode when reaching
top left or bottom right corners.
Also make the selection start xpointer more accurate by
getting them in onHold(), instead of possibly too late in
onHoldPan() where we have already moved.
Adds a toggle switch in the Orientation bottom config menu to
allow showing 1 page or 2 pages when in landscape mode.
Previously, this was hardcoded to be in 2-pages modes only
in some circumstances (device resolution + user dpi).
By scrolling last page a little bit further.
Also fix a few other scroll mode issues, all related
to doc_height not being updated (eg, in the middle of
a book, and doubling the font size, one would not
be able to turn pages and read the 2nd half of the
book...)
Avoids a use-after-free in mupdf.scaleBlitBuffer
Fixkoreader/koreader-base#821
Thanks for the hint, @poire-z ;).
* As @poire-z suggested, the original unscaled bb should probably be free'd.
Includes:
- Update french hyphenation pattern
- epub.css: update style for 'blockquote'
- DrawBuf: count nb of images and surface drawn
Adds Document:getDrawnImagesStatistics() to help deciding
if refresh with (possibly costly) dithering should be used
(on devices with HW dithering capabilities) with CreDocument:
when a page contains enough images to benefit from it.
When a main document is opened for displaying, some other
document openings (for getting metadata or cover image)
could affect the main document.
Split some code from CreDocument:init() into another new
method CreDocument:setupDefaultView(), that will only be
called by ReaderUI when opening the main document (and not
by these other openings like Book inforation, View cover...)
Also speed up some of these other openings (Search, Screensaver)
by using doc:loadDocument(false) to load only metadata and
avoid parsing the HTML.
Details in https://github.com/koreader/koreader/issues/4346#issuecomment-440036496
Take full/top status bar's height into account when
computing coordinates.
Also wrap getPageMargins() in CreDocument, so we
don't use cre.cpp _document directly anywhere else.
Adds new options to the Links> submenu, for now only
available and used with CRE documents.
- Allow larger tap area around links
- Ignore external links
- Show footnotes in popup
- Show more links as footnotes
(This last item is mostly for testing and loosening the
footnote detection algorithm, and see how it would behave
with glossary-like links and inter glossary terms links.)
Fix distance computation from gesture position to link by
using segments.
Code for detecting if a link is a footnote is in cre.cpp, and
tweakable a bit with flags in ReaderLink:showAsFoonotePopup().
Footnotes HTML content is displayed by a new FootnoteWidget,
which uses MuPDF for its rendering.
From it, swipe south or tap outside to close, swipe to the left
to follow the original link and jump to the footnote location
in the book.
Also fix tap on highlights after the recent change to use segments
for displaying: use segments also when checking taps.
Setting the hyph algo before loading the document may save crengine
from re-doing some expensive work at render time (the hyph algo
is accounted in the nodeStyleHash, and would cause a mismatch if it is
different at render time from how it was at load time - "English US" by
default - causing a full re-init of the nodes styles.)
We will only re-set it on pre-render (only then, after loading, we
know the document language) if it's really needed: when no algo saved
in book settings, no default algo, and book has some language defined.
* Better KOA2 support
* Fix FL
* Add physical pageturn keys
* Add accelerometer support
* Possibly better NaturalLight handling on the Clara
* Fix WiFi module name used in some(?) Mk7 Kobos
* A bunch of kodev fixes
Adds a new Hyphenation method: Soft-hyphens only.
Adds a new toggable option, usable with Algorithm and language
based methods: Trust soft hyphens (if enabled, if soft hyphens
found in word: use them and skip regular method).
Default in crengine is 50%, and this may be too much for some
people (words can look too much stuck to each other on some lines).
This manual setting may help testing if 70% or 80% is better, with:
"cre_min_space_condensing_percent" = 80
Includes:
- Avoid page break between a node and its borders
- Allow building an alternative TOC from document headings
On CRE documents, allows toggling between original TOC and an
alternative TOC with long-press on the "Table of content" menu
item.
Also update the Wikipedia stylesheet to further avoid
page-breaks between images and their caption, so a full
bordered wikipedia thumbnail is always full on a single page.
Use epub.css as the main default style, with all file formats
except FB2 (which needs fb2.css).
(epub.css has been cleaned recently to be more conforming to HTML
specs and to not include class name based styles - with conditional
compatiblity styles for previously opened documents. It should be
usable on all HTML based documents, except FB2 which has some
incompatible specs.)
Consider all shipped css files other than "epub.css" and "fb2.css"
obsolete, and put them in a sub-menu (these other css files have
not been updated in the same way, and are kept as-is for when a
previously opened document requests one of them).
Add an icon indicating which style is set as default (like it's
been done for the Font and Style tweaks menus).
Also set the font size of the full status bar (available with
cre documents) when the setting "cre_header_status_font_size"
is present in settings.reader.lua (to add manually).
* Switch all initial highlights to "fast" update
i.e., everything that does an invert
Plus a few other things that refresh small UI elements onTap
Re #3130
* Tweak refreshtype for a number of widgets:
* Fix iconbutton dimen
* Make touchmenu flash on close & initial menu popup. Full-screen on close.
* Use flashing updates when opening/closing dictionary popup. Full-screen on close.
* Switch FileManager to partial.
It's mostly text, and we want flash promotion there.
* Make configdialog & menu flash on exit
* Make FLWidget flash on close
* virtualkeyboard: flash on layout change & popup.
* Potentially not that great workaround to ensure we actually see the
highlights in the FM's chevrons
* Flash when closing BookStatus Widget
* Optimize away a quirk of the dual "fast" update in touchmenu
* Promote updates to flashing slightly more agressively.
* Document what each refreshtype actually does.
With a few guidelines on their optimal usecases.
* Switch remaining scheduleIn(0.0) to nextTick()
* Tighter scheduling timers
Shaving a hundred ms off UI callbacks...
* Cache FFI C Library namespace
* Ask MuPDF to convert pixmaps to BGR on Kobo
Fix#3949
* Mention koxtoolchain in the README
re #3972
* Kindle: Handle *all* fonts via EXT_FONT_DIR instead of bind mounts insanity
* Make black flashes in UI elements user-configurable
(All or nothing).
* Jot down some random KOA2 sysfs path
Books previously opened (that have a metadata.lua, whether with
highlights or not, but because they all have a last_xpointer that
points to the current page, that can be broken too by the changes)
will continue to have the old wrong behaviour, so they won't be
impacted by the breaking changes.
Update cre_dom_version code to use the new functions from cre.cpp.
Remove these Workarounds from Style tweaks.
Alternate rendering of list items as block elements (instead of
inline/final).
Check the built DOM is still coherent with styles on stylesheet
change, and propose to reload the document if not. Also
invalidate cache on close in that case so a new DOM is built
at next opening.
ReaderStyleTweak: fix tweak css duplication; when globally
enabled, when disabling and re-enabling locally, the css would
be added twice to the final CSS.
Also adds !important to "sub_sup_smaller" tweak, and increase
its priority so it can override "font_size_all_inherit".
Adds CreDocument:getCacheFilePath() for futur features.
Also allows for updating the fallback font and see results
in real-time on the underlying document.
Bump crengine for: fix updating fallback font
Also adds a menu item to generate a html document showing some
sample text rendered with each available font (only if the user has
a file koreader/settings/fonts-test-sample.html with some HTML
snippet of his choice).
Includes:
- [build] FreeType: bump to 2.9.1
- [build] bump libjpeg-turbo to 1.5.3
- bump crengine:
- Fix CSS selector specificity computation
- [fix] crengine/src/lvfntman.cpp: improve letter_spacing limit
- [fix] crengine/src/lvxml.cpp: add all empty HTML elements
- html head styles: fix styles not being applied
- Hyphenation: increase MAX_PATTERN_SIZE from 16 to 35
- html documents: proper handling of <BR> tags
- Allows requesting old (broken) XML/DOM building code
- cre.cpp: setStyleSheet(): accept filepath and/or css text,
and added getIntProperty() and getStringProperty()
credocument/readerrolling: request older XML/DOM building code
for books previously opened, to not lose bookmarks and highlights
(unit tests update because of new handling of consecutive BR via css
pushed links down in the book)
New module RenderImage (alongside existing RenderText) to provides
image rendering and scaling facilities.
Uses MuPDF, but tries first giflib on GIF.
Allows for getting all the frames from an animated GIF.
When hyphenating a word, the existing algorithms enforced a minimal
length of 2 for each word fragments on left or right side.
This adds a widget to allow changing these minimal sizes (from 1 to 10).
bump crengine:
Allow for quicker loading when interested in metadata only.
Allow for more than 65535 different attribute values.
Increase gamma values range.
Bump crengine for cache re-use fix and warnings when
cache sizes reached.
Allow for increasing some crengine cache sizes with a new
setting: "cre_storage_size_factor" = 1.5 (or 2, or 4...)
With ["cre_compress_cached_data"] = false in settings.reader.lua.
Not using compression for cache files does indeed take more space,
but it does speed up opening, rendering, page turns, and closing,
with big documents.
Bump base and crengine.
cre:loadDocument() may fail in recognizing the document format, and
koreader would previously keep calling other methods on it, which would
make crengine segfaults. We now check loadDocument success at the
various places it is called, and try to deal the best way we can when it fails.
This is done by/for kosync plugin at each opening, because
the docsettings was re-opened and saved for this, but later
overwritten by the current koreader docsettings - so it was
redone each time. This correctly adds this partial_md5_checksum
to the current koreader docsettings, which will be saved on
document closing - so it will not be redone next time.
Note: this partial_md5_checksum is not (yet) used by anything.
Avoid unnecessary work in ReaderView:onSetViewMode() and
ReaderRolling:updatePos() that would result in TOC being reset
and rebuilt (which can take time on books with huge TOC) during
reader setup.
* Allow for colored rendering
Available with all engines (CRE, PDF, Images).
Needs to manually add setting: "color_rendering" = true
* Disable color for djvudocument
* Use Screen:isColorEnabled()
* Bump base
Base bumped.
Previsouly, parts of document in <p style="font-family: serif">
would be rendered with the first font set, and would not
follow change of font (unline other texts without styles).
That was observable with Embedded Styles set to On, but is even
more noticiable with the new Embedded Fonts setting set to off.
Just under the existing "Embedded Styles". Support for this
was already there in crengine, and interfacing is similar to
what is done with Embedded Styles.
So an image file can have a cover in coverbrowser's display.
It also allows for an alternative viewer (ImageViewer widget)
when holding on file / View full size cover.
They may already be available (with other statistics stuff) under
the 'stats' key, but not if Statistics plugin is disabled. As this
plugin may be migrated to SQLite, readerui and readerfooter will now
set two new properties: doc_props (containing all and only the metadata
returned by document:getDocumentProps()) and doc_pages (updated
by readerfooter on any display or font change).
PDF document: returns additional PDF properties: Keywords
and Subject (as Description).
CRE cache, hyphdict and fonts can be initialized only once
when first credocument is opened. Previously, they were
recreated for each document, and as previous instances were probably
not free'd, this caused memory leaks.
When such blank page has only a few black pixels (dust), it would result in
a really small cropped area, than when scaled to fit screen, would cause a
crash with :
./ffi/mupdf.lua:63: could not allocate pixmap:
malloc of array (451342 x 612558 bytes) failed (integer overflow) (1)
Fix similar to what is done in KoptInterface:getAutoBBox(): when cropped area too small,
fall back to whole page (or here: to user's manually set area).
* Start battery stat plugin
* BatteryStat & kobolight
* Several minor improvements
* Remove a useless function
* flush settings
* Some review feedbacks
* Resolve review comments
* Remaining Minutes -> Remaining Hours
* Add dump_file
* typo
* realpath
* realpath on folder
* Remove useless os.time()
* Resolve review comments
* warning
* Add BatteryStat.debugging flag
* treat log as txt
* Minor improvement
* Charging hour should be positive
* Use warn instead of info
* onSuspend in Kobo
* Charging events for kobo and kindle
* More events
* dumpOrLog
* Warnings
* Typo
* More space
* Singleton
* slightly format change
* BatteryStat singleton
* Init
* Remove debugging flag
* sleeping percentage is still negative
* Read settings
* Do not need to change was_suspending and was_charging
* Typo
* Remove debugging flag
* Not charging should happen before suspend
* Resolve review comments
* was_suspend and was_charging should be updated each time in onCallback()
this should implement feature request of zoom mode for multi-columns
page in #501
This PR depends on koreader/koreader-base#435
How to use?
1. Tap the top left corner of a PDF/Djvu page to get into the flipping
mode
2. Double-tap on text block will zoom in to that column
3. Double-tap on any area will zoom out to an overview of the page
4. repeat step 2 to focus to another page block
How does it work?
1. We first find the mask of text blocks in the page. (Pic 1)
2. Then we intersect page boxes with user tap to form a page block. (Pic 2)
3. Finally we zoom the page to the page block and center current view to
that block. (Pic 3)
Currently only tested on Ubuntu-touch emulator with framework
ubuntu-sdk-14.10 for armhf.
The ubuntu-touch port is binary compatible with the Kobo port
major changes in this PR are:
1. rename the emulator device to sdl device since both the emulator
and the ubuntu-touch target use libsdl to handle input/output.
2. ubuntu-touch app has no write access to the installation dir so
all write-outs should be in a seperate dir definded in `datastorage`.
On kindle, kobo and pocketbook the data directory is the current
running directory but on Android the app is installed in system
defined location and users may have no access to that location.
The same circumstances should be true for the upcoming Koreader for
Ubuntu touch, so the data directory (in which tessdata, dictionaries,
global settings, persistant defaults and probably history data are
stored) could be stored in another place.
to support custom font directory for EPUB documents
Now Koreader could find fonts in the "fonts" directory in the USB root
directory of kindle, Kobo and PocketBook devices, thus no need to copy
fonts to "koreader/fonts" directory.
