2
0
mirror of https://github.com/koreader/koreader synced 2024-11-10 01:10:34 +00:00
Commit Graph

52 Commits

Author SHA1 Message Date
poire-z
626864f856 [chore] replace utf8 bytes with Unicode escape sequence 2023-08-02 01:28:24 +02:00
poire-z
08e9c27a32 Wikipedia EPUBs: re-add ToC in content
The ToC is no longer in the HTML we get from the
Wikipedia API. So, add it ourselves.
Also, as we can't get anywhere the Wikipedia localized
string for "Contents" (ie. "Sommaire" in French), use
thick <HR> to mark the start and end of this ToC.
2023-08-02 01:28:24 +02:00
poire-z
4fa278ff0b Wikipedia: handle images in changed Wikipedia HTML
The HTML we get from Wikipedia has recently changed,
which has caused images to not be detected and shown
in DictQuickLookup, and to be missing or ugly in
saved EPUBs. Update our code and CSS to display them
again.
2023-07-26 01:05:15 +02:00
poire-z
011c168686
Wikipedia EPUBs: strip out invalid-XHTML <link> (#10462) 2023-05-19 16:48:36 +02:00
poire-z
fc81c7db24 DictQuickLookup: add button to show list of results
Add a left button to the title bar to show the list
of results as a popup.
Dictionary: tap or long-press on that button give
different view of the results.
Wikipedia: request 30 results instead of 20, so we
can show 15, 10 or 6 of them per page of that popup.
2023-05-12 20:12:10 +02:00
NiLuJe
5c24470ea9
Logger: Use serpent instead of dump (#9588)
* Persist: support serpent, and use by default over dump (as we assume consistency > readability in Persist).
* Logger/Dbg: Use serpent instead of dump to dump tables (it's slightly more compact, honors __tostring, and will tag tables with their ref, which can come in handy when debugging).
* Dbg: Don't duplicate Logger's log function, just use it directly.
* Fontlist/ConfigDialog: Use serpent for the debug dump.
* Call `os.setlocale(C, "numeric")` on startup instead of peppering it around dump calls. It's process-wide, so it didn't make much sense.
* Trapper: Use LuaJIT's serde facilities instead of dump. They're more reliable in the face of funky input, much faster, and in this case, the data never makes it to human eyes, so a human-readable format didn't gain us anything.
2022-10-06 02:21:03 +02:00
NiLuJe
62059f8d68
Misc: Get rid of the legacy defaults.lua globals (#9546)
* This removes support for the following deprecated constants: `DTAP_ZONE_FLIPPING`, `DTAP_ZONE_BOOKMARK`, `DCREREADER_CONFIG_DEFAULT_FONT_GAMMA`
* The "Advanced settings" panel now highlights modified values in bold (think about:config in Firefox ;)).
* LuaData: Isolate global table lookup shenanigans, and fix a few issues in unused-in-prod codepaths.
* CodeStyle: Require module locals for Lua/C modules, too.
* ScreenSaver: Actually garbage collect our widget on close (ScreenSaver itself is not an instantiated object).
* DateTimeWidget: Code cleanups to ensure child widgets can be GC'ed.
2022-09-28 01:10:50 +02:00
NiLuJe
83a2965d6b
Misc: Unify error logging on network errors (#9523)
Making sure we get the relevant information in the logs, and that the UI feedback (if any), is meaningful and readable.
2022-09-17 00:08:00 +02:00
poire-z
c3cd88e019 Wikipedia EPUBs: keep math SVG images 2022-09-12 00:40:08 +02:00
poire-z
9cba1904ee
CRe, MuPDF, PicDocument: add support for WebP images (#9402)
Bump base, includes:
- thirdparty/luasec 1.2.0
- Bump NanoSVG: various small fixes
- Add thirdparty/libwebp and ffi/webp.lua wrapper
- MuPDF: add support for webp images
- bump crengine: add support for webp images
- thirdparty/harfbuzz 5.1.0
RenderImage:renderImageData(): render webp images
with ffi/webp.lua.
2022-08-03 23:19:31 +02:00
poire-z
94b76097aa Style tweaks: reorganize menu items
Make them a bit more logically grouped by topic.
Update a few priorities.
2022-07-07 00:32:07 +02:00
NiLuJe
23cd7e24bb
Minor Lua I/O cleanups (#8921)
Mostly making sure we always explicitly close io handles.
2022-03-19 23:57:31 +01:00
poire-z
402b0cf895 Wikipedia: tweak EPUB css and footnote styletweaks
EPUB: have pages with footnotes marked as non-linear
so they can be hidden with "Hide non-linear flows".
In-page footnotes: keep the footnote number near the
left margin (a recent crengine renders them near the
text by default).
2022-01-21 21:36:10 +01:00
NiLuJe
2f9db25969
Unify LuaSocket usage (#7405)
* Add a new socketutil module with a few helper functions that allow us to:
  * Always use a sane User-Agent (previously, only Wikipedia did so)
  * Set timeouts in an almost sane manner. Doing it explicitly prevents an interaction with KOSync that does crazy stuff I don't even want to try to understand.
* Unified said timeouts based on the request's intended usage (except for Wikipedia, which already had meaningful timeout values).
* Stopped using LuaSec directly, LuaSocket defers to LuaSec sanely on its own. Everything now transparently supports HTTPS without code duplication.
2021-03-15 01:25:10 +01:00
NiLuJe
dffe86dfe9
Cleanup eye-gouging madness around io.read calls (#7149)
* Don't reinvent the wheel when reading a one-line int or string from sysfs

* Simplify a whole other bunch of read calls
2021-01-16 04:41:46 +01:00
NiLuJe
1f994f8ede
Floor dimension computations (mul/div). (#6264)
* floor most every dimension computations involving MUL or DIV
Should avoid passing nonsensical floating point coordinates/dimensions
to the UI code.

* Update base

* https://github.com/koreader/koreader-base/pull/1113
* https://github.com/koreader/koreader-base/pull/1114
* https://github.com/koreader/koreader-base/pull/1115

* Bump android-luajit-launcher

https://github.com/koreader/android-luajit-launcher/pull/230
https://github.com/koreader/android-luajit-launcher/pull/231
2020-06-13 01:56:36 +02:00
poire-z
03140f505c Wikipedia EPUBs: keep links in bold text bold 2020-05-29 15:07:14 +02:00
poire-z
8ffb9aff64 Wikipedia EPUBs: keep original float left/right in web mode
Previous cleanup resurrected multiple class names, so
we know now the original alignment of a float.
Use them in 'web' mode, as it allows floats to be nearer
the section they are refering to, and it may limit
floats truncation in page splitting.
But keep floats aligned on a single side in 'book' mode,
as it might allow easier reading.
2020-02-10 19:17:20 +01:00
poire-z
6d2cada96e Wikipedia Save as EPUB: better gallery rendering
Switch Wikipedia galleries from floats to inline-block.
Also remove some crengine bugs workaround code as these
bugs have been fixed for a long time.
2020-01-31 15:33:13 +01:00
poire-z
d6d49a64a7 [RTL UI] use auto or LTR text direction in some specific cases
Allow TextBoxWidget new text direction/lang parameters to be
set on upper widgets, and propagate them all the way to it
(ScrollTextWidget, InputText, InputDialog, TextViewer).

Use specific non-default ones in some specific cases:
- Force LTR text direction when showing HTML and CSS, and
  configuration files (in some plugins).
- Use Wikipedia server language and text direction when
  showing an article.
- Use auto with Dictionary results, as we don't know the
  dictionary language, and they may contain mixed content.
- Force LTR when showing some paths (still needs more of them)

TextEditor plugin: add 2 new options "Auto paragraph direction"
and "Force paragraph direction LTR".

Footnotes popup: grab HTML direction, and forward it
to MuPDF for proper display.
2019-12-08 15:10:51 +01:00
poire-z
4462380048
Wikipedia lookup results: fix page turning when image loading (#5560)
Tapping to turn page while some image is loading was
interrupting the image loading, but not turning the page,
since a82d7832 where using trap_widget=false is to be
used to not resend the event (while =nil does resend it).
2019-11-02 02:30:12 +01:00
poire-z
3e3b00429f
EPUB: more support for RTL documents (#5436)
bump crengine: enhanced RTL block rendering.
Includes:
- Re-enable: Use the same FT load flags in HB as in FT
- Avoid redundant calls to getStyle()
- CSS: parse and store "direction:"
- Enhanced block rendering: handle RTL direction
- Enhanced block rendering: handle RTL tables
- Pagesplitting: account for lines' text direction
- CSS: add support for more pseudo-classes
- epub.css: a few updates for RTL documents
- Fix LVDocView::getBookmark() which could be slow or wrong

Add a few style tweaks useful to RTL readers.

Wikipedia Save as EPUB: build proper RTL documents from
persian, arabic, hebrew (and others) Wikipedia articles.
2019-09-29 23:02:54 +02:00
Frans de Jonge
a2dcfe9aec
[doc] Tag @todo, @fixme and @warning (#5244)
This commit standardizes the various todos around the code a bit in a manner recognized by LDoc.

Besides drawing more attention by being displayed in the developer docs, they're also extractable with LDoc on the command line:

```sh
ldoc --tags todo,fixme *.lua
```

However, whether that particular usage offers any advantage over other search tools is questionable at best.

* and some random beautification
2019-08-23 19:53:53 +02:00
poire-z
8749a3c5b6 Wikipedia download: increase timeout 2019-08-03 20:38:57 +02:00
poire-z
642aed240c Update Wikipedia EPUB stylesheet to allow floating thumbnails
Main thumbnails and galleries items will float in "book" and
"web" modes.
Also use a honest User-Agent when requesting Wikipedia to
comply with their policy and avoid rate limitation (and
broken images).
2019-07-15 20:14:33 +02:00
poire-z
f8d8863712 Style tweaks: split in-page footnote tweaks
Split the In-page footnotes tweak into 3 distinct ones,
mainly because I want the 3rd one with classic class names
to be disabled'able, while keeping the others, in case
these classic class names are not used for footnotes.

Also fix footnotes list-style-type in Wikipedia EPUBs,
which may have been wrong (but it was less noticable when
following page links because of the little black marker,
or showing them in popup footnotes where the number/letter
is not shown).
2019-01-06 23:06:04 +01:00
poire-z
ea946d52d6 bump crengine: tables rendering improvements and others
bump crengine, which includes:
- Adds support for symbol fonts (local or embedded)
- Fix some issues when rendering text in constrained width
- Page splitting: fix possible missing blocks
- CSS: adds support for 'auto', ignore % for borders
- Fix right border drawing position
- Fix: adds missing properties in copystyle()
- Adds comments, erm_killed rendering method
- Adds getRenderedWidths(): get node min/max node content width
- Tables rendering: fixes and improvements
- getRenderedWidths: enable min_width to be a single CJK char
- Fix wrong text wrap avoid in some case
- epub.css: add style for 'blockquote'
- Fix rendering issue when line ends with an image

Adds a few style tweaks related to tables.

Enforce table width: 100% in Wikipedia EPUBs to keep
previous look, which feels better with the various kinds
of tables in Wikipedia pages.

Fix unit tests as juliet.epub (full of blockquotes), grew quite
a few pages with the epub.css update.
2018-12-07 14:11:46 +01:00
poire-z
c3a938aad6
bump crengine: alternative TOC, no page break on borders (#4011)
Includes:
- Avoid page break between a node and its borders
- Allow building an alternative TOC from document headings

On CRE documents, allows toggling between original TOC and an
alternative TOC with long-press on the "Table of content" menu
item.

Also update the Wikipedia stylesheet to further avoid
page-breaks between images and their caption, so a full
bordered wikipedia thumbnail is always full on a single page.
2018-06-14 00:46:52 +02:00
poire-z
bdbaa7dfdd
bump crengine: HTML and rendering fixes (#3992)
- Fix getting xpointer to current page
- epub.css cleanup
- fb2def.h update
- Fix a few edge-case rendering issues
- Fix border being drawn in spite of 'border-width: 0'

Also update Wikipedia epub.css stylesheets regarding
recent HTML fixes and enhancements.
Remove "Remove all borders" workaround tweaks, as it's
now fixed.
(Unit tests failure to be fixed in upcoming commit)
2018-06-02 13:30:49 +02:00
poire-z
b498d872be
bump crengine: LI and CITE fixes are now the defaults (#3976)
Books previously opened (that have a metadata.lua, whether with
highlights or not, but because they all have a last_xpointer that
points to the current page, that can be broken too by the changes)
will continue to have the old wrong behaviour, so they won't be
impacted by the breaking changes.
Update cre_dom_version code to use the new functions from cre.cpp.
Remove these Workarounds from Style tweaks.
2018-05-25 00:22:08 +02:00
poire-z
4bb3999cbc RenderImage: factorize all image rendering and scaling code
New module RenderImage (alongside existing RenderText) to provides
image rendering and scaling facilities.
Uses MuPDF, but tries first giflib on GIF.
Allows for getting all the frames from an animated GIF.
2018-04-22 17:00:29 +02:00
poire-z
58f6352d60 Wiki saved as EPUB: stylesheet fix (#3894)
Removed width:80% as it was never supported by crengine (the effect
was achieved thanks to margins and paddings).
Wikipedia saved articles from now on won't be effected by a double
effect if we later add full support for width: into crengine.
2018-04-18 22:13:44 +02:00
poire-z
305e75c5ea Wikipedia Save as EPUB: various encoding fixes (#3851)
* Wiki Save as EPUB: various encoding fixes

Fix display of & in article titles
Fix display of &, <, > in TOC entries and in targeted anchor (the mismatch
with the target id made these TOC entries invalid and simply not displayed).
Remove percent-encoded URLs tweaks for crengine now that crengine
correctly supports them (each percent encode handled as an UTF8 byte).
Bump crengine for that.
Don't include <meta name="cover"> when no cover present.

* bump base/crengine
2018-04-10 18:30:27 +02:00
poire-z
5cfe9992ad Wiki Save as EPUB: better handling of math formulas (#3639)
Also fix some bad filename and extension in epub's OEBPS/content.opf,
Store image files (jpg, png, gif) uncompressed in the .epub as they
don't need compression.
2018-01-29 20:50:26 +01:00
poire-z
1e9457c9b4 Wikipedia full & epub: swap symbols for H3 and H4 (#3627) 2018-01-22 14:45:16 +01:00
poire-z
775b7a364f Wikipedia: show images and allow interrupting queries
Reword some english messages
2018-01-17 21:36:21 +01:00
poire-z
69ec6c07c5 Wikipedia Save as EPUB: fix some wiki links and speedup download (#3551)
- Fix links to wikipedia pages containing '?oldid=123', that
  wouldn't work when used back thru the API.
- Make external links to other wikipedias with a different langage
  work (they didn't work when non-ascii page titles)
- Speedup download of images by using a newly added 'fast_refresh'
  option to Trapper:info() to bypass UIManager.
2017-12-20 15:26:01 +01:00
poire-z
8886e16baa Small fix to wikipedia epub stylesheet (#3534) 2017-12-05 17:43:34 +01:00
poire-z
de54ef5ae6 Added Trapper module, for simple interaction with UI
This module provides methods for simple interaction with UI, without
the need for explicit callbacks, for use by linear jobs between
their steps.
Uses coroutines, but their usage is hidden by a simple API.
Factored out of Wikipedia:createEpubWithUI().
2017-09-20 23:09:59 +02:00
poire-z
80d8e44bde Fix Wiki save as epub crash on certain pages (#3231)
* Fix Wiki save as epub crash on certain pages

When first images don't have a height or width attribute
2017-09-17 15:48:47 +02:00
poire-z
79dca8e245 [UX] Wikipedia save as EPUB: allow for aborting download (#3224) 2017-09-16 17:25:47 +02:00
poire-z
0d38547990 Wikipedia save as epub: fix some image download failures 2017-08-07 13:37:08 +02:00
Qingping Hou
fc12ee6537 decouple reader footer from dictquicklookup widget & translation fix (#2664)
* chore: fix strings for translations

* dictionary(chore): decouple reader footer from dictquicklookup widget
2017-03-24 00:20:37 -07:00
Frans de Jonge
6257bae77f Language: mostly Wikipedia-related changes 2017-02-02 02:09:51 -08:00
poire-z
2ded2bce5e Wikipedia: save full page as epub (html, with optional images)
Also for the current text-only wikipedia full page: replace
the ==s in section titles with some unicode symbols for a better
visual feeling of hierarchy. These same symbols are also used
in the .epub.
Both can be disabled by adding ["wikipedia_prettify"] = false
to settings.reader.lua

readerhighlight: close ButtonTable (like other buttons do) when looking
up wikipedia, otherwise if we save as epub, and swtich to new document,
this ButtonTable will never be closed and stays in UI.
2017-01-24 01:16:23 -08:00
Qingping Hou
f95ad00b9e feat: add logger module & rewrite kobo suspend script in lua 2017-01-02 19:52:09 -08:00
poire-z
1708fd5e1c Dictionary and wikipedia enhancements (#2393)
Stardict:
- remove duplicate results
- better cleaning of selection
- append results from a 2nd query of a 2nd set of dictionaries
  in data/dict_ext/

Wikipedia:
- use the search API for better results
- allow viewing the full page content of a result in a bigger window
- allow queries for multiple languages
- available languages can be set in settings.reader.lua :
    ["wikipedia_languages"] = {"en", "fr", "it"}
- "Wikipedia lookup" added to Tools menu

For both:
- allow selection of multiple words for a new lookup (so one can
  actually browse wikipedia)
- allow continuous reading with Tap
- display "current result / total number of results"

Details in #2393
2016-12-11 14:32:02 -08:00
Qingping Hou
3c647ff244 minor: various luacheck fixes 2016-02-15 18:08:04 -08:00
Frans de Jonge
ba994f41ab Fix Wikipedia (HTTP is now HTTPS) and some failsafes
Rudimentary JSON validity check by seeing if the first character is {. The JSON decode function will crash in spite of pcall if it's not.
2015-08-23 21:20:19 +02:00
chrox
82393f0ed3 fix loading frontend/JSON.lua for json.lua on Kindle
The VFAT file system of `/mnt/us` is case insensitive so that
require("json") from Spore which intends to load luajson module in
rock directory actually will load `frontend/JSON.lua`, the two JSON
parser libraries have slightly different APIs.

This patch remove the pure Lua JSON library in favor of `luajson` which
uses LPeg pattern marching backend for better performance.
2015-03-21 13:19:00 +08:00