You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
koreader/frontend
Aleksa Sarai 5709b4c2f1
kopt: correctly handle CJK character detection for space insertion (#8438)
Previously getTextFromBoxes would just pass the first and last three
bytes of the current and previous words when trying to detect CJK
characters (which shouldn't have spaces inserted).

However, this handling was not correct because CJK characters can be
longer than 3 bytes, and internally BaseUtil.utf8charcode doesn't ensure
that it was only given a single utf8 character (it blindly does the bit
operations on whatever length code you give it).

As a result, before this patch selections in PDF documents would have
lots of spaces stripped because getTextFromBoxes would think that almost
all characters were CJK characters.

Fixes: 6f1b70e5eb ("util.utf8: improve CJK character detection")
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
3 years ago
..
apps ReaderDictionary: Colons in menus (#8431) 3 years ago
device Kobo: On sunxi, add a delay between the white flash and the sleep 3 years ago
document kopt: correctly handle CJK character detection for space insertion (#8438) 3 years ago
ui [Status bar] Add custom text to footer (#8419) 3 years ago
cache.lua DocCache: Allow disabling it (again) (#8198) 3 years ago
cacheitem.lua Cache: Some more tweaks after #7624 3 years ago
configurable.lua KOPTInterface: Minor optimization when hashing the configurable status 3 years ago
dbg.lua [doc] Documentation stub for Dbg module (#7677) 3 years ago
depgraph.lua Tame a few tests that relied on `pairs` being somewhat deterministic (#6371) 4 years ago
device.lua Truly silence the attempt at loading SDL2 3 years ago
dispatcher.lua Dispatcher: fix horizontal margins (#8344) 3 years ago
docsettings.lua DocSettings/Purge .sdr: reword, don't purge other books (#8348) 3 years ago
dump.lua Order keys in settings.reader.lua (#6868) 4 years ago
fontlist.lua fontlist: disable/enable some Kindle fonts (#8233) 3 years ago
gettext.lua [fix] gettext: unescape \\ for ID matching (#8357) 3 years ago
httpclient.lua build: enforce luacheck in travis build 8 years ago
languagesupport.lua *: luacheck fixes (#8368) 3 years ago
logger.lua use android log categories 5 years ago
luadata.lua Use fsync() for more robust setting files saving 5 years ago
luasettings.lua LuaSettings: Add a method to initialize a setting properly (#7371) 3 years ago
luxl.lua [fix] Don't break OPDS parsing on HR tags (#5949) 4 years ago
optmath.lua [[doc] Add documentation to optmath (#6258) 4 years ago
persist.lua Cache: Fix a whole lot of things. 3 years ago
pluginloader.lua crash.log: write plugin regular information only in debug mode (#8230) 3 years ago
pluginshare.lua Move PluginShare.backgroundJobs into PluginShare module (#3128) 7 years ago
random.lua Faster blitting @ BB8/BBRGB32 when no processing is needed (#4847) 5 years ago
readcollection.lua Minor util & ffi/util cleanups (#6657) 4 years ago
readhistory.lua ReadHistory: nil guard a Document instance access 3 years ago
socketutil.lua Unify LuaSocket usage (#7405) 3 years ago
util.lua util: add reversible table method wrapping helper 3 years ago
version.lua Centralize one time migration code after updates (#7531) 3 years ago