* Bump base
includes:
koreader/koreader-base#1691koreader/koreader-base#1692koreader/koreader-base#1689koreader/koreader-base#1690koreader/koreader-base#1693
* Integrate decoding of SSIDs within wpa_supplicant
The UTF-8 decoding of SSIDs is specific to wpa_supplicant. In this
patch, we move all of this decoding logic to the wpa_supplicant module.
We expose the raw bytes of the SSID to the NetworkMgr code, and make
sure to always fix bad UTF-8 before we display the SSID to the user.
Within the wpa_supplicant module, we replace the call to the
wpa_passphrase binary to get the PSK with a direct function call to
OpenSSL. This allows us to calculate the PSK over any arbitrary bytes,
including UTF-8. In the same vein, we use the hex-encoded SSID to
communicate with wpa_supplicant when setting up the network to support
arbitrary bytes in the SSID.
Unfortunately, we also remove the tests, as there is no way to unit test
local functions.
* Device: Add a `hasSeamlessWifiToggle` devcap to complement `hasWifiToggle`, to denote platforms where we can toggle WiFi without losing focus, as this has obvious UX impacts, and less obvious technical impacts on some of the NetworkMgr innards...
* Android: Mark as `!hasSeamlessWifiToggle`, as it requires losing focus to the system settings. Moreover, `turnOnWifi` returns *immediately* and we *still* run in the background during that time, for extra spiciness...
* NetworkMgr: Ensure only *one* call to `turnOnWifi` will actually go on when stuff gets re-scheduled by the `beforeWifiAction` framework.
* NetworkMgr: Ensure the `beforeWifiAction` framework will not re-schedule the same thing *ad vitam aeternam* if a previous connection attempt is still ongoing. (i.e., previously, on Android, if you backed out of the system settings, you entered the Benny Hill dimension, as NetworkMgr would keep throwing you back into the system settings ;p). This has a few implications on callbacks requested by subsequent connection attempts, though. Generally, we'll try to honor *explicitly interactive* callbacks, but `beforeWifiAction` stuff will be dropped (only the original cb is preserved). That's what prevents the aforementioned infinite loop, as the `beforeWifiAction` framework was based on the assumption that `turnOnWifi` somewhat guaranteed `isConnected` to be true on return, something which is only actually true on `hasWifiManager` platforms.
* NetworkMgr: In `prompt` mode, the above implies that the prompt will not even be shown for concurrent attempts, as it's otherwise extremely confusing (KOSync on Android being a prime example, as it has a pair of Suspend/Resume handlers, so the initial attempt trips those two because of the focus switch >_<").
* NetworkMgr: Don't attempt to kill wifi when aborting a connection attempt on `!hasSeamlessWifiToggle` (because, again, it'll break UX, and also because it might run at very awkward times (e.g., I managed to go back to KOReader *between* a FM/Reader switch at one point, which promptly caused `UIManager` to exit because there was nothing to show ;p).
* NetworkMgr: Don't drop the connectivity callback when `beforeWifiAction` is set to prompt and the target happens to use a connectivity check in its `turnOnWifi` implementation (e.g., on Kindle).
* Android: Add an `"ignore"` `beforeWifiAction` mode, that'll do nothing but schedule the connectivity check with its callback (with the intent being the system will eventually enable wifi on its own Soon(TM)). If you're already online, the callback will run immediately, obviously. If you followed the early discussions on this PR, this closely matches what happens on `!hasWifiToggle` platforms (as flagging Android that way was one of the possible approaches here).
* NetworkMgr: Bail out early in `goOnlineToRun` if `beforeWifiAction` isn't `"turn_on"`. Prompt cannot work there, and while ignore technically could, it would serve very little purpose given its intended use case.
* KOSync: Neuter the Resume/Suspend handlers early on `CloseDocument`, as this is how focus switches are handled on Android, and if `beforeWifiAction` is `turn_on` and you were offline at the time, we'd trip them because of the swap to system settings to enable wifi.
* KOSync: Allow `auto_sync` to be enabled regardless of the `beforeWifiAction` mode on `!hasSeamlessWifiToggle` platforms. Prompt is still a terrible idea, but given that `goOnlineToRun` now aborts early if the mode is not supported, it's less of a problem.
wpa_supplicant returns all non-ASCII SSIDs as raw bytes in the form
\x0a. We interpret these bytes as UTF-8, and make sure that all invalid
characters are replaced with a �.
Assume the host system does things right otherwise.
Should be sane on Kindle
On PocketBook, who knows, but assuming the device actually suspends,
that should effectively kill our keepalive
Irrelevant for Android, as we skipped it because the wifi toggling
methods are interactive.
Fix#10823
(For reference, we *enable* wifi_was_on no matter *how* wifi is enabled,
but we only toggle it off when it's killed by a *direct* user interaction,
the intent being that if *something* non-interactive enabled wifi,
you'll probably silently need it on resume too).
As early as turnOnWifi.
Implement it on hasWifiManager platforms, preventing useless
connectivity checks to run when they're obviously never going to work
because you're out of range of your AP.
Also implemented a flag to notify the backend if the connection attempt
was interactive or not.
Right now, interactive is extremely restricted, it basically means the
menu checkmark, or a gesture.
The intent being that for stuff like the beforeWifiAction framework, we
don't want the backend to spawn extra UI.
Specifically, for hasWifiManager platforms, we no longer spawn the AP
scan list on failure unless the caller was interactive.
TL;DR: beforeWifiAction is now much less obnoxious when you're obviously
not able to connect.
* Enable before_wifi_action & after_wifi_action on hasWifiToggle platforms (which is basically all of 'em except naked SDL).
* Decouple restoreWifiAsync from hasWifiManger, because we can do that on other platforms (namely, Kindle. Probably PB, too, but WiFi is already a mess there, and I can't test it).
* Implement restoreWifiAsync on Kindle.
* Properly flag rM as hasWifiManager & hasFastWifiStatusQuery, because it is actually both of those (it uses our wpa_supplicant backend).
* Update the KOSync checks to take these changes into account, to properly disable auto_sync if necessary.
* Really made the Network* event signaling consistent. For realz this time.
* In an effort to make the whole beforeWifiAction framework somewhat usable there, we now assume connectivity is always available on !hasWifiToggle platforms...
Fix: #10539, and for context #6489, #6733, #6534
Reorganize and reword most of the settings to make it clear what actually ties into auto sync, and what doesn't. (Specifically, what happens when a pull attempts to sync forward or backward has nothing to do with auto sync, it applies in all cases; while the periodic sync *does* require auto sync).
The main point of contention, though, is that auto sync will now *always* attempt to setup network connectivity (i.e., on resume/suspend/close). Periodic sync will *not* though (the intent being that, if you use periodic sync, you're relying on the activity check to actually keep wifi on at all times)).
Since this may lead to a large amount of nagging about wifi toggles on devices w/ NetworkManager support, it is now *disabled* by default on those devices. (And given that it wouldn't have worked because of the lack of connectivity, that doesn't really make any practical difference ;p).
Additionally, given the fact that there's no way to make this behavior viable if the "before wifi" action is left at its default of "prompt", this feature now *requires* that to be set to "turn_on" (on devices where it can, of course); attempting to toggle it on will warn about that if necessary.
This change is retroactive (OTM).
Includes an assortment of fixes and cleanups, including migrating to the new LuaSettings API, which is why there's no longer a smattering of superfluous flushes.
Much easier to deal with thanks to the cleanup work done in #10062 ;).
* `carrier` is set to 1 as soon as the device is *administratively* up (in practice, as soon as we run `ifconfig up`). This is perfectly fine for `isWifiOn`, but absolutely not for `isConnected`, because we are not, actually, connected to *anything*, no attempt at associating has even been made at that point. Besides being semantically wrong, in practice, this will horribly break the connectivity check, because it expects that `isConnected` means we can talk to at least the LAN.
* Delving into the Linux docs reveals that `operstate` looks like a better candidate, as it reflects *operational status*; for Wi-Fi, that means associated and successfully authenticated. That's... closer, but still not it, because we still don't have an IP, so we technically can't talk to anything other than the AP.
* So, I've brought out the big guns (`getifaddrs`), and replicated a bit of code that I already use in the USBNetwork hack on Kindle, to detect whether we actually have an IP assigned. (Other approaches, like `/proc/net/route`, may not be entirely fool-proof, and/or get complicated when IPv6 enters the fray (which it does, on Kobo, Mk. 8+ devices are IPv6-enabled)).
TL;DR: Bunch of C via ffi, and `isConnected` now returns true only when the device is operationally up *and* we have an IP assigned.
Pulls in https://github.com/koreader/koreader-base/pull/1579 & https://github.com/koreader/lj-wpaclient/pull/10
cervantes kindle kobo remarkable: use sysfs carrier file to determine connection state
cleanup hasWifiManager checks
gateway check: use ip if available
Fixes: #10087Closes: #10092
isWifiOn for kindle currently returns if the interface is connected, change this to doing what is says isWifiOn the file is only present if the wireless interface is up.
isConnected pings the gateway, rely on the kernel for a more reliable check.
Whan connecting to my android phone's wifi hotspot to remote debug from my phone, the network is connected yet the phone(gateway) does not respond to pings leading koreader to shut down the connection thinking it is unsuccessful
Basically:
* Use `extend` for class definitions
* Use `new` for object instantiations
That includes some minor code cleanups along the way:
* Updated `Widget`'s docs to make the semantics clearer.
* Removed `should_restrict_JIT` (it's been dead code since https://github.com/koreader/android-luajit-launcher/pull/283)
* Minor refactoring of LuaSettings/LuaData/LuaDefaults/DocSettings to behave (mostly, they are instantiated via `open` instead of `new`) like everything else and handle inheritance properly (i.e., DocSettings is now a proper LuaSettings subclass).
* Default to `WidgetContainer` instead of `InputContainer` for stuff that doesn't actually setup key/gesture events.
* Ditto for explicit `*Listener` only classes, make sure they're based on `EventListener` instead of something uselessly fancier.
* Unless absolutely necessary, do not store references in class objects, ever; only values. Instead, always store references in instances, to avoid both sneaky inheritance issues, and sneaky GC pinning of stale references.
* ReaderUI: Fix one such issue with its `active_widgets` array, with critical implications, as it essentially pinned *all* of ReaderUI's modules, including their reference to the `Document` instance (i.e., that was a big-ass leak).
* Terminal: Make sure the shell is killed on plugin teardown.
* InputText: Fix Home/End/Del physical keys to behave sensibly.
* InputContainer/WidgetContainer: If necessary, compute self.dimen at paintTo time (previously, only InputContainers did, which might have had something to do with random widgets unconcerned about input using it as a baseclass instead of WidgetContainer...).
* OverlapGroup: Compute self.dimen at *init* time, because for some reason it needs to do that, but do it directly in OverlapGroup instead of going through a weird WidgetContainer method that it was the sole user of.
* ReaderCropping: Under no circumstances should a Document instance member (here, self.bbox) risk being `nil`ed!
* Kobo: Minor code cleanups.
* UIManager: Support more specialized update modes for corner-cases:
* A2, which we'll use for the VirtualKeyboards keys (they'd... inadvertently switched to UI with the highlight refactor).
* NO_MERGE variants of ui & partial (for sunxi). Use `[ui]` in ReaderHighlight's popup, because of a Sage kernel bug that could otherwise make it translucent, sometimes completely so (*sigh*).
* UIManager: Assorted code cleanups & simplifications.
* Logger & dbg: Unify logging style, and code cleanups.
* SDL: Unbreak suspend/resume outside of the emulator (fix#9567).
* NetworkMgr: Cache the network status, and allow it to be queried. (Used by AutoSuspend to avoid repeatedly poking the system when computing the standby schedule delay).
* OneTimeMigration: Don't forget about `NETWORK_PROXY` & `STARDICT_DATA_DIR` when migrating `defaults.persistent.lua` (fix#9573)
* WakeupMgr: Workaround an apparent limitation of the RTC found on i.MX5 Kobo devices, where setting a wakealarm further than UINT16_MAX seconds in the future would apparently overflow and wraparound... (fix#8039, many thanks to @yfede for the extensive deep-dive and for actually accurately pinpointing the issue!).
* Kobo: Handle standby transitions at full CPU clock speeds, in order to limit the latency hit.
* UIManager: Properly quit on reboot & exit. This ensures our exit code is preserved, as we exit on our own terms (instead of being killed by the init system). This is important on platforms where exit codes are semantically meaningful (e.g., Kobo).
* UIManager: Speaking of reboot & exit, make sure the Screensaver shows in all circumstances (e.g., autoshutdown, re: #9542)), and that there aren't any extraneous refreshes triggered. (Additionally, fix a minor regression since #9448 about tracking this very transient state on Kobo & Cervantes).
* Kindle: ID the upcoming Scribe.
* Bump base (https://github.com/koreader/koreader-base/pull/1524)
* This removes support for the following deprecated constants: `DTAP_ZONE_FLIPPING`, `DTAP_ZONE_BOOKMARK`, `DCREREADER_CONFIG_DEFAULT_FONT_GAMMA`
* The "Advanced settings" panel now highlights modified values in bold (think about:config in Firefox ;)).
* LuaData: Isolate global table lookup shenanigans, and fix a few issues in unused-in-prod codepaths.
* CodeStyle: Require module locals for Lua/C modules, too.
* ScreenSaver: Actually garbage collect our widget on close (ScreenSaver itself is not an instantiated object).
* DateTimeWidget: Code cleanups to ensure child widgets can be GC'ed.
Rationale from: https://github.com/koreader/koreader/pull/9064#issuecomment-1116485065
>While we are dealing with WiFi, a slight hijack, on my kindle when the WiFi is on but not connected the ToggleWiFi gesture tries to connect so i have no way of turning it off. For eample I use the gesture to turn it on, then leave the range I can't use it to turn off. I think the more direct approach would be to toggle On -> Off states so if i am disconnected and want to reconnect i can always go Off then On, but I am able to turn it off.
>TLDR: the tooglewifi gesture should check for iswififup not isconnected
* Disable all non power management related input during suspend. (This prevents wonky touch events from being tripped when closing a sleep cover on an already-in-suspend device, among other things).
* Kobo: Use our WakeupMgr instance, not the class.
* WakupMgr: split `removeTask` in two:
* `removeTask`, which *only* takes a queue index as input, and only removes a single task. Greatly simplifies the function (i.e., it's just a `table.remove`).
* `removeTasks`, which takes an epoch or a cb ref, and removes *every* task that matches.
* Both of these will also *always* re-schedule the next task (if any) on exit, since we can have multiple WakeupMgr tasks queued, but we can only have a single RTC wake alarm set ;).
* `wakeupAction` now takes a `proximity` argument, which it passes on to its `validateWakeupAlarmByProximity` call, allowing call sites to avoir having to duplicate that call themselves when they want to use a custom proximity window.
* `wakeupAction` now re-schedules the next task (if any) on exit.
* Simplify `Kobo:checkUnexpectedWakeup`, by removing the duplicate `WakerupMgr:validateWakeupAlarmByProximity` call, now that we can pass a proximity window to `WakeuoMgr:wakeupAction`.
* The various network activity timeouts are now halved when autostandby is enabled.
* Autostandby: get rid of the dummy deadline_guard task, as it's no longer necessary since #9009.
* UIManager: The previous change allows us to simplify `getNextTaskTimes` into a simpler `getNextTaskTime` variant, getting rid of a table & a loop.
* ReaderFooter & ReaderHeader: Make sure we only perform a single refresh when exiting standby.
* Kobo: Rewrite sysfs writes to use ANSI C via FFI instead of stdio via Lua, as it obscured some common error cases (e.g., EBUSY on /sys/power/state).
* Kobo: Simplify `suspend`, now that we have sane error handling in sysfs writes.
* Kobo.powerd: Change `isCharging` & `isAuxCharging` behavior to match the behavior of the NTX ioctl (i.e., Charging == Plugged-in). This has the added benefit of making the AutoSuspend checks behave sensibly in the "fully-charged but still plugged in" scenario (because being plugged in is enough to break PM on `!canPowerSaveWhileCharging` devices).
* AutoSuspend: Disable our `AllowStandby` handler when auto standby is disabled, so as to not interfere with other modules using `UIManager:allowStandby` (fix#9038).
* PowerD: Allow platforms to implement `isCharged`, indicating that the battery is full while still plugged in to a power source (battery icon becomes a power plug icon).
* Kobo.powerd: Implement `isCharged`, and kill charging LEDs once battery is full.
* Kindle.powerd: Implement `isCharged` on post-Wario devices. (`isCharging` is still true in that state, as it ought to).
If we had a local prefered AP with a higher RSSI,
we attempted to associate with it over wpa_supplicant
being already attempting to associate with its own preferred AP.
That... failed horribly.
Also adapt to the new lj-wpaclient API, fixing a few other edge-cases,
and making the whole thing slightly faster because we no longer
uselessly sleep.
And more reliable because we now actually wait for replies to our
requests.
Bump base
https://github.com/koreader/koreader-base/pull/1424
successful authentication.
Fully tearing down Wi-Fi was a bit optimistic, as the AP list can
technically still be up, so the user might want to try again and/or
connect to another AP.
Fix#5912, regression since #4616.
The reasoning behind #4616 doesn't really apply anymore anyway, as the
Wi-Fi prompt now handles this inconsistent state properly.
The whole codepath should be *extremely* rare anyway (and/or require
super-broken network conditions).
* Hold "Wi-Fi connection" to show network connection options.
* Honor backend connections (e.g., if wpa_supplicant found a matching AP in its own config first).
* When user clicks "Wi-Fi connection" in menu, only prompt if state is ambiguous.
* LuaSettings/DocSettings: Updated readSetting API to allow proper initialization to default.
Use it to initialize tables, e.g., fixing corner-cases in readerFooter that could prevent settings from being saved.
(Fixes an issue reported on Gitter).
* LuaSettings/DocSettings: Add simpler API than the the flip* ones to toggle boolean settings.
* Update LuaSettings/DocSettigns usage throughout the codebase to use the dedicated boolean methods wher appropriate, and clean up some of the more mind-bending uses.
* FileChooser: Implement an extended default exclusion list (fix#2360)
* ScreenSaver: Refactor to avoid the pile of kludges this was threatening to become. Code should be easier to follow and use, and fallbacks now behave as expected (fix#4418).
If we're in range of multiple known WiFi Access Points (including multiple instances of the same SSID), we don't need to keep trying to connect after the first successful connection.
Minimal change would have been replacing the return inside the foreach function with return [a non-nil value].
But foreach is deprecated, and since I was touching the code anyhow, I figured I'd do that tiny update as well.
If you're *extremely* unlucky, the scheduled check might run *right*
after a wakeup, before the network interface is up. So, you have a
previously stored packet count, but the new one is nil. Boom.
* Revamped most actions that require an internet connection to a new/fixed backend that allows forwarding the initial action and running it automatically once connected. (i.e., it'll allow you to set "Action when Wi-Fi is off" to "turn_on", and whatch stuff connect and do what you wanted automatically without having to re-click anywhere instead of showing you a Wi-Fi prompt and then not doing anything without any other feedback).
* Speaking of, fixed the "turn_on" beforeWifi action to, well, actually work. It's no longer marked as experimental.
* Consistently use "Wi-Fi" everywhere.
* On Kobo/Cervantes/Sony, implemented a "Kill Wi-Fi connection when inactive" system that will automatically disconnect from Wi-Fi after sustained *network* inactivity (i.e., you can keep reading, it'll eventually turn off on its own). This should be smart and flexible enough not to murder Wi-Fi while you need it, while still not keeping it uselessly on and murdering your battery.
(i.e., enable that + turn Wi-Fi on when off and enjoy never having to bother about Wi-Fi ever again).
* Made sending `NetworkConnected` / `NetworkDisconnected` events consistent (they were only being sent... sometimes, which made relying on 'em somewhat problematic).
* restoreWifiAsync is now only run when really needed (i.e., we no longer stomp on an existing working connection just for the hell of it).
* We no longer attempt to kill a bogus non-existent Wi-Fi connection when going to suspend, we only do it when it's actually needed.
* Every method of enabling Wi-Fi will now properly tear down Wi-Fi on failure, instead of leaving it in an undefined state.
* Fixed an issue in the fancy crash screen on Kobo/reMarkable that could sometime lead to the log excerpt being missing.
* Worked-around a number of sneaky issues related to low-level Wi-Fi/DHCP/DNS handling on Kobo (see the lengthy comments [below](https://github.com/koreader/koreader/pull/6424#issuecomment-663881059) for details). Fix#6421
Incidentally, this should also fix the inconsistencies experienced re: Wi-Fi behavior in Nickel when toggling between KOReader and Nickel (use NM/KFMon, and run a current FW for best results).
* For developers, this involves various cleanups around NetworkMgr and NetworkListener. Documentation is in-line, above the concerned functions.
bidi.lua:
- Revert "Alias everything to Bidi.nowrap() when in LTR UI,
as using LTR isolates seems uneeded when already LTR" (part
of a628714f) which was a wrong assumption: we need proper
wrappers for all things paths. Enhance some of these wrappers.
- Fix GetText RTL wrapping which was losing empty lines and
trailing \n.
- Wrap all paths, directories, filenames in the code with
these wrappers.
- Wrap all book metadata (title, authors...) with BD.auto(),
as it helps fixing some edge cases (like open/close quotation
marks which are not considered as bracket types by FriBiDi).
(Needed some minor logic changes in CoverBrowser.)
- Tweak hyphenation menu text
- Update forgotten SortWidget for UI mirroring
- KoptConfig: update "justification" index for RTL re-ordering,
following the recent addition of the page_gap_height option.
Thanks to <https://github.com/koreader/koreader/pull/5237> we can now extract the knowledge currently embedded in Transifex and put it directly in our source. This positively affects <https://github.com/koreader/koreader/issues/3754>.
Translation instructions and knowledge that comes out of localization-related questions should be preserved in the source, because Transifex is too ephemeral. For example, the links from <https://github.com/koreader/koreader/pull/2290> are no longer accessible. Even when they are, it's quite useful to have this information around while dealing with the code as well, and I also hope it'll be informative to contributors who seldom visit Transifex.
This commit also makes a few minor changes to obviate the need for comments where possible.
