Prevent crash:
```
./luajit: frontend/util.lua:1092: attempt to get length of local 'str' (a nil value)
stack traceback:
frontend/util.lua:1092: in function 'fixUtf8'
frontend/ui/network/manager.lua:1109: in function 'requestToTurnOnWifi'
[…]
```
Namely, in the face of wpa_supplicant doing multiple scans. Note that a backend PR will make this largely redundant, by actually fixing the core issue in our backend ;).
Also add more logging around the authentication status.
* Kindle: Implement a NetworkMgr backend loosely based on WpaClient in order to allow feature-parity with hasWifiManager platforms. This involves dealing with the native wifid over lipc (the native IPC system, based on DBus), through custom Lua bindings (https://github.com/notmarek/openlipclua), since the stock ones lack support for the needed hasharray data type.
* NetworkMgr: Clear up leftover hallucinations from #10669, making `enableWifi` much simpler (and much more similar to `turnOnWifiAndWaitForConnection`).
* NetworkMgr: Made it clearer that `turnOnWifi` implementations *must* deal with `complete_callback`, as part of the aforementioned changes mean that it's *always* wrapped in a connectivity check, and we need that for proper event signaling.
* Android, Emu: Run `complete_callback` properly in `turnOnWifi`.
* Kindle: Support `powerd:isCharged()` on the PW2 (yes, this is random, it just happened to be my test device :D).
* NetworkMgr:disableWifi: Properly tear down any potential ongoing connection attempt (e.g., connectivity check).
* NetworkMgr:promptWifi: Make the "wifi enabled but not connected" popup clearer if there's an ongoing connection attempt, and gray out the "Connect" button in this case (as it would only lead to another "connection already in progress" popup anyway).
* NetworkMgr:reconnectOrShowNetworkMenu: Make *total* scanning failures fatal (they will lead to an immediate wifi teardown).
* NetworkMgr:reconnectOrShowNetworkMenu: Clear up the long-press behavior (which *always* shows the network list popup) so that it doesn't weirdly break all the things (technical term!).
* NetworkMgr:reconnectOrShowNetworkMenu: When we manage to connect to a preferred network on our own *before* showing the network list, make sure it is flagged as "connected" in said list.
* NetworkMgr:reconnectOrShowNetworkMenu: Make connection failures fatal in non-interactive workflows (they'll lead to a wifi teardown).
* NetworkSetting (the aforementioned network list widget): Clear NetworkMgr's "connection pending" flag on dismiss when nothing else will (i.e., when there's no connectivity check ticking).
Device:getDefaultRoute parses /proc/net/route and converts the hex
addresses to textual IP addresses, but in `isOnline` we don't care what
address the gateway actually has, we only care about whether we have a
default route into the Internet.
This provides a simpler alternative that does the equivalent of
"ip route get 203.0.113.1 || ip route get 2001:db8::1" (note that it
does support IPv6-only connectivity as opposed to
Device:getDefaultRoute) and returns true if we have a route.
Inspired by https://github.com/pavel-odintsov/get_default_outgoing_ip_linux
Doing the `isOnline` check (`socket.dns.toip("dns.msftncsi.com")`)
without having internet connectivity (`!isConnected`) results in the
`isOnline` check never succeeding again even if connectivity is later
acquired. This is most likely caused by /etc/resolv.conf only being
parsed once - https://sourceware.org/bugzilla/show_bug.cgi?id=984, an
issue that was fixed in glibc 2.26 (PocketBook firmware U740.6.8.2461
has glibc 2.23).
This fix works around the problem by checking if we have a default route
first before even attempting to check `isOnline`. If we don't, then
`isOnline` is (almost) guaranteed to fail anyway.
We could alternatively check `isConnected` instead, but that only checks
wireless connectivity on many platforms, and we could have internet
access via USBNet instead. Checking for the default route via any
interface should work reliably for both wireless and USBNet
connectivity.
Another alternative fix is to add a fallback nameserver to
/etc/resolv.conf like we do for the Kobo platform [1]. Unfortunately,
this fix would not work in the following (rather common) scenario:
1. PocketBook boots, connects to WiFi
2. KOReader starts, /etc/resolv.conf looks all right, no fallback needed
3. PocketBook goes to sleep, disconnects from WiFi, clears resolv.conf
4. PocketBook wakes up, stays disconnected
5. KOReader user does a Wikipedia lookup, networking freezes
[1]: https://github.com/koreader/koreader/pull/6424/files#diff-be863601c59a2d6607af6b04b3be2392ec4494df6d25dae48250fae57b737f61R216-R224
Fixes: https://github.com/koreader/koreader/issues/10183
Related: https://github.com/koreader/koreader/issues/6421
* Bump base
includes:
koreader/koreader-base#1691koreader/koreader-base#1692koreader/koreader-base#1689koreader/koreader-base#1690koreader/koreader-base#1693
* Integrate decoding of SSIDs within wpa_supplicant
The UTF-8 decoding of SSIDs is specific to wpa_supplicant. In this
patch, we move all of this decoding logic to the wpa_supplicant module.
We expose the raw bytes of the SSID to the NetworkMgr code, and make
sure to always fix bad UTF-8 before we display the SSID to the user.
Within the wpa_supplicant module, we replace the call to the
wpa_passphrase binary to get the PSK with a direct function call to
OpenSSL. This allows us to calculate the PSK over any arbitrary bytes,
including UTF-8. In the same vein, we use the hex-encoded SSID to
communicate with wpa_supplicant when setting up the network to support
arbitrary bytes in the SSID.
Unfortunately, we also remove the tests, as there is no way to unit test
local functions.
* Device: Add a `hasSeamlessWifiToggle` devcap to complement `hasWifiToggle`, to denote platforms where we can toggle WiFi without losing focus, as this has obvious UX impacts, and less obvious technical impacts on some of the NetworkMgr innards...
* Android: Mark as `!hasSeamlessWifiToggle`, as it requires losing focus to the system settings. Moreover, `turnOnWifi` returns *immediately* and we *still* run in the background during that time, for extra spiciness...
* NetworkMgr: Ensure only *one* call to `turnOnWifi` will actually go on when stuff gets re-scheduled by the `beforeWifiAction` framework.
* NetworkMgr: Ensure the `beforeWifiAction` framework will not re-schedule the same thing *ad vitam aeternam* if a previous connection attempt is still ongoing. (i.e., previously, on Android, if you backed out of the system settings, you entered the Benny Hill dimension, as NetworkMgr would keep throwing you back into the system settings ;p). This has a few implications on callbacks requested by subsequent connection attempts, though. Generally, we'll try to honor *explicitly interactive* callbacks, but `beforeWifiAction` stuff will be dropped (only the original cb is preserved). That's what prevents the aforementioned infinite loop, as the `beforeWifiAction` framework was based on the assumption that `turnOnWifi` somewhat guaranteed `isConnected` to be true on return, something which is only actually true on `hasWifiManager` platforms.
* NetworkMgr: In `prompt` mode, the above implies that the prompt will not even be shown for concurrent attempts, as it's otherwise extremely confusing (KOSync on Android being a prime example, as it has a pair of Suspend/Resume handlers, so the initial attempt trips those two because of the focus switch >_<").
* NetworkMgr: Don't attempt to kill wifi when aborting a connection attempt on `!hasSeamlessWifiToggle` (because, again, it'll break UX, and also because it might run at very awkward times (e.g., I managed to go back to KOReader *between* a FM/Reader switch at one point, which promptly caused `UIManager` to exit because there was nothing to show ;p).
* NetworkMgr: Don't drop the connectivity callback when `beforeWifiAction` is set to prompt and the target happens to use a connectivity check in its `turnOnWifi` implementation (e.g., on Kindle).
* Android: Add an `"ignore"` `beforeWifiAction` mode, that'll do nothing but schedule the connectivity check with its callback (with the intent being the system will eventually enable wifi on its own Soon(TM)). If you're already online, the callback will run immediately, obviously. If you followed the early discussions on this PR, this closely matches what happens on `!hasWifiToggle` platforms (as flagging Android that way was one of the possible approaches here).
* NetworkMgr: Bail out early in `goOnlineToRun` if `beforeWifiAction` isn't `"turn_on"`. Prompt cannot work there, and while ignore technically could, it would serve very little purpose given its intended use case.
* KOSync: Neuter the Resume/Suspend handlers early on `CloseDocument`, as this is how focus switches are handled on Android, and if `beforeWifiAction` is `turn_on` and you were offline at the time, we'd trip them because of the swap to system settings to enable wifi.
* KOSync: Allow `auto_sync` to be enabled regardless of the `beforeWifiAction` mode on `!hasSeamlessWifiToggle` platforms. Prompt is still a terrible idea, but given that `goOnlineToRun` now aborts early if the mode is not supported, it's less of a problem.
wpa_supplicant returns all non-ASCII SSIDs as raw bytes in the form
\x0a. We interpret these bytes as UTF-8, and make sure that all invalid
characters are replaced with a �.
Assume the host system does things right otherwise.
Should be sane on Kindle
On PocketBook, who knows, but assuming the device actually suspends,
that should effectively kill our keepalive
Irrelevant for Android, as we skipped it because the wifi toggling
methods are interactive.
Fix#10823
(For reference, we *enable* wifi_was_on no matter *how* wifi is enabled,
but we only toggle it off when it's killed by a *direct* user interaction,
the intent being that if *something* non-interactive enabled wifi,
you'll probably silently need it on resume too).
As early as turnOnWifi.
Implement it on hasWifiManager platforms, preventing useless
connectivity checks to run when they're obviously never going to work
because you're out of range of your AP.
Also implemented a flag to notify the backend if the connection attempt
was interactive or not.
Right now, interactive is extremely restricted, it basically means the
menu checkmark, or a gesture.
The intent being that for stuff like the beforeWifiAction framework, we
don't want the backend to spawn extra UI.
Specifically, for hasWifiManager platforms, we no longer spawn the AP
scan list on failure unless the caller was interactive.
TL;DR: beforeWifiAction is now much less obnoxious when you're obviously
not able to connect.
* Enable before_wifi_action & after_wifi_action on hasWifiToggle platforms (which is basically all of 'em except naked SDL).
* Decouple restoreWifiAsync from hasWifiManger, because we can do that on other platforms (namely, Kindle. Probably PB, too, but WiFi is already a mess there, and I can't test it).
* Implement restoreWifiAsync on Kindle.
* Properly flag rM as hasWifiManager & hasFastWifiStatusQuery, because it is actually both of those (it uses our wpa_supplicant backend).
* Update the KOSync checks to take these changes into account, to properly disable auto_sync if necessary.
* Really made the Network* event signaling consistent. For realz this time.
* In an effort to make the whole beforeWifiAction framework somewhat usable there, we now assume connectivity is always available on !hasWifiToggle platforms...
Fix: #10539, and for context #6489, #6733, #6534
Reorganize and reword most of the settings to make it clear what actually ties into auto sync, and what doesn't. (Specifically, what happens when a pull attempts to sync forward or backward has nothing to do with auto sync, it applies in all cases; while the periodic sync *does* require auto sync).
The main point of contention, though, is that auto sync will now *always* attempt to setup network connectivity (i.e., on resume/suspend/close). Periodic sync will *not* though (the intent being that, if you use periodic sync, you're relying on the activity check to actually keep wifi on at all times)).
Since this may lead to a large amount of nagging about wifi toggles on devices w/ NetworkManager support, it is now *disabled* by default on those devices. (And given that it wouldn't have worked because of the lack of connectivity, that doesn't really make any practical difference ;p).
Additionally, given the fact that there's no way to make this behavior viable if the "before wifi" action is left at its default of "prompt", this feature now *requires* that to be set to "turn_on" (on devices where it can, of course); attempting to toggle it on will warn about that if necessary.
This change is retroactive (OTM).
Includes an assortment of fixes and cleanups, including migrating to the new LuaSettings API, which is why there's no longer a smattering of superfluous flushes.
Much easier to deal with thanks to the cleanup work done in #10062 ;).
* `carrier` is set to 1 as soon as the device is *administratively* up (in practice, as soon as we run `ifconfig up`). This is perfectly fine for `isWifiOn`, but absolutely not for `isConnected`, because we are not, actually, connected to *anything*, no attempt at associating has even been made at that point. Besides being semantically wrong, in practice, this will horribly break the connectivity check, because it expects that `isConnected` means we can talk to at least the LAN.
* Delving into the Linux docs reveals that `operstate` looks like a better candidate, as it reflects *operational status*; for Wi-Fi, that means associated and successfully authenticated. That's... closer, but still not it, because we still don't have an IP, so we technically can't talk to anything other than the AP.
* So, I've brought out the big guns (`getifaddrs`), and replicated a bit of code that I already use in the USBNetwork hack on Kindle, to detect whether we actually have an IP assigned. (Other approaches, like `/proc/net/route`, may not be entirely fool-proof, and/or get complicated when IPv6 enters the fray (which it does, on Kobo, Mk. 8+ devices are IPv6-enabled)).
TL;DR: Bunch of C via ffi, and `isConnected` now returns true only when the device is operationally up *and* we have an IP assigned.
Pulls in https://github.com/koreader/koreader-base/pull/1579 & https://github.com/koreader/lj-wpaclient/pull/10
cervantes kindle kobo remarkable: use sysfs carrier file to determine connection state
cleanup hasWifiManager checks
gateway check: use ip if available
Fixes: #10087Closes: #10092
isWifiOn for kindle currently returns if the interface is connected, change this to doing what is says isWifiOn the file is only present if the wireless interface is up.
isConnected pings the gateway, rely on the kernel for a more reliable check.
Whan connecting to my android phone's wifi hotspot to remote debug from my phone, the network is connected yet the phone(gateway) does not respond to pings leading koreader to shut down the connection thinking it is unsuccessful
Basically:
* Use `extend` for class definitions
* Use `new` for object instantiations
That includes some minor code cleanups along the way:
* Updated `Widget`'s docs to make the semantics clearer.
* Removed `should_restrict_JIT` (it's been dead code since https://github.com/koreader/android-luajit-launcher/pull/283)
* Minor refactoring of LuaSettings/LuaData/LuaDefaults/DocSettings to behave (mostly, they are instantiated via `open` instead of `new`) like everything else and handle inheritance properly (i.e., DocSettings is now a proper LuaSettings subclass).
* Default to `WidgetContainer` instead of `InputContainer` for stuff that doesn't actually setup key/gesture events.
* Ditto for explicit `*Listener` only classes, make sure they're based on `EventListener` instead of something uselessly fancier.
* Unless absolutely necessary, do not store references in class objects, ever; only values. Instead, always store references in instances, to avoid both sneaky inheritance issues, and sneaky GC pinning of stale references.
* ReaderUI: Fix one such issue with its `active_widgets` array, with critical implications, as it essentially pinned *all* of ReaderUI's modules, including their reference to the `Document` instance (i.e., that was a big-ass leak).
* Terminal: Make sure the shell is killed on plugin teardown.
* InputText: Fix Home/End/Del physical keys to behave sensibly.
* InputContainer/WidgetContainer: If necessary, compute self.dimen at paintTo time (previously, only InputContainers did, which might have had something to do with random widgets unconcerned about input using it as a baseclass instead of WidgetContainer...).
* OverlapGroup: Compute self.dimen at *init* time, because for some reason it needs to do that, but do it directly in OverlapGroup instead of going through a weird WidgetContainer method that it was the sole user of.
* ReaderCropping: Under no circumstances should a Document instance member (here, self.bbox) risk being `nil`ed!
* Kobo: Minor code cleanups.
* UIManager: Support more specialized update modes for corner-cases:
* A2, which we'll use for the VirtualKeyboards keys (they'd... inadvertently switched to UI with the highlight refactor).
* NO_MERGE variants of ui & partial (for sunxi). Use `[ui]` in ReaderHighlight's popup, because of a Sage kernel bug that could otherwise make it translucent, sometimes completely so (*sigh*).
* UIManager: Assorted code cleanups & simplifications.
* Logger & dbg: Unify logging style, and code cleanups.
* SDL: Unbreak suspend/resume outside of the emulator (fix#9567).
* NetworkMgr: Cache the network status, and allow it to be queried. (Used by AutoSuspend to avoid repeatedly poking the system when computing the standby schedule delay).
* OneTimeMigration: Don't forget about `NETWORK_PROXY` & `STARDICT_DATA_DIR` when migrating `defaults.persistent.lua` (fix#9573)
* WakeupMgr: Workaround an apparent limitation of the RTC found on i.MX5 Kobo devices, where setting a wakealarm further than UINT16_MAX seconds in the future would apparently overflow and wraparound... (fix#8039, many thanks to @yfede for the extensive deep-dive and for actually accurately pinpointing the issue!).
* Kobo: Handle standby transitions at full CPU clock speeds, in order to limit the latency hit.
* UIManager: Properly quit on reboot & exit. This ensures our exit code is preserved, as we exit on our own terms (instead of being killed by the init system). This is important on platforms where exit codes are semantically meaningful (e.g., Kobo).
* UIManager: Speaking of reboot & exit, make sure the Screensaver shows in all circumstances (e.g., autoshutdown, re: #9542)), and that there aren't any extraneous refreshes triggered. (Additionally, fix a minor regression since #9448 about tracking this very transient state on Kobo & Cervantes).
* Kindle: ID the upcoming Scribe.
* Bump base (https://github.com/koreader/koreader-base/pull/1524)
* This removes support for the following deprecated constants: `DTAP_ZONE_FLIPPING`, `DTAP_ZONE_BOOKMARK`, `DCREREADER_CONFIG_DEFAULT_FONT_GAMMA`
* The "Advanced settings" panel now highlights modified values in bold (think about:config in Firefox ;)).
* LuaData: Isolate global table lookup shenanigans, and fix a few issues in unused-in-prod codepaths.
* CodeStyle: Require module locals for Lua/C modules, too.
* ScreenSaver: Actually garbage collect our widget on close (ScreenSaver itself is not an instantiated object).
* DateTimeWidget: Code cleanups to ensure child widgets can be GC'ed.
Rationale from: https://github.com/koreader/koreader/pull/9064#issuecomment-1116485065
>While we are dealing with WiFi, a slight hijack, on my kindle when the WiFi is on but not connected the ToggleWiFi gesture tries to connect so i have no way of turning it off. For eample I use the gesture to turn it on, then leave the range I can't use it to turn off. I think the more direct approach would be to toggle On -> Off states so if i am disconnected and want to reconnect i can always go Off then On, but I am able to turn it off.
>TLDR: the tooglewifi gesture should check for iswififup not isconnected
* Disable all non power management related input during suspend. (This prevents wonky touch events from being tripped when closing a sleep cover on an already-in-suspend device, among other things).
* Kobo: Use our WakeupMgr instance, not the class.
* WakupMgr: split `removeTask` in two:
* `removeTask`, which *only* takes a queue index as input, and only removes a single task. Greatly simplifies the function (i.e., it's just a `table.remove`).
* `removeTasks`, which takes an epoch or a cb ref, and removes *every* task that matches.
* Both of these will also *always* re-schedule the next task (if any) on exit, since we can have multiple WakeupMgr tasks queued, but we can only have a single RTC wake alarm set ;).
* `wakeupAction` now takes a `proximity` argument, which it passes on to its `validateWakeupAlarmByProximity` call, allowing call sites to avoir having to duplicate that call themselves when they want to use a custom proximity window.
* `wakeupAction` now re-schedules the next task (if any) on exit.
* Simplify `Kobo:checkUnexpectedWakeup`, by removing the duplicate `WakerupMgr:validateWakeupAlarmByProximity` call, now that we can pass a proximity window to `WakeuoMgr:wakeupAction`.
* The various network activity timeouts are now halved when autostandby is enabled.
* Autostandby: get rid of the dummy deadline_guard task, as it's no longer necessary since #9009.
* UIManager: The previous change allows us to simplify `getNextTaskTimes` into a simpler `getNextTaskTime` variant, getting rid of a table & a loop.
* ReaderFooter & ReaderHeader: Make sure we only perform a single refresh when exiting standby.
* Kobo: Rewrite sysfs writes to use ANSI C via FFI instead of stdio via Lua, as it obscured some common error cases (e.g., EBUSY on /sys/power/state).
* Kobo: Simplify `suspend`, now that we have sane error handling in sysfs writes.
* Kobo.powerd: Change `isCharging` & `isAuxCharging` behavior to match the behavior of the NTX ioctl (i.e., Charging == Plugged-in). This has the added benefit of making the AutoSuspend checks behave sensibly in the "fully-charged but still plugged in" scenario (because being plugged in is enough to break PM on `!canPowerSaveWhileCharging` devices).
* AutoSuspend: Disable our `AllowStandby` handler when auto standby is disabled, so as to not interfere with other modules using `UIManager:allowStandby` (fix#9038).
* PowerD: Allow platforms to implement `isCharged`, indicating that the battery is full while still plugged in to a power source (battery icon becomes a power plug icon).
* Kobo.powerd: Implement `isCharged`, and kill charging LEDs once battery is full.
* Kindle.powerd: Implement `isCharged` on post-Wario devices. (`isCharging` is still true in that state, as it ought to).
If we had a local prefered AP with a higher RSSI,
we attempted to associate with it over wpa_supplicant
being already attempting to associate with its own preferred AP.
That... failed horribly.
Also adapt to the new lj-wpaclient API, fixing a few other edge-cases,
and making the whole thing slightly faster because we no longer
uselessly sleep.
And more reliable because we now actually wait for replies to our
requests.
Bump base
https://github.com/koreader/koreader-base/pull/1424
successful authentication.
Fully tearing down Wi-Fi was a bit optimistic, as the AP list can
technically still be up, so the user might want to try again and/or
connect to another AP.
Fix#5912, regression since #4616.
The reasoning behind #4616 doesn't really apply anymore anyway, as the
Wi-Fi prompt now handles this inconsistent state properly.
The whole codepath should be *extremely* rare anyway (and/or require
super-broken network conditions).
* Hold "Wi-Fi connection" to show network connection options.
* Honor backend connections (e.g., if wpa_supplicant found a matching AP in its own config first).
* When user clicks "Wi-Fi connection" in menu, only prompt if state is ambiguous.
* LuaSettings/DocSettings: Updated readSetting API to allow proper initialization to default.
Use it to initialize tables, e.g., fixing corner-cases in readerFooter that could prevent settings from being saved.
(Fixes an issue reported on Gitter).
* LuaSettings/DocSettings: Add simpler API than the the flip* ones to toggle boolean settings.
* Update LuaSettings/DocSettigns usage throughout the codebase to use the dedicated boolean methods wher appropriate, and clean up some of the more mind-bending uses.
* FileChooser: Implement an extended default exclusion list (fix#2360)
* ScreenSaver: Refactor to avoid the pile of kludges this was threatening to become. Code should be easier to follow and use, and fallbacks now behave as expected (fix#4418).
If we're in range of multiple known WiFi Access Points (including multiple instances of the same SSID), we don't need to keep trying to connect after the first successful connection.
Minimal change would have been replacing the return inside the foreach function with return [a non-nil value].
But foreach is deprecated, and since I was touching the code anyhow, I figured I'd do that tiny update as well.
If you're *extremely* unlucky, the scheduled check might run *right*
after a wakeup, before the network interface is up. So, you have a
previously stored packet count, but the new one is nil. Boom.
