When we enable/disable exit mode on this restarts the unbound DNS
responder with the DNS trampoline (or restores upstream, when disabling)
to properly route DNS requests through the tunnel (because libunbound's
direct requests don't get tunneled because unbound is inside the network
extension).
This runs a DNS listener on localhost:1053 that bounces requests to the
upstream DNS through the tunnel. The idea here is that, when we turn on
exit mode, we start libunbound bouncing the requests through the
trampoline (since if it makes direct requests they won't go through the
tunnel).
(The actual libunbound configuration is still to follow).
Thus when a user goes looking for it they'll find the (commented out)
default in the right place and can edit it.
(That right place is: ~/Library/Containers/com.loki-project.lokinet.network-extension/Data/lokinet.ini)
Don't squash this commit so that the swift version stays around in
history in case we need to resurrect it again some day (i.e. when Apple
decides to kill off Objective-C support).
- Add a C callback interface (context_wrapper.h) between lokinet and the
objective-C code so that:
- we can use objective-C (rather than objective-C++), which seems more
likely to be supported by Apple into the future;
- we minimize the amount of code that needs to be aware of the Apple
APIs.
- this replaces apple logger objective c++ implementation with a plain
c++ implementation that takes a very simple C callback (provided
from the obj-c code) to actually make the call to NSLog.
- Add various documentation to the code of what is going on.
- Send all DNS traffic to the primary IP on the tun interface. The
match prefixes simply don't work as advertised, and have weird shit
(like even if you get it working for some domains, "instagram.com"
still doesn't because of god-knows-what Apple internal politics).
- Drop the dns proxy code as we don't need it anymore.
- Don't use 9.9.9.9 for default DNS. (We might consider the unfiltered
9.9.9.10 as an alternative default, but if we do it should be a global
lokinet change rather than a Mac-specific change).
- Parse a lokinet.ini in the data directory, if it exists. (Since we
are sandboxed, it is an app-specific "home" directory so is probably
buried god knows where, but at least the GUI ought to be able to get
it to let users add things to it).
- This commit also adds a swift version of the PacketTunnelProvider
glue, which ought to work in theory, but the *tooling* for cmake is so
underdeveloped that I couldn't find any way to actually get the damn
thing working. So I'm committing it here anyway (and will revert it
away in the next commit) in case we someday want to switch to it.
-
- Added contrib/macos/README.txt with description of the cancer
happening here.
- Add provisioningprofiles that Apple wants to make things work properly
- Made the entitlements files match the provisioningprofiles
- Remove configured entitlements files; we *can't* change any of the
things here because they are closedly tied to the provisioningprofiles
-- which means if someone wants to build their own Lokinet, they have
to replace a bunch of crap and change application IDs throughout.
This is the hostile-to-open-source Apple way.
- Remove unused old lokinet binary, as we're no longer using it on macos
- Use a POST_BUILD rather than install to copy things around into the
right places
- Convert all the configure_file's to consistently use @ONLY
- Misc cleanups
PR #1725 reversed argument orders but UnboundResolver was still using
(from,to) ordering in its callbacks, which leaked through to make a
wrong order in our reply function (which simply forwards arguments).
This fixes that bug by making UnboundResolver callback argument order
consistent (i.e. using to, from) with the PacketHandler argument order.
The reason the dns fix on android didn't work is that the DnsInterceptor
had a reversed to/from argument order for its
`SendServerMessageBufferTo` overload, and so android/mac needed the
to/from to be reversed so that the second reverse cancelled out the
first one.
Upon review, the DnsInterceptor order (to, from) is more intuitive than
the base order (from, to), so this reapplies the dns fix and swaps
everything *except* DnsInterceptor to match the (to, from) argument
order.
This reverts commit dace0224ec.
This reportedly didn't fix things on Android, and most definitely breaks
macOS (with this we get a bunch of errors about expecting inbound when
we have outbound).
- Reduce buffer size to INET6_ADDRSTRLEN, and use a single buf rather
than two identical ones in each branch.
- Don't pre-reserve because doing so is usually going to over-allocate,
but also because it prevents SSO, especially for the IPv4 case which
should fit in SSO for all IPv4 addresses.
The default upstream DNS was being set to 1.1.1.1:0, which doesn't work.
This fixes it to also set the port so that default upstream resolution
(i.e. with an empty config) works again.
This function had a bug in stable (fixed in dev) when `last` returns
npos, but the function also appears to basically be duplicating what the
next split version can do, so this just removes it and uses the single
more generic split(strview, strview) method.
* make it so that we don't set up unbound resolver when we have no resolvers provided by config
* clean up dns codepath and make it use llarp::SockAddr instead of llarp::IpAddress
* add option to persist address mappings between restarts using [network]:persist-addrmap-file
* make it work
* only persist address map for inbound convos
* turn persisting address map on by default
* dont load addrmap file if it has been modified last over a minute ago to prevent foot cannons fired from loading a really old version of it
* when a path build times out, shitlist every router in the path except the first hop, this way eventually we get the nodedb pruned to only the routers that are currently actually alive, any ones we nuke that we need later we can always do lookups for.
* introset path haodver tweaks
* improve warn/error messages to convey more information
* dont block on queue insertion
* reset convotag on decrypt/verify fail
* add multiple ready hooks on outbound context
* lookup introsets from close routers on dht
* continue to tick dead sessions so they expire their paths
* introset spacing
* reduce lns lookup diversity requirement for speed
* add a function to send reset convotag message
* only have 1 outbound context at a time
* include stricter router profiling checks in path::Builder hop slection algorithm
* make intro selection function nicer by returning a std::optional instead of a bool with an "out" variable
* do FEC for latency tests so if we fail one test it doesn't kill the entire path
* ignore FEC'd responses on latency tests
* track latency history and report the mean latency instead of just the last sample
* add remove_node_from_failing to remove a node by pubkey from the failing set
* if a router is deregistered we remove it from the failing set so we don't retest it
* remove a router from the failing set if we get a test success
currently creating an outbound session will cancel if we have any session
at all with the relay. instead, only cancel if we have an outbound session
to that relay. this is useful for reachability testing.
It's there for polling, which we aren't doing anymore; we just got the
hash from oxend's push notification, so if it pass it in then we will
always get an "unchanged" result because we're telling oxend that we
already have the data for that hash updated.
This just drops the hash completely because we don't need it anymore.
* immediately poke routes when we are told to use an exit so that packets get pushed which makes an exit path happen
* fix up cmake oddity in nsis section
* not gossip our rc
* not explore the network to prevent outbound session attempts
* not establish sessions to other service nodes
* close all open sessions we have to tell clients we don't want them
* catch exceptions flushing peerdb in disk thread
* don't connect out to non allowed routers
* simplify logic in RCLookupHandler::RemoteIsAllowed()
* add HaveReceivedWhitelist to I_RCLookupHandler base type
* add LooksDeregistered to Router type that tells us if we think we are deregistered
* don't allow building paths over us if we are deregistered
so it doesn't time out and get into a state that's totally screwed.
add virtual function service::Endpont::DefaultPathAlignmentTimeout() to get the timeout for path alignment
and use it for resetablishing outbound sessions
* increase publish introset timeout so that it does not time out on the network
* remove pedantic log warn
* make sure the path we are using for replying on inbound sessions is alive
* include convotag in log message so we know wtf is going on
* appease tom's autism, improve log message text
adds [network] section parameter called path-alignment-timeout that allows configring the timeout
for optional name lookup + introset lookup + aligned path build, used by tun endpoint dns, provided
as milliseconds.
Fixes a subtle memory leak that was a result of outbound messages which
were in the shared queue (not yet sorted into a per-path queue) when a
path was removed, resulting in a ghost path queue (and thus round-robin
order entry as well).
Adds much needed documentation to the outbound message handler class.
Wires up systemd support to configure DNS on startup and when
enabling/disabling exit mode.
On startup (and when turning off an exit) we tell systemd-resolved to
direct .loki and .snode lookups to lokinet (leaving other DNS traffic
alone).
On exit enabling, we reconfigure it to resolve "." (i.e. the root DNS
domain) so that all lookups come into it.
* add srv records in RCs if we have any
* add mechanism to add SRV records for plainquic exposed ports
* resign and republish rc or introset on srv record changes
* llarp::service::NameIsValid was not checking that the tld was .loki, add this check.
* make link layer initial connection timeout 5s not the session activity timeout which happens to be 60 god damn seconds.
* add lokinet_add_bootstrap_rc function for adding an rc from memory
* prevent stack overflow on error closing connection in quic
* add in memory nodedb
* refactor how convotags are set as active
* add initial stubs for endpoint statistics
* refactor time stuff to be a bit cleaner
* update lnproxy script with more arguments
Replace stream_reset (which typically isn't called) with a stream_close
handler (which is already called whether or not it was a reset). Most
importantly, the server side needs to extend the max bidi streams
counter during stream_close (otherwise we run out when we hit the
limit and new connections just stall).
Stream forward on the client-side TCP connection gets set up within in
initial_client_data_handler (which also handles reading the initial
stream version byte).
Also fix a potential transmission delay because `again()` wasn't being
called when the expiry is already passed (i.e. meaning we should run
immediately).
Make stream closing with expiring connections work better. Fixes an
issue where the stream's uv_async could outlive the stream and/or
connection and segfault.
In the standalone plainquic code we triggered a retransmit when the
socket became writeable again, but that doesn't work here, so just
schedule it right away to let ngtcp2 worry about retrying.
For now we still steal buffers from uvw.
In the future I'd like to change that, but it's still uvw
work-in-progress to support custom data allocators, and so for now we
still steal data buffers from uvw.
ngtcp2 was rejecting them because we have the port when constructing,
but then it was 0 on the return packet (which ngtcp2 drops because it's
coming from an unknown/invalid path).
* wire up last of the quic stuff
* clean up udp packet generation code
* pass EndpointBase not quic tunnel for quic stuff
* add {n,h}uint16_t::FromString
* add nuint_t::FromString
* make AlignedBuffer::IsZero non constant time call for speed
Refactors how quic packets get handled: the actual tunnels now live in
tunnel.hpp's TunnelManager which holds and manages all the quic<->tcp
tunnelling. service::Endpoint now holds a TunnelManager rather than a
quic::Server. We only need one quic server, but we need a separate quic
client instance per outgoing quic tunnel, and TunnelManager handles all
that glue now.
Adds QUIC packet handling to get to the right tunnel code. This
required multiplexing incoming quic packets, as follows:
Adds a very small quic tunnel packet header of 4 bytes:
[1, SPORT, ECN] for client->server packets, where SPORT is our
source "port" (really: just a uint16_t unique quic instance
identifier)
or
[2, DPORT, ECN] for server->client packets where the DPORT is the SPORT
from above.
(This also reworks ECN bits to get properly carried over lokinet.)
We don't need a destination/source port for the server-side because
there is only ever one quic server (and we know we're going to it when
the first byte of the header is 1).
Removes the config option for quic exposing ports; a full lokinet will
simply accept anything incoming on quic and tunnel it to the requested
port on the the local endpoint IP (this handler will come in a following
commit).
Replace ConvoTags with full addresses: we need to carry the port, as
well, which the ConvoTag can't give us, so change those to more general
SockAddrs from which we can extract both the ConvoTag *and* the port.
Add a pending connection queue along with new quic-side handlers to call
when a stream becomes available (TunnelManager uses this to wire up
pending incoming conns with quic streams as streams open up).
Completely get rid of tunnel_server/tunnel_client.cpp code; it is now
moved to tunnel.hpp.
Add listen()/forget() methods in TunnelManager for setting up quic
listening sockets (for liblokinet usage).
Add open()/close() methods in TunnelManager for spinning up new quic
clients for outgoing quic connections.
They were failing to compile if output because implicit integer
promotion doesn't work for std::byte's (but rather needs an explicit
std::to_integer call).
- LogTrace() (and LogTraceTag, etc.) are now no-ops for release builds.
(hoping there are no side effects in trace logging!)
- renamed llarp::_Log to llarp::_log because _Log is a reserved keyword
- change logging code to implicitly convert 1-byte types (char, unsigned
char, uint8_t) to ints so that we print them as numeric values rather
than raw chars because, more often than not, printing a single char is
trying to log an 8-bit value.
- Move uvw target to external instead of llarp/CMakeLists.txt
- Add sqlite_orm headers via interface library rather than shoving it
into the global include search path.
- Find sqlite3 as part of the sqlite_orm target deps rather than linking
it separately. (Also don't link it into lokinet-util, which doesn't
appear to actually need it).
* add ip6-range option to network section to control which range we want to use for ipv6 traffic
* make ip6-range able to disable ipv6 exits ( i dont like this but eh )
- Make SockAddr endian arguments explicit
- Consolidate port-less contructors and port constructors into one with
a default port of 0.
- Add {h,n}uint128_t ctors for construction from IPv6 addrs
- De-lambda a lambda that is only called once, immediately after it is
defined.
- C++17 modernize key-value for loops
- Remove unused (and useless) method
All #ifndef guards on headers have been removed, I think,
in favor of #pragma once
Headers are now included as `#include "filename"` if the included file
resides in the same directory as the file including it, or any
subdirectory therein. Otherwise they are included as
`#include <project/top/dir/relative/path/filename>`
The above does not include system/os headers.
ip packet on read error was returning an uninitialized ip packet with non zero size so it was thinking it was a valid packet.
value initialize it to set the packet size to zero by default.
this fixed it in an emulator at least...
call(), when invoked from the logic thread, wasn't triggering a wakeup
which stalled some traffic (such as client-to-snode packets).
Fix it by triggering a wakeup on `call()`, and expose it because this is
a useful thing to be able to do.
The `const sockaddr*` conversion was returning a pointer to the
sockaddr_in6, but with its family set to AF_INET, so the receiving code
would have treated interpreting the result as a sockaddr_in pointer --
but it isn't, and trying to get the IPv4 from the mis-reinterpreted
sockaddr_in6 fails; this then prevented uvw UDP binding (which uses the
sockaddr* operator) from working.
This changes it to always leave the family value at the proper value and
using the ipv6_is_mapped_ipv4 check to determine when we have (and
should return) a pointer to the sockaddr_in rather than the
sockaddr_in6.
Also fixes toString() to compare the full prefix instead of just bytes
10 and 11.
loop->call(...) is similar to the old logic->Call(...), but is smart
about the current thread: if called from within the event loop it simply
runs the argument directly, otherwise it queues it.
Similarly most of the other event loop calls are also now thread-aware:
for example, `call_later(...)` can queue the job directly when called if
in the event loop rather than having to double-queue through the even
loop (once to call, then inside the call to initiate the time).
- removes all the llarp_ev_* functions, replacing with methods/classes/functions in the llarp
namespace.
- banish ev/ev.h to the void
- Passes various things by const lvalue ref, especially shared_ptr's that don't need to be copied
(to avoid an atomic refcount increment/decrement).
- Add a llarp::UDPHandle abstract class for UDP handling
- Removes the UDP tick handler; code that needs tick can just do a separate handler on the event
loop outside the UDP socket.
- Adds an "OwnedBuffer" which owns its own memory but is implicitly convertible to a llarp_buffer_t.
This is mostly needed to take over ownership of buffers from uvw without copying them as,
currently, uvw does its own allocation (pending some open upstream issues/PRs).
- Logic:
- add `make_caller`/`call_forever`/`call_every` utility functions to abstract Call wrapping and
dependent timed tasks.
- Add inLogicThread() so that code can tell its inside the logic thread (typically for
debugging assertions).
- get rid of janky integer returns and dealing with cancellations on call_later: the other methods
added here and the event loop code remove the need for them.
- Event loop:
- redo everything with uvw instead of libuv
- rename EventLoopWakeup::Wakeup to EventLoopWakeup::Trigger to better reflect what it does.
- add EventLoopRepeater for repeated events, and replace the code that reschedules itself every
time it is called with a repeater.
- Split up `EventLoop::run()` into a non-virtual base method and abstract `run_loop()` methods;
the base method does a couple extra setup/teardown things that don't need to be in the derived class.
- udp_listen is replaced with ev->udp(...) which returns a new UDPHandle object rather that
needing gross C-style-but-not-actually-C-compatible structs.
- Remove unused register_poll_fd_(un)readable
- Use shared_ptr for EventLoopWakeup rather than returning a raw pointer; uvw lets us not have to
worry about having the event loop class maintain ownership of it.
- Add factory EventLoop::create() function to create a default (uvw-based) event loop (previously
this was one of the llarp_ev_blahblah unnamespaced functions).
- ev_libuv: this is mostly rewritten; all of the glue code/structs, in particular, are gone as
they are no longer needed with uvw.
- DNS:
- Rename DnsHandler to DnsInterceptor to better describe what it does (this is the code that
intercepts all DNS to the tun IP range for Android).
- endpoint:
- remove unused "isolated network" code
- remove distinct (but actually always the same) variables for router/endpoint logic objects
- llarp_buffer_t
- make constructors type-safe against being called with points to non-size-1 values
- tun packet reading:
- read all available packets off the device/file descriptor; previously we were reading one packet
at a time then returning to the event loop to poll again.
- ReadNextPacket() now returns a 0-size packet if the read would block (so that we can implement
the previous point).
- ReadNextPacket() now throws on I/O error
- Miscellaneous code cleanups/simplifications
when retransmissions happen, they will be executed in a deterministic order by their delivery id.
this reduces the jitter from delayed message delivery on link layer as with an unordered map the order of iteration is "random",
so when we iterate all messages for retransmission we do it lowest id first (the messages queued first).
- Previous android java and jni code updated to work, but with much love
still needed to make it work nicely, e.g. handling when the VPN is
turned off.
- DNS handling refactored to allow android to intercept and handle DNS
requests as we can't set the system DNS to use a high port
(and apparently Chrome ignores system DNS settings anyway)
- add packet router structure to allow separate handling of specific
intercepted traffic, e.g. UDP traffic to port 53 gets handled by our
DNS handler rather than being naively forwarded as exit traffic.
- For now, android lokinet is exit-only and hard-coded to use exit.loki
as its exit. The exit will be configurable before release, but
allowing to not use exit-only mode is more of a challenge.
- some old gitignore remnants which were matching to things we don't
want them to (and are no longer relevant) removed
- some minor changes to CI configuration
