Commit Graph

4720 Commits (ubuntu/impish)

Author SHA1 Message Date
Jeff Becker b830eeb535
initial lokinet router testing:
* report via rpc to oxen core connection stats on success and failure
* connect to random service node by pubkey every 5 seconds for testing
3 years ago
Jeff Becker f172db75c5
don't defer call to ready hook with timer call it right away because we are ready 3 years ago
Jeff Becker 432e7f2a5a
update introset if we get a discard message from a possibly expired intro 3 years ago
Jeff Becker 970ccfcd4e
send a keep alive to keep an idle session alive so it doesn't get removed 3 years ago
Jeff Becker 7c6bb9a24c
simplify loop and correct logic, we want to bail out of the loop on first match. 3 years ago
Jeff Becker ae97b87111
exits don't rewrite ip addresses, fix this and add note 3 years ago
Jeff Becker f1efcc29ea
mark inbound traffic as convotag active 3 years ago
Jeff bd9db35975
Merge pull request #1643 from majestrate/win32-rpc-fixups-2021-05-13
win32 fixups
3 years ago
Jeff Becker 499bb38e6f
fix route poking via rpc:
* immediately poke routes when we are told to use an exit so that packets get pushed which makes an exit path happen
* fix up cmake oddity in nsis section
3 years ago
Jeff Becker 209bcc39dd
make liblokinet target work on nsis cpack 3 years ago
Jeff Becker 8c0f448e12
add liblokinet option to nsis cpack installer 3 years ago
Jeff Becker 2458b5fd71
rpc server fixups for win32:
* RoutePoker::Enable calls RoutePoker::Up so remove additional call to RoutePoker::Up
* allow specifying null exit via rpc
3 years ago
Jeff Becker 3c2334112c
when we stop a path builder we want to expire all of their paths so they go away 3 years ago
Jeff Becker 51b7566a46
if we look deregistered we will now:
* not gossip our rc
* not explore the network to prevent outbound session attempts
* not establish sessions to other service nodes
* close all open sessions we have to tell clients we don't want them
* catch exceptions flushing peerdb in disk thread
* don't connect out to non allowed routers
* simplify logic in RCLookupHandler::RemoteIsAllowed()
* add HaveReceivedWhitelist to I_RCLookupHandler base type
* add LooksDeregistered to Router type that tells us if we think we are deregistered
* don't allow building paths over us if we are deregistered
3 years ago
Jeff Becker c5350ab2c4
DefaultPathAlignmentTimeout -> PathAlignmentTimeout 3 years ago
Jeff Becker c834414b47
when we have to reestablish an outbound session use a much much higher timeout for restablishment
so it doesn't time out and get into a state that's totally screwed.

add virtual function service::Endpont::DefaultPathAlignmentTimeout() to get the timeout for path alignment
and use it for resetablishing outbound sessions
3 years ago
Jeff Becker 38cc130dc3
shift off bad intros when we get a discard message, this happens if the
other side's intro expires or the pivot restarts for whatever reason.
3 years ago
Jeff Becker 3c22e01d7c
in the event that a session is removed and then gets more traffic we would re-add the session with everything blank if there was more traffic in the same tick.
this remedies this behavior by only increment usage timestamps on sessions if they exist.
3 years ago
Jeff Becker 6bb31468d7
don't send nx if we have an address already mapped 3 years ago
Jeff Becker d3d929efa7
fixups in service endpoint
* increase publish introset timeout so that it does not time out on the network
* remove pedantic log warn
* make sure the path we are using for replying on inbound sessions is alive
* include convotag in log message so we know wtf is going on
* appease tom's autism, improve log message text
3 years ago
Jeff d53945b011
Merge pull request #1621 from majestrate/path-ptr-leak-2021-05-02
try fixing std::shared_ptr leak with paths
3 years ago
Jeff 34eb254959
Merge pull request #1626 from majestrate/edge-limiter-2021-05-05
limit path builds across all builders
3 years ago
Jeff ea4529af92
Merge pull request #1628 from majestrate/unfug-network-path-builds-2021-05-05
always do path tests
3 years ago
Jeff 2bb5a7ccea
Merge pull request #1622 from majestrate/stricter-profiling-2021-05-03
stricter profiling
3 years ago
Jeff Becker b1afe0f596
always do path tests
this reverts some stupid bullshit that broke 0.9.0
3 years ago
Jeff Becker ec62228149
limit path builds across all builders 3 years ago
Jeff Becker 955071ba5c
make profiling far stricter for path builds 3 years ago
Jeff Becker e916c9610c
format 3 years ago
Jeff Becker 554a44c8bf
report block height reported by oxend in systemd status 3 years ago
Jeff Becker f955bec5da
re-add parans 3 years ago
Jeff Becker 0826a557d6
use correct intro 3 years ago
Jeff Becker f108af3d8b
only use alive paths for lookups
only track usage from sessions and don't care about intro timeouts
3 years ago
Jeff Becker 0005bee196
allow more than one outbound context per endpoint for fallback in case one outbound context takes a shit 3 years ago
Jeff Becker 83e70d95e0
dont invalidate cache on lns timeout, only on explicit negative 3 years ago
Jeff Becker d563e3b340
if a path's latency is zero dont use it because it's not actually a zero latency path it's probably about to be failed or timed out
increase default path alignment timeout
3 years ago
Jeff Becker 397d8b01fc
try fixing std::shared_ptr leak with paths 3 years ago
Jeff d514f30eaa
Merge pull request #1620 from majestrate/default-log-level-warn-2021-05-01
make default log level warn
3 years ago
Jeff Becker ed707eecf9
change granularity to seconds 3 years ago
Jeff Becker 2968caf7af
make default log level warn fixes #1593 3 years ago
Jeff Becker 54f9e1b44e
make path alignment timeout configuable
adds [network] section parameter called path-alignment-timeout that allows configring the timeout
for optional name lookup + introset lookup + aligned path build, used by tun endpoint dns, provided
as milliseconds.
3 years ago
Jeff Becker f9fe6f7e3b
consmetic fixes
* print names in lookup jobs instead of yyyyyy
* update syntax in construction of objects
3 years ago
Jeff Becker f2e8b5547d
compat for lokinet 0.8.x
* don't send messages back that aren't expected
3 years ago
Jeff Becker e0185bab09
don't rehash decaying hashsets that is no bueno and probably leaks like the pipes in tom's apartment 3 years ago
Jeff Becker c6320724d3
parameterize traffic alignement timeout in dns lookups 3 years ago
Jeff Becker f69ccb73a8
limit path reanimation
* wait for a limited time for dead paths to reanimate and then remove them after that forever
3 years ago
Jeff Becker b31a484bc9
fix up outbound session
* prevent introset lookup spam
* change return values to void becuase bool doesn't mean fucking shit at all
3 years ago
Jeff Becker 6b115913bc
lokinetmon updates
* add country flags to lokinetmon
* expose hop ip addresses via rpc introspection for geoip in lokinetmon
3 years ago
Thomas Winget cecbddc912 Fixes subtle memory leak, adds comments
Fixes a subtle memory leak that was a result of outbound messages which
were in the shared queue (not yet sorted into a per-path queue) when a
path was removed, resulting in a ghost path queue (and thus round-robin
order entry as well).

Adds much needed documentation to the outbound message handler class.
3 years ago
Jeff ef28de8c9f
Merge pull request #1610 from majestrate/android-fixes-2021-04-26
Android fixes
3 years ago
Jason Rhinelander 7963cd0181 Fix headers & grammar for non-systemd 3 years ago
Jason Rhinelander 1d48cd6d35 format 3 years ago
Jason Rhinelander b9363c8d21 Put systemd ifdef around the headers, too 3 years ago
Jason Rhinelander 4ef25ef679 Add systemd-resolved dynamic DNS updating
Wires up systemd support to configure DNS on startup and when
enabling/disabling exit mode.

On startup (and when turning off an exit) we tell systemd-resolved to
direct .loki and .snode lookups to lokinet (leaving other DNS traffic
alone).

On exit enabling, we reconfigure it to resolve "." (i.e. the root DNS
domain) so that all lookups come into it.
3 years ago
Jason Rhinelander 35e4e8817b Add missing getIPv6 implementation 3 years ago
Jeff Becker 8a74b55af3
limit calls to service node list updates for when we are synching the chain we dont spam with list updates 3 years ago
Jeff Becker d750f68328
prepare for ipv6 on android
* remove 21/8 from ipv4 bogon ranges as it is being sold by DoD
* start adding ipv6 bogon ranges
3 years ago
Jeff Becker 3e7137ad96
fix regression: llarp::Context::CallSafe was not deferring call on startup 3 years ago
Jeff Becker 6d12a7a712
fixes for android jni 3 years ago
Jason Rhinelander 86dcdde8d3 Add missing <optional> header
Reported by TechnicalTumbleweed as needed to fix a build.
3 years ago
Jason Rhinelander b2b0e904d7 😡 3 years ago
Jason Rhinelander 41e5cf09ed ngtcp2 compat update 3 years ago
Jeff 4273fb0208
Merge pull request #1602 from majestrate/rpc-block-notify-fix-2021-04-20
add_command instead of add_request_command
3 years ago
Jeff Becker 7f93b95080
add_command instead of add_request_command 3 years ago
Jeff Becker a3d7590fc8
correct header 3 years ago
Jeff Becker 714f6b0183
macos variant fixes 3 years ago
Jason Rhinelander 869b362572 Attempt fix to work around gcc-8 warning 3 years ago
Jeff Becker 76eb2277f8
deconstexpr ip range bogon check 3 years ago
Jeff Becker 71391957c5
set up accept any plainquic handlers 3 years ago
Jeff Becker 83b0f25075
cleanup after rebase 3 years ago
Jeff Becker f63122272d
fix some compiler errors 3 years ago
Jeff Becker b721b8d65b
finish wiring up config for #1595 3 years ago
Jeff Becker 0be62f0ad4
win32 fix
* windows does not have uint
3 years ago
Jeff Becker fedc56e3f1
initial commit for #1595 3 years ago
Jeff Becker 545021aa3d
temp commit 3 years ago
Jeff Becker bb7b46c43d
fixes for testnet 3 years ago
Jeff Becker 118bc9c2fb
formatting 3 years ago
Jeff Becker 230fb15b24
ensure dns srv data is valid after decoding 3 years ago
Jeff Becker a7a886e670
fix infinite iteration 3 years ago
Jeff Becker 07eaeb681a
try fixing deadlock 3 years ago
Jeff Becker 4889b8cddf
fix up client to snode codepath 3 years ago
Jeff Becker 8b2ede5fc5
use rxid 3 years ago
Jeff Becker 1ea77e6a68
more client to snode quic fixes 3 years ago
Jeff Becker 0ee40c1720
use rxid 3 years ago
Jeff Becker e1f90e5e5a
use rx id on snode side of paths
set protocol on reply routing messages
3 years ago
Jeff Becker e058b873b6
Revert "tweak timeout parameters for link layer."
This reverts commit eef192b37aec070d624e84cedfc94f522fd0a346.
3 years ago
Jeff Becker 1677ed40f6
dont persist sessions if we failed to build 3 years ago
Jeff Becker 657c39a91c
tweak timeout parameters for link layer.
have different timeouts for service node vs client
3 years ago
Jeff Becker 6b2c311800
cover special case for 1 hop path builders 3 years ago
Jeff Becker 2e8f47a7fa
various cleanups 3 years ago
Jeff Becker b36f87a4d2
stop crashing when we send dns reply for snode on service nodes 3 years ago
Jeff Becker b01e20b4cb
add forgotten file again 3 years ago
Jeff Becker 2fa24b5eae
add untracked file 3 years ago
Jeff Becker 5eda4addc2
correctly discard invalid data on transfer traffic message 3 years ago
Jeff Becker 25e338d621
srv records
* add srv records in RCs if we have any
* add mechanism to add SRV records for plainquic exposed ports
* resign and republish rc or introset on srv record changes
3 years ago
Jeff Becker a2285730b8
reduce log level 3 years ago
Jeff Becker de1b0c9d14
add forgotten header 3 years ago
Jeff Becker 94ecf02a62
add option to run lokinet traffic only with no exit and drop all non lokinet traffic
triggered by: exit-node=null
3 years ago
Jeff Becker 2ed0ab1ca1
liblokinet and lnproxy updates
* add lokinet_status function to get the current status of the context, aka are we ready to send or not.
* catch some exceptions in lnproxy
3 years ago
Jeff Becker e005b34169
fix up some retarded bullshit
* llarp::service::NameIsValid was not checking that the tld was .loki, add this check.
* make link layer initial connection timeout 5s not the session activity timeout which happens to be 60 god damn seconds.
3 years ago
Jeff Becker 0046de3e7a
service node logic change: simplify transit hop behavior to send protocol frames directly from a path transfer message 3 years ago
Jeff Becker 3f9e51e391
more liblokinet api functions
* get/set netid
* set log level
3 years ago
Jeff Becker cf7603f20e
temp commit, closeReset -> close 3 years ago
Jeff Becker 48ce77ed1f
XXX: single threaded mode 3 years ago
Jeff Becker bac9341ae0
fix: if we have not used a session yet we only check the intro for expiration 3 years ago
Jeff Becker 2188b443e9
more liblokinet api stuff
* add lokinet_wait_for_ready to wait until we have published introsets and such
* optionally expose ports in lnproxy
* dont save profiles by default
3 years ago
Jeff Becker 95cd275cdd
liblokinet additions:
* add lokinet_add_bootstrap_rc function for adding an rc from memory
* prevent stack overflow on error closing connection in quic
* add in memory nodedb
* refactor how convotags are set as active
* add initial stubs for endpoint statistics
* refactor time stuff to be a bit cleaner
* update lnproxy script with more arguments
3 years ago
Jason Rhinelander 354f0f63e5
Reduce max number of per-connection active streams to 32
This is the max number of simultaneous open connections to the same port
on the remote.  100 was fairly arbitrary and seems a bit high.
3 years ago
Jason Rhinelander 5e912600f8
Fix connection close handling
Replace stream_reset (which typically isn't called) with a stream_close
handler (which is already called whether or not it was a reset).  Most
importantly, the server side needs to extend the max bidi streams
counter during stream_close (otherwise we run out when we hit the
limit and new connections just stall).
3 years ago
Jeff Becker 8bc60a59ac
fix up liblokinet api to be more friendly to ffi
add lnproxy exmaple of use of liblokinet
3 years ago
Jeff Becker 853cc52efb
wire up liblokinet 3 years ago
Jason Rhinelander 2a809c7a30
Add trace debugging of inbound packet type 3 years ago
Jason Rhinelander 35ead9dc6b
Remove superfluous block (FIXME: squash this with earlier "FIXME") 3 years ago
Jason Rhinelander a2045cd966
Fix debug message 3 years ago
Jason Rhinelander 80d3738531
Increase null_iv to 8 bytes to avoid ngtcp2 stack corruption 3 years ago
Jeff Becker 309a899c9f
mark loopback convotags as forever
make null handler mark discard data as activity for convotag
3 years ago
Jeff Becker f310778f81
dont regenerate convotag on loopback when we have one already 3 years ago
Jeff Becker abf8d2fb17
fix loopback convotag 3 years ago
Jeff Becker 5b05d22bad
refactors
* add path sequence numbers on routing messages
* reduce log level in debug mode
* wire up loopback style sending to ourself
3 years ago
Jeff Becker f86a2daf83
fixes
* Add service::Endpoint::HasOutboundConvo
* dont mark outbound convos as inbound
* order quic packets
3 years ago
Jeff Becker 59c9e997f2
build paths faster and limit path builds at edge router instead of via a time counter for all edges 3 years ago
Jason Rhinelander ff2e79ce38
Remove superfluous log statement
The send_packet() we call here already prints the buffer.
3 years ago
Jason Rhinelander 312222e881
Properly handle ngtcp2 returning UINT64_MAX 3 years ago
Jason Rhinelander 2ea84a61f9
Replace timer->again() with stop/start()
repeat()/again() sets a repeating timer, which we don't want; stop and
start it instead so that it fires just once.
3 years ago
Jason Rhinelander 68e96a4e5d
Remove obsolete fixme
Remote addr and ecn are handled.
3 years ago
Jason Rhinelander 7982581cfd
Revisit/reduce quic logging
Demote many things to Trace.
3 years ago
Jason Rhinelander 3c630b260a
Don't install stream forwarding here
Stream forward on the client-side TCP connection gets set up within in
initial_client_data_handler (which also handles reading the initial
stream version byte).
3 years ago
Jason Rhinelander f534103971
Simplify/fix expiry timer logic
Also fix a potential transmission delay because `again()` wasn't being
called when the expiry is already passed (i.e. meaning we should run
immediately).
3 years ago
Jason Rhinelander ac34835c12
Fix/refactor stream closing
Make stream closing with expiring connections work better.  Fixes an
issue where the stream's uv_async could outlive the stream and/or
connection and segfault.
3 years ago
Jason Rhinelander 60c813d306
Schedule retransmit after blocked send
In the standalone plainquic code we triggered a retransmit when the
socket became writeable again, but that doesn't work here, so just
schedule it right away to let ngtcp2 worry about retrying.
3 years ago
Jason Rhinelander 71beae5923
Stop retransmit timer when destroying a connection
Otherwise we segfault because it outlives the Connection.
3 years ago
Jason Rhinelander 027243cbcc
Remove obsolete fixme 3 years ago
Jason Rhinelander 233cb86191
Sever tunnel links earlier
When we get an error on the tcp connection immediately sever the link to
the quic tunnel so that it doesn't keep trying to forward data to it.
3 years ago
Jeff Becker 73572b317b
stuff 3 years ago
Jason Rhinelander 4cea33a139
Fix Server's missing opt-out of ring buffer
For now we still steal buffers from uvw.

In the future I'd like to change that, but it's still uvw
work-in-progress to support custom data allocators, and so for now we
still steal data buffers from uvw.
3 years ago
Jason Rhinelander 99954f7501
Add some more useful debug logging 3 years ago
Jason Rhinelander b8be889291
Add missing client TCP accept/forwarding handlers
Somehow the TCP client connection accept and stream forwarding got
dropped in the quic refactor.
3 years ago
Jason Rhinelander 738f16366b
Try outbound convos if no inbound convos work (FIXME)
FIXME - eliminate this block of code that used to be an `else` before
PR'ing.
3 years ago
Jason Rhinelander 183abd58aa
Add more checks and logging
Most of the logging here is Trace level so needs a Debug build to not
get compiled away.
3 years ago
Jason Rhinelander 44fc941c32
Set port properly in server-to-client reply packets
ngtcp2 was rejecting them because we have the port when constructing,
but then it was 0 on the return packet (which ngtcp2 drops because it's
coming from an unknown/invalid path).
3 years ago
Jeff Becker 6bb23aa956
patch 3 years ago
Jeff Becker 3977c12da6
dont try to build untracked file 3 years ago
Jeff Becker 708e408c30
various fixups and cleanups
* wire up last of the quic stuff
* clean up udp packet generation code
* pass EndpointBase not quic tunnel for quic stuff
* add {n,h}uint16_t::FromString
* add nuint_t::FromString
* make AlignedBuffer::IsZero non constant time call for speed
3 years ago
Jeff Becker 100a953a23
make the lokinet endpoint code work
still needs the quic sides to work
3 years ago
Jeff Becker 7098f46d60
rpc endpoint for triggering quic tunnel 3 years ago
Jeff Becker 838b968d8f
re add static build for all internal lokinet libs 3 years ago
Jeff Becker c88406fb27
move lokinet-quic into the big giant amalgum of liblokinet 3 years ago
Jeff Becker 4dafe973c2
comment out member 3 years ago
Jeff Becker 883005d7db
variable is unused for now but i dont want to remove it yet 3 years ago
Jeff Becker 2360700911
add virtual default destructor to appease clang 3 years ago
Jeff Becker 8b95f9945a
fix compiler errors 3 years ago
Jeff Becker f5700c560e
try inbound paths first 3 years ago
Jeff Becker 59891d5d5f
wire up snode traffic to quic 3 years ago
Jeff Becker 58bec09232
fix compile error 3 years ago
Jason Rhinelander 752879d712
QUIC lokinet integration refactor
Refactors how quic packets get handled: the actual tunnels now live in
tunnel.hpp's TunnelManager which holds and manages all the quic<->tcp
tunnelling.  service::Endpoint now holds a TunnelManager rather than a
quic::Server.  We only need one quic server, but we need a separate quic
client instance per outgoing quic tunnel, and TunnelManager handles all
that glue now.

Adds QUIC packet handling to get to the right tunnel code.  This
required multiplexing incoming quic packets, as follows:

Adds a very small quic tunnel packet header of 4 bytes:

    [1, SPORT, ECN] for client->server packets, where SPORT is our
    source "port" (really: just a uint16_t unique quic instance
    identifier)

or

    [2, DPORT, ECN] for server->client packets where the DPORT is the SPORT
    from above.

(This also reworks ECN bits to get properly carried over lokinet.)

We don't need a destination/source port for the server-side because
there is only ever one quic server (and we know we're going to it when
the first byte of the header is 1).

Removes the config option for quic exposing ports; a full lokinet will
simply accept anything incoming on quic and tunnel it to the requested
port on the the local endpoint IP (this handler will come in a following
commit).

Replace ConvoTags with full addresses: we need to carry the port, as
well, which the ConvoTag can't give us, so change those to more general
SockAddrs from which we can extract both the ConvoTag *and* the port.

Add a pending connection queue along with new quic-side handlers to call
when a stream becomes available (TunnelManager uses this to wire up
pending incoming conns with quic streams as streams open up).

Completely get rid of tunnel_server/tunnel_client.cpp code; it is now
moved to tunnel.hpp.

Add listen()/forget() methods in TunnelManager for setting up quic
listening sockets (for liblokinet usage).

Add open()/close() methods in TunnelManager for spinning up new quic
clients for outgoing quic connections.
3 years ago
Jason Rhinelander 158ea4a951
Add isIPv4/isIPv6 to SockAddr 3 years ago
Jason Rhinelander 2ca7ef7f5f
Rename isRouter -> isSNode
The isRouter wording was quite confusing, especially in a call such as:

    router->Configure(config, opts.isRouter, nodedb)
3 years ago
Jason Rhinelander d588f777c7
Fix logging of std::byte values
They were failing to compile if output because implicit integer
promotion doesn't work for std::byte's (but rather needs an explicit
std::to_integer call).
3 years ago
Jason Rhinelander e27fdf4c9a
Simplify/unify SendTo* methods
Overload them to the same name to make it easier to call and/or visit
them with either a RouterID, Address, Variant, or ConvoTag.
3 years ago
Jason Rhinelander 104f63543f
Change MaybeGetLoop to a virtual member func
Minor simplification.
3 years ago
Jeff Becker a1a1b2e2bc
remove superflous include 3 years ago
Jeff Becker a61f846d33
* add convotags to ensure path to snode
* add service::Endpoint::EnsurePathTo which gives you a std::optional<Convotag>
3 years ago
Jeff Becker fe32475cad
use more variants 3 years ago
Jeff Becker 9d483a12db
* add protocol type to snode traffic
* make path::PathSet::SendPacketToRemote know about protocol type
3 years ago
Jeff Becker e4ca7ce4de
get rid of uneeded type 3 years ago
Jeff Becker 618881e65c
update ngtcp2 address/path structs 3 years ago
Jeff Becker fc9b09bdbc
clean up address/ip management code to use std::variant and std::optional 3 years ago
Jeff Becker 1885b1cae9
more 3 years ago
Jeff Becker 4446f2fc16
fix and format.
* start moving quic code to use lokinet internals
3 years ago
Jason Rhinelander 39d31df059
no-op LogTrace(); print chars as ints
- LogTrace() (and LogTraceTag, etc.) are now no-ops for release builds.
(hoping there are no side effects in trace logging!)
- renamed llarp::_Log to llarp::_log because _Log is a reserved keyword
- change logging code to implicitly convert 1-byte types (char, unsigned
char, uint8_t) to ints so that we print them as numeric values rather
than raw chars because, more often than not, printing a single char is
trying to log an 8-bit value.
3 years ago
Jason Rhinelander b61bd82b4b
Add llarp::buffer_printer for pretty buffer logs 3 years ago
Jason Rhinelander 818b4042e9
Migrate plainquic logging to llarp logging 3 years ago
Jason Rhinelander 5d897781ef
Replace custom random with llarp/sodium random 3 years ago
Jason Rhinelander aa0f54fa07
WIP plainquic tunnels 3 years ago
Jason Rhinelander 46c619c49f
Move external/ cmake bits into external/CMakeLists.txt
- Move uvw target to external instead of llarp/CMakeLists.txt

- Add sqlite_orm headers via interface library rather than shoving it
  into the global include search path.

- Find sqlite3 as part of the sqlite_orm target deps rather than linking
  it separately.  (Also don't link it into lokinet-util, which doesn't
  appear to actually need it).
3 years ago
Jeff Becker b45a268dbc
fix windows dll link-ism 3 years ago
Jeff Becker 2fa7a54e86
fix pybind layer compile errors from missing llarp.h header 3 years ago
Jeff Becker e4841917ba
redo cmake parts
* add liblokinet.so target
* make every library we build except liblokinet.so static
* wire up parts of liblokinet
3 years ago
Jeff Becker a6715a1f75
remove old vpnio bits 3 years ago
Jeff Becker c92894804c
Merge remote-tracking branch 'origin/stable' into dev 4 years ago
Jeff Becker a0d3ab125a
throw on failed writing and port fs namespace changes from oxen-core 4 years ago
Jeff Becker e8178a032e
lokinet-bootstrap native binary 4 years ago
Jeff Becker 0c37cc7f60
lokinet-bootstrap native binary 4 years ago
Jeff Becker 7a1ffc2df4
* lessen replay filter window
* dont have transit hops use a replay filter
* formatting
4 years ago
Jeff Becker 9c742b36eb
call delivery status hook on overload 4 years ago
Jeff 6bd53484da
Merge pull request #1541 from majestrate/lns-consensus-2021-02-19
lns lookup consensus
4 years ago
Jeff cb2254ba46
Merge pull request #1573 from jagerman/more-code-refactors
More code refactors
4 years ago
Jeff Becker dfa84081c9
forgot a type 4 years ago
Jeff Becker 202c9348fa
use std::variant 4 years ago
Jeff Becker b7f72601ce
move stuff to var::variant 4 years ago
Jeff Becker ba9dca6c5d
use var::variant and var::visit shims because apple is sux 4 years ago
Jeff Becker f10269b03e
remove std::visit because apple is functionally retarded and does not support it despite claiming to support c++14 4 years ago
Jeff Becker c07358f084
allow failure of setting ipv6 address when it is not allowed by kernel on linux 4 years ago
Jeff Becker 6c1d18412c
seperate logic into IsDone() 4 years ago
Jeff Becker 88ddcceca4
make lookup name async return optional variant 4 years ago
Jeff Becker 4e14567d67
make Endpoint::LookupNameAsync return void 4 years ago
Jeff Becker 8e57572b37
dont reference iterator in lambda. 4 years ago
Jeff Becker 95eba18e40
cache result on success 4 years ago
Jeff Becker 7a11f3b1e3
redunant lookups for lns.
* request lns from all endpoints we have on our pathset
* make sure all snodes agree on the lns name being looked up
4 years ago
Jeff ec242447a0
Merge pull request #1581 from majestrate/ipv6-configurable-range-2021-03-18
make ipv6 range used by exit traffic configurable.
4 years ago
Jeff Becker cf85ea6f2b
add warning and note in documenation of config option 4 years ago
Jeff Becker 1acea95ced
fix issue #1562
* use csrng for std::shuffle
4 years ago
Jeff Becker b2687a53ee
dont use .value() because macos is absolutely broken, thanks apple. 4 years ago
Jeff Becker a83428297e
make ipv6 range used by exit traffic configurable.
* add ip6-range option to network section to control which range we want to use for ipv6 traffic
* make ip6-range able to disable ipv6 exits ( i dont like this but eh )
4 years ago
Jeff Becker ad6dcd683b
fix segfault 4 years ago
Jeff Becker 80a76c9386
fix segfault 4 years ago
Jason Rhinelander 181953b4a6 Replace ::Hash nested structs with std::hash specializations 4 years ago
Jason Rhinelander d1dadb530c SockAddr endian cleanups & add uint128 ctors
- Make SockAddr endian arguments explicit
- Consolidate port-less contructors and port constructors into one with
  a default port of 0.
- Add {h,n}uint128_t ctors for construction from IPv6 addrs
4 years ago
Jason Rhinelander fe0d099e86 Code cleanups
- De-lambda a lambda that is only called once, immediately after it is
  defined.
- C++17 modernize key-value for loops
- Remove unused (and useless) method
4 years ago
Jason Rhinelander 0339bd941a Convert protocol type to enum class 4 years ago
Jeff 9bf002cc8d
Merge pull request #1574 from tewinget/header_fixup
standardize include format and pragma once
4 years ago
Jason Rhinelander f47e0cf7a2 Fix UB -- use after move 4 years ago
Thomas Winget 7caa87862e standardize include format and pragma once
All #ifndef guards on headers have been removed, I think,
in favor of #pragma once

Headers are now included as `#include "filename"` if the included file
resides in the same directory as the file including it, or any
subdirectory therein.  Otherwise they are included as
`#include <project/top/dir/relative/path/filename>`

The above does not include system/os headers.
4 years ago
Thomas Winget 35fb54d131 add project source dir to util include directories 4 years ago
Jeff Becker 324498fe9a
flip order of parameters so it works 4 years ago
Jeff Becker 3d09b94590
make android work again
ip packet on read error was returning an uninitialized ip packet with non zero size so it was thinking it was a valid packet.
value initialize it to set the packet size to zero by default.

this fixed it in an emulator at least...
4 years ago
Jeff Becker 6550aa844c
try to unbreak more 4 years ago
Jeff Becker 7af4807302 correctly set size of OwnedBuffer when using OwnedBuffer::copy_used 4 years ago
Jason Rhinelander 91d6698a9d Reformat with clang-format-11 4 years ago
Jason Rhinelander c00de1e4ec Add wakeup call into transit_hop UpstreamHandler 4 years ago
Jason Rhinelander 89b582bf60 Modernize [[maybe_unused]] attribute 4 years ago
Jason Rhinelander 4b5387a078 Remove unused ShouldFlushNow method 4 years ago
Jason Rhinelander f9dc308f75 Tick event loop on data queuing 4 years ago
Jason Rhinelander 895acc45ff EventLoop: add public wakeup() method, and call it from call()
call(), when invoked from the logic thread, wasn't triggering a wakeup
which stalled some traffic (such as client-to-snode packets).

Fix it by triggering a wakeup on `call()`, and expose it because this is
a useful thing to be able to do.
4 years ago
Jason Rhinelander f8d5e106a0 Uncurse SockAddr's sockaddr* operator
The `const sockaddr*` conversion was returning a pointer to the
sockaddr_in6, but with its family set to AF_INET, so the receiving code
would have treated interpreting the result as a sockaddr_in pointer --
but it isn't, and trying to get the IPv4 from the mis-reinterpreted
sockaddr_in6 fails; this then prevented uvw UDP binding (which uses the
sockaddr* operator) from working.

This changes it to always leave the family value at the proper value and
using the ipv6_is_mapped_ipv4 check to determine when we have (and
should return) a pointer to the sockaddr_in rather than the
sockaddr_in6.

Also fixes toString() to compare the full prefix instead of just bytes
10 and 11.
4 years ago
Jason Rhinelander 62fc1bbc48 Make llarp_buffer_t constructible from an rvalue reference 4 years ago
Jason Rhinelander f8486604c4 Export UDP file descriptor (for android) 4 years ago
Jason Rhinelander d995766436 Android & hive fixes 4 years ago
Jason Rhinelander 673d065ed8 Remove HasNextPacket() (and fix win32)
We don't use it anymore (instead we just ReadNextPacket until it returns
an empty result).
4 years ago
Jason Rhinelander 6e05dc693f Remove unused arguments (and ants) 4 years ago
Jason Rhinelander 2c827870c9 Simplify code of many f's
We don't need to go through the contention killer here, that was mainly
for (long ago) debugging reasons.
4 years ago
Jason Rhinelander 36edabd9b0 Rename EventLoop::udp to EventLoop::make_udp
Makes it consistent with the make_waker, etc.

Also removes some debugging.
4 years ago
Jason Rhinelander afd856bca0 Remove unused junk (also fix win32 compilation fail) 4 years ago
Jason Rhinelander 096f37fe4e Fix linux compilation & narrowing conversion warning 4 years ago
Jason Rhinelander e198bfd3e1 Remove unused variable 4 years ago
Jason Rhinelander c4559d158e Make format 4 years ago
Jason Rhinelander ccc7b5c9e9 Merge Logic functions into EventLoop
loop->call(...) is similar to the old logic->Call(...), but is smart
about the current thread: if called from within the event loop it simply
runs the argument directly, otherwise it queues it.

Similarly most of the other event loop calls are also now thread-aware:
for example, `call_later(...)` can queue the job directly when called if
in the event loop rather than having to double-queue through the even
loop (once to call, then inside the call to initiate the time).
4 years ago
Jason Rhinelander 5b555ee5aa Replace libuv with uvw & related refactoring
- removes all the llarp_ev_* functions, replacing with methods/classes/functions in the llarp
  namespace.
- banish ev/ev.h to the void
- Passes various things by const lvalue ref, especially shared_ptr's that don't need to be copied
  (to avoid an atomic refcount increment/decrement).
- Add a llarp::UDPHandle abstract class for UDP handling
- Removes the UDP tick handler; code that needs tick can just do a separate handler on the event
  loop outside the UDP socket.
- Adds an "OwnedBuffer" which owns its own memory but is implicitly convertible to a llarp_buffer_t.
  This is mostly needed to take over ownership of buffers from uvw without copying them as,
  currently, uvw does its own allocation (pending some open upstream issues/PRs).
- Logic:
  - add `make_caller`/`call_forever`/`call_every` utility functions to abstract Call wrapping and
    dependent timed tasks.
  - Add inLogicThread() so that code can tell its inside the logic thread (typically for
    debugging assertions).
  - get rid of janky integer returns and dealing with cancellations on call_later: the other methods
    added here and the event loop code remove the need for them.
- Event loop:
  - redo everything with uvw instead of libuv
  - rename EventLoopWakeup::Wakeup to EventLoopWakeup::Trigger to better reflect what it does.
  - add EventLoopRepeater for repeated events, and replace the code that reschedules itself every
    time it is called with a repeater.
  - Split up `EventLoop::run()` into a non-virtual base method and abstract `run_loop()` methods;
    the base method does a couple extra setup/teardown things that don't need to be in the derived class.
  - udp_listen is replaced with ev->udp(...) which returns a new UDPHandle object rather that
    needing gross C-style-but-not-actually-C-compatible structs.
  - Remove unused register_poll_fd_(un)readable
  - Use shared_ptr for EventLoopWakeup rather than returning a raw pointer; uvw lets us not have to
    worry about having the event loop class maintain ownership of it.
  - Add factory EventLoop::create() function to create a default (uvw-based) event loop (previously
    this was one of the llarp_ev_blahblah unnamespaced functions).
  - ev_libuv: this is mostly rewritten; all of the glue code/structs, in particular, are gone as
    they are no longer needed with uvw.
- DNS:
  - Rename DnsHandler to DnsInterceptor to better describe what it does (this is the code that
    intercepts all DNS to the tun IP range for Android).
- endpoint:
  - remove unused "isolated network" code
  - remove distinct (but actually always the same) variables for router/endpoint logic objects
- llarp_buffer_t
  - make constructors type-safe against being called with points to non-size-1 values
- tun packet reading:
  - read all available packets off the device/file descriptor; previously we were reading one packet
    at a time then returning to the event loop to poll again.
  - ReadNextPacket() now returns a 0-size packet if the read would block (so that we can implement
    the previous point).
  - ReadNextPacket() now throws on I/O error
- Miscellaneous code cleanups/simplifications
4 years ago
Jason Rhinelander c71d3527bd C++17 simplification 4 years ago
Jason Rhinelander 8b00eacabf Fix signed/unsigned char comparison bug
These functions would not working properly if given a high bit char
value.
4 years ago
Jeff abf057d6e3
Merge pull request #1536 from majestrate/memleak-fix-2021-02-15
plug super slow memory leak
4 years ago
Jeff Becker ec0f45eeb7
plug memory leak in outbound message queue.
sometimes we use rxid in our pathid so we need to clear those queues too.
if we don't it'll leak from never clearing those queues.
4 years ago
Jeff Becker 7dbd25f271
don't leave old multi ack codepath in 4 years ago
Jeff Becker c9ff917e0d
revert priority queue ordering change 4 years ago
Jeff Becker ad3c23ba2b
simplify call 4 years ago
Jeff Becker 5cffc3b0f8
consolidate rx message handling in iwp
* add toggle flag for using multi ack
* check replay filter before processing message
4 years ago
Jeff Becker e6ac7e721d
use a std::map instead of a std::unordered_map for holding message buffers by their sequentially increasing message id.
when retransmissions happen, they will be executed in a deterministic order by their delivery id.
this reduces the jitter from delayed message delivery on link layer as with an unordered map the order of iteration is "random",
so when we iterate all messages for retransmission we do it lowest id first (the messages queued first).
4 years ago
Jeff Becker 74f707ee01
only do single ack packets 4 years ago
Jeff 4992629f20
Merge pull request #1563 from majestrate/dont-multiplex-from-outbound-context-2021-03-03
use lowest latency path for send context
4 years ago
Jeff Becker 7060a60c84
enable route poker poking before going "up" 4 years ago
Jeff Becker f77cbfb48e
fixes for ipv4 traffic: dont use new range because that breaks backwards comapt 4 years ago
Jeff Becker f67668f0a6
use lowest latency path for send context 4 years ago
Jeff f2b234d6c3
Merge pull request #1539 from majestrate/path-algorithm-flavors-2021-02-18
add option to enforce unique netblocks per path.
4 years ago
Jeff b4b8b84e57
Merge pull request #1548 from majestrate/lns-mx-records-2021-02-25
add mx records to lns names
4 years ago
Jeff Becker d6567614e3
use mutable lambda instead of shared_ptr for handling mx dns reply for lns 4 years ago
Thomas Winget 4c630e0437 Large collection of changes to make android work
- Previous android java and jni code updated to work, but with much love
  still needed to make it work nicely, e.g. handling when the VPN is
  turned off.

- DNS handling refactored to allow android to intercept and handle DNS
  requests as we can't set the system DNS to use a high port
  (and apparently Chrome ignores system DNS settings anyway)

- add packet router structure to allow separate handling of specific
  intercepted traffic, e.g. UDP traffic to port 53 gets handled by our
  DNS handler rather than being naively forwarded as exit traffic.

- For now, android lokinet is exit-only and hard-coded to use exit.loki
  as its exit.  The exit will be configurable before release, but
  allowing to not use exit-only mode is more of a challenge.

- some old gitignore remnants which were matching to things we don't
  want them to (and are no longer relevant) removed

- some minor changes to CI configuration
4 years ago
Jeff Becker b59a92aa01
add mx records to lns names 4 years ago
Jeff Becker 0fa91381ba
add llarp::CSRNG for cryptographically secure random number generator for use in std::shuffle 4 years ago
Jeff Becker a88aec5f01
fix typo 4 years ago
Jeff Becker a3585caa70
add value 0 to disable unique-range-size 4 years ago
Jeff Becker f7424b8bbe
dont use std::optional 4 years ago
Jeff Becker 4478cf6f72
use pointers to entries instead of lookups by key 4 years ago
Jeff Becker b865ffb736
use 64-bit Mersenne Twister instead of 32 bit flavor in nodedb 4 years ago
Jeff Becker e584cbe44d
make non 64 bit platforms happy 4 years ago
Jeff Becker 3425069b41
feedback from jason
* use emplace in PeerSectionConfig::Acceptable use insert everywhere else
* use const l-value references
* fix typos and spelling mistakes
4 years ago
Jeff Becker 5d465264a8
truly random access in nodedb iteration 4 years ago
Jeff Becker 9457da27d9
add option to enforce unique netblocks per path. 4 years ago
Jason Rhinelander 56cbef6086 SockAddr fixes and cleanups
- Remove SIIT from method names & comments because we're doing IPv4
mapped addresses (::ffff:0:0/96) rather than actual SIIT
(::ffff:0:0:0/96).

- add constructor taking a string+numeric port (and then don't allow a
port in the string).

- simplify IP string parsing by using parse_int()

- replace addrIsV4 with call to ipv6_is_mapped_ipv4 (this also fixes a
bug where addrIsV4 was not checking for leading 0s and so could return
true for a public IPv6 that happened to have ffff in the wrong spot).
4 years ago
Jeff Becker dcdd147e74
use static_cast with enum class 4 years ago
Jeff Becker 5fe7e00765
make AuthResultCode an enum class 4 years ago
Jeff Becker 94b8743b1c
fix typo 4 years ago
Jeff Becker 5b2abce86f
add string reason to auth messages 4 years ago
Jeff Becker 2cb889013b
revise lambda captures 4 years ago
Jeff Becker 6d5f1b0c6a
feedback auth result via lmq rpc 4 years ago
Jeff c76f2fe8de
Merge pull request #1496 from majestrate/rpc-kill-endpoint-sessions-2020-11-28
add llarp.endpoint rpc command
4 years ago
Jeff Becker 785b4258ed
only set public address when provided 4 years ago
Jeff Becker d7842d18ee
add missing header 4 years ago
Jeff Becker cb3b89457f
remove pointless operator overload 4 years ago
Jeff Becker be3faeb402
feedback from tom.
* remove unneeded header.
* fix typo in docs
4 years ago
Jeff Becker 1c3020a05e
drop on overload 4 years ago
Jeff Becker bb9bd25059
get rid of use of llarp::IpAddress from hotpaths 4 years ago
Jeff Becker 01262d1c4d
flush queue when full 4 years ago
Jeff Becker 42ffbcca0a
try coleasing inbound packets from iwp 4 years ago
Jeff Becker dc68ff6de9
prevent bad side effects, dont use iterator after use and use operator[] 4 years ago
Jeff Becker b2b68c413c
actually set disabled when disabiling route poker 4 years ago
Jeff Becker 6a29cc7380
call closed hook when we explicitly close a link session 4 years ago
Jeff Becker 6a2d35d907
always use most recent convotag 4 years ago
Jeff Becker b484d9bebb
update route poker logic to include "up" 4 years ago
Jeff Becker e46204b068
dont wire up router thaw to sigusr1 4 years ago
Jeff Becker 07b599430e
thaw router when connectivity changes 4 years ago
Jeff Becker 676ca7f511
close stale sessions on thaw because they are probably already gone. this removes them and forces the sessions to be renegotiated. 4 years ago
Jeff Becker ffef3bc48f
wire up sigusr1 to reset libunbound dns resolver 4 years ago
Jeff Becker 8239efa71b
clarify new member's purpose and name. add comment. 4 years ago
Jeff Becker 1526941f3a
make it compile again 4 years ago
Jeff Becker 0600f42814
fix up names of functions to be less hidious. 4 years ago
Jeff Becker 1ee545d6d2
fix up macos route command for ipv6 4 years ago
Jeff Becker 35137913b9
windows and macos route pokers for ipv6 4 years ago
Jeff Becker 1c7079dbab
fix up route poking with lokinet to work with ipv6 4 years ago
Jeff Becker 063e982ef6
fix exit range default value 4 years ago
Jeff Becker bd93a8f828
initial v6 exits 4 years ago
Jeff Becker 1f1db29816
GetIf -> GetRandom 4 years ago
Jeff Becker f24cdb305b
clean up calls to NodeDB::GetIf to be cleaner and more concise 4 years ago
Jeff Becker c8aba0af31
make llarp::NodeDB::GetIf always randmize location 4 years ago
Jeff Becker d55705362a
use good routers 4 years ago
Jeff Becker b6dbbb4bef
consolidate pathbuilder logic
* deduplicate code for path hops selection
4 years ago
Jeff Becker 242e56507c
proper lokimq -> oxenmq refactor 4 years ago
Jeff Becker 1e58715e0b
fix pybind layer 4 years ago
Jeff Becker 922f4fb724
fix iteration bug 4 years ago
Jeff Becker df4ea34a56
nodedb refactor
* bump zmq static dep
* lokimq -> oxenmq
* llarp_nodedb -> llarp::NodeDB
* remove all crufty api parts of NodeDB
* make NodeDB rc selection api not suck
* make path builder api not suck
* propagate all above changes so that unit tests work and it all compiles
4 years ago
Jeff 5f0f4e0c70
prevent crash by removing duplicate logic (#1515)
when a path expires or fails or something causes its lifecycle to end we have cleanup
logic that handles clean up for it. this removes a code path that was crashing that
is duplicated elsewhere and is thus probably safe to bin. yolo.
4 years ago
Jeff Becker 82b3ec19e8
unbreak the build
* remove llarp/ev/pipe.cpp from cmake
* fix typo in llarp/ev/ev_libuv.cpp
4 years ago
Jeff 49b9ad7197
tun code refactor (#1495)
* partial tun code refactor

* take out the trash

* move vpn platform code into llarp/vpn/platform.cpp

* fix hive build

* fix win32

* fix memory leak on win32

* reduce cpu use

* make macos compile

* win32 patches:

* use wepoll for zmq
* use all cores on windows iocp read loop

* fix zmq patch for windows

* clean up cmake for win32

* add uninstall before reinstall option to win32 installer

* more ipv6 stuff

* make it compile

* fix up route poker

* remove an unneeded code block in macos wtf

* always use call to system

* fix route poker behavior on macos

* disable ipv6 on windows for now

* cpu perf improvement:

* colease calls to Router::PumpLL to 1 per event loop wakeup

* set up THEN add addresses

* emulate proactor event loop on win32

* remove excessively verbose error message

* fix issue #1499

* exclude uv_poll from win32 so that it can start up

* update logtag to include directory

* create minidump on windows if there was a crash

* make windows happy

* use dmp suffix on minidump files

* typo fix

* address feedback from jason
* use PROJECT_SOURCE_DIR instead of CMAKE_SOURCE_DIR
* quote $@ in apply-patches in case path has spaces in it

* address feedback from tom

* remove llarp/ev/pipe
* add comments for clairification
* make event loop queue size constant named
4 years ago
Jeff Becker 7ee9d0281c
add llarp.endpoint rpc command with kill option to kill all active convotags given
a list of or a single remote service address.
4 years ago
Jeff 6407733c8e
* make llarp.halt rpc call actually return a reply (#1488)
* add --kill flag to lokinet-vpn that calls llarp.halt
* make macos uninstaller kill lokinet via lokinet-vpn --kill
4 years ago
Jason Rhinelander 384b968cf0
Merge pull request #1437 from majestrate/key-manager-fixes-2020-10-30
key manager should throw on failure to load keys
4 years ago
Jeff db7abe8f09
do /1 route poker split on linux (#1480)
* do /1 route poker split on linux

i hate netlink so god damn much

* use an enum class for the gateway mode
4 years ago
Jeff dad66c50cb
poke route hole for loopback on windows because that OS is made by brain damaged non programmers. (#1473)
this fixes the gui bug where it hangs on adding an exit becuase loopback gets routed over the exit.
before it was causing the lmq connection to hang becuase of that route change.
4 years ago
Jeff a0ed303d7b
try to handle router deregistrations on the network better (#1461)
* don't send messages to de-registered relays but allow traffic to clients

* actively close sessions to de-registered relays
4 years ago
Jeff a7518057cb Revert "deny traffic to non whitelisted routers. (#1444)"
This reverts commit ff02e62a79.
4 years ago
Jeff c743022cfb
try fixing deadlock (#1454)
* fix deadlock in #1453

* remove use of shared_lock in profiling
* clean up Save and Load to acquire lock
* bencode and bdecode are now non locking

* remove all places using std::shared_lock

* bail if write fails
4 years ago
Jeff e66522b9dd
fix path reference leak (#1451) 4 years ago
Jeff 34c7f0da0d
profiles.dat is always relative so if you spawn lokinet in / then it will error about permissions. (#1449)
* use absolute path for profiles.dat.

* remove old bencoding bits to use const fs::path instead of const char *
4 years ago
Jeff 8be7c46531
try fixing memory leak in path builder (#1442) 4 years ago
Jeff ff02e62a79
deny traffic to non whitelisted routers. (#1444)
* deny message delivery from non registered nodes when whitelist is enabled and applicable

* use DoCallback and return true so that the event gets fired right
4 years ago
Jeff 87c76a6769
add exit.localhost.loki cname record (#1440)
* add exit.localhost.loki cname record

* return cname in localhost.loki A/AAAA lookup reply

* correctly handle case where we have no exit

* typofix

* simplify logic
4 years ago
Jeff 3ec727d844
I have no words. (#1438)
So apparently dnsConfig in Router was never set.
Router::networkConfig was set, but only used in the function
which set it and otherwise only passed to another function once.
dnsConfig was only used in one place.

both member variables have been removed in favor of just
referencing Router::m_Config.

Co-authored-by: Thomas Winget <tewinget@gmail.com>
4 years ago
Jeff Becker 5233b26552
format 4 years ago
Jeff Becker bf1a692005
key manager should throw on failures 4 years ago
Jeff 0e1e0aaef3
allow for relative file paths in config (#1432)
* llarp::Config constructor can take an empty fs::path now and it will default to fs::current_path
* llarp::ensureConfig now throws on fs error
* updates to daemon/main.cpp to reflect changes to llarp::ensureConfig's new throwability
* dowse parts of the code with holy water for the 0.8.1 tag
4 years ago
Jeff c81b6049f1
fix crashes on shutdown (#1433) 4 years ago
Jason Rhinelander e47b70a82f
CMake fixes: libuv static build, base_libs, shared lib install dir (#1431)
* Update how we build libuv

- Update submoduled libuv to latest stable (1.40.0)
- Don't look for a system libuv if we're under BUILD_STATIC_DEPS
- Add a libuv interface library rather than using globals
- Make the windows build fall back to the submodule if not explicitly
  given a LIBUV_ROOT

* Replace ${LIBS} global with `base_libs` interface

This simplifies linking and include handling a bit.

* Remove unneeded header

* Add missing csignal header

(This was previously being pulled in incredibly indirectly via some
stuff that eventually includes some other stuff that eventually included
uv.h)

* Use GNUInstallDirs to get lib dir instead of hard-coding lib

Fixes #1429
4 years ago
Jeff dcb48db5fe
enable profiling on clients by default (#1421)
* enable client relay profiling by default

* macos dns fixes

* improve peer profiling algorithm to track timeouts vs failures

* remove debug ioctl call in tuntap code

* use ub_wait instead of ub_process as that was what was there before

* const correctness

* DRY out checking for SIIT

* typofix

* correct name
4 years ago
Jeff 4c7d52ac20
more aggressive path building. (#1423)
* more aggressive path building.

* do more than one in parallel path builds at a time

* correct last commit's logic

* rename numPaths -> numDesiredPaths to clarify intent

* revert string change as it will break a lot

* don't prematurly short circuit on snode builds

Co-authored-by: Thomas Winget <tewinget@gmail.com>
4 years ago
Jeff 7a1207dc47
dont bind inbound links as client (#1419)
* dont bind inbound links as client

* fix broken test

* logic call MUST defer call
4 years ago
Thomas Winget a91bb35dbf
Some Windows fixes (#1415)
* Should fix some windows service issues

* fix return condition inversion

* Add some Trace level logging

also make the logger actually respect the log level you set.

* event loop should not queue things to itself...

at present, logic thread queue continues until it is empty, so
queueing things onto itself is just wasteful.

* call_later(foreach thing) is better than foreach thing (call later)

also if you already queued those things but they have not happened yet,
there is no sense to queue them to happen again.

* do not queue read on write finish, only on read finish

* failure to start DNS server should be proper startup failure.

without the DNS server working lokinet is...kinda pointless, right?

* format

* don't queue stuff to logic thread if in logic thread
the thing that clears the queue...clears it.  So you're just delaying and adding overhead.

* windows unbound thread sleep instead of just busy-waiting

also clang-format decided I can't have a blank line for some reason...

* fix unbound async worker on windows
4 years ago
Jeff 12eb32a816
add conf.d directory for config overrides (#1410)
* add conf.d directory for config overrides

* refactor llarp::Config

* add explicit constructor with datadir as parameter
* have all config files be passed as std::optional
* make Config::LoadDefault private and use std::optional in Config::Load to remove ambiguity
* update rest of codebase to reflect above changes

* fix pybind

* rename bootstrap config skipBootstrap to seednode as it's more descriptive
* make seednode configurable
* make pybind layer compile
* make pybind layer run
4 years ago
Jeff 22acf0a537
Lns exits (#1407)
* lns exits

* try appeasing the clang

* clean up lambda
4 years ago
Thomas Winget df36ed953d
Route Poker changes (#1416)
If not using an exit, messing with the routing table is not good.
As such, the ability to keep track of routes we *might* want is good,
but the ability to set/unset those routes is necessary, to correspond
to enabling/disabling exit functionality.
4 years ago
Jeff 98e022ea21
Unflaky-ify lns dns (#1406)
* always ensure path to service on dns lookup

* deprecate profiles option
4 years ago
Jeff 38f13533df
always use data directory for bootstrap.signed (#1405) 4 years ago
Jeff 5b5bd6b44e
dns features (#1404)
* add some dns txt records for stuff we want to expose

* fix txt records

* txt records for snode info

* dont send cname as it mangles the response

* check for 3 parts not 3 characters
4 years ago
Jason Rhinelander d10fee87c7
Don't report a huge negative elapsed time on startup (#1401) 4 years ago
Jeff 3b70b99dd2
fix empty config case (#1400)
* fix empty config case

* * fix case for empty ifname / ifaddr on relay
* bail if no dns server bound

* use AssignmentAcceptor
4 years ago
Jeff 50aea744f6
order packets when writing to network interface (#1372)
* order packet writes on userside

* make it compile

* fix pybind
4 years ago
Jason Rhinelander af6caf776a
Config file improvements (#1397)
* Config file API/comment improvements

API improvements:
=================

Make the config API use position-independent tag parameters (Required,
Default{123}, MultiValue) rather than a sequence of bools with
overloads.  For example, instead of:

    conf.defineOption<int>("a", "b", false, true, 123, [] { ... });

you now write:

    conf.defineOption<int>("a", "b", MultiValue, Default{123}, [] { ... });

The tags are:
- Required
- MultiValue
- Default{value}
plus new abilities (see below):
- Hidden
- RelayOnly
- ClientOnly
- Comment{"line1", "line2", "line3"}

Made option definition more powerful:
=====================================

- `Hidden` allows you to define an option that won't show up in the
  generated config file if it isn't set.

- `RelayOnly`/`ClientOnly` sets up an option that is only accepted and
  only shows up for relay or client configs.  (If neither is specified
  the option shows up in both modes).

- `Comment{...}` lets the option comments be specified as part of the
  defineOption.

Comment improvements
====================

- Rewrote comments for various options to expand on details.
- Inlined all the comments with the option definitions.
- Several options that were missing comments got comments added.
- Made various options for deprecated and or internal options hidden by
  default so that they don't show up in a default config file.
- show the section comment (but not option comments) *after* the
  [section] tag instead of before it as it makes more sense that way
  (particularly for the [bind] section which has a new long comment to
  describe how it works).

Disable profiling by default
============================

We had this weird state where we use and store profiling by default but
never *load* it when starting up.  This commit makes us just not use
profiling at all unless explicitly enabled.

Other misc changes:
===================

- change default worker threads to 0 (= num cpus) instead of 1, and fix
  it to allow 0.
- Actually apply worker-threads option
- fixed default data-dir value erroneously having quotes around it
- reordered ifname/ifaddr/mapaddr (was previously mapaddr/ifaddr/ifname)
  as mapaddr is a sort of specialization of ifaddr and so makes more
  sense to come after it (particularly because it now references ifaddr
  in its help message).
- removed peer-stats option (since we always require it for relays and
  never use it for clients)
- removed router profiles filename option (this doesn't need to be
  configurable)
- removed defunct `service-node-seed` option
- Change default logging output file to "" (which means stdout), and
  also made "-" work for stdout.

* Router hive compilation fixes

* Comments for SNApp SRV settings in ini file

* Add extra blank line after section comments

* Better deprecated option handling

Allow {client,relay}-only options in {relay,client} configs to be
specified as implicitly deprecated options: they warn, and don't set
anything.

Add an explicit `Deprecated` tag and move deprecated option handling
into definition.cpp.

* Move backwards compat options into section definitions

Keep the "addBackwardsCompatibleConfigOptions" only for options in
sections that no longer exist.

* Fix INI parsing issues & C++17-ify

- don't allow inline comments because it seems they aren't allowed in
ini formats in general, and is going to cause problems if there is a
comment character in a value (e.g. an exit auth string).  Additionally
it was breaking on a line such as:

    # some comment; see?

because it was treating only `; see?` as the comment and then producing
an error message about the rest of the line being invalid.

- make section parsing stricter: the `[` and `]` have to be at the
beginning at end of the line now (after stripping whitespace).

- Move whitespace stripping to the top since everything in here does it.

- chop off string_view suffix/prefix rather than maintaining position
values

- fix potential infinite loop/segfault when given a line such as `]foo[`

* Make config parsing failure fatal

Load() LogError's and returns false on failure, so we weren't aborting
on config file errors.

* Formatting: allow `{}` for empty functions/structs

Instead of using two lines when empty:

    {
    }

* Make default dns bind 127.0.0.1 on non-Linux

* Don't show empty section; fix tests

We can conceivably have sections that only make sense for clients or
relays, and so want to completely omit that section if we have no
options for the type of config being generated.

Also fixes missing empty lines between tests.

Co-authored-by: Thomas Winget <tewinget@gmail.com>
4 years ago
Jeff b0088b3298
dont segfault on exit (#1396)
* dont segfault on exit

* initialize m_isServiceNode earlier.
4 years ago
Jeff ff23106852
don't allow running lokid-rpc as client (#1394) 4 years ago
Jason Rhinelander 753d989f7c Set default inbound link to best public interface
Fixes default config not working.
4 years ago
Jason Rhinelander 7aa4566016 Make format 4 years ago
Jason Rhinelander 4faaf9082c Fix broken unbound resolver when including upstream port
unbound breaks when given "1.2.3.4:53" as it expects only an IP.
4 years ago
Jason Rhinelander 01013c1963 Make upstream= override work; reject non-default upstream port
If you specify upstream= then you get no upstream, if you give one then
you use that instead of the default, but you can still list multiple.

unbound doesn't support an upstream port, so bail if the user gives a
non-port-53 response.
4 years ago
Jason Rhinelander d129b0432a Allow empty values so that upstream= can disable the default 4 years ago
Jason Rhinelander c97b9ef31b Simplify k=v parsing code 4 years ago
Jeff 9d6dc40f81 format 4 years ago
Jeff 2b5b3ce8e1 default upstream dns to cloudflare 4 years ago
Jeff 0f7e848903
disable rpc on relays by default (#1383)
* disable rpc on relays by default

* add default inbound link as service node

* throw if public-ip and public-port are not set
4 years ago
Jason Rhinelander fbfa8ca89c
make format (#1381) 4 years ago
Jason Rhinelander b121933374
Make public-ip option work (#1380)
0.7 supported both public-address and public-ip, but only the former was
being allowed in 0.8.

This makes `public-ip=...` work again, and makes it the canonical option
(because it makes a bit more sense than address when we also have the
public-port parameter), while making `public-address=` accepted with a
deprecation warning.

Also adds default descriptions for port and ip for the generated ini.
4 years ago
Jeff f797405318 ping lokid every 30 second after starting up 4 years ago
Thomas Winget 4f3ed5d1c4
remove unnecessary code
the signed_bt_dict field is not used for a Version 0 RC anyway, so no need to handle it here.  That was my bad.
4 years ago
Jeff Becker c04cbfe34f
fix decode version 0 rc when in version 1 mode 4 years ago
Jeff Becker 79dfb6d357
kill lmq before stopping links/ 4 years ago
Jeff Becker bb67b42f0b
put pubkey_ed25519 in fields 4 years ago
Jeff Becker 23302cd10c
use size_left 4 years ago
Jeff 503a92c992 un limit lmq 4 years ago
Jeff 7852735b36 tweak configuration defaults 4 years ago
Jeff 28bfaf0372
ammend log levels for common spammy log events (#1375) 4 years ago
Jeff 8943be6c3a
serialize auth codes in rpc (#1369)
* serialize auth codes in rpc

* format

* format

* config and route fixups

* bump lokimq

* fix compile

* discard unknown members in service keys

* format
4 years ago
Thomas Winget 7f18df4e9a
fix format string for size_t (#1371) 4 years ago
Jeff 56f49a6980
make win32 exits work again (#1367) 4 years ago
Jeff 53598ec0e9
dont serialize ephemeral members of service identity (#1370)
* prepare for keytool script

* dont serialize ephemeral members in service key file
* regnerate ephemeral members in service identity on load

* add keygen script

* use nacl for generating keys

* format
4 years ago
Thomas Winget 15229ea7ff
New RC format (#1368)
* initial work for RC SRVs.

Needs tests for new RC format.

Needs SRVs added to new RC, and associated tests.

* convert rc sign/verify test to catch2, add test for new rc

Also fixes a mistake in new rc serialization

* bump loki-mq submodule

need support for viewing bt deserialize consumer buffer so we
know how much it has consumed.

* fix some behavior errors

need to advance llarp_buffer_t after consuming data from it

only rewind and modify size of llarp_buffer_t in owning context.

* Add test for router contact (de-)serialization

Adds a test that makes a list of RouterContact with mixed
versions and ensures it serializes and deserializes correctly.
4 years ago
Jeff 252692a55b
dont poke routes if we have no exit (#1352)
* * refactor route poking out of llarp::Router and into llarp::RoutePoker
* only poke routes when we have an exit enabled as a client

* add route_poker header so it compiles
4 years ago
Jeff 10e3d80559
resolve issue #1361 about peer stats excessive logging (#1365) 4 years ago
Jeff Becker 7a18d14fde make it compile 4 years ago
Jeff a9d23d3ac3
use absolute path for route command (#1354)
* ignore tun interfaces on windows for mitigating foot cannons

* add flag for git add -p when using format verifier as git hook

* use explicit path for route command on windows

* fix typo

* fix typo

* remove hunk for win32 route exclusion based off being a tun interface

* add metric to win32 route command

* * refactor win32 route poking to use a common function for iterating over routes

* put interface in route poking for default route

* mnake it compile

* use correct route command on windows

* use fs::path for service::Identity::EnsureKeys
4 years ago
Jeff 11ed8924c6
use fs::path for service::Identity::EnsureKeys (#1364) 4 years ago
Jason Rhinelander d03066ee05
Support [dns] no-resolvconf option (ignored by lokinet) for deb compatibility (#1362)
* Add ignored [dns]no-resolvconf option for deb backwards compat

The debs hack a `#no-resolvconf=1` into the config file in 0.7 to allow
a user to disable resolvconf setup during startup by uncommenting it.
That doesn't work anymore since 0.8 errors on invalid config options, so
add it as an ignored option so that 0.7 deb config files can still be
compatible.

* Fix `[dns]upstream` comment not being produced in generated conf file

* fix test

Co-authored-by: Jeff <jeff@i2p.rocks>
4 years ago
Jeff c91e4df856
make errors with service keyfile throw (#1358) 4 years ago
Jeff fe30193a97
revive android build system (#1339)
* it lives?

* clean up

* add readme and add x86_64 to abi filters

* disable route poking on android

* make it compile on android

* it compiles!!111

* typofix

* re-enable ccache for android
4 years ago
Jeff 75ef28531e allow lns subdomains 4 years ago
Jeff a035dbdb9a enable peer stats by default 4 years ago
Jeff 78b40d6725 wire up lns gotnamemessage handler to kitchen sink 4 years ago
Jeff 21930cf667
LNS (#1342)
* initial relay side lns

* fix typo

* add reserved names and refactor test for dns

* lns name decryption

* all wired up (allegedly)

* refact to use service::EncryptedName for LNS responses to include nonce with ciphertext

* fully rwemove tag_lookup_job

* replace lns cache with DecayingHashTable

* check for lns name validity against the following rules:

* not localhost.loki, loki.loki, or snode.loki

* if it contains no dash then max 32 characters long, not including the .loki tld (and also assuming a leading subdomain has been stripped)

* These are from general DNS requirements, and also enforced in
registrations:

* Must be all [A-Za-z0-9-]. (A-Z will be lower-cased by the RPC call).

* cannot start or end with a -

* max 63 characters long if it does contain a dash

* cannot contain -- in the third and fourth characters unless it starts with xn--

* handle timeout in name lookup job by calling the right handler with std::nullopt
4 years ago
Jeff c64c9093f2
make endpoints reachable by default (#1344) 4 years ago
Jeff 681459185f
remove locks (#1336)
* remove locks

* use tryPushBack to attempt to prevent deadlocks
4 years ago
Jeff 3ab7db7723
macos route poking (#1333)
* fix up macos route poker logic

* fix typo

* use string_view

* add forgotten header

* full paths

* add debugging

* catch exception on adding route

* workarround for macos

* typofix

* typofix

* fix for macos

* fix command for macos

* because we autopoke remove explicit route poking in rpc

* probably final fix of macos route poking

* split routes instead of deleting them

* dynamic route poking

* move log statement for introset lookup and dont consider bad sessions as able to send

* send convotag reset frame when we have no session

* add exit map to rpc

* use split_any
4 years ago
Jeff 60f4d96ba5
proper route poking (#1330)
* route poking:

* remove popen() call, replace with reading /proc/net/route for getting default route
* dynamically poke and unpoke routes on runtime

* swap intros and fix rpc endpoint for version to return what the ui expects

* use std::string::find_first_not_of instead of using a lambda
4 years ago
Thomas Winget 2c6e7b86c3
SRV records fixes (#1332)
* fix a log print log level

* correctly match SRV record service and protocol...

* tests for new dns question functions
4 years ago
Thomas Winget b1c14af938
SRV Record handling for introsets (#1331)
* update loki-mq submodule for tuple support

* srv record reply implementation

still need to encode srv records into intro sets / router contacts
as well as decode from them and match against queried service.proto

* inverted condition fix in config code

* SRV record struct (de-)serialization for intro sets

* parsing and using srv records from config (for/in introsets)

* adopt str utils from core and use for srv parsing

* changes to repeat requests

no longer drop repeat requests on the floor, but do not make
an *actual* request for them if one is in progress.

do not call reply hook for each reply for a request, as
each userland request is actually made into several lokinet
requests and this would result in duplicate replies.

* fetch SRVs from introsets for .loki

* make format

* dns and srv fixes, srv appears to be working
4 years ago
jeff 1537ad18df format 4 years ago
jeff 75f550434c change assert to throw an exception instead when service node has no peer stats enable.
update unit tests so it passes on debian sid
4 years ago
jeff 52d20968a7 remove llarp::StrEq 4 years ago
jeff 5391e6a66a initial config overriding on runtime 4 years ago
jeff e3bfe76f98 add exit auth token awareness to config 4 years ago
Jeff Becker 1b4f5ee5ea
add forgotten win32 header 4 years ago
jeff 39f5e7213f set send timeout to 5 seconds for exits 4 years ago
Jeff Becker d39ed26ff0
add headers and fix type for windows 4 years ago
Jeff Becker 68e72d88fd
add forgotten header on macos 4 years ago
Jeff Becker d4e1a44452
fix typo 4 years ago
Jeff Becker 184503dab1
change to unsigned char for CI 4 years ago