Commit Graph

358 Commits (a79a5d2ea00a8e0939b20f2209509818cd19a029)

Author SHA1 Message Date
Thomas Winget 4c630e0437 Large collection of changes to make android work
- Previous android java and jni code updated to work, but with much love
  still needed to make it work nicely, e.g. handling when the VPN is
  turned off.

- DNS handling refactored to allow android to intercept and handle DNS
  requests as we can't set the system DNS to use a high port
  (and apparently Chrome ignores system DNS settings anyway)

- add packet router structure to allow separate handling of specific
  intercepted traffic, e.g. UDP traffic to port 53 gets handled by our
  DNS handler rather than being naively forwarded as exit traffic.

- For now, android lokinet is exit-only and hard-coded to use exit.loki
  as its exit.  The exit will be configurable before release, but
  allowing to not use exit-only mode is more of a challenge.

- some old gitignore remnants which were matching to things we don't
  want them to (and are no longer relevant) removed

- some minor changes to CI configuration
4 years ago
Jeff Becker a88aec5f01
fix typo 4 years ago
Jeff Becker a3585caa70
add value 0 to disable unique-range-size 4 years ago
Jeff Becker f7424b8bbe
dont use std::optional 4 years ago
Jeff Becker 3425069b41
feedback from jason
* use emplace in PeerSectionConfig::Acceptable use insert everywhere else
* use const l-value references
* fix typos and spelling mistakes
4 years ago
Jeff Becker 9457da27d9
add option to enforce unique netblocks per path. 4 years ago
Jeff Becker 063e982ef6
fix exit range default value 4 years ago
Jeff Becker 242e56507c
proper lokimq -> oxenmq refactor 4 years ago
Jeff Becker df4ea34a56
nodedb refactor
* bump zmq static dep
* lokimq -> oxenmq
* llarp_nodedb -> llarp::NodeDB
* remove all crufty api parts of NodeDB
* make NodeDB rc selection api not suck
* make path builder api not suck
* propagate all above changes so that unit tests work and it all compiles
4 years ago
Jeff Becker 5233b26552
format 4 years ago
Jeff Becker bf1a692005
key manager should throw on failures 4 years ago
Jeff 0e1e0aaef3
allow for relative file paths in config (#1432)
* llarp::Config constructor can take an empty fs::path now and it will default to fs::current_path
* llarp::ensureConfig now throws on fs error
* updates to daemon/main.cpp to reflect changes to llarp::ensureConfig's new throwability
* dowse parts of the code with holy water for the 0.8.1 tag
4 years ago
Jeff dcb48db5fe
enable profiling on clients by default (#1421)
* enable client relay profiling by default

* macos dns fixes

* improve peer profiling algorithm to track timeouts vs failures

* remove debug ioctl call in tuntap code

* use ub_wait instead of ub_process as that was what was there before

* const correctness

* DRY out checking for SIIT

* typofix

* correct name
4 years ago
Jeff 7a1207dc47
dont bind inbound links as client (#1419)
* dont bind inbound links as client

* fix broken test

* logic call MUST defer call
4 years ago
Jeff 12eb32a816
add conf.d directory for config overrides (#1410)
* add conf.d directory for config overrides

* refactor llarp::Config

* add explicit constructor with datadir as parameter
* have all config files be passed as std::optional
* make Config::LoadDefault private and use std::optional in Config::Load to remove ambiguity
* update rest of codebase to reflect above changes

* fix pybind

* rename bootstrap config skipBootstrap to seednode as it's more descriptive
* make seednode configurable
* make pybind layer compile
* make pybind layer run
4 years ago
Jeff 22acf0a537
Lns exits (#1407)
* lns exits

* try appeasing the clang

* clean up lambda
4 years ago
Jeff 98e022ea21
Unflaky-ify lns dns (#1406)
* always ensure path to service on dns lookup

* deprecate profiles option
4 years ago
Jeff 3b70b99dd2
fix empty config case (#1400)
* fix empty config case

* * fix case for empty ifname / ifaddr on relay
* bail if no dns server bound

* use AssignmentAcceptor
4 years ago
Jason Rhinelander af6caf776a
Config file improvements (#1397)
* Config file API/comment improvements

API improvements:
=================

Make the config API use position-independent tag parameters (Required,
Default{123}, MultiValue) rather than a sequence of bools with
overloads.  For example, instead of:

    conf.defineOption<int>("a", "b", false, true, 123, [] { ... });

you now write:

    conf.defineOption<int>("a", "b", MultiValue, Default{123}, [] { ... });

The tags are:
- Required
- MultiValue
- Default{value}
plus new abilities (see below):
- Hidden
- RelayOnly
- ClientOnly
- Comment{"line1", "line2", "line3"}

Made option definition more powerful:
=====================================

- `Hidden` allows you to define an option that won't show up in the
  generated config file if it isn't set.

- `RelayOnly`/`ClientOnly` sets up an option that is only accepted and
  only shows up for relay or client configs.  (If neither is specified
  the option shows up in both modes).

- `Comment{...}` lets the option comments be specified as part of the
  defineOption.

Comment improvements
====================

- Rewrote comments for various options to expand on details.
- Inlined all the comments with the option definitions.
- Several options that were missing comments got comments added.
- Made various options for deprecated and or internal options hidden by
  default so that they don't show up in a default config file.
- show the section comment (but not option comments) *after* the
  [section] tag instead of before it as it makes more sense that way
  (particularly for the [bind] section which has a new long comment to
  describe how it works).

Disable profiling by default
============================

We had this weird state where we use and store profiling by default but
never *load* it when starting up.  This commit makes us just not use
profiling at all unless explicitly enabled.

Other misc changes:
===================

- change default worker threads to 0 (= num cpus) instead of 1, and fix
  it to allow 0.
- Actually apply worker-threads option
- fixed default data-dir value erroneously having quotes around it
- reordered ifname/ifaddr/mapaddr (was previously mapaddr/ifaddr/ifname)
  as mapaddr is a sort of specialization of ifaddr and so makes more
  sense to come after it (particularly because it now references ifaddr
  in its help message).
- removed peer-stats option (since we always require it for relays and
  never use it for clients)
- removed router profiles filename option (this doesn't need to be
  configurable)
- removed defunct `service-node-seed` option
- Change default logging output file to "" (which means stdout), and
  also made "-" work for stdout.

* Router hive compilation fixes

* Comments for SNApp SRV settings in ini file

* Add extra blank line after section comments

* Better deprecated option handling

Allow {client,relay}-only options in {relay,client} configs to be
specified as implicitly deprecated options: they warn, and don't set
anything.

Add an explicit `Deprecated` tag and move deprecated option handling
into definition.cpp.

* Move backwards compat options into section definitions

Keep the "addBackwardsCompatibleConfigOptions" only for options in
sections that no longer exist.

* Fix INI parsing issues & C++17-ify

- don't allow inline comments because it seems they aren't allowed in
ini formats in general, and is going to cause problems if there is a
comment character in a value (e.g. an exit auth string).  Additionally
it was breaking on a line such as:

    # some comment; see?

because it was treating only `; see?` as the comment and then producing
an error message about the rest of the line being invalid.

- make section parsing stricter: the `[` and `]` have to be at the
beginning at end of the line now (after stripping whitespace).

- Move whitespace stripping to the top since everything in here does it.

- chop off string_view suffix/prefix rather than maintaining position
values

- fix potential infinite loop/segfault when given a line such as `]foo[`

* Make config parsing failure fatal

Load() LogError's and returns false on failure, so we weren't aborting
on config file errors.

* Formatting: allow `{}` for empty functions/structs

Instead of using two lines when empty:

    {
    }

* Make default dns bind 127.0.0.1 on non-Linux

* Don't show empty section; fix tests

We can conceivably have sections that only make sense for clients or
relays, and so want to completely omit that section if we have no
options for the type of config being generated.

Also fixes missing empty lines between tests.

Co-authored-by: Thomas Winget <tewinget@gmail.com>
4 years ago
Jason Rhinelander 753d989f7c Set default inbound link to best public interface
Fixes default config not working.
4 years ago
Jason Rhinelander 7aa4566016 Make format 4 years ago
Jason Rhinelander 01013c1963 Make upstream= override work; reject non-default upstream port
If you specify upstream= then you get no upstream, if you give one then
you use that instead of the default, but you can still list multiple.

unbound doesn't support an upstream port, so bail if the user gives a
non-port-53 response.
4 years ago
Jason Rhinelander d129b0432a Allow empty values so that upstream= can disable the default 4 years ago
Jason Rhinelander c97b9ef31b Simplify k=v parsing code 4 years ago
Jeff 9d6dc40f81 format 4 years ago
Jeff 2b5b3ce8e1 default upstream dns to cloudflare 4 years ago
Jeff 0f7e848903
disable rpc on relays by default (#1383)
* disable rpc on relays by default

* add default inbound link as service node

* throw if public-ip and public-port are not set
4 years ago
Jason Rhinelander fbfa8ca89c
make format (#1381) 4 years ago
Jason Rhinelander b121933374
Make public-ip option work (#1380)
0.7 supported both public-address and public-ip, but only the former was
being allowed in 0.8.

This makes `public-ip=...` work again, and makes it the canonical option
(because it makes a bit more sense than address when we also have the
public-port parameter), while making `public-address=` accepted with a
deprecation warning.

Also adds default descriptions for port and ip for the generated ini.
4 years ago
Jeff 7852735b36 tweak configuration defaults 4 years ago
Jeff 8943be6c3a
serialize auth codes in rpc (#1369)
* serialize auth codes in rpc

* format

* format

* config and route fixups

* bump lokimq

* fix compile

* discard unknown members in service keys

* format
4 years ago
Jason Rhinelander d03066ee05
Support [dns] no-resolvconf option (ignored by lokinet) for deb compatibility (#1362)
* Add ignored [dns]no-resolvconf option for deb backwards compat

The debs hack a `#no-resolvconf=1` into the config file in 0.7 to allow
a user to disable resolvconf setup during startup by uncommenting it.
That doesn't work anymore since 0.8 errors on invalid config options, so
add it as an ignored option so that 0.7 deb config files can still be
compatible.

* Fix `[dns]upstream` comment not being produced in generated conf file

* fix test

Co-authored-by: Jeff <jeff@i2p.rocks>
4 years ago
Jeff a035dbdb9a enable peer stats by default 4 years ago
Jeff 78b40d6725 wire up lns gotnamemessage handler to kitchen sink 4 years ago
Jeff c64c9093f2
make endpoints reachable by default (#1344) 4 years ago
Thomas Winget b1c14af938
SRV Record handling for introsets (#1331)
* update loki-mq submodule for tuple support

* srv record reply implementation

still need to encode srv records into intro sets / router contacts
as well as decode from them and match against queried service.proto

* inverted condition fix in config code

* SRV record struct (de-)serialization for intro sets

* parsing and using srv records from config (for/in introsets)

* adopt str utils from core and use for srv parsing

* changes to repeat requests

no longer drop repeat requests on the floor, but do not make
an *actual* request for them if one is in progress.

do not call reply hook for each reply for a request, as
each userland request is actually made into several lokinet
requests and this would result in duplicate replies.

* fetch SRVs from introsets for .loki

* make format

* dns and srv fixes, srv appears to be working
4 years ago
jeff 5391e6a66a initial config overriding on runtime 4 years ago
jeff e3bfe76f98 add exit auth token awareness to config 4 years ago
Jeff Becker d8dcffe80a
resolve conflict with dev 4 years ago
jeff 30a6e901f5
add router.enable-peer-stats config option unconditionally 4 years ago
Jeff Becker ff130e2b5e
die if we are a relay and don't have any inbound links 4 years ago
Jeff Becker 8c3621cead
fix issue #1328 4 years ago
Jeff Becker 301cb6d32f
fix issue #1320 (allow providing ip/port in bind section of config) 4 years ago
Stephen Shelton b0d8568452
Remove llarp C API usage from RouterHive 4 years ago
Stephen Shelton 023e061146
Make [router]:enable-peer-stats only valid for client 4 years ago
Stephen Shelton 2a30e7dac2
Add PeerDb to Router 4 years ago
Jeff Becker d85c7bc628
remove wrapper and just use a lambda that throws on non empty value 4 years ago
Jeff Becker a0dc2e795c
add a wrapper that always throws with a custom error message
use that wrapper for lokid.jsonrpc
4 years ago
Jeff Becker a9109aa91b
* deprecate old unused lokid rpc options
* use lokimq::address in config and in lokid rpc
* bump lokimq submodule
4 years ago
Jeff Becker 80919a3b76
address feedback.
* use exceptions when fetching identity key instead of std::optional, will throw on fail
* fix up config options for endpoint auth and add better docs
* add llarp::serive::AuthType enum for controlling what kind of endpoint auth to use
4 years ago
Jeff Becker 0f21eeccb0
* rework exit codepath to allow multiple exits
* rework net code for ip ranges to be cleaner
* clean up endpoint auth code
* refactor config to validate network configs before setting up endpoints
* remove buildone from path/pathbuilder.cpp so we don't spam connection attempts
4 years ago
Jeff Becker 0d6e3bcd89
fix crashing tests 4 years ago
Jeff Becker e95b9d530e
fix unit tests, make them pass 4 years ago
Jeff Becker 5abf111159
endpoint auth whitelist 4 years ago
Jeff Becker a42d3d51c3
more 4 years ago
Jeff Becker 749c5a813c
require at least 2 hops because otherwise stuff wont publish 4 years ago
Jeff Becker ad882d0d70
initial working code 4 years ago
Stephen Shelton daecd6eda5
Remove llarp_config and llarp_main 4 years ago
Jeff Becker 800668348a
add regression test for key backup bug 4 years ago
Stephen Shelton cecafb3215
Undo unintentional revert? 4 years ago
Stephen Shelton d8b12ce764
Let KeyManager assemble key filenames 4 years ago
Stephen Shelton 1497b829bd
Ignore our self.signed file if we're a client 4 years ago
Jeff 45cda241f0
Merge pull request #1272 from majestrate/exit-nodes-2020-05-16
exit traffic via snapps
4 years ago
Rick V 1340cd0dce remove some string conversions entirely 4 years ago
Rick V 5529371637 RouterContact::[Read|Write] now take a fs::path const ref 4 years ago
Rick V f62214cf8c
first pass fix wstring path conversions 4 years ago
Jeff Becker 3a776b3ed1
bounds checks and such 4 years ago
Jeff Becker 6af498092b
exit traffic via loki addresses 4 years ago
Jason Rhinelander 854619f04f Workaround for ghc::filesystem (mac) compatibility 4 years ago
Jason Rhinelander d05e6716cb Remove llarp::str(string_view)
It was a workaround for pre-C++17 std::string which didn't support
passing a string_view to various functions/operators.  There's only one
place left that needs an explicit conversion, and that's where it is
used as a map key; so just be explicit there and remove llarp::str()
everywhere else.
4 years ago
Jason Rhinelander ebd2142114 Don't use std::optional::value() because f u macos
This replaces all use of std::optional's `opt.value()` with `*opt`
because macOS is great and the ghost of Steve Jobs says that actually
supporting std::optional's value() method is not for chumps before macOS
10.14.  So don't use it because Apple is great.

Pretty much all of our use of it actually is done better with operator*
anyway (since operator* doesn't do a check that the optional has a
value).

Also replaced *most* of the `has_value()` calls with direct bool
context, except for one in the config section which looked really
confusing at a glance without a has_value().
4 years ago
Jason Rhinelander be9ddf2ae1 Bring back ghc::filesystem for broke AF macos
macOS doesn't provide `<filesystem>` support when targetting anything
earlier than 10.15.
4 years ago
Stephen Shelton ffdeca5378
Initialize all primitive values in Config 4 years ago
Stephen Shelton ac225eb65e
Testing: explicitly initialize variable 4 years ago
Jason Rhinelander 28362fd6a6 Clean up curl define
It doesn't have to be win32-specific, we could (for example) use this on
mac as well to build without curl/SN support.
4 years ago
Jason Rhinelander 26ecf23ad8 Silence various warnings 4 years ago
Jeff ad9edc4ecf
Merge pull request #1266 from majestrate/issue-1264-fix-setting-upstream-dns-2020-05-14
default to port 53 in dns section entries
4 years ago
Stephen Shelton ed31cdaacf
Cosmetic fixes 4 years ago
Jeff Becker 09a356a8a4
dont use static local lambda 4 years ago
Jeff Becker ebed21c392
DRY 4 years ago
Jeff Becker 3753dfbddb
IpAddresses in dns section config were not defaulting to port 53 when no port was provided.
Fixes #1264
4 years ago
Jeff a5b09c47e1
Merge pull request #1262 from notlesh/bandaids-for-sockaddr-refactor-2020-05-11
Bandaids for sockaddr refactor 2020 05 11
4 years ago
Stephen Shelton aee96e53a3
Refactor Addr -> IpAddress/SockAddr 4 years ago
Jeff Becker 635dc4fe13
unbreak router hive
llarp/config/config.cpp:
respect [network]:type option

llarp/handlers/exit.cpp:
when [network]:type is null dont init tun interface

llarp/service/context.cpp:
respect [network]:type option
change endpoint name back to "default"

llarp/tooling/router_hive.cpp:
dont use LogicCall for obtaining RCs from underlying relays, it crashes the mainloop and it's probably safe to readonly access RCs.

pybind/common.hpp:
remove typecasters as we use C++17 now

pybind/llarp/config.cpp:
remove SnappConfig
wire up NetworkConfig

pybind/llarp/handlers/pyhandler.hpp:
remove SnappConfig from constructor

pybind/llarp/handlers/pyhandler.cpp:
update constructor implementation to match header

test/hive/hive.py:
remove broke endpoint related code
wire up null endpoint option using NetworkConfig
use index at 0 for relays and clients instead of 1
dont add a python endpoint to all clients
4 years ago
Jeff Becker f1edca9fa1
additional cleanup of cruft 4 years ago
Jeff Becker c4b5b641bb
default to default data dir given by params instead 4 years ago
Jeff Becker e33a1a2e0f
proper autodetect of ifname and ifaddr 4 years ago
Jason Rhinelander 4f09090c58 Add [network]:enabled to ignored settings list
No one knows what this option does or did, but it is in some generated
config files.
4 years ago
Jason Rhinelander 1d98632fd3
Merge pull request #1254 from majestrate/network-ifaddr-ifname-2020-05-04
add ifname and ifaddr to network config parsing
4 years ago
Jeff Becker 96a6d81380
add ifname and ifaddr to network config parsing 4 years ago
Jason Rhinelander 4be7b2c8a5
Merge pull request #1253 from notlesh/better-config-error-messages-2020-05-04
Use more user-friendly error messages in config
4 years ago
Stephen Shelton f96fded525
Use more user-friendly error messages in config 4 years ago
Stephen Shelton 78eb0d976d
nonstd::optional -> std::optional 4 years ago
Jeff Becker 7c12347e80
upstream-dns -> upstream 4 years ago
Stephen Shelton 42a7bcd434
Less invasive fix for Addr 4 years ago
Stephen Shelton 15918ff1c9
log-- 4 years ago
Stephen Shelton f2a26adcaa
Move all [endpoint] options to [network] 4 years ago
Stephen Shelton 559610ec94
Properly handle empty config default values 4 years ago
Stephen Shelton dd9ab0f1d5
Remove ability to have multiple endpoints/snodes from config 4 years ago
Stephen Shelton b7451b7632
Rip out tag, prefetch tags/addrs from Endpoint 4 years ago
Stephen Shelton 1d12e9d14e
Lokinet_INIT() is already checked in main(), so remove from config 4 years ago
Stephen Shelton 3198a8b139
Handle blacklist-snode validation in config 4 years ago
Stephen Shelton 2730491641
Remove NetworkConfig's freehand options 4 years ago
Stephen Shelton 2bc908cd40
Remove [dns]:local-dns in favor of [dns]:bind 4 years ago
Stephen Shelton 779658edd0
Refactor ExitEndpoint's config, mv dns ops from [network] to [dns] 4 years ago
Stephen Shelton 21ad442b55
Remove NetConfig_t 4 years ago
Stephen Shelton df01770466
Rename SnappConfig -> EndpointConfig 4 years ago
Stephen Shelton 93b35c92a7
Eradicate service::Config 4 years ago
Stephen Shelton cbd93fcb09
Find "-snapp" conf sections, add more snapp conf options 4 years ago
Jason Rhinelander 3efaf9ff76 Add string header (reportedly needed for libc++) 4 years ago
Jason Rhinelander 5cf2126bb9 make format 4 years ago
Jason Rhinelander 1697bf90fe C++17
Compiles with C++17, replaces ghc::filesystem with std::filesystem,
nonstd::optional with std::optional, and llarp::string_view with
std::string_view.
4 years ago
Stephen Shelton f0f9b1abdd
Allow 0 as outbound port 4 years ago
Stephen Shelton f8439dab61
Misc aesthetic tweaks 4 years ago
Stephen Shelton 7d83611a0a
Default outbound port to 0, not 1090 4 years ago
Stephen Shelton c4de0f8493
Add more ignored config options 4 years ago
Stephen Shelton 517922e4c3
Reintroduce support for configurable key files 4 years ago
Stephen Shelton 320564d792
Specialize ConfigOption for bool to accept "truthy" / "falsy" values 4 years ago
Stephen Shelton 331770b348
Add ability to tolerate (ignore) old config options
This allows us to explicitly ignore (with a warning) old options
that are no longer supported.
4 years ago
Stephen Shelton cdd4439cff
Temporary fix for [dns]bind and [dns]local-dns 4 years ago
Stephen Shelton 936fbb2424
Fix config not falling back to undeclared handler for missing option 4 years ago
Stephen Shelton 526b1320b7
Some maybe-fixes for RouterHive post config cleanup 4 years ago
Stephen Shelton dfcf8fb62e
Update RouterHive to reflect config changes 5 years ago
Stephen Shelton de8e44ba21
Re-apply clang-format rules after rebasing 5 years ago
Stephen Shelton a66f502ed6
Remove [netdb] conf and place it under data-dir
This does three things:

1) Remove the [netdb] conf section,
2) Rename the subdir 'netdb' -> 'nodedb'
3) Place 'nodedb' under 'data-dir'
5 years ago
Stephen Shelton be014175e9
Consolidate logging initialization logic 5 years ago
Stephen Shelton 6909e20588
Fix logging initialization and flush at program exit 5 years ago
Stephen Shelton d3bcc05aa6
Organize some constants and default values 5 years ago
Stephen Shelton da6bda15a1
Where did rc.signed come from?! 5 years ago
Stephen Shelton 7ea8d62640
Use fs::path over std::string for files 5 years ago
Stephen Shelton 7ce256d846
Prefer fs::path over std::string @ KeyManager 5 years ago
Stephen Shelton 0a9515a94a
Proper support for multiple values @ ConfigDefinition 5 years ago
Stephen Shelton 28c1ca9c7a
Handle missing IP address in conf properly 5 years ago
Stephen Shelton 0697d57190
Important TODO 5 years ago
Stephen Shelton 0e5e9a1222
Silently ignore config file if it exists already 5 years ago
Stephen Shelton 1fc7c61d1f
Share common conf comments for client and relay 5 years ago
Stephen Shelton 3739fc1d70
Reinstate multi-valued option 5 years ago
Stephen Shelton a6787657be
Refactor config comments to take list of strings 5 years ago
Stephen Shelton a8671cf9c7
Rename config classes for clarity
ConfigDefinition -> OptionDefiniton
Configuration -> ConfigDefinition
5 years ago
Stephen Shelton f250b7adcb
Remove pidfile support 5 years ago
Stephen Shelton 4f77080f75
Remove optional<> from block-bogons 5 years ago
Stephen Shelton 4d053231d3
Define config defaults as constexpr 5 years ago
Stephen Shelton adfcbd1d0b
Slight cleanup of k:v option maps 5 years ago
Stephen Shelton e06c2e4502
Check that [router]min-connections <= [router]max 5 years ago
Stephen Shelton 6fd270ba09
Rip out link type spec and factory
We only support IWP now, so this is all effectively dead code.
5 years ago
Stephen Shelton 9e850705b4
Add 'AssignmentAcceptor' convenience for simple config acceptors 5 years ago
Stephen Shelton f2a8dd10ce
Modernize Snapp config 5 years ago
Stephen Shelton 733efbab40
Reflect removal of accessors on config structs 5 years ago
Stephen Shelton c8c940b0f5
Replace individual file locations with dataDir 5 years ago
Stephen Shelton 5d0b53e1be
Remove environment variable support in config 5 years ago