Error out unit tests if DEBUG_PROXY enabled, because we debug print
line_num if DEBUG_PROXY enabled. We cannot support line_num in expected
debug output, it would make things complicated with very little benefit.
So that we can replace the SSL/TLS configuration of the conn with the
one in the matching filtering rule. Otherwise, once the server conn is
established, we cannot change the SSL config, or would risk confusing
the SSL routines.
Now all connection oriented proxy options possible to specify
per-proxyspec or globally can be specified in structured filtering rules
to be selectively applied to connections too. One line filtering rules
can specify filter and log actions only.
For example, we can enable/disable user authentication, protocol
validation, server ssl verification, and many other options
per-connection, or configure SSL/TLS connection options per-connection.
So, now we replace the conn_opts struct of a connection's ctx with the
conn_opts struct of the matching structured filtering rule. (One line
filtering rules have a NULL conn_opts, so we first check if the
conn_opts is not NULL.)
Now, all of the 'to' site fields in filtering rules can specify a port,
not just the dstip sites.
Fix the precedence of sites in the same type of rules. For example, if
we find a match with an sni site, we should not stop searching for a
match in cn, because a matching cn site may have a higher precedence
than the matching sni site. We should apply the action of the cn site,
although sni rules have precedence over cn. The same applies to http
host and uri rules too.
Fix the precedence of dstip rules.
Improve and update unit and e2e tests accordingly.
Now, the filter uses B-trees for exact string matching and Aho-Corasick
machines for substring matching. B-trees and AC machines are exported to
linked lists for debug logging only.
Also,
- Separate all_sites and all_ports filters from substring filters. They
are not related with substring filters actually, and ACM keywords cannot
be empty strings anyway. So now they should be handled separately too.
- Improve debug logging of filtering rules.
- Update unit tests accordingly, and improve.
- Fix pxyconn_filter(), keep searching for a match in substring filters
if exact match does not have a matching site rule.
- Increase common names max len and tokens. weather.gov has 73 tokens.
- Rename keyword to desc.
- Update documentation.
- Clean up.
So, for 'to' fields too, we use two separate data structures: binary
search trees (BST) for exact match and linked lists for substring match.
Now all 'from' and 'to' fields in filtering rules use these two data
structures.
To repeat, filtering rules should be written with exact matches instead
of substring matches, as much as possible. Because BST search must be
much faster than substring search over linked lists.
To repeat, we have modifed kbtree to support complex data structures in
from fields.
Also, update the unit tests accordingly.
Now the target IP address filters can use port specs too.
Refactor for code reuse, create filter_action struct used by rules,
sites, and ports.
Also, improve code and documentation.
Filtering rules can enable/disable or don't change logging. If a rule
does not mention a log action, its logging should not change. So, binary
log action fields were not enough to represent those 3 possibilities,
hence we have increased the size of those fields to 2-bits.
We should obey the order of rules as they are written in the conf file,
because latter rules should be able to override the log actions of
earlier rules. So, keep the order.
We should defer pass and/or block actions as long as possible, because a
higher precedence rule in SSL filter should be able to override (cancel)
deferred pass and block actions taken by a lower precedence rule in Dst
Host filter. And in HTTP filter the same applies to deferred block
actions taken by Dst Host and SSL filters.
Also, thanks to this new deferred actions, now HTTP filter can keep
enabled divert and split modes. In other words, a higher precedence HTTP
filter rule can cancel a deferred block action set by a lower precedence
rule earlier, which was not possible before without deferred actions and
rule precedence.
And other improvements.
Now filtering rules can disable log actions too. This is possible thanks
to the newly added precedence field of rules. Log actions of filtering
rules at higher precedence can modify logging now. In other words, more
specific rules can change the log actions of more general rules.
HTTP filtering rules can only disable logging.
Now we assign precedence to each filtering rule. More specific rules
have higher precedence. So, filtering rules at lower precedence cannot
override the actions applied to a conn by filtering rules at higher
precedence.
The other precedence rules still apply.
Log actions specified in HTTP filter rules can never enable disabled
logging, because their loggers would not be initialized.
Perhaps we should initialize them in the log submit function, if they
are initialized yet.
- Match action is added to be used with log actions only, the other
filter actions can specify log actions too
- Log actions do not configure any loggers. Global loggers for
respective log actions should have been configured for those log actions
to have any effect.
- If no filter rules are defined for a proxyspec, all log actions are
enabled. Otherwise, all log actions are disabled, and filtering rules
should enable them specifically.
- Fix max number of tokens in proxyspec and filter parsers
- Fix issues with rejecting unknown args in filter rule parser
- Do not use filter_rules field of proxyspec after config finished, it
is used for filter configuration and freed afterwards
- SSL and Dst Host filters can take all of the actions.
- HTTP filter can only take block action, not divert, split, or pass.
Because, we cannot tear a conn down and reconnect its src, after the
processing of HTTP request header is complete, e.g. SSLproxy line has
already been added to its dst buffer. Also, any change in child conns
would affect listening programs too.
- The precedence of filters is as Dst Host > SSL > HTTP.
- The precedence of actions is as Divert > Split > Pass > Block. This is
only for the same type of filter.
- The precedence of match sites is as sni > cn for ssl filter and host >
uri for http filter.
For example, pass action of dst host filter is taken before split action
of ssl filter, due to the precedence order of filters.
For example, pass action of sni site is taken before split action of cn,
due to the precedence order of sites.
We now create src ssl before enabling src to be able to take divert or
split actions of SSL filter. Otherwise, we wouldn't be able to switch
between divert and split while enabling src, only pass or block action
could be taken at that stage.
Also, refactor and clean up.
(Divert|Split|Pass|Block)
([from (
user (username|*) [desc keyword]|
ip (clientaddr|*)|
*)]
[to (
sni (servername[*]|*)|
cn (commonname[*]|*)|
host (host[*]|*)|
uri (uri[*]|*)|
ip (serveraddr|*)|
*)]
|*)
Also, fix a couple of issues with filter rule handling
Clean up
The -n command line option enables split mode for all proxyspecs,
effectively making sslproxy behave like sslsplit.
Divert option can be set/unset globally and per-proxyspec.
Add e2e tests for split mode, and update make file for tests
accordingly.
Update documentation accordingly.
Improve code reuse, remove duplicate functions.
This change deserves a release of its own, hence v0.8.4.
We don't do anything in srvdst writecb except for passhtrough mode.
We handle srvdst and dst connect tasks in connectcb for them by
arranging connect events correctly, so we don't need any extra flags.
Correct connect ordering helps us remove code checking if bev exists.
There were a lot of unnecessary code in autossl. Tcp and ssl code are
decoupled now.
Because we directly relay the packets from the server to the client
until we receive the first packet from the client, at which time we xfer
srvdst to the first child conn and effectively disable this readcb,
hence start diverting packets to the listening program.
Improve documentation.
Create function macros for fine* debug logs
Fix a few memory leaks when DEBUG_PROXY enabled
Add main.mk to MKFS list
Put a few function params within DEBUG_PROXY directives
Check retval of a snprintf() call
Fix segfault with -w/-W options if no ssl proxyspec specified, also fixed in sslsplit develop: https://github.com/droe/sslsplit/issues/271
Various clean-up
Soner Tari