Commit Graph

59 Commits (master)

Author SHA1 Message Date
Soner Tari dfb783d7ba Release v0.9.5
Bump version to 0.9.5
Update copyright year to 2024
Update NEWS
4 months ago
Soner Tari 01d0b156d6 Decouple autossl code handling underlying bufs
And clean up redundant child callbacks
2 years ago
Soner Tari 63a48308cd Fix autossl without STARTTLS in divert mode
In the previous implementation, the use case for autossl was assumed to
be STARTTLS with POP3 or SMTP. But there are users who use autossl with
HTTP too. The split mode was fine, but the divert mode was broken. This
change makes autossl a generic upgrade mechanism.
Also fix sslproxy line in autossl, change p to s if upgraded.
Add e2e tests for autossl in divert and split mode.
2 years ago
Soner Tari e43564635a Add WUNRES and NONNULL attribs to function types with return values
Otherwise gcc does not issue warnings for them.
2 years ago
Soner Tari 76ea48f2d0 Remove srvdst_xferred flag
Setting srvdst.bev to NULL can be used as a flag to indicate that we
have reused the srvdst of the parent as the dst of the first child conn.
This also prevents access to srvdst.bev after we do the xfer, without
any extra flag.
2 years ago
Soner Tari 02a6cc12e6 Remove redundant srvdst_xferred flag in child ctx 2 years ago
Soner Tari dc40f74c13 Update copyright year to 2022 2 years ago
Soner Tari 2e8e677bac Discard underlying bufs while closing, sending errors, and denying ocsp
Also refactor for code reuse.
Since we are closing in all such cases, performance is not important.
2 years ago
Soner Tari 7143102efa Create callback functions for set/unset watermark
Watermarking for underlying bevs is for autossl only. The other
protocols should not waste time with it.
2 years ago
Soner Tari 59c28f6ad9 Fix enabling pcap and mirror logging, issue #37
Reported by @Leonschmitt
3 years ago
Soner Tari 775ae774ea Set conn term flag only, do not free conn in eventcb
Rename reconnected_ssl flag to reconnected
Improve e2e tests
3 years ago
Soner Tari 8f63ec7f82 Add ReconnectSSL option to enforce SSL options in struct filtering rules
The ReconnectSSL option allows rule developers to write struct filtering
rules using SNI and CN SSL specifications to override the SSL
configuration of a connection.

Otherwise, without this new option, filtering rules cannot change SSL
options using SSL filtering fields to match connections (the SSL config
in the rule would not have any effect on the server side of the matching
connection). Without ReconnectSSL, only DstIP and DstPort fields can be
used to override the SSL config of a connection.

If the ReconnectSSL option in a struct filtering rule is set, we
disconnect and free the server side of the matching SSL connection, and
reconnect it with the SSL options in the matching struct filtering rule.
This enforces the SSL config in the rule.

Do not use the ReconnectSSL option if server disconnect is not desirable
or acceptable in your case.
3 years ago
Soner Tari e8f35ce587 Reapply dsthost filter after user auth
We have to apply the DstHost filter both (1) as early as in
pxy_conn_connect() and also (2) after user owner of the conn is
determined in srvdst connected callback functions for tcp and ssl.

Otherwise, we cannot override SSL options of conns if we don't apply it
before SSL establishment (1), and we cannot apply user auth filtering
rules if we don't apply after determining the user owner of conn (2).

This commit actually adds the same calls in the same places as they were
before the structured filtering rules were introduced.

So for example, now we have to apply filters 4x for an HTTPS conn: 2x
dsthost, 1x ssl, and 1x http.
3 years ago
Soner Tari 05d5412515 Fix warning for unused ctx debug param in pxy_conn_set_filter_action() 3 years ago
Soner Tari 18fb6f4dd7 Apply dstip filter before ssl server conn setup
So that we can replace the SSL/TLS configuration of the conn with the
one in the matching filtering rule. Otherwise, once the server conn is
established, we cannot change the SSL config, or would risk confusing
the SSL routines.
3 years ago
Soner Tari 6c988b0f4a Add structured filtering rules to specify conn options
Now all connection oriented proxy options possible to specify
per-proxyspec or globally can be specified in structured filtering rules
to be selectively applied to connections too. One line filtering rules
can specify filter and log actions only.

For example, we can enable/disable user authentication, protocol
validation, server ssl verification, and many other options
per-connection, or configure SSL/TLS connection options per-connection.

So, now we replace the conn_opts struct of a connection's ctx with the
conn_opts struct of the matching structured filtering rule. (One line
filtering rules have a NULL conn_opts, so we first check if the
conn_opts is not NULL.)
3 years ago
Soner Tari 14c8d417c9 Move connection oriented options to a new conn_opts struct 3 years ago
Soner Tari dff3f90c62 Add DEBUG_PROXY around debug func params 3 years ago
Soner Tari f056f699c1 Add port option to all site specs, fix precedences in filtering rules
Now, all of the 'to' site fields in filtering rules can specify a port,
not just the dstip sites.

Fix the precedence of sites in the same type of rules. For example, if
we find a match with an sni site, we should not stop searching for a
match in cn, because a matching cn site may have a higher precedence
than the matching sni site. We should apply the action of the cn site,
although sni rules have precedence over cn. The same applies to http
host and uri rules too.

Fix the precedence of dstip rules.

Improve and update unit and e2e tests accordingly.
3 years ago
Soner Tari 4f36a21c78 Use kbtree BST for exact match in site and port 'to' fields
So, for 'to' fields too, we use two separate data structures: binary
search trees (BST) for exact match and linked lists for substring match.

Now all 'from' and 'to' fields in filtering rules use these two data
structures.

To repeat, filtering rules should be written with exact matches instead
of substring matches, as much as possible. Because BST search must be
much faster than substring search over linked lists.

To repeat, we have modifed kbtree to support complex data structures in
from fields.

Also, update the unit tests accordingly.
3 years ago
Soner Tari e844f30886 Convert warning and info logs in filtering rules to fine debug logs 3 years ago
Soner Tari fd6c852355 Move filtering rules to filter.c/h
Also, fix certain and possible memory leaks in debug printing
And improve code
3 years ago
Soner Tari c8f09d162a Add port field to Dst Host filter rules, and refactor for code reuse
Now the target IP address filters can use port specs too.
Refactor for code reuse, create filter_action struct used by rules,
sites, and ports.
Also, improve code and documentation.
3 years ago
Soner Tari 39e1d87783 Fix deferred pass filter action in SSL filter
Do not call protopassthrough_engage() twice. If we call it to apply the
deferred pass action and also set the ctx->pass flag, it will be called
again in protossl_setup_src_ssl(). So, just set the flag and leave the
rest to protossl_setup_src_ssl().
3 years ago
Soner Tari 446cc29f5f Don't change log action if not specified, and obey the order of filtering rules
Filtering rules can enable/disable or don't change logging. If a rule
does not mention a log action, its logging should not change. So, binary
log action fields were not enough to represent those 3 possibilities,
hence we have increased the size of those fields to 2-bits.

We should obey the order of rules as they are written in the conf file,
because latter rules should be able to override the log actions of
earlier rules. So, keep the order.
3 years ago
Soner Tari ac3607a841 Add deferred pass and block actions
We should defer pass and/or block actions as long as possible, because a
higher precedence rule in SSL filter should be able to override (cancel)
deferred pass and block actions taken by a lower precedence rule in Dst
Host filter. And in HTTP filter the same applies to deferred block
actions taken by Dst Host and SSL filters.

Also, thanks to this new deferred actions, now HTTP filter can keep
enabled divert and split modes. In other words, a higher precedence HTTP
filter rule can cancel a deferred block action set by a lower precedence
rule earlier, which was not possible before without deferred actions and
rule precedence.

And other improvements.
3 years ago
Soner Tari cc7bd4a332 Add precedence to filtering rules
Now we assign precedence to each filtering rule. More specific rules
have higher precedence. So, filtering rules at lower precedence cannot
override the actions applied to a conn by filtering rules at higher
precedence.
The other precedence rules still apply.
3 years ago
Soner Tari 8ec97d779f Do not take log actions in HTTP filtering rules
Log actions specified in HTTP filter rules can never enable disabled
logging, because their loggers would not be initialized.

Perhaps we should initialize them in the log submit function, if they
are initialized yet.
3 years ago
Soner Tari 8a57d52f62 Add master and cert log actions
Also, improve documentation
3 years ago
Soner Tari f0c2ca6819 Add Match action and connect|content|pcap|mirror log actions in filtering rules
- Match action is added to be used with log actions only, the other
filter actions can specify log actions too
- Log actions do not configure any loggers. Global loggers for
respective log actions should have been configured for those log actions
to have any effect.
- If no filter rules are defined for a proxyspec, all log actions are
enabled. Otherwise, all log actions are disabled, and filtering rules
should enable them specifically.
- Fix max number of tokens in proxyspec and filter parsers
- Fix issues with rejecting unknown args in filter rule parser
- Do not use filter_rules field of proxyspec after config finished, it
is used for filter configuration and freed afterwards
3 years ago
Soner Tari 99c852972e Implement filter actions
- SSL and Dst Host filters can take all of the actions.

- HTTP filter can only take block action, not divert, split, or pass.
Because, we cannot tear a conn down and reconnect its src, after the
processing of HTTP request header is complete, e.g. SSLproxy line has
already been added to its dst buffer. Also, any change in child conns
would affect listening programs too.

- The precedence of filters is as Dst Host > SSL > HTTP.

- The precedence of actions is as Divert > Split > Pass > Block. This is
only for the same type of filter.

- The precedence of match sites is as sni > cn for ssl filter and host >
uri for http filter.

For example, pass action of dst host filter is taken before split action
of ssl filter, due to the precedence order of filters.

For example, pass action of sni site is taken before split action of cn,
due to the precedence order of sites.

We now create src ssl before enabling src to be able to take divert or
split actions of SSL filter. Otherwise, we wouldn't be able to switch
between divert and split while enabling src, only pass or block action
could be taken at that stage.

Also, refactor and clean up.
3 years ago
Soner Tari 85fb1bd214 Add new data structure for general filtering rules
Now PassSite rules are considered as pass rule and added to the new data
structure. PassSite option will be deprecated in the future.
3 years ago
Soner Tari f2d4ef61c9 Add support for passsite substring match
Now the site field in PassSite option can have an '*' suffix to search
for a match anywhere in sni or common names. Note that this is not a
regex or wildcard search.

Previously, we only supported exact matches in sni and between slashes
in common names. This change makes it possible to cover multiple sites
in one PassSite option. In fact, without this change, certain sites
could not be added as passsite, because it was impossible to know their
subdomain names beforehand, for example *.fbcdn.net, which may have many
subdomain names in place of asterisk.

So to use substring match, append an '*' to a site name in PassSite
option (the asterisk is removed before substring search). For example,
use ".fbcdn.net*" to match all subdomains of fbcdn.net, notice the
asterisk at the end.

We also add a warning log starting with "Closing on ssl error without
passsite match" to report sites that can be added as passsite, which is
expected to help in writing PassSite rules.

Also, we now set dstaddr_str earlier in conn handling, so we can print
it in debug logs. This also helps in IDLE and EXPIRED conn logs.
3 years ago
Soner Tari 45b34678de Prepend sslproxy line using evbuffer_add_printf() in non-debug mode for non-http protos
This prevents unnecessary malloc and memmove calls in non-debug mode.
This change is for correctness not for speed, because it improves
conn handling only of the first packet and for non-http protos.
3 years ago
Soner Tari 9123732739 Fix possible segfaults with srvdst_xferred, autossl and passthrough
- Fix segfault introduced in previous commit to prevent extra eof event.
We should NULL srvdst.bev after terminating child dst xferred from
srvdst of parent, so that we don't try to access srvdst.bev. This
happens if child conn with dst xferred from parent srvdst is terminated
before parent conn.

- Fix autossl crash trying to engage passthrough mode. We cannot engage
passthrough mode in autossl, because src is already enabled. But we
shouldn't crash either. These changes are expected to fix other possible
segfaults if passthrough is engaged on eventcb of a child conn.
3 years ago
Soner Tari 7ed4748d96 Fix clang warning due to -Wpointer-bool-conversion, remove NONNULL for list
pxyconn.c:1590:9: warning: comparison of nonnull parameter 'list' not
equal to a null pointer is 'true' on first encounter
3 years ago
Soner Tari 596aebb2f3 Update version to 0.8.3 and copyright year to 2021 3 years ago
Soner Tari 65b7fb9847 Reduce code for user lists 4 years ago
Soner Tari 80d10a94c3 Move classify_user into identify_user
Otherwise, we cannot classify user if we need to issue identify_user
events, in case database is busy or locked. We should call classify_user
callback right after the user is identified.
So we introduce classify_user callback to achieve that, which fixes the
classify_user behavior for autssl proto too.

Return void in pxy_userauth
Fix typo in clasify
4 years ago
Soner Tari 4f3ce763ac Add DivertUsers and PassUsers options
Update documentation
4 years ago
Soner Tari 6f5a7ceeb1 Add WITHOUT_USERAUTH switch 4 years ago
Soner Tari ac4285cef1 Fix POP3 and SMTP protocol validation, thanks to the new testproxy e2e tests
Add testproxy e2e tests for POP3 and SMTP protocol validation.

We have detected that POP3 and SMTP protocol validation was broken
thanks to these new testproxy e2e tests. This is yet another example why
e2e tests are important.
4 years ago
Soner Tari f3ac5ee4f2 Move passsite flag to sslctx
The passsite flag is ssl specific.
4 years ago
Soner Tari aba07a53ee Disable conn ids unless debugging
We don't need parent or child ids unless debugging. IDLE and EXPIRED
conn logs do not need to report ids either. Ids are useful only in
detailed debug logs.
4 years ago
Soner Tari 11f92e3ce8 Add unique child id, set to the children count of parent conn
This is necessary to uniquely identify child conns. The src fd of child
conns was possibly not unique. We use this id in debug logs only.
Also relocate the update code related with this id.
4 years ago
Soner Tari 757ed35687 Do not use privsep to open socket for child listener
We don't need a privsep call to open a socket for child listener,
because listener port of child conns are assigned by the system, hence
are from non-privileged range above 1024.
So the open privsep socket is used only to update user atime now.
4 years ago
Soner Tari d3c3059c26 Remove ctx evbase and dnsbase
We already have thr in conn ctx to access its evbase and dnsbase.
4 years ago
Soner Tari 8786b9fda7 Remove pending ssl conns list and in_thr_conns flag
We have carried almost all conn init tasks from thrmgr to conn handling
thread. So we immediately add the conn to the conn list of its thr,
which renders both pending ssl conns list and in_thr_conns flag useless.
4 years ago
Soner Tari c8371e7b3d Decouple thrmgr and conn handling thr on error too
Also add/fix comments, fix debug logs and mistakes.
4 years ago
Soner Tari 229ea2fe0a Convert linked lists to doubly linked lists, so keep track of previous node
The only time we go over the linked list is to check idle or expired
connections, or to print debug info. Otherwise, mostly what we need is
to add and remove list nodes. Removing a list node becomes a very simple
task if we keep track of the previous node too. So now we also keep
record of prev node, and update prev node as we add and remove nodes.
All three linked lists we use benefit from this data structure
improvement, making it very fast to remove a list node.

Another benefit of this change is that we don't need to identify conns
with their id numbers or child conns with their src fds. So now we
directly delete them, without needing to check their ids or fds.
4 years ago