Add testproxy e2e tests for POP3 and SMTP protocol validation.
We have detected that POP3 and SMTP protocol validation was broken
thanks to these new testproxy e2e tests. This is yet another example why
e2e tests are important.
We don't need parent or child ids unless debugging. IDLE and EXPIRED
conn logs do not need to report ids either. Ids are useful only in
detailed debug logs.
This is necessary to uniquely identify child conns. The src fd of child
conns was possibly not unique. We use this id in debug logs only.
Also relocate the update code related with this id.
Do not pass pxy_thr_print_children() or bufferevent_getfd() to MAX() or
util_max() macro functions as params, or else they are called twice.
Since MAX() macro call duplicates params, do not call it nested either,
or else we get very long macro expansions.
ce5f409dbe
("Zero all bytes when passing file descriptors over AF_UNIX sockets",
2018-11-12)
Also, bufferevent_getfd() returns -1 if no file descriptor is associated
with the bufferevent.
Free vars.
Finalize sqlite3 statements.
Close sqlite3 db.
Init memory.
Do not close fd -1.
Some of these may be harmless, but we fix them anyway. Now valgrind
reports 0 "lost" memory, but some "still reachable", both for sslproxy
and lp.
We don't need a privsep call to open a socket for child listener,
because listener port of child conns are assigned by the system, hence
are from non-privileged range above 1024.
So the open privsep socket is used only to update user atime now.
We have carried almost all conn init tasks from thrmgr to conn handling
thread. So we immediately add the conn to the conn list of its thr,
which renders both pending ssl conns list and in_thr_conns flag useless.
The only time we go over the linked list is to check idle or expired
connections, or to print debug info. Otherwise, mostly what we need is
to add and remove list nodes. Removing a list node becomes a very simple
task if we keep track of the previous node too. So now we also keep
record of prev node, and update prev node as we add and remove nodes.
All three linked lists we use benefit from this data structure
improvement, making it very fast to remove a list node.
Another benefit of this change is that we don't need to identify conns
with their id numbers or child conns with their src fds. So now we
directly delete them, without needing to check their ids or fds.
Do we need a thr mutex? This mutex is for thread-safe access to
thr.load. But thrmgr read-accesses thr.load, and write-accesses are by
thr only. So can we really live without it?
So now we do a couple of expensive tasks on conn handling threads, not
on thrmgr: Add the conn to its thread conn list, check fd usage, nat
lookup dst, and make string src addr.
This prevents possible multithreading issues between thrmgr and conn
handling threads. So we can remove and clean up the code and comments
related with such possible issues now. For example, we can add the conn
to its thread list earlier, and we can handle errors immediately, thanks
to this early switch to conn handling threads. This also helps achieve
cleaner code.
Enable dst r/w events before socket connect.
Improve verbose debug logs using common header fields to better identify
connections.
Create function macros for fine* debug logs.
This happens if there was no autossl handshake prior to ClientHello,
e.g. no STARTTLS message. This is perhaps due to the SSL handshake of a
direct SSL connection, i.e. invalid protocol.
We should not crash upon protocol errors, hence the need for fuzzing
tests.
We don't do anything in srvdst writecb except for passhtrough mode.
We handle srvdst and dst connect tasks in connectcb for them by
arranging connect events correctly, so we don't need any extra flags.
Correct connect ordering helps us remove code checking if bev exists.
There were a lot of unnecessary code in autossl. Tcp and ssl code are
decoupled now.
Because we directly relay the packets from the server to the client
until we receive the first packet from the client, at which time we xfer
srvdst to the first child conn and effectively disable this readcb,
hence start diverting packets to the listening program.
Improve documentation.
