Support chat marker (XEP-0333) stanzas when the "--chat-marker" switch
is provided on the command line. A chat marker is prefixed by 'M':
~~~
M:groupchat:test@conference.example.com/msantos:me@example.com/162315501161646113068402:
~~~
The idea is to allow scripts to react based on whether a message has
been read, for example, escalating via other channels.
strcmp(3) segfaulted when comparing the from address to NULL.
Checking messages originated from the output account only makes sense
with groupchat. Remove the check for type of "chat".
This reverts commit 7090ef09fb.
fstat/ioctl are checking whether the program is attached to tty for
setting buffering. Since xmppipe explicitly enables line buffering, the
capsicum sandbox can ignore these tests.
An empty string in the type, to and from uses a default value. For
example to send a message to the groupchat specified on the command
line:
~~~
m::::this is a test message
~~~
Rough implementation to allow input to be formatted as colon separated
values in the same way as output:
* percent decoding of the input is not supported yet
* only message stanzas supported
Using formatted input lets the script respond to other users aside from
the default channel assigned to stdout:
~~~
m:chat:to@example.com:from@example.com:message-body
~~~
TODO:
* does the default stdout channel always need to be formatted?
~~~
m:chat:to@example.com:from@example.com:message-body
m:groupchat:default@conference.example.com:from@example.com:message-body
~~~
Otherwise it could be ambiguous.
* support presence and iq stanzas
For example, a bot could respond to groupchat invitations.
* percent decoding: require the input to be percent encoded
Support binary and multiline data.
* format naming: choose better names for the format types
Instead of supplying the output JID as an optional argument:
xmppipe --output foo@conference.example.com
Use the first argument:
xmppipe foo@conference.example.com
The -o/--output switches are still accepted.
Running xmppipe in chat mode without a full jid:
xmppipe --chat --output foo
Caused a segfault when a NULL domain was passed to
xmpp_stanza_set_attribute(). The libstrophe functions do not check for
NULL and so crash calling strlen(NULL).
Set the jid's domain from the user's username. If the user's jid is
user@example.com:
# expanded to foo@example.com
xmppipe --chat --output foo
Document usage of one to one chat:
xmppipe --chat --output me@example.com
Only provide the long option until a few quirks have been worked out:
* if only a username is provided, it will be expanded to a conference
name
# expands to me@conference.example.com
xmppipe --chat --output me
* should "normal" and "headline" message types also be supported?
* tests
* always add id in iq stanzas.
* presence: response from muc may contain more than one 'x' element,
match stanza by namespace
* debug: print out current state
Add preliminary support to one to one chats. No XEPs were read in the
preparation of this change:
xmppipe -C example@example.com
TODO
* clean up
* state change is hardcoded
* if (GROUPCHAT) branches
* autodetect MUC
* in chat mode, ctrl-D can cause a loop
libstrophe 0.9.2 supports TLS certificate verification. Tested by:
* valid certificate: verified using strace that xmppipe is reading the
system SSL cert store
* invalid certificate:
sudo chmod 700 /usr/lib/ssl
Verified xmppipe rejected the cert as invalid without the local CA
root.
* valid certificate, invalid domain
Verified a subdomain hosted on the XMPP node but not included in the
TLS certificate is rejected.
Terminate long option list so xmppipe doesn't segfault when passed an
unknown option.
Correct the usage. Revert to using --ouput for the MUC name instead of
--stdout to avoid confusion.
libstrophe 0.9.2 uses OpenSSL to validate TLS certs by checking against
the system cert store.
Allow llseek(2). Probably a better way of handling syscalls is to allow
classes of syscalls based on OpenBSD's pledge.
Add a sandbox enforced before options are parsed and the connection is
established to the XMPP server. This sandbox will allow network
operations.
The post-connect sandbox is unchanged and restricts operations to stdio.
The commit just adds the infrastructure for the pre-connect sandbox.
Use the UID of the xmppipe process instead of the PID in the default
name. This allows many processes running under the same user on a host
to share the same output channel and makes it easier to pre-create the
MUC if the xmppipe XMPP user does not have MUC creation privs.
Whether gethostname(2) returns an error depends on the implementation.
Some implementations:
* truncate the hostname if length is less than the hostname, with or
without a trailing NULL
* return -1 if length is less than hostname
* return -1 if length is 0
Set a default name if gethostanme() returns error.
Some errors will cause the XMPP file descriptor to be closed before
xmppipe exits. Return EBADF if close is called since the process will
terminate anyway.
When base64 encoding is enabled, ignore any messages that fail base64
decoding.
Previously signed-unsigned integer conversion would cause the return
value of b64_pton() on error (a negative integer) to be converted to a
large value. The attempt to allocate this value would force xmppipe to
exit.
Add a BPF seccomp syscall filter on Linux. Not enabled by default. To
compile:
XMPPIPE_SANDBOX=XMPPIPE_SANDBOX_SECCOMP make
The sandbox is derived from OpenSSH's seccomp sandbox by Will Drewry and
Kees Cook's tutorial on seccomp:
http://outflux.net/teach-seccomp/
The number of file descriptors enforced by setrlimit() can now be set at
compile time using a flag. The flag defaults to 0 on Linux and -1
everywhere else:
XMPPIPE_SANDBOX=XMPPIPE_SANDBOX_RLIMIT \
XMPPIPE_SANDBOX_RLIMIT_NOFILE=-1 \
make
The meaning of the XMPPIPE_SANDBOX_RLIMIT_NOFILE is:
* -1 : set rlim_cur/rlim_max to the lowest allocated file desciptor
* >=0: set rlim_cur/rlim_max to this number
On some platforms, setting rlim_cur below the value of the highest
allocated fd may interfere with polling. See commit a34d5766c5 for
details.
The rlimit sandbox disables forking processes and opening files.
The rlimit sandbox is not used by default yet. To compile it:
XMPPIPE_SANDBOX=XMPPIPE_SANDBOX_RLIMIT make
The rlimit sandbox should work on any platform. However the interaction
of RLIMIT_NOFILE with poll(2) (and select(2)?) on some platforms (FreeBSD
but really any OS besides Linux) is problematic:
* opening a number of fd's, setting RLIMIT_NOFILE to 0, calling
poll(2) on the fdset
Linux: works
FreeBSD: fails
* opening a number of fd's, setting RLIMIT_NOFILE to maxfd+1, calling
poll(2) on the fdset
Linux: works
FreeBSD: works
The issue with the second option is that a library may have opened a
sequence of file descriptors then closed the lower numbered fd's:
open() => 3
open() => 4
open() => 5
close(3)
close(4)
maxfd = 5
RLIMIT_NOFILE would be set to 6 (stdin, stdout, stderr, 3, 4, 5) and the
sandbox would allow re-opening fd's 3 and 4.
One possible fix would be to run through the sequence of fd's before
entering the rlimit sandbox:
* test if the fd is closed
* if the fd is closed, dup2(STDIN_FILENO, fd)
Since the closed fd's are not part of the pollset, they will not be
polled and should be ignored.
Note we can't simply move maxfd to the lowest unused fd because
libstrophe maintains the fd number as internal, opaque state.
Empirically, the xmpp fd is always 3. Another option would be to abort
the process if the fd does not equal 3.
Prepare for sandboxing the xmppipe process by adding a function called
after all file descriptors are allocated.
The intent of the sandbox is to limit the xmppipe process to the role
of a component in a shell pipeline: reading from stdin, reading/writing
to the XMPP socket and writing to stdout. Any activity not involved with
using stdio should force the process to exit.
The sandbox function will vary based on the capabilities of the
platform. The default sandbox function does nothing.
Limitations of the sandbox:
Probably the biggest risk is in session establishment:
* the TLS handshake
* the XML parsing
The sandbox is enforced after the TLS connection is established, i.e.,
after the file descriptor for the XMPP session is allocated and so has no
effect on the TLS handshake or the initial XMPP handshake.
Possibly an initial sandbox could be setup for the connection phase
followed by a stricter sandbox for the stdio phase.
Use uuid_create(3) and uuid_to_string(3) to create the message id on
BSDs. Only tested on FreeBSD but should work on OpenBSD and NetBSD.
Add untested support for compiling on Solaris and Mac OS X:
* SmartOS has libuuid installed by default with rsyslog via pkgsrc
* Mac OS X has libuuid as part of libSystem:
http://lists.apple.com/archives/unix-porting/2009/Aug/msg00006.html
Use a fixed ping handler rather than scheduling a ping handler per ping
request.
If the XMPP client or server is busy, many outstanding ping requests
can be waiting, resulting in a number of ping handlers being queued.
Each of the ping handlers does the same thing (resets the failure count
to 0) but any dropped pings resulted in the handlers being queued
forever.
For ejabberd and the mongooseim fork of ejabbered, 4097 bytes is around
the maximum message size before messages are dropped (the extra byte is
for libstrophe to put the terminating NULL. Otherwise, with a buffer
of 4096, the message will be broken up into 2 parts, with the second
consisting of the last byte of the message).
With base64 encoding, the message size will increase to 4097 * 4 / 3
bytes. When sending streams of binary data (cat'ing a file over XMPP,
ssh over XMPP, ...), stream management will get backlogged and messages
will disappear.
So halve the default buffer size. Is it too big for other jabber
servers? It is one of the mysteries but the "-b" option exists for those.
Implement flow control by monitoring the number of unacked stream
requests and the number of unprocessed stanzas. If either exceeds a user
controlled limit, xmppipe will stop reading from stdin. Further stream
requests will be generated whenever the keepalive timeout is reached and
an XMPP ping is sent.
ejabberd/mongooseim will close the connection if there are about 5
unacked stream requests. xmppipe should probably do the same thing.
With these changes, binaries can be reliably (but slowly) sent over
XMPP. ssh over XMPP is also working reliably (with light testing). Because
of the increase in size due to base64 encoding, the buffer size needs
to be reduced so the encoded message is under 4k (-m 1024 is ok).
When stdin is closed with stream management enabled, xmppipe waits for
the ack to match the number of stanzas sent to ensure all the data has
been flushed to the server.
The server may not respond to every stream request and a stream request
message is generated only when a stanza is sent. Since stdin is closed,
the only traffic is the XMPP pings which are sent out every 60 seconds
by default. The result is, when stdin is closed, it may take several
minutes for xmppipe to exit.
Work around the problem by sending a stream management request for every
stanza.
Optionally base64 encode data sent to and from a conference, allowing
transfer of binary data.
This could also be accomplished by piping in base64 encoded data:
base64 /bin/ls | xmppipe
However the base64 command does not work with streams of data.
Check for errors on libstrophe functions that return non-void. Some of
the funs have undocumented failure conditions (e.g.,
xmpp_stanza_copy()). Others return less than zero or enum values. To
account for future discoveries, use a wrapper instead of a macro.
Request stream management for the session by default. The rate of requests
and acks is controlled by the -I option. With stream management, the
client can check when all stanzas have been flushed to the server. For
example, the following works:
cat /path/textfile | xmppipe -o transfer -r sender
# on another system
xmppipe -o transfer -r receiver > tmpfile
The behaviour is to always attempt enabling stream management. The XEP
is ambiguous about the response to an enable when stream management is
not offered. Presumably the server will respond with "failure".
So the code should probably have 2 checks:
* if stream management is offered as a feature, send an enable
* if the server responds with "enabled", handle requests
Set a limit on the number of pings without a reply and exit if the limit
is reached. Useful for detecing broken network connections due to
network changes.
The default configuration will take about 4 minutes to detect failure
(1 keepalive every 60 seconds, tolerate 3 failures).
No point shutting down nicely if the connection has failed: exit
immediately and print out an error.
The error is the same whether the connection has failed due to a DNS
lookup, TCP connection or authentication failure. Running in debug mode
can show the reason for failure. libstrophe has an enum indicating the
connection state; maybe this can be used for better error reporting.
Add a ID per message. Seems to fix the duplicate messages seen with MUCs
on the Conversations android client whenever the network is switched
(e.g., from wifi to cell).
Introduce a dependency on Linux's libuuid which introduces portability
issues. However, it should be simple to add support for the BSD UUID
interface.
libuuid was used because it is a fast and simple way of generating
a unique id. Realistically, the ID only needs to be unique within
the MUC and so could probably be replaced with an sprintf() combining
time of day, PID and a random number.
Force exiting on nick conflicts. xmppipe could generate a unique resource
(e.g., xmppipe-<pid>) but it prevents accidentally having two instances
running in a room.
Michael Santos
Dmitry Podgorny