The PTS received from MediaCodec are expressed relative to an arbitrary
clock origin. We consider the PTS of the first frame to be 0, and the
PTS of every other frame is relative to this first PTS (note that the
PTS is only used for recording, it is ignored for mirroring).
For simplicity, this relative PTS was computed on the server-side.
To prepare support for multiple stream (video and audio), send the
packet with its original PTS, and handle the PTS offset on the
client-side (by the recorder).
Since we can't know in advance which stream will produce the first
packet with the lowest PTS (a packet received later on one stream may
have a PTS lower than a packet received earlier on another stream),
computing the PTS on the server-side would require unnecessary waiting.
On click event, only the whole buttons state was passed to the device.
In addition, on ACTION_DOWN and ACTION_UP, pass the button associated to
the action.
Refs #3635 <https://github.com/Genymobile/scrcpy/issues/3635>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
For the initial connection between the device and the computer, an adb
tunnel is established (with "adb reverse" or "adb forward").
The device-side of the tunnel is a local socket having the hard-coded
name "scrcpy". This may cause issues when several scrcpy instances are
started in a few seconds for the same device, since they will try to
bind the same name.
To avoid conflicts, make the client generate a random UID, and append
this UID to the local socket name ("scrcpy_01234567").
Use av_packet_ref() to reference the packet without copy.
This also simplifies the logic, by making the "offset" variable and the
memcpy() call local to the if-block.
Right click and middle click require the source device to be a mouse,
not a touchscreen. Therefore, the source device was changed only when a
button other than the primary button was pressed (see
adc547fa6e).
However, this led to inconsistencies between the ACTION_DOWN when a
secondary button is pressed (with a mouse as source device) and the
matching ACTION_UP when the secondary button is released (with a
touchscreen as source device, because then there is no button pressed).
To avoid the problem in all cases, force a mouse as source device when
--forward-all-clicks is set.
Concretely, for mouse events in --forward-all-clicks mode:
- device source is set to InputDevice.SOURCE_MOUSE;
- motion event toolType is set to MotionEvent.TOOL_TYPE_MOUSE;
Otherwise (when --forward-all-clicks is unset, or for real touch
events), finger events are injected:
- device source is set to InputDevice.SOURCE_TOUCHSCREEN;
- motion event toolType is set to MotionEvent.TOOL_TYPE_FINGER.
Fixes#3568 <https://github.com/Genymobile/scrcpy/issues/3568>
PR #3579 <https://github.com/Genymobile/scrcpy/pull/3579>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
If the current adb port is not 5555 (typically 0 because it is not in
TCP/IP mode), --tcpip automatically executes (among other commands):
adb tcpip 5555
In case adb was already listening on another port, this command forced
to listen on 5555, and the connection should still succeed.
But this reconfiguration might be inconvenient for the user. If adb is
already in TCP/IP mode, use the current enabled port without
reconfiguration.
Fixes#3591 <https://github.com/Genymobile/scrcpy/issues/3591>
On Windows and macOS, resizing blocks the event loop. Handling it
properly would require the same workaround as done in screen.c.
This reverts commit 436b368f9d.
We must distinguish 3 cases for await_for_server():
- an error occurred
- no error occurred, the device is connected
- no error occurred, the device is not connected (user requested to
quit)
For this purpose, use an additional output parameter to indicate if the
device is connected (only set when no error occurs).
Refs #3085 <https://github.com/Genymobile/scrcpy/pull/3085>
For simplicity, the parsing of `adb devices -l` output is performed in a
single pass on the whole output.
This output was limited to 4096 bytes. Since there are about 100 chars
per device line, this limited the number of connected devices to ~40.
Increase to 65536 bytes to avoid a limitation in practice.
PR #3035 <https://github.com/Genymobile/scrcpy/pull/3035>
All adb commands are executed with an "interruptor", so that they can be
interrupted on Ctrl+C.
Make this interruptor optional, so that we could call "adb kill-server"
in OTG mode. This command always returns almost immediately anyway.
Ideally, we should make all blocking calls interruptible (including
libusb calls, by using the asynchronous API), but it's a lot of work,
and in practice it works well enough.
PR #3011 <https://github.com/Genymobile/scrcpy/pull/3011>
USB device disconnection is detected via a hotplug callback when it is
supported.
In addition, report disconnection on libusb calls returning
LIBUSB_ERROR_NO_DEVICE or LIBUSB_ERROR_NOT_FOUND. This allows to detect
disconnection after a libusb call when hotplug is not available.
PR #3011 <https://github.com/Genymobile/scrcpy/pull/3011>
The value of sc_tick_now() has microsecond precision, but
sc_cond_timedwait() has only millisecond precision.
To guarantee that sc_tick_now() >= deadline when sc_cond_timedwait()
returns due to timeout, round up to the next millisecond.
This avoids to call a non-blocking sc_cond_timedwait() in a loop for no
reason until a target deadline during up to 1 millisecond.
Refs 682a691173
The type uint32_t is not sufficient to store the result of
sc_tick_now().
As a consequence, the FPS counter entered a live loop and caused a lock
starvation (deadlock in practice).
Refs ec871dd3f5
Refs 682a691173
A special PTS value was used to encode a config packet.
To prepare for adding more flags, use the most significant bits of the
PTS field to store flags.
The relative mouse mode is tracked by SDL, and accessible via
SDL_GetRelativeMouseMode().
This is more robust in case SDL changes the relative mouse mode on its
own.
Without this log, the user would have no way to know that a USB device
is rejected because it could not be opened (typically due to
insufficient permissions).
If several devices are connected (as listed by `adb devices`), it was
necessary to provide the explicit serial via -s/--serial.
If only one device is connected via USB (respectively, via TCP/IP), it
might be convenient to select it automatically. For this purpose, two
new options are introduced:
- -d/--select-usb: select the single device connected over USB
- -e/--select-tcpip: select the single device connected over TCP/IP
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
Currently, a device is selected either from a specific serial, or if it
is the only one connected.
In order to support selecting the only device connected via USB or via
TCP/IP separately, introduce a new selection structure.
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
This does nothing if the adb daemon is already started, but allows to
print any output/errors to the console.
Otherwise, the daemon starting would occur during `adb devices`, which
does not output to the console because the result is parsed.
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
Since the previous commit, if a serial is given via -s/--serial (either
a real USB serial or an IP:port), a device is selected if its serial
matches exactly.
In addition, if the user pass an IP without a port, then select any
device with this IP, regardless of the port (so that "192.168.1.1"
matches any "192.168.1.1:port"). This is also the default behavior of
adb.
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
List all USB devices in a first step, then select the matching one(s).
This allows to report a user-friendly log message containing the list of
devices, with the matching one(s) highlighted.
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
One log macro was provided for each log level (LOGV(), LOGD(), LOGI(),
LOGW(), LOGE()).
Add a generic macro LOG(LEVEL, ...) accepting a log level as parameter,
so that it is possible to write logging wrappers.
PR #3005 <https://github.com/Genymobile/scrcpy/pull/3005>
This util function was error-prone:
- it accepted a buffer as parameter (not necessarily a NUL-terminated
string) and its length (including the NUL char, if any);
- it wrote '\0' over the last character of the buffer, so the last
character was lost if the buffer was not a NUL-terminated string, and
even worse, it caused undefined behavior if the length was empty;
- it returned the length of the resulting NUL-terminated string,
which was inconsistent with the input buffer length.
In addition, it was not necessarily optimal:
- it wrote '\0' twice;
- it required to know the buffer length, that is the input string
length + 1, in advance.
Remove this function, and let the client use strcspn() manually.
The function assumed that the raw output of "adb connect" was a
NUL-terminated string, but it is not the case.
It this output did not end with a space or a new line character, then
sc_str_truncate() would write '\0' over the last character. Even worse,
if the output was empty, then sc_str_truncate() would write
out-of-bounds.
Avoid the error-prone sc_str_truncate() util function.
The function assumed that the raw output of "adb get-serialno" was a
NUL-terminated string, but it is not the case.
It this output did not end with a space or a new line character, then
sc_str_truncate() would write '\0' over the last character. Even worse,
if the output was empty, then sc_str_truncate() would write
out-of-bounds.
Avoid the error-prone sc_str_truncate() util function.
The function assumed that the raw output of "adb getprop" was a
NUL-terminated string, but it is not the case.
It this output did not end with a space or a new line character, then
sc_str_truncate() would write '\0' over the last character. Even worse,
if the output was empty, then sc_str_truncate() would write
out-of-bounds.
Avoid the error-prone sc_str_truncate() util function.
The parser assumed that its input was a NUL-terminated string, but it
was not the case: it is just the raw output of "adb devices ip route".
In practice, it was harmless, since the output always ended with '\n'
(which was replaced by '\0' on truncation), but it was incorrect
nonetheless.
Always write a '\0' at the end of the buffer, and explicitly parse as a
NUL-terminated string. For that purpose, avoid the error-prone
sc_str_truncate() util function.
Before starting the server, the actual device serial (possibly its
ip:port if it's over TCP/IP) must be known.
A serial might be requested via -s/--serial (stored in the
sc_server_params), but the actual serial may change afterwards:
- if none is provided, then it is retrieved with "adb get-serialno";
- if --tcpip is requested, then the final serial will be the target
ip:port.
The requested serial was overwritten by the actual serial in the
sc_server_params struct, which was a bit hacky.
Instead, store a separate serial field in sc_server (and rename the one
from sc_server_params to "req_serial" to avoid confusion).
Now that providing a serial is mandatory for adb commands where it is
relevant, the whole argv array may be built statically, without
allocations at runtime.
If no serial is passed, then the command would work if there is exactly
one device connected, but will fail with multiple devices.
To avoid such cases, ensure that a serial is always provided.
On Windows, adb is provided in the release archive. Most missing adb
issues come from users setting the ADB environment variable to an
incorrect value (on all platforms).
Suggesting to install platform-tools to solve the problem will just make
things worse (there will be one more adb in yet another location).
The function was initially implemented to truncate lines, but was later
generalized to accept custom delimiters. The whole documentation has not
been updated accordingly.
Refs 9619ade706
Add an option --otg to run scrcpy with only physical keyboard and mouse
simulation (HID over AOA), without mirroring and without requiring adb.
To avoid adding complexity into the scrcpy initialization and screen
implementation, OTG mode is implemented totally separately, with a
separate window.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
Input events helpers to convert from SDL events to scrcpy events were
implemented in input_manager. To reuse them for OTG mode, move them to
input_events.h.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
The device disconnection is detected when the video socket closes.
In order to introduce an OTG mode (HID events) without mirroring (and
without server), we must be able to detect USB device disconnection.
This feature will only be used in OTG mode.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
Acksync is used to delay HID events until some request (in practice,
device clipboard synchronization) is acknowledged by the device.
This mechanism will not be necessary for OTG mode.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
The device was automatically found by sc_usb_connect(). Instead, expose
a function to find a device from a serial, and let the caller connect to
the device found (if any).
This will allow to list all devices first, then select one device to
connect to.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
Use it from accept_device() to simplify (at the cost an additional
allocation for each serial, but it is not important).
It will also be useful in other functions in further commits.
PR #2974 <https://github.com/Genymobile/scrcpy/pull/2974>
The server needs to interrupt the sockets on stop, but it must not close
them while other threads may attempt to read from or write to them.
In particular, the video_socket is read by the stream thread, and the
control_socket is written by the controller and read by receiver.
Therefore, close the socket only on sc_server_destroy(), which is called
after all other threads are joined.
Reported by TSAN on close:
WARNING: ThreadSanitizer: data race (pid=3287612)
Write of size 8 at 0x7ba000000080 by thread T1:
#0 close ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1690 (libtsan.so.0+0x359d8)
#1 net_close ../app/src/util/net.c:280 (scrcpy+0x23643)
#2 run_server ../app/src/server.c:772 (scrcpy+0x20047)
#3 <null> <null> (libSDL2-2.0.so.0+0x905a0)
Previous read of size 8 at 0x7ba000000080 by thread T16:
#0 recv ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:6603 (libtsan.so.0+0x4f4a6)
#1 net_recv_all ../app/src/util/net.c:228 (scrcpy+0x234a9)
#2 stream_recv_packet ../app/src/stream.c:33 (scrcpy+0x2045c)
#3 run_stream ../app/src/stream.c:228 (scrcpy+0x21169)
#4 <null> <null> (libSDL2-2.0.so.0+0x905a0)
Refs ddb9396743
If --no-control is requested, then the controller instance is not
initialized. However, its reference was still passed to screen and
input_manager.
Instead, pass NULL if no controller is available.
The optimal initial size was computed from the expected dimensions, sent
immediately by the server before encoding any video frame.
However, the actual frame size may be different, for example when the
device encoder does not support the requested size.
To always handle this case properly, position and size the window only
once the first frame size is known.
PR #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
Show the window only after the actual frame size is known (and if no
error has occurred).
This will allow to properly position and size the window when the size
of the first frame is different from the size initially announced by the
server.
PR #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
The components should be configurable independently of the global
scrcpy_options instance: their configuration could be provided
separately, like it is the case for example for some screen parameters.
For consistency, keyboard injection should not depend on scrcpy_options.
In relative mouse mode, the mouse pointer must be "captured" from the
computer.
Toggle (disable/enable) relative mouse mode using any of the hardcoded
capture keys:
- left-Alt
- left-Super
- right-Super
These capture keys do not conflict with shortcuts, since a shortcut is
always a combination of the MOD key and some other key, while the
capture key triggers an action only if it is pressed and released alone.
The relative mouse mode is also automatically enabled on any click in
the window, and automatically disabled on focus lost (it is possible to
lose focus even without the mouse).
The decision to not send motion events when no click is pressed is
specific to Android mouse injection. Other mouse processors (e.g. for
HID mouse) will need to receive all events.
The default mouse injection works in absolute mode: it forwards clicks
at a specific position on screen.
To support HID mouse, add a flag to indicate that the mouse processor
works in relative mode: it forwards mouse motion vectors, without any
absolute reference to the screen.
A scroll event might be produced when a mouse button is pressed (for
example when scrolling while selecting a text). For consistency, pass
the actual buttons state (instead of 0).
In practice, it seems that this use case does not work properly with
Android event injection, but it will work with HID mouse.
The input_manager is strongly tied to the screen, it could not work
independently of the specific screen implementation.
To implement a user-friendly HID mouse behavior, some SDL events
will need to be handled both by the screen and by the input manager. For
example, a click must typically be handled by the input_manager so that
it is forwarded to the device, but in HID mouse mode, the first click
should be handled by the screen to capture the mouse (enable relative
mouse mode).
Make the input_manager a descendant of the screen, so that the screen
decides what to do on SDL events.
Concretely, replace this structure hierarchy:
+- struct scrcpy
+- struct input_manager
+- struct screen
by this one:
+- struct scrcpy
+- struct screen
+- struct input_manager
This avoids to directly pass the options instance (which contains more
data than strictly necessary), and limit the number of parameters for
the init function.
Not all key processors support text injection (HID keyboard does not
support it).
Instead of providing a dummy op function, set it to NULL and check on
the caller side before calling it.
Pass scrcpy input events instead of SDL input events to mouse
processors.
These events represent exactly what mouse processors need, abstracted
from any visual orientation and scaling applied on the SDL window.
This makes the mouse processors independent of the "screen" instance,
and the implementation source code independent of the SDL API.
This aims to make the key/mouse processors independent of the "screen",
by processing scrcpy-specific input events instead of SDL events.
In particular, these scrcpy events are not impacted by any UI window
scaling or rotation (contrary to SDL events).
The input manager exposed functions taking an "actions" parameter,
containing a bitmask-OR of ACTION_UP and ACTION_DOWN.
But they are never called with both actions simultaneously anymore, so
simplify.
Refs 964b6d2243
Refs d0739911a3
This allows to report a meaningful error message if an unsupported
feature is used on an incompatible platform. This is consistent with the
behavior of -K/--hid-keyboard.
The "resize to fit" feature (MOD+w or double-click on black borders)
computed the "optimal size" using the same function computing the
initial window size on start.
However, on "resize to fit", only the black borders must be removed (the
content size must be preserved), so the display bounds must not be
considered.
