Mention that the Debian package is obsolete

It cannot be updated until the android-framework-XX Debian package is
fixed.

Refs <https://tracker.debian.org/pkg/scrcpy>

.gitignore

@@ -7,3 +7,4 @@ build/
 .gradle/
 /x/
 local.properties
+/scrcpy-server

DEVELOP.md

@@ -1,309 +0,0 @@
-# scrcpy for developers
-
-## Overview
-
-This application is composed of two parts:
- - the server (`scrcpy-server`), to be executed on the device,
- - the client (the `scrcpy` binary), executed on the host computer.
-
-The client is responsible to push the server to the device and start its
-execution.
-
-Once the client and the server are connected to each other, the server initially
-sends device information (name and initial screen dimensions), then starts to
-send a raw H.264 video stream of the device screen. The client decodes the video
-frames, and display them as soon as possible, without buffering, to minimize
-latency. The client is not aware of the device rotation (which is handled by the
-server), it just knows the dimensions of the video frames.
-
-The client captures relevant keyboard and mouse events, that it transmits to the
-server, which injects them to the device.
-
-
-
-## Server
-
-
-### Privileges
-
-Capturing the screen requires some privileges, which are granted to `shell`.
-
-The server is a Java application (with a [`public static void main(String...
-args)`][main] method), compiled against the Android framework, and executed as
-`shell` on the Android device.
-
-[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123
-
-To run such a Java application, the classes must be [_dexed_][dex] (typically,
-to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
-`classes.dex`, pushed to the device in `/data/local/tmp`, then it can be run
-with:
-
-    adb shell CLASSPATH=/data/local/tmp/classes.dex \
-        app_process / my.package.MainClass
-
-_The path `/data/local/tmp` is a good candidate to push the server, since it's
-readable and writable by `shell`, but not world-writable, so a malicious
-application may not replace the server just before the client executes it._
-
-Instead of a raw _dex_ file, `app_process` accepts a _jar_ containing
-`classes.dex` (e.g. an [APK]). For simplicity, and to benefit from the gradle
-build system, the server is built to an (unsigned) APK (renamed to
-`scrcpy-server`).
-
-[dex]: https://en.wikipedia.org/wiki/Dalvik_(software)
-[apk]: https://en.wikipedia.org/wiki/Android_application_package
-
-
-### Hidden methods
-
-Although compiled against the Android framework, [hidden] methods and classes are
-not directly accessible (and they may differ from one Android version to
-another).
-
-They can be called using reflection though. The communication with hidden
-components is provided by [_wrappers_ classes][wrappers] and [aidl].
-
-[hidden]: https://stackoverflow.com/a/31908373/1987178
-[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers
-[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view
-
-
-### Threading
-
-The server uses 3 threads:
-
- - the **main** thread, encoding and streaming the video to the client;
- - the **controller** thread, listening for _control messages_ (typically,
-   keyboard and mouse events) from the client;
- - the **receiver** thread (managed by the controller), sending _device messages_
-   to the clients (currently, it is only used to send the device clipboard
-   content).
-
-Since the video encoding is typically hardware, there would be no benefit in
-encoding and streaming in two different threads.
-
-
-### Screen video encoding
-
-The encoding is managed by [`ScreenEncoder`].
-
-The video is encoded using the [`MediaCodec`] API. The codec takes its input
-from a [surface] associated to the display, and writes the resulting H.264
-stream to the provided output stream (the socket connected to the client).
-
-[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
-[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
-[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69
-
-On device [rotation], the codec, surface and display are reinitialized, and a
-new video stream is produced.
-
-New frames are produced only when changes occur on the surface. This is good
-because it avoids to send unnecessary frames, but there are drawbacks:
-
- - it does not send any frame on start if the device screen does not change,
- - after fast motion changes, the last frame may have poor quality.
-
-Both problems are [solved][repeat] by the flag
-[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
-
-[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
-[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
-[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
-
-
-### Input events injection
-
-_Control messages_ are received from the client by the [`Controller`] (run in a
-separate thread). There are several types of input events:
- - keycode (cf [`KeyEvent`]),
- - text (special characters may not be handled by keycodes directly),
- - mouse motion/click,
- - mouse scroll,
- - other commands (e.g. to switch the screen on or to copy the clipboard).
-
-Some of them need to inject input events to the system. To do so, they use the
-_hidden_ method [`InputManager.injectInputEvent`] (exposed by our
-[`InputManager` wrapper][inject-wrapper]).
-
-[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81
-[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
-[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
-[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
-[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
-
-
-
-## Client
-
-The client relies on [SDL], which provides cross-platform API for UI, input
-events, threading, etc.
-
-The video stream is decoded by [libav] (FFmpeg).
-
-[SDL]: https://www.libsdl.org
-[libav]: https://www.libav.org/
-
-### Initialization
-
-On startup, in addition to _libav_ and _SDL_ initialization, the client must
-push and start the server on the device, and open two sockets (one for the video
-stream, one for control) so that they may communicate.
-
-Note that the client-server roles are expressed at the application level:
-
- - the server _serves_ video stream and handle requests from the client,
- - the client _controls_ the device through the server.
-
-However, the roles are reversed at the network level:
-
- - the client opens a server socket and listen on a port before starting the
-   server,
- - the server connects to the client.
-
-This role inversion guarantees that the connection will not fail due to race
-conditions, and avoids polling.
-
-_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
-reverse`. See commit [1038bad] and [issue #5].)_
-
-Once the server is connected, it sends the device information (name and initial
-screen dimensions). Thus, the client may init the window and renderer, before
-the first frame is available.
-
-To minimize startup time, SDL initialization is performed while listening for
-the connection from the server (see commit [90a46b4]).
-
-[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
-[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
-[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
-
-
-### Threading
-
-The client uses 4 threads:
-
- - the **main** thread, executing the SDL event loop,
- - the **stream** thread, receiving the video and used for decoding and
-   recording,
- - the **controller** thread, sending _control messages_ to the server,
- - the **receiver** thread (managed by the controller), receiving _device
-   messages_ from the server.
-
-In addition, another thread can be started if necessary to handle APK
-installation or file push requests (via drag&drop on the main window) or to
-print the framerate regularly in the console.
-
-
-
-### Stream
-
-The video [stream] is received from the socket (connected to the server on the
-device) in a separate thread.
-
-If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
-to decode the H.264 stream from the socket, and notifies the main thread when a
-new frame is available.
-
-There are two [frames][video_buffer] simultaneously in memory:
- - the **decoding** frame, written by the decoder from the decoder thread,
- - the **rendering** frame, rendered in a texture from the main thread.
-
-When a new decoded frame is available, the decoder _swaps_ the decoding and
-rendering frame (with proper synchronization). Thus, it immediately starts
-to decode a new frame while the main thread renders the last one.
-
-If a [recorder] is present (i.e. `--record` is enabled), then it muxes the raw
-H.264 packet to the output video file.
-
-[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h
-[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h
-[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h
-[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h
-
-```
-                                   +----------+      +----------+
-                              ---> | decoder  | ---> |  screen  |
-             +---------+     /     +----------+      +----------+
- socket ---> | stream  | ----
-             +---------+     \     +----------+
-                              ---> | recorder |
-                                   +----------+
-```
-
-### Controller
-
-The [controller] is responsible to send _control messages_ to the device. It
-runs in a separate thread, to avoid I/O on the main thread.
-
-On SDL event, received on the main thread, the [input manager][inputmanager]
-creates appropriate [_control messages_][controlmsg]. It is responsible to
-convert SDL events to Android events (using [convert]). It pushes the _control
-messages_ to a queue hold by the controller. On its own thread, the controller
-takes messages from the queue, that it serializes and sends to the client.
-
-[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h
-[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h
-[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h
-[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h
-
-
-### UI and event loop
-
-Initialization, input events and rendering are all [managed][scrcpy] in the main
-thread.
-
-Events are handled in the [event loop], which either updates the [screen] or
-delegates to the [input manager][inputmanager].
-
-[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c
-[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201
-[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h
-
-
-## Hack
-
-For more details, go read the code!
-
-If you find a bug, or have an awesome idea to implement, please discuss and
-contribute ;-)
-
-
-### Debug the server
-
-The server is pushed to the device by the client on startup.
-
-To debug it, enable the server debugger during configuration:
-
-```bash
-meson setup x -Dserver_debugger=true
-# or, if x is already configured
-meson configure x -Dserver_debugger=true
-```
-
-If your device runs Android 8 or below, set the `server_debugger_method` to
-`old` in addition:
-
-```bash
-meson setup x -Dserver_debugger=true -Dserver_debugger_method=old
-# or, if x is already configured
-meson configure x -Dserver_debugger=true -Dserver_debugger_method=old
-```
-
-Then recompile.
-
-When you start scrcpy, it will start a debugger on port 5005 on the device.
-Redirect that port to the computer:
-
-```bash
-adb forward tcp:5005 tcp:5005
-```
-
-In Android Studio, _Run_ > _Debug_ > _Edit configurations..._ On the left, click on
-`+`, _Remote_, and fill the form:
-
- - Host: `localhost`
- - Port: `5005`
-
-Then click on _Debug_.

FAQ.md

@@ -7,7 +7,7 @@ Here are the common reported problems and their status.
 If you encounter any error, the first step is to upgrade to the latest version.
 
 
-## `adb` issues
+## `adb` and USB issues
 
 `scrcpy` execute `adb` commands to initialize the connection with the device. If
 `adb` fails, then scrcpy will not work.
@@ -133,6 +133,21 @@ Try with another USB cable or plug it into another USB port. See [#281] and
 [#283]: https://github.com/Genymobile/scrcpy/issues/283
 
 
+## OTG issues on Windows
+
+On Windows, if `scrcpy --otg` (or `--keyboard=aoa`/`--mouse=aoa`) results in:
+
+>     ERROR: Could not find any USB device
+
+(or if only unrelated USB devices are detected), there might be drivers issues.
+
+Please read [#3654], in particular [this comment][#3654-comment1] and [the next
+one][#3654-comment2].
+
+[#3654]: https://github.com/Genymobile/scrcpy/issues/3654
+[#3654-comment1]: https://github.com/Genymobile/scrcpy/issues/3654#issuecomment-1369278232
+[#3654-comment2]: https://github.com/Genymobile/scrcpy/issues/3654#issuecomment-1369295011
+
 
 ## Control issues
 
@@ -144,6 +159,8 @@ In developer options, enable:
 > **USB debugging (Security settings)**  
 > _Allow granting permissions and simulating input via USB debugging_
 
+Rebooting the device is necessary once this option is set.
+
 [simulating input]: https://github.com/Genymobile/scrcpy/issues/70#issuecomment-373286323
 
 
@@ -153,43 +170,17 @@ The default text injection method is [limited to ASCII characters][text-input].
 A trick allows to also inject some [accented characters][accented-characters],
 but that's all. See [#37].
 
-Since scrcpy v1.20 on Linux, it is possible to simulate a [physical
-keyboard][hid] (HID).
+To avoid the problem, [change the keyboard mode to simulate a physical
+keyboard][hid].
 
 [text-input]: https://github.com/Genymobile/scrcpy/issues?q=is%3Aopen+is%3Aissue+label%3Aunicode
 [accented-characters]: https://blog.rom1v.com/2018/03/introducing-scrcpy/#handle-accented-characters
 [#37]: https://github.com/Genymobile/scrcpy/issues/37
-[hid]: README.md#physical-keyboard-simulation-hid
+[hid]: doc/keyboard.md#physical-keyboard-simulation
 
 
 ## Client issues
 
-### The quality is low
-
-If the definition of your client window is smaller than that of your device
-screen, then you might get poor quality, especially visible on text (see [#40]).
-
-[#40]: https://github.com/Genymobile/scrcpy/issues/40
-
-This problem should be fixed in scrcpy v1.22: **update to the latest version**.
-
-On older versions, you must configure the [scaling behavior]:
-
-> `scrcpy.exe` > Properties > Compatibility > Change high DPI settings >
-> Override high DPI scaling behavior > Scaling performed by: _Application_.
-
-[scaling behavior]: https://github.com/Genymobile/scrcpy/issues/40#issuecomment-424466723
-
-Also, to improve downscaling quality, trilinear filtering is enabled
-automatically if the renderer is OpenGL and if it supports mipmapping.
-
-On Windows, you might want to force OpenGL to enable mipmapping:
-
-```
-scrcpy --render-driver=opengl
-```
-
-
 ### Issue with Wayland
 
 By default, SDL uses x11 on Linux. The [video driver] can be changed via the
@@ -224,102 +215,15 @@ As a workaround, [disable "Block compositing"][kwin].
 
 ### Exception
 
-There may be many reasons. One common cause is that the hardware encoder of your
-device is not able to encode at the given definition:
-
-> ```
-> ERROR: Exception on thread Thread[main,5,main]
-> android.media.MediaCodec$CodecException: Error 0xfffffc0e
-> ...
-> Exit due to uncaughtException in main thread:
-> ERROR: Could not open video stream
-> INFO: Initial texture: 1080x2336
-> ```
-
-or
-
-> ```
-> ERROR: Exception on thread Thread[main,5,main]
-> java.lang.IllegalStateException
->         at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
-> ```
-
-Just try with a lower definition:
+If you get any exception related to `MediaCodec`:
 
 ```
-scrcpy -m 1920
-scrcpy -m 1024
-scrcpy -m 800
+ERROR: Exception on thread Thread[main,5,main]
+java.lang.IllegalStateException
+        at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
 ```
 
-Since scrcpy v1.22, scrcpy automatically tries again with a lower definition
-before failing. This behavior can be disabled with `--no-downsize-on-error`.
-
-You could also try another [encoder](README.md#encoder).
-
-
-If you encounter this exception on Android 12, then just upgrade to scrcpy >=
-1.18 (see [#2129]):
-
-```
-> ERROR: Exception on thread Thread[main,5,main]
-java.lang.AssertionError: java.lang.reflect.InvocationTargetException
-    at com.genymobile.scrcpy.wrappers.SurfaceControl.setDisplaySurface(SurfaceControl.java:75)
-    ...
-Caused by: java.lang.reflect.InvocationTargetException
-    at java.lang.reflect.Method.invoke(Native Method)
-    at com.genymobile.scrcpy.wrappers.SurfaceControl.setDisplaySurface(SurfaceControl.java:73)
-    ... 7 more
-Caused by: java.lang.IllegalArgumentException: displayToken must not be null
-    at android.view.SurfaceControl$Transaction.setDisplaySurface(SurfaceControl.java:3067)
-    at android.view.SurfaceControl.setDisplaySurface(SurfaceControl.java:2147)
-    ... 9 more
-```
-
-[#2129]: https://github.com/Genymobile/scrcpy/issues/2129
-
-
-## Command line on Windows
-
-Since v1.22, a "shortcut" has been added to directly open a terminal in the
-scrcpy directory. Double-click on `open_a_terminal_here.bat`, then type your
-command. For example:
-
-```
-scrcpy --record file.mkv
-```
-
-You could also open a terminal and go to the scrcpy folder manually:
-
- 1. Press <kbd>Windows</kbd>+<kbd>r</kbd>, this opens a dialog box.
- 2. Type `cmd` and press <kbd>Enter</kbd>, this opens a terminal.
- 3. Go to your _scrcpy_ directory, by typing (adapt the path):
-
-    ```bat
-    cd C:\Users\user\Downloads\scrcpy-win64-xxx
-    ```
-
-    and press <kbd>Enter</kbd>
- 4. Type your command. For example:
-
-    ```bat
-    scrcpy --record file.mkv
-    ```
-
-If you plan to always use the same arguments, create a file `myscrcpy.bat`
-(enable [show file extensions] to avoid confusion) in the `scrcpy` directory,
-containing your command. For example:
-
-```bat
-scrcpy --prefer-text --turn-screen-off --stay-awake
-```
-
-Then just double-click on that file.
-
-You could also edit (a copy of) `scrcpy-console.bat` or `scrcpy-noconsole.vbs`
-to add some arguments.
-
-[show file extensions]: https://www.howtogeek.com/205086/beginner-how-to-make-windows-show-file-extensions/
+then try with another [encoder](doc/video.md#encoder).
 
 
 ## Translations
@@ -328,4 +232,4 @@ Translations of this FAQ in other languages are available in the [wiki].
 
 [wiki]: https://github.com/Genymobile/scrcpy/wiki
 
-Only this README file is guaranteed to be up-to-date.
+Only this FAQ file is guaranteed to be up-to-date.

LICENSE

@@ -188,7 +188,7 @@
       identification within third-party archives.
 
    Copyright (C) 2018 Genymobile
-   Copyright (C) 2018-2022 Romain Vimont
+   Copyright (C) 2018-2024 Romain Vimont
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

app/data/bash-completion/scrcpy

@@ -2,57 +2,90 @@ _scrcpy() {
     local cur prev words cword
     local opts="
         --always-on-top
-        -b --bit-rate=
-        --codec-options=
+        --audio-bit-rate=
+        --audio-buffer=
+        --audio-codec=
+        --audio-codec-options=
+        --audio-encoder=
+        --audio-source=
+        --audio-output-buffer=
+        -b --video-bit-rate=
+        --camera-ar=
+        --camera-id=
+        --camera-facing=
+        --camera-fps=
+        --camera-high-speed
+        --camera-size=
         --crop=
         -d --select-usb
         --disable-screensaver
-        --display=
         --display-buffer=
+        --display-id=
+        --display-orientation=
         -e --select-tcpip
-        --encoder=
+        -f --fullscreen
         --force-adb-forward
         --forward-all-clicks
-        -f --fullscreen
-        -K --hid-keyboard
         -h --help
+        -K
+        --keyboard=
+        --kill-adb-on-close
         --legacy-paste
+        --list-camera-sizes
+        --list-cameras
+        --list-displays
+        --list-encoders
         --lock-video-orientation
         --lock-video-orientation=
-        --max-fps=
-        -M --hid-mouse
         -m --max-size=
+        -M
+        --max-fps=
+        --mouse=
+        -n --no-control
+        -N --no-playback
+        --no-audio
+        --no-audio-playback
         --no-cleanup
-        --no-clipboard-on-error
+        --no-clipboard-autosync
         --no-downsize-on-error
-        -n --no-control
-        -N --no-display
         --no-key-repeat
         --no-mipmaps
         --no-power-on
+        --no-video
+        --no-video-playback
+        --orientation=
         --otg
         -p --port=
+        --pause-on-exit
+        --pause-on-exit=
         --power-off-on-close
         --prefer-text
         --print-fps
         --push-target=
-        --raw-key-events
         -r --record=
+        --raw-key-events
         --record-format=
+        --record-orientation=
         --render-driver=
+        --require-audio
         --rotation=
         -s --serial=
-        --shortcut-mod=
         -S --turn-screen-off
+        --shortcut-mod=
         -t --show-touches
         --tcpip
         --tcpip=
+        --time-limit=
         --tunnel-host=
         --tunnel-port=
         --v4l2-buffer=
         --v4l2-sink=
-        -V --verbosity=
         -v --version
+        -V --verbosity=
+        --video-codec=
+        --video-codec-options=
+        --video-encoder=
+        --video-source=
         -w --stay-awake
         --window-borderless
         --window-title=
@@ -64,8 +97,48 @@ _scrcpy() {
     _init_completion -s || return
 
     case "$prev" in
+        --video-codec)
+            COMPREPLY=($(compgen -W 'h264 h265 av1' -- "$cur"))
+            return
+            ;;
+        --audio-codec)
+            COMPREPLY=($(compgen -W 'opus aac flac raw' -- "$cur"))
+            return
+            ;;
+        --video-source)
+            COMPREPLY=($(compgen -W 'display camera' -- "$cur"))
+            return
+            ;;
+        --audio-source)
+            COMPREPLY=($(compgen -W 'output mic' -- "$cur"))
+            return
+            ;;
+        --camera-facing)
+            COMPREPLY=($(compgen -W 'front back external' -- "$cur"))
+            return
+            ;;
+        --keyboard)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
+        --mouse)
+            COMPREPLY=($(compgen -W 'disabled sdk uhid aoa' -- "$cur"))
+            return
+            ;;
+        --orientation|--display-orientation)
+            COMPREPLY=($(compgen -W '0 90 180 270 flip0 flip90 flip180 flip270' -- "$cur"))
+            return
+            ;;
+        --record-orientation)
+            COMPREPLY=($(compgen -W '0 90 180 270' -- "$cur"))
+            return
+            ;;
         --lock-video-orientation)
-            COMPREPLY=($(compgen -W 'unlocked initial 0 1 2 3' -- "$cur"))
+            COMPREPLY=($(compgen -W 'unlocked initial 0 90 180 270' -- "$cur"))
+            return
+            ;;
+        --pause-on-exit)
+            COMPREPLY=($(compgen -W 'true false if-error' -- "$cur"))
             return
             ;;
         -r|--record)
@@ -73,17 +146,13 @@ _scrcpy() {
             return
             ;;
         --record-format)
-            COMPREPLY=($(compgen -W 'mkv mp4' -- "$cur"))
+            COMPREPLY=($(compgen -W 'mp4 mkv m4a mka opus aac flac wav' -- "$cur"))
             return
             ;;
         --render-driver)
             COMPREPLY=($(compgen -W 'direct3d opengl opengles2 opengles metal software' -- "$cur"))
             return
             ;;
-        --rotation)
-            COMPREPLY=($(compgen -W '0 1 2 3' -- "$cur"))
-            return
-            ;;
         --shortcut-mod)
             # Only auto-complete a single key
             COMPREPLY=($(compgen -W 'lctrl rctrl lalt ralt lsuper rsuper' -- "$cur"))
@@ -98,20 +167,30 @@ _scrcpy() {
             COMPREPLY=($(compgen -W "$("${ADB:-adb}" devices | awk '$2 == "device" {print $1}')" -- ${cur}))
             return
             ;;
-        -b|--bitrate \
-        |--codec-options \
+        --audio-bit-rate \
+        |--audio-buffer \
+        |-b|--video-bit-rate \
+        |--audio-codec-options \
+        |--audio-encoder \
+        |--audio-output-buffer \
+        |--camera-ar \
+        |--camera-id \
+        |--camera-fps \
+        |--camera-size \
         |--crop \
-        |--display \
+        |--display-id \
         |--display-buffer \
-        |--encoder \
         |--max-fps \
         |-m|--max-size \
         |-p|--port \
         |--push-target \
+        |--rotation \
         |--tunnel-host \
         |--tunnel-port \
         |--v4l2-buffer \
         |--v4l2-sink \
+        |--video-codec-options \
+        |--video-encoder \
         |--tcpip \
         |--window-*)
             # Option accepting an argument, but nothing to auto-complete

app/data/scrcpy-console.bat

@@ -1,4 +1,2 @@
 @echo off
-scrcpy.exe %*
-:: if the exit code is >= 1, then pause
-if errorlevel 1 pause
+scrcpy.exe --pause-on-exit=if-error %*

app/data/scrcpy-console.desktop

@@ -5,7 +5,7 @@ Comment=Display and control your Android device
 # For some users, the PATH or ADB environment variables are set from the shell
 # startup file, like .bashrc or .zshrc… Run an interactive shell to get
 # environment correctly initialized.
-Exec=/bin/bash --norc --noprofile -i -c '"$SHELL" -i -c scrcpy || read -p "Press any key to quit..."'
+Exec=/bin/sh -c "\\$SHELL -i -c 'scrcpy --pause-on-exit=if-error'"
 Icon=scrcpy
 Terminal=true
 Type=Application

app/data/scrcpy.desktop

@@ -5,7 +5,7 @@ Comment=Display and control your Android device
 # For some users, the PATH or ADB environment variables are set from the shell
 # startup file, like .bashrc or .zshrc… Run an interactive shell to get
 # environment correctly initialized.
-Exec=/bin/sh -c '"$SHELL" -i -c scrcpy'
+Exec=/bin/sh -c "\\$SHELL -i -c scrcpy"
 Icon=scrcpy
 Terminal=false
 Type=Application

app/data/zsh-completion/_scrcpy

@@ -9,55 +9,86 @@ local arguments
 
 arguments=(
     '--always-on-top[Make scrcpy window always on top \(above other windows\)]'
-    {-b,--bit-rate=}'[Encode the video at the given bit-rate]'
-    '--codec-options=[Set a list of comma-separated key\:type=value options for the device encoder]'
+    '--audio-bit-rate=[Encode the audio at the given bit-rate]'
+    '--audio-buffer=[Configure the audio buffering delay (in milliseconds)]'
+    '--audio-codec=[Select the audio codec]:codec:(opus aac flac raw)'
+    '--audio-codec-options=[Set a list of comma-separated key\:type=value options for the device audio encoder]'
+    '--audio-encoder=[Use a specific MediaCodec audio encoder]'
+    '--audio-source=[Select the audio source]:source:(output mic)'
+    '--audio-output-buffer=[Configure the size of the SDL audio output buffer (in milliseconds)]'
+    {-b,--video-bit-rate=}'[Encode the video at the given bit-rate]'
+    '--camera-ar=[Select the camera size by its aspect ratio]'
+    '--camera-high-speed=[Enable high-speed camera capture mode]'
+    '--camera-id=[Specify the camera id to mirror]'
+    '--camera-facing=[Select the device camera by its facing direction]:facing:(front back external)'
+    '--camera-fps=[Specify the camera capture frame rate]'
+    '--camera-size=[Specify an explicit camera capture size]'
     '--crop=[\[width\:height\:x\:y\] Crop the device screen on the server]'
     {-d,--select-usb}'[Use USB device]'
     '--disable-screensaver[Disable screensaver while scrcpy is running]'
-    '--display=[Specify the display id to mirror]'
     '--display-buffer=[Add a buffering delay \(in milliseconds\) before displaying]'
+    '--display-id=[Specify the display id to mirror]'
+    '--display-orientation=[Set the initial display orientation]:orientation values:(0 90 180 270 flip0 flip90 flip180 flip270)'
     {-e,--select-tcpip}'[Use TCP/IP device]'
-    '--encoder=[Use a specific MediaCodec encoder \(must be a H.264 encoder\)]'
+    {-f,--fullscreen}'[Start in fullscreen]'
     '--force-adb-forward[Do not attempt to use \"adb reverse\" to connect to the device]'
     '--forward-all-clicks[Forward clicks to device]'
-    {-f,--fullscreen}'[Start in fullscreen]'
-    {-K,--hid-keyboard}'[Simulate a physical keyboard by using HID over AOAv2]'
     {-h,--help}'[Print the help]'
+    '-K[Use UHID keyboard (same as --keyboard=uhid)]'
+    '--keyboard[Set the keyboard input mode]:mode:(disabled sdk uhid aoa)'
+    '--kill-adb-on-close[Kill adb when scrcpy terminates]'
     '--legacy-paste[Inject computer clipboard text as a sequence of key events on Ctrl+v]'
-    '--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 1 2 3)'
-    '--max-fps=[Limit the frame rate of screen capture]'
-    {-M,--hid-mouse}'[Simulate a physical mouse by using HID over AOAv2]'
+    '--list-camera-sizes[List the valid camera capture sizes]'
+    '--list-cameras[List cameras available on the device]'
+    '--list-displays[List displays available on the device]'
+    '--list-encoders[List video and audio encoders available on the device]'
+    '--lock-video-orientation=[Lock video orientation]:orientation:(unlocked initial 0 90 180 270)'
     {-m,--max-size=}'[Limit both the width and height of the video to value]'
+    '-M[Use UHID mouse (same as --mouse=uhid)]'
+    '--max-fps=[Limit the frame rate of screen capture]'
+    '--mouse[Set the mouse input mode]:mode:(disabled sdk uhid aoa)'
+    {-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
+    {-N,--no-playback}'[Disable video and audio playback]'
+    '--no-audio[Disable audio forwarding]'
+    '--no-audio-playback[Disable audio playback]'
     '--no-cleanup[Disable device cleanup actions on exit]'
     '--no-clipboard-autosync[Disable automatic clipboard synchronization]'
     '--no-downsize-on-error[Disable lowering definition on MediaCodec error]'
-    {-n,--no-control}'[Disable device control \(mirror the device in read only\)]'
-    {-N,--no-display}'[Do not display device \(during screen recording or when V4L2 sink is enabled\)]'
     '--no-key-repeat[Do not forward repeated key events when a key is held down]'
     '--no-mipmaps[Disable the generation of mipmaps]'
     '--no-power-on[Do not power on the device on start]'
+    '--no-video[Disable video forwarding]'
+    '--no-video-playback[Disable video playback]'
+    '--orientation=[Set the video orientation]:orientation values:(0 90 180 270 flip0 flip90 flip180 flip270)'
     '--otg[Run in OTG mode \(simulating physical keyboard and mouse\)]'
     {-p,--port=}'[\[port\[\:port\]\] Set the TCP port \(range\) used by the client to listen]'
+    '--pause-on-exit=[Make scrcpy pause before exiting]:mode:(true false if-error)'
     '--power-off-on-close[Turn the device screen off when closing scrcpy]'
     '--prefer-text[Inject alpha characters and space as text events instead of key events]'
     '--print-fps[Start FPS counter, to print frame logs to the console]'
     '--push-target=[Set the target directory for pushing files to the device by drag and drop]'
-    '--raw-key-events[Inject key events for all input keys, and ignore text events]'
     {-r,--record=}'[Record screen to file]:record file:_files'
-    '--record-format=[Force recording format]:format:(mp4 mkv)'
+    '--raw-key-events[Inject key events for all input keys, and ignore text events]'
+    '--record-format=[Force recording format]:format:(mp4 mkv m4a mka opus aac flac wav)'
+    '--record-orientation=[Set the record orientation]:orientation values:(0 90 180 270)'
     '--render-driver=[Request SDL to use the given render driver]:driver name:(direct3d opengl opengles2 opengles metal software)'
-    '--rotation=[Set the initial display rotation]:rotation values:(0 1 2 3)'
+    '--require-audio=[Make scrcpy fail if audio is enabled but does not work]'
     {-s,--serial=}'[The device serial number \(mandatory for multiple devices only\)]:serial:($("${ADB-adb}" devices | awk '\''$2 == "device" {print $1}'\''))'
-    '--shortcut-mod=[\[key1,key2+key3,...\] Specify the modifiers to use for scrcpy shortcuts]:shortcut mod:(lctrl rctrl lalt ralt lsuper rsuper)'
     {-S,--turn-screen-off}'[Turn the device screen off immediately]'
+    '--shortcut-mod=[\[key1,key2+key3,...\] Specify the modifiers to use for scrcpy shortcuts]:shortcut mod:(lctrl rctrl lalt ralt lsuper rsuper)'
     {-t,--show-touches}'[Show physical touches]'
     '--tcpip[\(optional \[ip\:port\]\) Configure and connect the device over TCP/IP]'
+    '--time-limit=[Set the maximum mirroring time, in seconds]'
     '--tunnel-host=[Set the IP address of the adb tunnel to reach the scrcpy server]'
     '--tunnel-port=[Set the TCP port of the adb tunnel to reach the scrcpy server]'
     '--v4l2-buffer=[Add a buffering delay \(in milliseconds\) before pushing frames]'
     '--v4l2-sink=[\[\/dev\/videoN\] Output to v4l2loopback device]'
-    {-V,--verbosity=}'[Set the log level]:verbosity:(verbose debug info warn error)'
     {-v,--version}'[Print the version of scrcpy]'
+    {-V,--verbosity=}'[Set the log level]:verbosity:(verbose debug info warn error)'
+    '--video-codec=[Select the video codec]:codec:(h264 h265 av1)'
+    '--video-codec-options=[Set a list of comma-separated key\:type=value options for the device video encoder]'
+    '--video-encoder=[Use a specific MediaCodec video encoder]'
+    '--video-source=[Select the video source]:source:(display camera)'
     {-w,--stay-awake}'[Keep the device on while scrcpy is running, when the device is plugged in]'
     '--window-borderless[Disable window decorations \(display borderless window\)]'
     '--window-title=[Set a custom window title]'

app/deps/.gitignore

@@ -0,0 +1 @@
+/work

app/deps/README

@@ -0,0 +1,27 @@
+This directory (app/deps/) contains:
+
+*.sh          : shell scripts to download and build dependencies
+
+patches/      : patches to fix dependencies (used by scripts)
+
+work/sources/ : downloaded tarballs and extracted folders
+                    ffmpeg-6.1.1.tar.xz
+                    ffmpeg-6.1.1/
+                    libusb-1.0.27.tar.gz
+                    libusb-1.0.27/
+                    ...
+work/build/   : build dirs for each dependency/version/architecture
+                    ffmpeg-6.1.1/win32/
+                    ffmpeg-6.1.1/win64/
+                    libusb-1.0.27/win32/
+                    libusb-1.0.27/win64/
+                    ...
+work/install/ : install dirs for each architexture
+                    win32/bin/
+                    win32/include/
+                    win32/lib/
+                    win32/share/
+                    win64/bin/
+                    win64/include/
+                    win64/lib/
+                    win64/share/

app/deps/adb.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=34.0.5
+FILENAME=platform-tools_r$VERSION-windows.zip
+PROJECT_DIR=platform-tools-$VERSION
+SHA256SUM=3f8320152704377de150418a3c4c9d07d16d80a6c0d0d8f7289c22c499e33571
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://dl.google.com/android/repository/$FILENAME" "$FILENAME" "$SHA256SUM"
+    mkdir -p "$PROJECT_DIR"
+    cd "$PROJECT_DIR"
+    ZIP_PREFIX=platform-tools
+    unzip "../$FILENAME" \
+        "$ZIP_PREFIX"/AdbWinApi.dll \
+        "$ZIP_PREFIX"/AdbWinUsbApi.dll \
+        "$ZIP_PREFIX"/adb.exe
+    mv "$ZIP_PREFIX"/* .
+    rmdir "$ZIP_PREFIX"
+fi
+
+mkdir -p "$INSTALL_DIR/$HOST/bin"
+cd "$INSTALL_DIR/$HOST/bin"
+cp -r "$SOURCES_DIR/$PROJECT_DIR"/. "$INSTALL_DIR/$HOST/bin/"

app/deps/common

@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# This file is intended to be sourced by other scripts, not executed
+
+if [[ $# != 1 ]]
+then
+    # <host>: win32 or win64
+    echo "Syntax: $0 <host>" >&2
+    exit 1
+fi
+
+HOST="$1"
+
+if [[ "$HOST" = win32 ]]
+then
+    HOST_TRIPLET=i686-w64-mingw32
+elif [[ "$HOST" = win64 ]]
+then
+    HOST_TRIPLET=x86_64-w64-mingw32
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+
+PATCHES_DIR="$PWD/patches"
+
+WORK_DIR="$PWD/work"
+SOURCES_DIR="$WORK_DIR/sources"
+BUILD_DIR="$WORK_DIR/build"
+INSTALL_DIR="$WORK_DIR/install"
+
+mkdir -p "$INSTALL_DIR" "$SOURCES_DIR" "$WORK_DIR"
+
+checksum() {
+    local file="$1"
+    local sum="$2"
+    echo "$file: verifying checksum..."
+    echo "$sum  $file" | sha256sum -c
+}
+
+get_file() {
+    local url="$1"
+    local file="$2"
+    local sum="$3"
+    if [[ -f "$file" ]]
+    then
+        echo "$file: found"
+    else
+        echo "$file: not found, downloading..."
+        wget "$url" -O "$file"
+    fi
+    checksum "$file" "$sum"
+}

app/deps/ffmpeg.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=6.1.1
+FILENAME=ffmpeg-$VERSION.tar.xz
+PROJECT_DIR=ffmpeg-$VERSION
+SHA256SUM=8684f4b00f94b85461884c3719382f1261f0d9eb3d59640a1f4ac0873616f968
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://ffmpeg.org/releases/$FILENAME" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+    patch -d "$PROJECT_DIR" -p1 < "$PATCHES_DIR"/ffmpeg-6.1-fix-build.patch
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+if [[ "$HOST" = win32 ]]
+then
+    ARCH=x86
+elif [[ "$HOST" = win64 ]]
+then
+    ARCH=x86_64
+else
+    echo "Unsupported host: $HOST" >&2
+    exit 1
+fi
+
+# -static-libgcc to avoid missing libgcc_s_dw2-1.dll
+# -static to avoid dynamic dependency to zlib
+export CFLAGS='-static-libgcc -static'
+export CXXFLAGS="$CFLAGS"
+export LDFLAGS='-static-libgcc -static'
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --enable-cross-compile \
+        --target-os=mingw32 \
+        --arch="$ARCH" \
+        --cross-prefix="${HOST_TRIPLET}-" \
+        --cc="${HOST_TRIPLET}-gcc" \
+        --extra-cflags="-O2 -fPIC" \
+        --enable-shared \
+        --disable-static \
+        --disable-programs \
+        --disable-doc \
+        --disable-swscale \
+        --disable-postproc \
+        --disable-avfilter \
+        --disable-avdevice \
+        --disable-network \
+        --disable-everything \
+        --enable-swresample \
+        --enable-decoder=h264 \
+        --enable-decoder=hevc \
+        --enable-decoder=av1 \
+        --enable-decoder=pcm_s16le \
+        --enable-decoder=opus \
+        --enable-decoder=aac \
+        --enable-decoder=flac \
+        --enable-decoder=png \
+        --enable-protocol=file \
+        --enable-demuxer=image2 \
+        --enable-parser=png \
+        --enable-zlib \
+        --enable-muxer=matroska \
+        --enable-muxer=mp4 \
+        --enable-muxer=opus \
+        --enable-muxer=flac \
+        --enable-muxer=wav \
+        --disable-vulkan
+fi
+
+make -j
+make install

app/deps/libusb.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=1.0.27
+FILENAME=libusb-$VERSION.tar.bz2
+PROJECT_DIR=libusb-$VERSION
+SHA256SUM=ffaa41d741a8a3bee244ac8e54a72ea05bf2879663c098c82fc5757853441575
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libusb/libusb/releases/download/v$VERSION/libusb-$VERSION.tar.bz2" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+make install-strip

app/deps/patches/ffmpeg-6.1-fix-build.patch

@@ -0,0 +1,27 @@
+From 03c80197afb324da38c9b70254231e3fdcfa68fc Mon Sep 17 00:00:00 2001
+From: Romain Vimont <rom@rom1v.com>
+Date: Sun, 12 Nov 2023 17:58:50 +0100
+Subject: [PATCH] Fix FFmpeg 6.1 build
+
+Build failed on tag n6.1 With --enable-decoder=av1 but without
+--enable-muxer=av1.
+---
+ libavcodec/Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/libavcodec/Makefile b/libavcodec/Makefile
+index 580a8d6b54..aff19b670c 100644
+--- a/libavcodec/Makefile
++++ b/libavcodec/Makefile
+@@ -249,7 +249,7 @@ OBJS-$(CONFIG_ATRAC3PAL_DECODER)       += atrac3plusdec.o atrac3plus.o \
+ OBJS-$(CONFIG_ATRAC9_DECODER)          += atrac9dec.o
+ OBJS-$(CONFIG_AURA_DECODER)            += cyuv.o
+ OBJS-$(CONFIG_AURA2_DECODER)           += aura.o
+-OBJS-$(CONFIG_AV1_DECODER)             += av1dec.o
++OBJS-$(CONFIG_AV1_DECODER)             += av1dec.o av1_parse.o
+ OBJS-$(CONFIG_AV1_CUVID_DECODER)       += cuviddec.o
+ OBJS-$(CONFIG_AV1_MEDIACODEC_DECODER)  += mediacodecdec.o
+ OBJS-$(CONFIG_AV1_MEDIACODEC_ENCODER)  += mediacodecenc.o
+-- 
+2.42.0
+

app/deps/sdl.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+set -ex
+DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
+cd "$DEPS_DIR"
+. common
+
+VERSION=2.28.5
+FILENAME=SDL-$VERSION.tar.gz
+PROJECT_DIR=SDL-release-$VERSION
+SHA256SUM=9f0556e4a24ef5b267010038ad9e9948b62f236d5bcc4b22179f95ef62d84023
+
+cd "$SOURCES_DIR"
+
+if [[ -d "$PROJECT_DIR" ]]
+then
+    echo "$PWD/$PROJECT_DIR" found
+else
+    get_file "https://github.com/libsdl-org/SDL/archive/refs/tags/release-$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
+    tar xf "$FILENAME"  # First level directory is "$PROJECT_DIR"
+fi
+
+mkdir -p "$BUILD_DIR/$PROJECT_DIR"
+cd "$BUILD_DIR/$PROJECT_DIR"
+
+export CFLAGS='-O2'
+export CXXFLAGS="$CFLAGS"
+
+if [[ -d "$HOST" ]]
+then
+    echo "'$PWD/$HOST' already exists, not reconfigured"
+    cd "$HOST"
+else
+    mkdir "$HOST"
+    cd "$HOST"
+
+    "$SOURCES_DIR/$PROJECT_DIR"/configure \
+        --prefix="$INSTALL_DIR/$HOST" \
+        --host="$HOST_TRIPLET" \
+        --enable-shared \
+        --disable-static
+fi
+
+make -j
+# There is no "make install-strip"
+make install
+# Strip manually
+${HOST_TRIPLET}-strip "$INSTALL_DIR/$HOST/bin/SDL2.dll"

app/meson.build

@@ -4,44 +4,59 @@ src = [
     'src/adb/adb_device.c',
     'src/adb/adb_parser.c',
     'src/adb/adb_tunnel.c',
+    'src/audio_player.c',
     'src/cli.c',
     'src/clock.c',
     'src/compat.c',
     'src/control_msg.c',
     'src/controller.c',
     'src/decoder.c',
+    'src/delay_buffer.c',
     'src/demuxer.c',
     'src/device_msg.c',
+    'src/display.c',
     'src/icon.c',
     'src/file_pusher.c',
     'src/fps_counter.c',
     'src/frame_buffer.c',
     'src/input_manager.c',
-    'src/keyboard_inject.c',
-    'src/mouse_inject.c',
+    'src/keyboard_sdk.c',
+    'src/mouse_sdk.c',
     'src/opengl.c',
     'src/options.c',
+    'src/packet_merger.c',
     'src/receiver.c',
     'src/recorder.c',
     'src/scrcpy.c',
     'src/screen.c',
     'src/server.c',
     'src/version.c',
-    'src/video_buffer.c',
+    'src/hid/hid_keyboard.c',
+    'src/hid/hid_mouse.c',
+    'src/trait/frame_source.c',
+    'src/trait/packet_source.c',
+    'src/uhid/keyboard_uhid.c',
+    'src/uhid/mouse_uhid.c',
+    'src/uhid/uhid_output.c',
     'src/util/acksync.c',
+    'src/util/audiobuf.c',
+    'src/util/average.c',
     'src/util/file.c',
     'src/util/intmap.c',
     'src/util/intr.c',
     'src/util/log.c',
+    'src/util/memory.c',
     'src/util/net.c',
     'src/util/net_intr.c',
     'src/util/process.c',
     'src/util/process_intr.c',
+    'src/util/rand.c',
     'src/util/strbuf.c',
     'src/util/str.c',
     'src/util/term.c',
     'src/util/thread.c',
     'src/util/tick.c',
+    'src/util/timeout.c',
 ]
 
 conf = configuration_data()
@@ -78,8 +93,8 @@ usb_support = get_option('usb')
 if usb_support
     src += [
         'src/usb/aoa_hid.c',
-        'src/usb/hid_keyboard.c',
-        'src/usb/hid_mouse.c',
+        'src/usb/keyboard_aoa.c',
+        'src/usb/mouse_aoa.c',
         'src/usb/scrcpy_otg.c',
         'src/usb/screen_otg.c',
         'src/usb/usb.c',
@@ -88,81 +103,24 @@ endif
 
 cc = meson.get_compiler('c')
 
-crossbuild_windows = meson.is_cross_build() and host_machine.system() == 'windows'
-
-if not crossbuild_windows
-
-    # native build
-    dependencies = [
-        dependency('libavformat', version: '>= 57.33'),
-        dependency('libavcodec', version: '>= 57.37'),
-        dependency('libavutil'),
-        dependency('sdl2', version: '>= 2.0.5'),
-    ]
-
-    if v4l2_support
-        dependencies += dependency('libavdevice')
-    endif
-
-    if usb_support
-        dependencies += dependency('libusb-1.0')
-    endif
-
-else
-    # cross-compile mingw32 build (from Linux to Windows)
-    prebuilt_sdl2 = meson.get_cross_property('prebuilt_sdl2')
-    sdl2_bin_dir = meson.current_source_dir() + '/prebuilt-deps/data/' + prebuilt_sdl2 + '/bin'
-    sdl2_lib_dir = meson.current_source_dir() + '/prebuilt-deps/data/' + prebuilt_sdl2 + '/lib'
-    sdl2_include_dir = 'prebuilt-deps/data/' + prebuilt_sdl2 + '/include'
-
-    sdl2 = declare_dependency(
-        dependencies: [
-            cc.find_library('SDL2', dirs: sdl2_bin_dir),
-            cc.find_library('SDL2main', dirs: sdl2_lib_dir),
-        ],
-        include_directories: include_directories(sdl2_include_dir)
-    )
-
-    prebuilt_ffmpeg = meson.get_cross_property('prebuilt_ffmpeg')
-    ffmpeg_bin_dir = meson.current_source_dir() + '/prebuilt-deps/data/' + prebuilt_ffmpeg + '/bin'
-    ffmpeg_include_dir = 'prebuilt-deps/data/' + prebuilt_ffmpeg + '/include'
-
-    # ffmpeg versions are different for win32 and win64 builds
-    ffmpeg_avcodec = meson.get_cross_property('ffmpeg_avcodec')
-    ffmpeg_avformat = meson.get_cross_property('ffmpeg_avformat')
-    ffmpeg_avutil = meson.get_cross_property('ffmpeg_avutil')
-
-    ffmpeg = declare_dependency(
-        dependencies: [
-            cc.find_library(ffmpeg_avcodec, dirs: ffmpeg_bin_dir),
-            cc.find_library(ffmpeg_avformat, dirs: ffmpeg_bin_dir),
-            cc.find_library(ffmpeg_avutil, dirs: ffmpeg_bin_dir),
-        ],
-        include_directories: include_directories(ffmpeg_include_dir)
-    )
-
-    prebuilt_libusb = meson.get_cross_property('prebuilt_libusb')
-    prebuilt_libusb_root = meson.get_cross_property('prebuilt_libusb_root')
-    libusb_bin_dir = meson.current_source_dir() + '/prebuilt-deps/data/' + prebuilt_libusb
-    libusb_include_dir = 'prebuilt-deps/data/' + prebuilt_libusb_root + '/include'
-
-    libusb = declare_dependency(
-        dependencies: [
-            cc.find_library('msys-usb-1.0', dirs: libusb_bin_dir),
-        ],
-        include_directories: include_directories(libusb_include_dir)
-    )
+dependencies = [
+    dependency('libavformat', version: '>= 57.33'),
+    dependency('libavcodec', version: '>= 57.37'),
+    dependency('libavutil'),
+    dependency('libswresample'),
+    dependency('sdl2', version: '>= 2.0.5'),
+]
 
-    dependencies = [
-        ffmpeg,
-        sdl2,
-        libusb,
-        cc.find_library('mingw32')
-    ]
+if v4l2_support
+    dependencies += dependency('libavdevice')
+endif
 
+if usb_support
+    dependencies += dependency('libusb-1.0')
 endif
 
 if host_machine.system() == 'windows'
+    dependencies += cc.find_library('mingw32')
     dependencies += cc.find_library('ws2_32')
 endif
 
@@ -170,6 +128,9 @@ check_functions = [
     'strdup',
     'asprintf',
     'vasprintf',
+    'nrand48',
+    'jrand48',
+    'reallocarray',
 ]
 
 foreach f : check_functions
@@ -197,10 +158,6 @@ conf.set('PORTABLE', get_option('portable'))
 conf.set('DEFAULT_LOCAL_PORT_RANGE_FIRST', '27183')
 conf.set('DEFAULT_LOCAL_PORT_RANGE_LAST', '27199')
 
-# the default video bitrate, in bits/second
-# overridden by option --bit-rate
-conf.set('DEFAULT_BIT_RATE', '8000000')  # 8Mbps
-
 # run a server debugger and wait for a client to be attached
 conf.set('SERVER_DEBUGGER', get_option('server_debugger'))
 
@@ -260,8 +217,10 @@ if get_option('buildtype') == 'debug'
         ['test_binary', [
             'tests/test_binary.c',
         ]],
-        ['test_cbuf', [
-            'tests/test_cbuf.c',
+        ['test_audiobuf', [
+            'tests/test_audiobuf.c',
+            'src/util/audiobuf.c',
+            'src/util/memory.c',
         ]],
         ['test_cli', [
             'tests/test_cli.c',
@@ -273,10 +232,6 @@ if get_option('buildtype') == 'debug'
             'src/util/strbuf.c',
             'src/util/term.c',
         ]],
-        ['test_clock', [
-            'tests/test_clock.c',
-            'src/clock.c',
-        ]],
         ['test_control_msg_serialize', [
             'tests/test_control_msg_serialize.c',
             'src/control_msg.c',
@@ -287,8 +242,9 @@ if get_option('buildtype') == 'debug'
             'tests/test_device_msg_deserialize.c',
             'src/device_msg.c',
         ]],
-        ['test_queue', [
-            'tests/test_queue.c',
+        ['test_orientation', [
+            'tests/test_orientation.c',
+            'src/options.c',
         ]],
         ['test_strbuf', [
             'tests/test_strbuf.c',
@@ -299,13 +255,18 @@ if get_option('buildtype') == 'debug'
             'src/util/str.c',
             'src/util/strbuf.c',
         ]],
+        ['test_vecdeque', [
+            'tests/test_vecdeque.c',
+            'src/util/memory.c',
+        ]],
         ['test_vector', [
             'tests/test_vector.c',
         ]],
     ]
 
     foreach t : tests
-        exe = executable(t[0], t[1],
+        sources = t[1] + ['src/compat.c']
+        exe = executable(t[0], sources,
                          include_directories: src_dir,
                          dependencies: dependencies,
                          c_args: ['-DSDL_MAIN_HANDLED', '-DSC_TEST'])

app/prebuilt-deps/.gitignore

@@ -1 +0,0 @@
-/data

app/prebuilt-deps/common

@@ -1,22 +0,0 @@
-PREBUILT_DATA_DIR=data
-
-checksum() {
-    local file="$1"
-    local sum="$2"
-    echo "$file: verifying checksum..."
-    echo "$sum  $file" | sha256sum -c
-}
-
-get_file() {
-    local url="$1"
-    local file="$2"
-    local sum="$3"
-    if [[ -f "$file" ]]
-    then
-        echo "$file: found"
-    else
-        echo "$file: not found, downloading..."
-        wget "$url" -O "$file"
-    fi
-    checksum "$file" "$sum"
-}

app/prebuilt-deps/prepare-adb.sh

@@ -1,32 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=platform-tools-33.0.3
-
-FILENAME=platform-tools_r33.0.3-windows.zip
-SHA256SUM=1e59afd40a74c5c0eab0a9fad3f0faf8a674267106e0b19921be9f67081808c2
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://dl.google.com/android/repository/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=platform-tools
-unzip "../$FILENAME" \
-    "$ZIP_PREFIX"/AdbWinApi.dll \
-    "$ZIP_PREFIX"/AdbWinUsbApi.dll \
-    "$ZIP_PREFIX"/adb.exe
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-ffmpeg-win32.sh

@@ -1,45 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=ffmpeg-win32-4.3.1
-
-FILENAME_SHARED=ffmpeg-4.3.1-win32-shared.zip
-SHA256SUM_SHARED=357af9901a456f4dcbacd107e83a934d344c9cb07ddad8aaf80612eeab7d26d2
-
-FILENAME_DEV=ffmpeg-4.3.1-win32-dev.zip
-SHA256SUM_DEV=230efb08e9bcf225bd474da29676c70e591fc94d8790a740ca801408fddcb78b
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/Genymobile/scrcpy/releases/download/v1.16/$FILENAME_SHARED" \
-    "$FILENAME_SHARED" "$SHA256SUM_SHARED"
-get_file "https://github.com/Genymobile/scrcpy/releases/download/v1.16/$FILENAME_DEV" \
-    "$FILENAME_DEV" "$SHA256SUM_DEV"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX_SHARED=ffmpeg-4.3.1-win32-shared
-unzip "../$FILENAME_SHARED" \
-    "$ZIP_PREFIX_SHARED"/bin/avutil-56.dll \
-    "$ZIP_PREFIX_SHARED"/bin/avcodec-58.dll \
-    "$ZIP_PREFIX_SHARED"/bin/avformat-58.dll \
-    "$ZIP_PREFIX_SHARED"/bin/swresample-3.dll \
-    "$ZIP_PREFIX_SHARED"/bin/swscale-5.dll
-
-ZIP_PREFIX_DEV=ffmpeg-4.3.1-win32-dev
-unzip "../$FILENAME_DEV" \
-    "$ZIP_PREFIX_DEV/include/*"
-
-mv "$ZIP_PREFIX_SHARED"/* .
-mv "$ZIP_PREFIX_DEV"/* .
-rmdir "$ZIP_PREFIX_SHARED" "$ZIP_PREFIX_DEV"

app/prebuilt-deps/prepare-ffmpeg-win64.sh

@@ -1,36 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-VERSION=5.1.2
-DEP_DIR=ffmpeg-win64-$VERSION
-
-FILENAME=ffmpeg-$VERSION-full_build-shared.7z
-SHA256SUM=d9eb97b72d7cfdae4d0f7eaea59ccffb8c364d67d88018ea715d5e2e193f00e9
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/GyanD/codexffmpeg/releases/download/$VERSION/$FILENAME" \
-    "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-ZIP_PREFIX=ffmpeg-$VERSION-full_build-shared
-7z x "../$FILENAME" \
-    "$ZIP_PREFIX"/bin/avutil-57.dll \
-    "$ZIP_PREFIX"/bin/avcodec-59.dll \
-    "$ZIP_PREFIX"/bin/avformat-59.dll \
-    "$ZIP_PREFIX"/bin/swresample-4.dll \
-    "$ZIP_PREFIX"/bin/swscale-6.dll \
-    "$ZIP_PREFIX"/include
-mv "$ZIP_PREFIX"/* .
-rmdir "$ZIP_PREFIX"

app/prebuilt-deps/prepare-libusb.sh

@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=libusb-1.0.26
-
-FILENAME=libusb-1.0.26-binaries.7z
-SHA256SUM=9c242696342dbde9cdc47239391f71833939bf9f7aa2bbb28cdaabe890465ec5
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://github.com/libusb/libusb/releases/download/v1.0.26/$FILENAME" "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-# include/ is the same in all folders of the archive
-7z x "../$FILENAME" \
-    libusb-1.0.26-binaries/libusb-MinGW-Win32/bin/msys-usb-1.0.dll \
-    libusb-1.0.26-binaries/libusb-MinGW-x64/bin/msys-usb-1.0.dll \
-    libusb-1.0.26-binaries/libusb-MinGW-x64/include/
-
-mv libusb-1.0.26-binaries/libusb-MinGW-Win32/bin MinGW-Win32
-mv libusb-1.0.26-binaries/libusb-MinGW-x64/bin MinGW-x64
-mv libusb-1.0.26-binaries/libusb-MinGW-x64/include .
-rm -rf libusb-1.0.26-binaries

app/prebuilt-deps/prepare-sdl.sh

@@ -1,32 +0,0 @@
-#!/usr/bin/env bash
-set -e
-DIR=$(dirname ${BASH_SOURCE[0]})
-cd "$DIR"
-. common
-mkdir -p "$PREBUILT_DATA_DIR"
-cd "$PREBUILT_DATA_DIR"
-
-DEP_DIR=SDL2-2.26.1
-
-FILENAME=SDL2-devel-2.26.1-mingw.tar.gz
-SHA256SUM=aa43e1531a89551f9f9e14b27953a81d4ac946a9e574b5813cd0f2b36e83cc1c
-
-if [[ -d "$DEP_DIR" ]]
-then
-    echo "$DEP_DIR" found
-    exit 0
-fi
-
-get_file "https://libsdl.org/release/$FILENAME" "$FILENAME" "$SHA256SUM"
-
-mkdir "$DEP_DIR"
-cd "$DEP_DIR"
-
-TAR_PREFIX="$DEP_DIR" # root directory inside the tar has the same name
-tar xf "../$FILENAME" --strip-components=1 \
-    "$TAR_PREFIX"/i686-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/i686-w64-mingw32/include/ \
-    "$TAR_PREFIX"/i686-w64-mingw32/lib/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/bin/SDL2.dll \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/include/ \
-    "$TAR_PREFIX"/x86_64-w64-mingw32/lib/ \

app/scrcpy-windows.rc

@@ -13,7 +13,7 @@ BEGIN
       VALUE "LegalCopyright", "Romain Vimont, Genymobile"
       VALUE "OriginalFilename", "scrcpy.exe"
       VALUE "ProductName", "scrcpy"
-      VALUE "ProductVersion", "1.25"
+      VALUE "ProductVersion", "2.4"
     END
   END
   BLOCK "VarFileInfo"

app/scrcpy.1

@@ -20,20 +20,94 @@ provides display and control of Android devices connected on USB (or over TCP/IP
 Make scrcpy window always on top (above other windows).
 
 .TP
-.BI "\-b, \-\-bit\-rate " value
-Encode the video at the given bit\-rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
+.BI "\-\-audio\-bit\-rate " value
+Encode the audio at the given bit rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
 
-Default is 8000000.
+Default is 128K (128000).
 
 .TP
-.BI "\-\-codec\-options " key[:type]=value[,...]
-Set a list of comma-separated key:type=value options for the device encoder.
+.BI "\-\-audio\-buffer " ms
+Configure the audio buffering delay (in milliseconds).
+
+Lower values decrease the latency, but increase the likelyhood of buffer underrun (causing audio glitches).
+
+Default is 50.
+
+.TP
+.BI "\-\-audio\-codec " name
+Select an audio codec (opus, aac, flac or raw).
+
+Default is opus.
+
+.TP
+.BI "\-\-audio\-codec\-options " key\fR[:\fItype\fR]=\fIvalue\fR[,...]
+Set a list of comma-separated key:type=value options for the device audio encoder.
 
 The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'.
 
-The list of possible codec options is available in the Android documentation
-.UR https://d.android.com/reference/android/media/MediaFormat
-.UE .
+The list of possible codec options is available in the Android documentation:
+
+<https://d.android.com/reference/android/media/MediaFormat>
+
+.TP
+.BI "\-\-audio\-encoder " name
+Use a specific MediaCodec audio encoder (depending on the codec provided by \fB\-\-audio\-codec\fR).
+
+The available encoders can be listed by \fB\-\-list\-encoders\fR.
+
+.TP
+.BI "\-\-audio\-source " source
+Select the audio source (output or mic).
+
+Default is output.
+
+.TP
+.BI "\-\-audio\-output\-buffer " ms
+Configure the size of the SDL audio output buffer (in milliseconds).
+
+If you get "robotic" audio playback, you should test with a higher value (10). Do not change this setting otherwise.
+
+Default is 5.
+
+.TP
+.BI "\-b, \-\-video\-bit\-rate " value
+Encode the video at the given bit rate, expressed in bits/s. Unit suffixes are supported: '\fBK\fR' (x1000) and '\fBM\fR' (x1000000).
+
+Default is 8M (8000000).
+
+.TP
+.BI "\-\-camera\-ar " ar
+Select the camera size by its aspect ratio (+/- 10%).
+
+Possible values are "sensor" (use the camera sensor aspect ratio), "\fInum\fR:\fIden\fR" (e.g. "4:3") and "\fIvalue\fR" (e.g. "1.6").
+
+.TP
+.B \-\-camera\-high\-speed
+Enable high-speed camera capture mode.
+
+This mode is restricted to specific resolutions and frame rates, listed by \fB\-\-list\-camera\-sizes\fR.
+
+.TP
+.BI "\-\-camera\-id " id
+Specify the device camera id to mirror.
+
+The available camera ids can be listed by \fB\-\-list\-cameras\fR.
+
+.TP
+.BI "\-\-camera\-facing " facing
+Select the device camera by its facing direction.
+
+Possible values are "front", "back" and "external".
+
+.TP
+.BI "\-\-camera\-fps " fps
+Specify the camera capture frame rate.
+
+If not specified, Android's default frame rate (30 fps) is used.
+
+.TP
+.BI "\-\-camera\-size " width\fRx\fIheight
+Specify an explicit camera capture size.
 
 .TP
 .BI "\-\-crop " width\fR:\fIheight\fR:\fIx\fR:\fIy
@@ -50,23 +124,30 @@ Use USB device (if there is exactly one, like adb -d).
 Also see \fB\-e\fR (\fB\-\-select\-tcpip\fR).
 
 .TP
-.BI "\-\-disable-screensaver"
+.BI "\-\-disable\-screensaver"
 Disable screensaver while scrcpy is running.
 
 .TP
-.BI "\-\-display " id
-Specify the display id to mirror.
+.BI "\-\-display\-buffer " ms
+Add a buffering delay (in milliseconds) before displaying. This increases latency to compensate for jitter.
 
-The list of possible display ids can be listed by "adb shell dumpsys display"
-(search "mDisplayId=" in the output).
+Default is 0 (no buffering).
+
+.TP
+.BI "\-\-display\-id " id
+Specify the device display id to mirror.
+
+The available display ids can be listed by \fB\-\-list\-displays\fR.
 
 Default is 0.
 
 .TP
-.BI "\-\-display\-buffer ms
-Add a buffering delay (in milliseconds) before displaying. This increases latency to compensate for jitter.
+.BI "\-\-display\-orientation " value
+Set the initial display orientation.
 
-Default is 0 (no buffering).
+Possible values are 0, 90, 180, 270, flip0, flip90, flip180 and flip270. The number represents the clockwise rotation in degrees; the "flip" keyword applies a horizontal flip before the rotation.
+
+Default is 0.
 
 .TP
 .B \-e, \-\-select\-tcpip
@@ -75,8 +156,8 @@ Use TCP/IP device (if there is exactly one, like adb -e).
 Also see \fB\-d\fR (\fB\-\-select\-usb\fR).
 
 .TP
-.BI "\-\-encoder " name
-Use a specific MediaCodec encoder (must be a H.264 encoder).
+.B \-f, \-\-fullscreen
+Start in fullscreen.
 
 .TP
 .B \-\-force\-adb\-forward
@@ -86,29 +167,36 @@ Do not attempt to use "adb reverse" to connect to the device.
 .B \-\-forward\-all\-clicks
 By default, right-click triggers BACK (or POWER on) and middle-click triggers HOME. This option disables these shortcuts and forward the clicks to the device instead.
 
-.TP
-.B \-f, \-\-fullscreen
-Start in fullscreen.
-
 .TP
 .B \-h, \-\-help
 Print this help.
 
 .TP
-.B \-K, \-\-hid\-keyboard
-Simulate a physical keyboard by using HID over AOAv2.
+.B \-K
+Same as \fB\-\-keyboard=uhid\fR.
 
-This provides a better experience for IME users, and allows to generate non-ASCII characters, contrary to the default injection method.
+.TP
+.BI "\-\-keyboard " mode
+Select how to send keyboard inputs to the device.
 
-It may only work over USB.
+Possible values are "disabled", "sdk", "uhid" and "aoa":
 
-The keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly:
+ - "disabled" does not send keyboard inputs to the device.
+ - "sdk" uses the Android system API to deliver keyboard events to applications.
+ - "uhid" simulates a physical HID keyboard using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical HID keyboard using the AOAv2 protocol. It may only work over USB.
+
+For "uhid" and "aoa", the keyboard layout must be configured (once and for all) on the device, via Settings -> System -> Languages and input -> Physical keyboard. This settings page can be started directly using the shortcut MOD+k (except in OTG mode), or by executing:
 
     adb shell am start -a android.settings.HARD_KEYBOARD_SETTINGS
 
-However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
+This option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
+
+Also see \fB\-\-mouse\fR.
 
-Also see \fB\-\-hid\-mouse\fR.
+.TP
+.B \-\-kill\-adb\-on\-close
+Kill adb when scrcpy terminates.
 
 .TP
 .B \-\-legacy\-paste
@@ -117,16 +205,30 @@ Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+S
 This is a workaround for some devices not behaving as expected when setting the device clipboard programmatically.
 
 .TP
-.BI "\-\-lock\-video\-orientation[=value]
-Lock video orientation to \fIvalue\fR. Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 1, 2 and 3. Natural device orientation is 0, and each increment adds a 90 degrees rotation counterclockwise.
+.B \-\-list\-camera\-sizes
+List the valid camera capture sizes.
 
-Default is "unlocked".
+.TP
+.B \-\-list\-cameras
+List cameras available on the device.
 
-Passing the option without argument is equivalent to passing "initial".
+.TP
+.B \-\-list\-encoders
+List video and audio encoders available on the device.
 
 .TP
-.BI "\-\-max\-fps " value
-Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
+.B \-\-list\-displays
+List displays available on the device.
+
+.TP
+\fB\-\-lock\-video\-orientation\fR[=\fIvalue\fR]
+Lock capture video orientation to \fIvalue\fR.
+
+Possible values are "unlocked", "initial" (locked to the initial orientation), 0, 90, 180, and 270. The values represent the clockwise rotation from the natural device orientation, in degrees.
+
+Default is "unlocked".
+
+Passing the option without argument is equivalent to passing "initial".
 
 .TP
 .BI "\-m, \-\-max\-size " value
@@ -135,16 +237,46 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
 Default is 0 (unlimited).
 
 .TP
-.B \-M, \-\-hid\-mouse
-Simulate a physical mouse by using HID over AOAv2.
+.B \-M
+Same as \fB\-\-mouse=uhid\fR.
+
+.TP
+.BI "\-\-max\-fps " value
+Limit the framerate of screen capture (officially supported since Android 10, but may work on earlier versions).
+
+.TP
+.BI "\-\-mouse " mode
+Select how to send mouse inputs to the device.
 
-In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
+Possible values are "disabled", "sdk", "uhid" and "aoa":
+
+ - "disabled" does not send mouse inputs to the device.
+ - "sdk" uses the Android system API to deliver mouse events to applications.
+ - "uhid" simulates a physical HID mouse using the Linux HID kernel module on the device.
+ - "aoa" simulates a physical mouse using the AOAv2 protocol. It may only work over USB.
+
+In "uhid" and "aoa" modes, the computer mouse is captured to control the device directly (relative mouse mode).
 
 LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
 
-It may only work over USB.
+Also see \fB\-\-keyboard\fR.
+
+
+.TP
+.B \-n, \-\-no\-control
+Disable device control (mirror the device in read\-only).
+
+.TP
+.B \-N, \-\-no\-playback
+Disable video and audio playback on the computer (equivalent to \fB\-\-no\-video\-playback \-\-no\-audio\-playback\fR).
 
-Also see \fB\-\-hid\-keyboard\fR.
+.TP
+.B \-\-no\-audio
+Disable audio forwarding.
+
+.TP
+.B \-\-no\-audio\-playback
+Disable audio playback on the computer.
 
 .TP
 .B \-\-no\-cleanup
@@ -164,14 +296,6 @@ By default, on MediaCodec error, scrcpy automatically tries again with a lower d
 
 This option disables this behavior.
 
-.TP
-.B \-n, \-\-no\-control
-Disable device control (mirror the device in read\-only).
-
-.TP
-.B \-N, \-\-no\-display
-Do not display device (only when screen recording is enabled).
-
 .TP
 .B \-\-no\-key\-repeat
 Do not forward repeated key events when a key is held down.
@@ -184,6 +308,18 @@ If the renderer is OpenGL 3.0+ or OpenGL ES 2.0+, then mipmaps are automatically
 .B \-\-no\-power\-on
 Do not power on the device on start.
 
+.TP
+.B \-\-no\-video
+Disable video forwarding.
+
+.TP
+.B \-\-no\-video\-playback
+Disable video playback on the computer.
+
+.TP
+.BI "\-\-orientation " value
+Same as --display-orientation=value --record-orientation=value.
+
 .TP
 .B \-\-otg
 Run in OTG mode: simulate physical keyboard and mouse, as if the computer keyboard and mouse were plugged directly to the device via an OTG cable.
@@ -199,11 +335,21 @@ It may only work over USB.
 See \fB\-\-hid\-keyboard\fR and \fB\-\-hid\-mouse\fR.
 
 .TP
-.BI "\-p, \-\-port " port[:port]
+.BI "\-p, \-\-port " port\fR[:\fIport\fR]
 Set the TCP port (range) used by the client to listen.
 
 Default is 27183:27199.
 
+.TP
+\fB\-\-pause\-on\-exit\fR[=\fImode\fR]
+Configure pause on exit. Possible values are "true" (always pause on exit), "false" (never pause on exit) and "if-error" (pause only if an error occured).
+
+This is useful to prevent the terminal window from automatically closing, so that error messages can be read.
+
+Default is "false".
+
+Passing the option without argument is equivalent to passing "true".
+
 .TP
 .B \-\-power\-off\-on\-close
 Turn the device screen off when closing scrcpy.
@@ -225,10 +371,6 @@ Set the target directory for pushing files to the device by drag & drop. It is p
 
 Default is "/sdcard/Download/".
 
-.TP
-.B \-\-raw\-key\-events
-Inject key events for all input keys, and ignore text events.
-
 .TP
 .BI "\-r, \-\-record " file
 Record screen to
@@ -236,11 +378,23 @@ Record screen to
 
 The format is determined by the
 .B \-\-record\-format
-option if set, or by the file extension (.mp4 or .mkv).
+option if set, or by the file extension.
+
+.TP
+.B \-\-raw\-key\-events
+Inject key events for all input keys, and ignore text events.
 
 .TP
 .BI "\-\-record\-format " format
-Force recording format (either mp4 or mkv).
+Force recording format (mp4, mkv, m4a, mka, opus, aac, flac or wav).
+
+.TP
+.BI "\-\-record\-orientation " value
+Set the record orientation.
+
+Possible values are 0, 90, 180 and 270. The number represents the clockwise rotation in degrees.
+
+Default is 0.
 
 .TP
 .BI "\-\-render\-driver " name
@@ -248,19 +402,22 @@ Request SDL to use the given render driver (this is just a hint).
 
 Supported names are currently "direct3d", "opengl", "opengles2", "opengles", "metal" and "software".
 
-.UR https://wiki.libsdl.org/SDL_HINT_RENDER_DRIVER
-.UE
+<https://wiki.libsdl.org/SDL_HINT_RENDER_DRIVER>
 
 .TP
-.BI "\-\-rotation " value
-Set the initial display rotation. Possibles values are 0, 1, 2 and 3. Each increment adds a 90 degrees rotation counterclockwise.
+.B \-\-require\-audio
+By default, scrcpy mirrors only the video if audio capture fails on the device. This option makes scrcpy fail if audio is enabled but does not work.
 
 .TP
 .BI "\-s, \-\-serial " number
 The device serial number. Mandatory only if several devices are connected to adb.
 
 .TP
-.BI "\-\-shortcut\-mod " key[+...]][,...]
+.B \-S, \-\-turn\-screen\-off
+Turn the device screen off immediately.
+
+.TP
+.BI "\-\-shortcut\-mod " key\fR[+...]][,...]
 Specify the modifiers to use for scrcpy shortcuts. Possible keys are "lctrl", "rctrl", "lalt", "ralt", "lsuper" and "rsuper".
 
 A shortcut can consist in several keys, separated by '+'. Several shortcuts can be specified, separated by ','.
@@ -270,7 +427,13 @@ For example, to use either LCtrl+LAlt or LSuper for scrcpy shortcuts, pass "lctr
 Default is "lalt,lsuper" (left-Alt or left-Super).
 
 .TP
-.BI "\-\-tcpip[=ip[:port]]
+.B \-t, \-\-show\-touches
+Enable "show touches" on start, restore the initial value on exit.
+
+It only shows physical touches (not clicks from scrcpy).
+
+.TP
+.BI "\-\-tcpip\fR[=\fIip\fR[:\fIport\fR]]
 Configure and reconnect the device over TCP/IP.
 
 If a destination address is provided, then scrcpy connects to this address before starting. The device must listen on the given TCP port (default is 5555).
@@ -278,27 +441,31 @@ If a destination address is provided, then scrcpy connects to this address befor
 If no destination address is provided, then scrcpy attempts to find the IP address and adb port of the current device (typically connected over USB), enables TCP/IP mode if necessary, then connects to this address before starting.
 
 .TP
-.B \-S, \-\-turn\-screen\-off
-Turn the device screen off immediately.
-
-.TP
-.B \-t, \-\-show\-touches
-Enable "show touches" on start, restore the initial value on exit.
-
-It only shows physical touches (not clicks from scrcpy).
+.BI "\-\-time\-limit " seconds
+Set the maximum mirroring time, in seconds.
 
 .TP
 .BI "\-\-tunnel\-host " ip
-Set the IP address of the adb tunnel to reach the scrcpy server. This option automatically enables --force-adb-forward.
+Set the IP address of the adb tunnel to reach the scrcpy server. This option automatically enables \fB\-\-force\-adb\-forward\fR.
 
 Default is localhost.
 
 .TP
 .BI "\-\-tunnel\-port " port
-Set the TCP port of the adb tunnel to reach the scrcpy server. This option automatically enables --force-adb-forward.
+Set the TCP port of the adb tunnel to reach the scrcpy server. This option automatically enables \fB\-\-force\-adb\-forward\fR.
 
 Default is 0 (not forced): the local port used for establishing the tunnel will be used.
 
+.TP
+.B \-v, \-\-version
+Print the version of scrcpy.
+
+.TP
+.BI "\-V, \-\-verbosity " value
+Set the log level ("verbose", "debug", "info", "warn" or "error").
+
+Default is "info" for release builds, "debug" for debug builds.
+
 .TP
 .BI "\-\-v4l2-sink " /dev/videoN
 Output to v4l2loopback device.
@@ -314,14 +481,34 @@ This option is similar to \fB\-\-display\-buffer\fR, but specific to V4L2 sink.
 Default is 0 (no buffering).
 
 .TP
-.BI "\-V, \-\-verbosity " value
-Set the log level ("verbose", "debug", "info", "warn" or "error").
+.BI "\-\-video\-codec " name
+Select a video codec (h264, h265 or av1).
 
-Default is "info" for release builds, "debug" for debug builds.
+Default is h264.
 
 .TP
-.B \-v, \-\-version
-Print the version of scrcpy.
+.BI "\-\-video\-codec\-options " key\fR[:\fItype\fR]=\fIvalue\fR[,...]
+Set a list of comma-separated key:type=value options for the device video encoder.
+
+The possible values for 'type' are 'int' (default), 'long', 'float' and 'string'.
+
+The list of possible codec options is available in the Android documentation:
+
+<https://d.android.com/reference/android/media/MediaFormat>
+
+.TP
+.BI "\-\-video\-encoder " name
+Use a specific MediaCodec video encoder (depending on the codec provided by \fB\-\-video\-codec\fR).
+
+The available encoders can be listed by \fB\-\-list\-encoders\fR.
+
+.TP
+.BI "\-\-video\-source " source
+Select the video source (display or camera).
+
+Camera mirroring requires Android 12+.
+
+Default is display.
 
 .TP
 .B \-w, \-\-stay-awake
@@ -382,6 +569,14 @@ Rotate display left
 .B MOD+Right
 Rotate display right
 
+.TP
+.B MOD+Shift+Left, MOD+Shift+Right
+Flip display horizontally
+
+.TP
+.B MOD+Shift+Up, MOD+Shift+Down
+Flip display vertically
+
 .TP
 .B MOD+g
 Resize window to 1:1 (pixel\-perfect)
@@ -458,13 +653,21 @@ Copy computer clipboard to device, then paste (inject PASTE keycode, Android >=
 .B MOD+Shift+v
 Inject computer clipboard text as a sequence of key events
 
+.TP
+.B MOD+k
+Open keyboard settings on the device (for HID keyboard only)
+
 .TP
 .B MOD+i
 Enable/disable FPS counter (print frames/second in logs)
 
 .TP
 .B Ctrl+click-and-move
-Pinch-to-zoom from the center of the screen
+Pinch-to-zoom and rotate from the center of the screen
+
+.TP
+.B Shift+click-and-move
+Tilt (slide vertically with two fingers)
 
 .TP
 .B Drag & drop APK file
@@ -483,7 +686,7 @@ Path to adb.
 
 .TP
 .B ANDROID_SERIAL
-Device serial to use if no selector (-s, -d, -e or --tcpip=<addr>) is specified.
+Device serial to use if no selector (\fB-s\fR, \fB-d\fR, \fB-e\fR or \fB\-\-tcpip=\fIaddr\fR) is specified.
 
 .TP
 .B SCRCPY_ICON_PATH
@@ -506,23 +709,14 @@ for the Debian Project (and may be used by others).
 
 
 .SH "REPORTING BUGS"
-Report bugs to
-.UR https://github.com/Genymobile/scrcpy/issues
-.UE .
+Report bugs to <https://github.com/Genymobile/scrcpy/issues>.
 
 .SH COPYRIGHT
-Copyright \(co 2018 Genymobile
-.UR https://www.genymobile.com
-Genymobile
-.UE
-
-Copyright \(co 2018\-2022
-.MT rom@rom1v.com
-Romain Vimont
-.ME
+Copyright \(co 2018 Genymobile <https://www.genymobile.com>
+
+Copyright \(co 2018\-2024 Romain Vimont <rom@rom1v.com>
 
 Licensed under the Apache License, Version 2.0.
 
 .SH WWW
-.UR https://github.com/Genymobile/scrcpy
-.UE
+<https://github.com/Genymobile/scrcpy>

app/src/adb/adb.c

@@ -70,7 +70,7 @@ argv_to_string(const char *const *argv, char *buf, size_t bufsize) {
 }
 
 static void
-show_adb_installation_msg() {
+show_adb_installation_msg(void) {
 #ifndef __WINDOWS__
     static const struct {
         const char *binary;
@@ -218,8 +218,16 @@ sc_adb_forward(struct sc_intr *intr, const char *serial, uint16_t local_port,
                const char *device_socket_name, unsigned flags) {
     char local[4 + 5 + 1]; // tcp:PORT
     char remote[108 + 14 + 1]; // localabstract:NAME
-    sprintf(local, "tcp:%" PRIu16, local_port);
-    snprintf(remote, sizeof(remote), "localabstract:%s", device_socket_name);
+
+    int r = snprintf(local, sizeof(local), "tcp:%" PRIu16, local_port);
+    assert(r >= 0 && (size_t) r < sizeof(local));
+
+    r = snprintf(remote, sizeof(remote), "localabstract:%s",
+                 device_socket_name);
+    if (r < 0 || (size_t) r >= sizeof(remote)) {
+        LOGE("Could not write socket name");
+        return false;
+    }
 
     assert(serial);
     const char *const argv[] =
@@ -233,7 +241,9 @@ bool
 sc_adb_forward_remove(struct sc_intr *intr, const char *serial,
                       uint16_t local_port, unsigned flags) {
     char local[4 + 5 + 1]; // tcp:PORT
-    sprintf(local, "tcp:%" PRIu16, local_port);
+    int r = snprintf(local, sizeof(local), "tcp:%" PRIu16, local_port);
+    assert(r >= 0 && (size_t) r < sizeof(local));
+    (void) r;
 
     assert(serial);
     const char *const argv[] =
@@ -249,8 +259,16 @@ sc_adb_reverse(struct sc_intr *intr, const char *serial,
                unsigned flags) {
     char local[4 + 5 + 1]; // tcp:PORT
     char remote[108 + 14 + 1]; // localabstract:NAME
-    sprintf(local, "tcp:%" PRIu16, local_port);
-    snprintf(remote, sizeof(remote), "localabstract:%s", device_socket_name);
+    int r = snprintf(local, sizeof(local), "tcp:%" PRIu16, local_port);
+    assert(r >= 0 && (size_t) r < sizeof(local));
+
+    r = snprintf(remote, sizeof(remote), "localabstract:%s",
+                 device_socket_name);
+    if (r < 0 || (size_t) r >= sizeof(remote)) {
+        LOGE("Could not write socket name");
+        return false;
+    }
+
     assert(serial);
     const char *const argv[] =
         SC_ADB_COMMAND("-s", serial, "reverse", remote, local);
@@ -263,7 +281,12 @@ bool
 sc_adb_reverse_remove(struct sc_intr *intr, const char *serial,
                       const char *device_socket_name, unsigned flags) {
     char remote[108 + 14 + 1]; // localabstract:NAME
-    snprintf(remote, sizeof(remote), "localabstract:%s", device_socket_name);
+    int r = snprintf(remote, sizeof(remote), "localabstract:%s",
+                     device_socket_name);
+    if (r < 0 || (size_t) r >= sizeof(remote)) {
+        LOGE("Device socket name too long");
+        return false;
+    }
 
     assert(serial);
     const char *const argv[] =
@@ -333,7 +356,9 @@ bool
 sc_adb_tcpip(struct sc_intr *intr, const char *serial, uint16_t port,
              unsigned flags) {
     char port_string[5 + 1];
-    sprintf(port_string, "%" PRIu16, port);
+    int r = snprintf(port_string, sizeof(port_string), "%" PRIu16, port);
+    assert(r >= 0 && (size_t) r < sizeof(port_string));
+    (void) r;
 
     assert(serial);
     const char *const argv[] =
@@ -433,6 +458,7 @@ sc_adb_list_devices(struct sc_intr *intr, unsigned flags,
         // in the buffer in a single pass
         LOGW("Result of \"adb devices -l\" does not fit in 64Kb. "
              "Please report an issue.");
+        free(buf);
         return false;
     }
 
@@ -628,8 +654,8 @@ sc_adb_select_device(struct sc_intr *intr,
         return false;
     }
 
-    LOGD("ADB device found:");
-    sc_adb_devices_log(SC_LOG_LEVEL_DEBUG, vec.data, vec.size);
+    LOGI("ADB device found:");
+    sc_adb_devices_log(SC_LOG_LEVEL_INFO, vec.data, vec.size);
 
     // Move devics into out_device (do not destroy device)
     sc_adb_device_move(out_device, device);

app/src/adb/adb_parser.c

@@ -7,7 +7,7 @@
 #include "util/log.h"
 #include "util/str.h"
 
-bool
+static bool
 sc_adb_parse_device(char *line, struct sc_adb_device *device) {
     // One device line looks like:
     // "0123456789abcdef	device usb:2-1 product:MyProduct model:MyModel "
@@ -204,6 +204,7 @@ sc_adb_parse_device_ip(char *str) {
     while (str[idx_line] != '\0') {
         char *line = &str[idx_line];
         size_t len = strcspn(line, "\n");
+        bool is_last_line = line[len] == '\0';
 
         // The same, but without any trailing '\r'
         size_t line_len = sc_str_remove_trailing_cr(line, len);
@@ -215,12 +216,12 @@ sc_adb_parse_device_ip(char *str) {
             return ip;
         }
 
-        idx_line += len;
-
-        if (str[idx_line] != '\0') {
-            // The next line starts after the '\n'
-            ++idx_line;
+        if (is_last_line) {
+            break;
         }
+
+        // The next line starts after the '\n'
+        idx_line += len + 1;
     }
 
     return NULL;

app/src/adb/adb_tunnel.c

@@ -7,8 +7,6 @@
 #include "util/net_intr.h"
 #include "util/process_intr.h"
 
-#define SC_SOCKET_NAME "scrcpy"
-
 static bool
 listen_on_port(struct sc_intr *intr, sc_socket socket, uint16_t port) {
     return net_listen_intr(intr, socket, IPV4_LOCALHOST, port, 1);
@@ -17,10 +15,11 @@ listen_on_port(struct sc_intr *intr, sc_socket socket, uint16_t port) {
 static bool
 enable_tunnel_reverse_any_port(struct sc_adb_tunnel *tunnel,
                                struct sc_intr *intr, const char *serial,
+                               const char *device_socket_name,
                                struct sc_port_range port_range) {
     uint16_t port = port_range.first;
     for (;;) {
-        if (!sc_adb_reverse(intr, serial, SC_SOCKET_NAME, port,
+        if (!sc_adb_reverse(intr, serial, device_socket_name, port,
                             SC_ADB_NO_STDOUT)) {
             // the command itself failed, it will fail on any port
             return false;
@@ -52,7 +51,7 @@ enable_tunnel_reverse_any_port(struct sc_adb_tunnel *tunnel,
         }
 
         // failure, disable tunnel and try another port
-        if (!sc_adb_reverse_remove(intr, serial, SC_SOCKET_NAME,
+        if (!sc_adb_reverse_remove(intr, serial, device_socket_name,
                                 SC_ADB_NO_STDOUT)) {
             LOGW("Could not remove reverse tunnel on port %" PRIu16, port);
         }
@@ -78,12 +77,13 @@ enable_tunnel_reverse_any_port(struct sc_adb_tunnel *tunnel,
 static bool
 enable_tunnel_forward_any_port(struct sc_adb_tunnel *tunnel,
                                struct sc_intr *intr, const char *serial,
+                               const char *device_socket_name,
                                struct sc_port_range port_range) {
     tunnel->forward = true;
 
     uint16_t port = port_range.first;
     for (;;) {
-        if (sc_adb_forward(intr, serial, port, SC_SOCKET_NAME,
+        if (sc_adb_forward(intr, serial, port, device_socket_name,
                            SC_ADB_NO_STDOUT)) {
             // success
             tunnel->local_port = port;
@@ -123,13 +123,14 @@ sc_adb_tunnel_init(struct sc_adb_tunnel *tunnel) {
 
 bool
 sc_adb_tunnel_open(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
-                   const char *serial, struct sc_port_range port_range,
-                   bool force_adb_forward) {
+                   const char *serial, const char *device_socket_name,
+                   struct sc_port_range port_range, bool force_adb_forward) {
     assert(!tunnel->enabled);
 
     if (!force_adb_forward) {
         // Attempt to use "adb reverse"
-        if (enable_tunnel_reverse_any_port(tunnel, intr, serial, port_range)) {
+        if (enable_tunnel_reverse_any_port(tunnel, intr, serial,
+                                           device_socket_name, port_range)) {
             return true;
         }
 
@@ -139,12 +140,13 @@ sc_adb_tunnel_open(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
         LOGW("'adb reverse' failed, fallback to 'adb forward'");
     }
 
-    return enable_tunnel_forward_any_port(tunnel, intr, serial, port_range);
+    return enable_tunnel_forward_any_port(tunnel, intr, serial,
+                                          device_socket_name, port_range);
 }
 
 bool
 sc_adb_tunnel_close(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
-                    const char *serial) {
+                    const char *serial, const char *device_socket_name) {
     assert(tunnel->enabled);
 
     bool ret;
@@ -152,7 +154,7 @@ sc_adb_tunnel_close(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
         ret = sc_adb_forward_remove(intr, serial, tunnel->local_port,
                                     SC_ADB_NO_STDOUT);
     } else {
-        ret = sc_adb_reverse_remove(intr, serial, SC_SOCKET_NAME,
+        ret = sc_adb_reverse_remove(intr, serial, device_socket_name,
                                     SC_ADB_NO_STDOUT);
 
         assert(tunnel->server_socket != SC_SOCKET_NONE);

app/src/adb/adb_tunnel.h

@@ -34,14 +34,14 @@ sc_adb_tunnel_init(struct sc_adb_tunnel *tunnel);
  */
 bool
 sc_adb_tunnel_open(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
-                   const char *serial, struct sc_port_range port_range,
-                   bool force_adb_forward);
+                   const char *serial, const char *device_socket_name,
+                   struct sc_port_range port_range, bool force_adb_forward);
 
 /**
  * Close the tunnel
  */
 bool
 sc_adb_tunnel_close(struct sc_adb_tunnel *tunnel, struct sc_intr *intr,
-                    const char *serial);
+                    const char *serial, const char *device_socket_name);
 
 #endif

app/src/audio_player.c

@@ -0,0 +1,487 @@
+#include "audio_player.h"
+
+#include <libavcodec/avcodec.h>
+#include <libavutil/opt.h>
+
+#include "util/log.h"
+
+#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
+
+/**
+ * Real-time audio player with configurable latency
+ *
+ * As input, the player regularly receives AVFrames of decoded audio samples.
+ * As output, an SDL callback regularly requests audio samples to be played.
+ * In the middle, an audio buffer stores the samples produced but not consumed
+ * yet.
+ *
+ * The goal of the player is to feed the audio output with a latency as low as
+ * possible while avoiding buffer underrun (i.e. not being able to provide
+ * samples when requested).
+ *
+ * The player aims to feed the audio output with as little latency as possible
+ * while avoiding buffer underrun. To achieve this, it attempts to maintain the
+ * average buffering (the number of samples present in the buffer) around a
+ * target value. If this target buffering is too low, then buffer underrun will
+ * occur frequently. If it is too high, then latency will become unacceptable.
+ * This target value is configured using the scrcpy option --audio-buffer.
+ *
+ * The player cannot adjust the sample input rate (it receives samples produced
+ * in real-time) or the sample output rate (it must provide samples as
+ * requested by the audio output callback). Therefore, it may only apply
+ * compensation by resampling (converting _m_ input samples to _n_ output
+ * samples).
+ *
+ * The compensation itself is applied by libswresample (FFmpeg). It is
+ * configured using swr_set_compensation(). An important work for the player
+ * is to estimate the compensation value regularly and apply it.
+ *
+ * The estimated buffering level is the result of averaging the "natural"
+ * buffering (samples are produced and consumed by blocks, so it must be
+ * smoothed), and making instant adjustments resulting of its own actions
+ * (explicit compensation and silence insertion on underflow), which are not
+ * smoothed.
+ *
+ * Buffer underflow events can occur when packets arrive too late. In that case,
+ * the player inserts silence. Once the packets finally arrive (late), one
+ * strategy could be to drop the samples that were replaced by silence, in
+ * order to keep a minimal latency. However, dropping samples in case of buffer
+ * underflow is inadvisable, as it would temporarily increase the underflow
+ * even more and cause very noticeable audio glitches.
+ *
+ * Therefore, the player doesn't drop any sample on underflow. The compensation
+ * mechanism will absorb the delay introduced by the inserted silence.
+ */
+
+/** Downcast frame_sink to sc_audio_player */
+#define DOWNCAST(SINK) container_of(SINK, struct sc_audio_player, frame_sink)
+
+#define SC_AV_SAMPLE_FMT AV_SAMPLE_FMT_FLT
+#define SC_SDL_SAMPLE_FMT AUDIO_F32
+
+#define TO_BYTES(SAMPLES) sc_audiobuf_to_bytes(&ap->buf, (SAMPLES))
+#define TO_SAMPLES(BYTES) sc_audiobuf_to_samples(&ap->buf, (BYTES))
+
+static void SDLCALL
+sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
+    struct sc_audio_player *ap = userdata;
+
+    // This callback is called with the lock used by SDL_LockAudioDevice()
+
+    assert(len_int > 0);
+    size_t len = len_int;
+    uint32_t count = TO_SAMPLES(len);
+
+#ifndef SC_AUDIO_PLAYER_NDEBUG
+    LOGD("[Audio] SDL callback requests %" PRIu32 " samples", count);
+#endif
+
+    bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
+    if (!played) {
+        uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
+        // Wait until the buffer is filled up to at least target_buffering
+        // before playing
+        if (buffered_samples < ap->target_buffering) {
+            LOGV("[Audio] Inserting initial buffering silence: %" PRIu32
+                 " samples", count);
+            // Delay playback starting to reach the target buffering. Fill the
+            // whole buffer with silence (len is small compared to the
+            // arbitrary margin value).
+            memset(stream, 0, len);
+            return;
+        }
+    }
+
+    uint32_t read = sc_audiobuf_read(&ap->buf, stream, count);
+
+    if (read < count) {
+        uint32_t silence = count - read;
+        // Insert silence. In theory, the inserted silent samples replace the
+        // missing real samples, which will arrive later, so they should be
+        // dropped to keep the latency minimal. However, this would cause very
+        // audible glitches, so let the clock compensation restore the target
+        // latency.
+        LOGD("[Audio] Buffer underflow, inserting silence: %" PRIu32 " samples",
+             silence);
+        memset(stream + TO_BYTES(read), 0, TO_BYTES(silence));
+
+        bool received = atomic_load_explicit(&ap->received,
+                                             memory_order_relaxed);
+        if (received) {
+            // Inserting additional samples immediately increases buffering
+            atomic_fetch_add_explicit(&ap->underflow, silence,
+                                      memory_order_relaxed);
+        }
+    }
+
+    atomic_store_explicit(&ap->played, true, memory_order_relaxed);
+}
+
+static uint8_t *
+sc_audio_player_get_swr_buf(struct sc_audio_player *ap, uint32_t min_samples) {
+    size_t min_buf_size = TO_BYTES(min_samples);
+    if (min_buf_size > ap->swr_buf_alloc_size) {
+        size_t new_size = min_buf_size + 4096;
+        uint8_t *buf = realloc(ap->swr_buf, new_size);
+        if (!buf) {
+            LOG_OOM();
+            // Could not realloc to the requested size
+            return NULL;
+        }
+        ap->swr_buf = buf;
+        ap->swr_buf_alloc_size = new_size;
+    }
+
+    return ap->swr_buf;
+}
+
+static bool
+sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
+                                const AVFrame *frame) {
+    struct sc_audio_player *ap = DOWNCAST(sink);
+
+    SwrContext *swr_ctx = ap->swr_ctx;
+
+    int64_t swr_delay = swr_get_delay(swr_ctx, ap->sample_rate);
+    // No need to av_rescale_rnd(), input and output sample rates are the same.
+    // Add more space (256) for clock compensation.
+    int dst_nb_samples = swr_delay + frame->nb_samples + 256;
+
+    uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
+    if (!swr_buf) {
+        return false;
+    }
+
+    int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
+                          (const uint8_t **) frame->data, frame->nb_samples);
+    if (ret < 0) {
+        LOGE("Resampling failed: %d", ret);
+        return false;
+    }
+
+    // swr_convert() returns the number of samples which would have been
+    // written if the buffer was big enough.
+    uint32_t samples = MIN(ret, dst_nb_samples);
+#ifndef SC_AUDIO_PLAYER_NDEBUG
+    LOGD("[Audio] %" PRIu32 " samples written to buffer", samples);
+#endif
+
+    uint32_t cap = sc_audiobuf_capacity(&ap->buf);
+    if (samples > cap) {
+        // Very very unlikely: a single resampled frame should never
+        // exceed the audio buffer size (or something is very wrong).
+        // Ignore the first bytes in swr_buf to avoid memory corruption anyway.
+        swr_buf += TO_BYTES(samples - cap);
+        samples = cap;
+    }
+
+    uint32_t skipped_samples = 0;
+
+    uint32_t written = sc_audiobuf_write(&ap->buf, swr_buf, samples);
+    if (written < samples) {
+        uint32_t remaining = samples - written;
+
+        // All samples that could be written without locking have been written,
+        // now we need to lock to drop/consume old samples
+        SDL_LockAudioDevice(ap->device);
+
+        // Retry with the lock
+        written += sc_audiobuf_write(&ap->buf,
+                                     swr_buf + TO_BYTES(written),
+                                     remaining);
+        if (written < samples) {
+            remaining = samples - written;
+            // Still insufficient, drop old samples to make space
+            skipped_samples = sc_audiobuf_read(&ap->buf, NULL, remaining);
+            assert(skipped_samples == remaining);
+
+            // Now there is enough space
+            uint32_t w = sc_audiobuf_write(&ap->buf,
+                                           swr_buf + TO_BYTES(written),
+                                           remaining);
+            assert(w == remaining);
+            (void) w;
+        }
+
+        SDL_UnlockAudioDevice(ap->device);
+    }
+
+    uint32_t underflow = 0;
+    uint32_t max_buffered_samples;
+    bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
+    if (played) {
+        underflow = atomic_exchange_explicit(&ap->underflow, 0,
+                                             memory_order_relaxed);
+
+        max_buffered_samples = ap->target_buffering
+                               + 12 * ap->output_buffer
+                               + ap->target_buffering / 10;
+    } else {
+        // SDL playback not started yet, do not accumulate more than
+        // max_initial_buffering samples, this would cause unnecessary delay
+        // (and glitches to compensate) on start.
+        max_buffered_samples = ap->target_buffering + 2 * ap->output_buffer;
+    }
+
+    uint32_t can_read = sc_audiobuf_can_read(&ap->buf);
+    if (can_read > max_buffered_samples) {
+        uint32_t skip_samples = 0;
+
+        SDL_LockAudioDevice(ap->device);
+        can_read = sc_audiobuf_can_read(&ap->buf);
+        if (can_read > max_buffered_samples) {
+            skip_samples = can_read - max_buffered_samples;
+            uint32_t r = sc_audiobuf_read(&ap->buf, NULL, skip_samples);
+            assert(r == skip_samples);
+            (void) r;
+            skipped_samples += skip_samples;
+        }
+        SDL_UnlockAudioDevice(ap->device);
+
+        if (skip_samples) {
+            if (played) {
+                LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
+                     " samples", skip_samples);
+#ifndef SC_AUDIO_PLAYER_NDEBUG
+            } else {
+                LOGD("[Audio] Playback not started, skipping %" PRIu32
+                     " samples", skip_samples);
+#endif
+            }
+        }
+    }
+
+    atomic_store_explicit(&ap->received, true, memory_order_relaxed);
+    if (!played) {
+        // Nothing more to do
+        return true;
+    }
+
+    // Number of samples added (or removed, if negative) for compensation
+    int32_t instant_compensation = (int32_t) written - frame->nb_samples;
+    // Inserting silence instantly increases buffering
+    int32_t inserted_silence = (int32_t) underflow;
+    // Dropping input samples instantly decreases buffering
+    int32_t dropped = (int32_t) skipped_samples;
+
+    // The compensation must apply instantly, it must not be smoothed
+    ap->avg_buffering.avg += instant_compensation + inserted_silence - dropped;
+    if (ap->avg_buffering.avg < 0) {
+        // Since dropping samples instantly reduces buffering, the difference
+        // is applied immediately to the average value, assuming that the delay
+        // between the producer and the consumer will be caught up.
+        //
+        // However, when this assumption is not valid, the average buffering
+        // may decrease indefinitely. Prevent it to become negative to limit
+        // the consequences.
+        ap->avg_buffering.avg = 0;
+    }
+
+    // However, the buffering level must be smoothed
+    sc_average_push(&ap->avg_buffering, can_read);
+
+#ifndef SC_AUDIO_PLAYER_NDEBUG
+    LOGD("[Audio] can_read=%" PRIu32 " avg_buffering=%f",
+         can_read, sc_average_get(&ap->avg_buffering));
+#endif
+
+    ap->samples_since_resync += written;
+    if (ap->samples_since_resync >= ap->sample_rate) {
+        // Recompute compensation every second
+        ap->samples_since_resync = 0;
+
+        float avg = sc_average_get(&ap->avg_buffering);
+        int diff = ap->target_buffering - avg;
+
+        // Enable compensation when the difference exceeds +/- 4ms.
+        // Disable compensation when the difference is lower than +/- 1ms.
+        int threshold = ap->compensation != 0
+                      ? ap->sample_rate     / 1000  /* 1ms */
+                      : ap->sample_rate * 4 / 1000; /* 4ms */
+
+        if (abs(diff) < threshold) {
+            // Do not compensate for small values, the error is just noise
+            diff = 0;
+        } else if (diff < 0 && can_read < ap->target_buffering) {
+            // Do not accelerate if the instant buffering level is below the
+            // target, this would increase underflow
+            diff = 0;
+        }
+        // Compensate the diff over 4 seconds (but will be recomputed after 1
+        // second)
+        int distance = 4 * ap->sample_rate;
+        // Limit compensation rate to 2%
+        int abs_max_diff = distance / 50;
+        diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
+        LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
+             " compensation=%d", ap->target_buffering, avg, can_read, diff);
+
+        if (diff != ap->compensation) {
+            int ret = swr_set_compensation(swr_ctx, diff, distance);
+            if (ret < 0) {
+                LOGW("Resampling compensation failed: %d", ret);
+                // not fatal
+            } else {
+                ap->compensation = diff;
+            }
+        }
+    }
+
+    return true;
+}
+
+static bool
+sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
+                                const AVCodecContext *ctx) {
+    struct sc_audio_player *ap = DOWNCAST(sink);
+#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
+    assert(ctx->ch_layout.nb_channels > 0);
+    unsigned nb_channels = ctx->ch_layout.nb_channels;
+#else
+    int tmp = av_get_channel_layout_nb_channels(ctx->channel_layout);
+    assert(tmp > 0);
+    unsigned nb_channels = tmp;
+#endif
+
+    assert(ctx->sample_rate > 0);
+    assert(!av_sample_fmt_is_planar(SC_AV_SAMPLE_FMT));
+    int out_bytes_per_sample = av_get_bytes_per_sample(SC_AV_SAMPLE_FMT);
+    assert(out_bytes_per_sample > 0);
+
+    ap->sample_rate = ctx->sample_rate;
+    ap->nb_channels = nb_channels;
+    ap->out_bytes_per_sample = out_bytes_per_sample;
+
+    ap->target_buffering = ap->target_buffering_delay * ap->sample_rate
+                                                      / SC_TICK_FREQ;
+
+    uint64_t aout_samples = ap->output_buffer_duration * ap->sample_rate
+                                                       / SC_TICK_FREQ;
+    assert(aout_samples <= 0xFFFF);
+    ap->output_buffer = (uint16_t) aout_samples;
+
+    SDL_AudioSpec desired = {
+        .freq = ctx->sample_rate,
+        .format = SC_SDL_SAMPLE_FMT,
+        .channels = nb_channels,
+        .samples = aout_samples,
+        .callback = sc_audio_player_sdl_callback,
+        .userdata = ap,
+    };
+    SDL_AudioSpec obtained;
+
+    ap->device = SDL_OpenAudioDevice(NULL, 0, &desired, &obtained, 0);
+    if (!ap->device) {
+        LOGE("Could not open audio device: %s", SDL_GetError());
+        return false;
+    }
+
+    SwrContext *swr_ctx = swr_alloc();
+    if (!swr_ctx) {
+        LOG_OOM();
+        goto error_close_audio_device;
+    }
+    ap->swr_ctx = swr_ctx;
+
+#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
+    av_opt_set_chlayout(swr_ctx, "in_chlayout", &ctx->ch_layout, 0);
+    av_opt_set_chlayout(swr_ctx, "out_chlayout", &ctx->ch_layout, 0);
+#else
+    av_opt_set_channel_layout(swr_ctx, "in_channel_layout",
+                              ctx->channel_layout, 0);
+    av_opt_set_channel_layout(swr_ctx, "out_channel_layout",
+                              ctx->channel_layout, 0);
+#endif
+
+    av_opt_set_int(swr_ctx, "in_sample_rate", ctx->sample_rate, 0);
+    av_opt_set_int(swr_ctx, "out_sample_rate", ctx->sample_rate, 0);
+
+    av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", ctx->sample_fmt, 0);
+    av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", SC_AV_SAMPLE_FMT, 0);
+
+    int ret = swr_init(swr_ctx);
+    if (ret) {
+        LOGE("Failed to initialize the resampling context");
+        goto error_free_swr_ctx;
+    }
+
+    // Use a ring-buffer of the target buffering size plus 1 second between the
+    // producer and the consumer. It's too big on purpose, to guarantee that
+    // the producer and the consumer will be able to access it in parallel
+    // without locking.
+    uint32_t audiobuf_samples = ap->target_buffering + ap->sample_rate;
+
+    size_t sample_size = ap->nb_channels * ap->out_bytes_per_sample;
+    bool ok = sc_audiobuf_init(&ap->buf, sample_size, audiobuf_samples);
+    if (!ok) {
+        goto error_free_swr_ctx;
+    }
+
+    size_t initial_swr_buf_size = TO_BYTES(4096);
+    ap->swr_buf = malloc(initial_swr_buf_size);
+    if (!ap->swr_buf) {
+        LOG_OOM();
+        goto error_destroy_audiobuf;
+    }
+    ap->swr_buf_alloc_size = initial_swr_buf_size;
+
+    // Samples are produced and consumed by blocks, so the buffering must be
+    // smoothed to get a relatively stable value.
+    sc_average_init(&ap->avg_buffering, 128);
+    ap->samples_since_resync = 0;
+
+    ap->received = false;
+    atomic_init(&ap->played, false);
+    atomic_init(&ap->received, false);
+    atomic_init(&ap->underflow, 0);
+    ap->compensation = 0;
+
+    // The thread calling open() is the thread calling push(), which fills the
+    // audio buffer consumed by the SDL audio thread.
+    ok = sc_thread_set_priority(SC_THREAD_PRIORITY_TIME_CRITICAL);
+    if (!ok) {
+        ok = sc_thread_set_priority(SC_THREAD_PRIORITY_HIGH);
+        (void) ok; // We don't care if it worked, at least we tried
+    }
+
+    SDL_PauseAudioDevice(ap->device, 0);
+
+    return true;
+
+error_destroy_audiobuf:
+    sc_audiobuf_destroy(&ap->buf);
+error_free_swr_ctx:
+    swr_free(&ap->swr_ctx);
+error_close_audio_device:
+    SDL_CloseAudioDevice(ap->device);
+
+    return false;
+}
+
+static void
+sc_audio_player_frame_sink_close(struct sc_frame_sink *sink) {
+    struct sc_audio_player *ap = DOWNCAST(sink);
+
+    assert(ap->device);
+    SDL_PauseAudioDevice(ap->device, 1);
+    SDL_CloseAudioDevice(ap->device);
+
+    free(ap->swr_buf);
+    sc_audiobuf_destroy(&ap->buf);
+    swr_free(&ap->swr_ctx);
+}
+
+void
+sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering,
+                     sc_tick output_buffer_duration) {
+    ap->target_buffering_delay = target_buffering;
+    ap->output_buffer_duration = output_buffer_duration;
+
+    static const struct sc_frame_sink_ops ops = {
+        .open = sc_audio_player_frame_sink_open,
+        .close = sc_audio_player_frame_sink_close,
+        .push = sc_audio_player_frame_sink_push,
+    };
+
+    ap->frame_sink.ops = &ops;
+}

app/src/audio_player.h

@@ -0,0 +1,84 @@
+#ifndef SC_AUDIO_PLAYER_H
+#define SC_AUDIO_PLAYER_H
+
+#include "common.h"
+
+#include <stdatomic.h>
+#include <stdbool.h>
+#include <libavformat/avformat.h>
+#include <libswresample/swresample.h>
+#include <SDL2/SDL.h>
+
+#include "trait/frame_sink.h"
+#include "util/audiobuf.h"
+#include "util/average.h"
+#include "util/thread.h"
+#include "util/tick.h"
+
+struct sc_audio_player {
+    struct sc_frame_sink frame_sink;
+
+    SDL_AudioDeviceID device;
+
+    // The target buffering between the producer and the consumer. This value
+    // is directly use for compensation.
+    // Since audio capture and/or encoding on the device typically produce
+    // blocks of 960 samples (20ms) or 1024 samples (~21.3ms), this target
+    // value should be higher.
+    sc_tick target_buffering_delay;
+    uint32_t target_buffering; // in samples
+
+    // SDL audio output buffer size.
+    sc_tick output_buffer_duration;
+    uint16_t output_buffer;
+
+    // Audio buffer to communicate between the receiver and the SDL audio
+    // callback
+    struct sc_audiobuf buf;
+
+    // Resampler (only used from the receiver thread)
+    struct SwrContext *swr_ctx;
+
+    // The sample rate is the same for input and output
+    unsigned sample_rate;
+    // The number of channels is the same for input and output
+    unsigned nb_channels;
+    // The number of bytes per sample for a single channel
+    size_t out_bytes_per_sample;
+
+    // Target buffer for resampling (only used by the receiver thread)
+    uint8_t *swr_buf;
+    size_t swr_buf_alloc_size;
+
+    // Number of buffered samples (may be negative on underflow) (only used by
+    // the receiver thread)
+    struct sc_average avg_buffering;
+    // Count the number of samples to trigger a compensation update regularly
+    // (only used by the receiver thread)
+    uint32_t samples_since_resync;
+
+    // Number of silence samples inserted since the last received packet
+    atomic_uint_least32_t underflow;
+
+    // Current applied compensation value (only used by the receiver thread)
+    int compensation;
+
+    // Set to true the first time a sample is received
+    atomic_bool received;
+
+    // Set to true the first time the SDL callback is called
+    atomic_bool played;
+
+    const struct sc_audio_player_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_audio_player_callbacks {
+    void (*on_ended)(struct sc_audio_player *ap, bool success, void *userdata);
+};
+
+void
+sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering,
+                     sc_tick audio_output_buffer);
+
+#endif

app/src/cli.h

@@ -7,10 +7,17 @@
 
 #include "options.h"
 
+enum sc_pause_on_exit {
+    SC_PAUSE_ON_EXIT_TRUE,
+    SC_PAUSE_ON_EXIT_FALSE,
+    SC_PAUSE_ON_EXIT_IF_ERROR,
+};
+
 struct scrcpy_cli_args {
     struct scrcpy_options opts;
     bool help;
     bool version;
+    enum sc_pause_on_exit pause_on_exit;
 };
 
 void

app/src/clock.c

@@ -1,111 +1,36 @@
 #include "clock.h"
 
+#include <assert.h>
+
 #include "util/log.h"
 
 #define SC_CLOCK_NDEBUG // comment to debug
 
+#define SC_CLOCK_RANGE 32
+
 void
 sc_clock_init(struct sc_clock *clock) {
-    clock->count = 0;
-    clock->head = 0;
-    clock->left_sum.system = 0;
-    clock->left_sum.stream = 0;
-    clock->right_sum.system = 0;
-    clock->right_sum.stream = 0;
-}
-
-// Estimate the affine function f(stream) = slope * stream + offset
-static void
-sc_clock_estimate(struct sc_clock *clock,
-                  double *out_slope, sc_tick *out_offset) {
-    assert(clock->count > 1); // two points are necessary
-
-    struct sc_clock_point left_avg = {
-        .system = clock->left_sum.system / (clock->count / 2),
-        .stream = clock->left_sum.stream / (clock->count / 2),
-    };
-    struct sc_clock_point right_avg = {
-        .system = clock->right_sum.system / ((clock->count + 1) / 2),
-        .stream = clock->right_sum.stream / ((clock->count + 1) / 2),
-    };
-
-    double slope = (double) (right_avg.system - left_avg.system)
-                 / (right_avg.stream - left_avg.stream);
-
-    if (clock->count < SC_CLOCK_RANGE) {
-        /* The first frames are typically received and decoded with more delay
-         * than the others, causing a wrong slope estimation on start. To
-         * compensate, assume an initial slope of 1, then progressively use the
-         * estimated slope. */
-        slope = (clock->count * slope + (SC_CLOCK_RANGE - clock->count))
-              / SC_CLOCK_RANGE;
-    }
-
-    struct sc_clock_point global_avg = {
-        .system = (clock->left_sum.system + clock->right_sum.system)
-                 / clock->count,
-        .stream = (clock->left_sum.stream + clock->right_sum.stream)
-                 / clock->count,
-    };
-
-    sc_tick offset = global_avg.system - (sc_tick) (global_avg.stream * slope);
-
-    *out_slope = slope;
-    *out_offset = offset;
+    clock->range = 0;
+    clock->offset = 0;
 }
 
 void
 sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
-    struct sc_clock_point *point = &clock->points[clock->head];
-
-    if (clock->count == SC_CLOCK_RANGE || clock->count & 1) {
-        // One point passes from the right sum to the left sum
-
-        unsigned mid;
-        if (clock->count == SC_CLOCK_RANGE) {
-            mid = (clock->head + SC_CLOCK_RANGE / 2) % SC_CLOCK_RANGE;
-        } else {
-            // Only for the first frames
-            mid = clock->count / 2;
-        }
-
-        struct sc_clock_point *mid_point = &clock->points[mid];
-        clock->left_sum.system += mid_point->system;
-        clock->left_sum.stream += mid_point->stream;
-        clock->right_sum.system -= mid_point->system;
-        clock->right_sum.stream -= mid_point->stream;
-    }
-
-    if (clock->count == SC_CLOCK_RANGE) {
-        // The current point overwrites the previous value in the circular
-        // array, update the left sum accordingly
-        clock->left_sum.system -= point->system;
-        clock->left_sum.stream -= point->stream;
-    } else {
-        ++clock->count;
+    if (clock->range < SC_CLOCK_RANGE) {
+        ++clock->range;
     }
 
-    point->system = system;
-    point->stream = stream;
-
-    clock->right_sum.system += system;
-    clock->right_sum.stream += stream;
-
-    clock->head = (clock->head + 1) % SC_CLOCK_RANGE;
-
-    if (clock->count > 1) {
-        // Update estimation
-        sc_clock_estimate(clock, &clock->slope, &clock->offset);
+    sc_tick offset = system - stream;
+    clock->offset = ((clock->range - 1) * clock->offset + offset)
+                  / clock->range;
 
 #ifndef SC_CLOCK_NDEBUG
-        LOGD("Clock estimation: %f * pts + %" PRItick,
-             clock->slope, clock->offset);
+    LOGD("Clock estimation: pts + %" PRItick, clock->offset);
 #endif
-    }
 }
 
 sc_tick
 sc_clock_to_system_time(struct sc_clock *clock, sc_tick stream) {
-    assert(clock->count > 1); // sc_clock_update() must have been called
-    return (sc_tick) (stream * clock->slope) + clock->offset;
+    assert(clock->range); // sc_clock_update() must have been called
+    return stream + clock->offset;
 }

app/src/clock.h

@@ -3,13 +3,8 @@
 
 #include "common.h"
 
-#include <assert.h>
-
 #include "util/tick.h"
 
-#define SC_CLOCK_RANGE 32
-static_assert(!(SC_CLOCK_RANGE & 1), "SC_CLOCK_RANGE must be even");
-
 struct sc_clock_point {
     sc_tick system;
     sc_tick stream;
@@ -21,40 +16,18 @@ struct sc_clock_point {
  *
  *     f(stream) = slope * stream + offset
  *
- * To that end, it stores the SC_CLOCK_RANGE last clock points (the timestamps
- * of a frame expressed both in stream time and system time) in a circular
- * array.
+ * Theoretically, the slope encodes the drift between the device clock and the
+ * computer clock. It is expected to be very close to 1.
  *
- * To estimate the slope, it splits the last SC_CLOCK_RANGE points into two
- * sets of SC_CLOCK_RANGE/2 points, and computes their centroid ("average
- * point"). The slope of the estimated affine function is that of the line
- * passing through these two points.
+ * Since the clock is used to estimate very close points in the future (which
+ * are reestimated on every clock update, see delay_buffer), the error caused
+ * by clock drift is totally negligible, so it is better to assume that the
+ * slope is 1 than to estimate it (the estimation error would be larger).
  *
- * To estimate the offset, it computes the centroid of all the SC_CLOCK_RANGE
- * points. The resulting affine function passes by this centroid.
- *
- * With a circular array, the rolling sums (and average) are quick to compute.
- * In practice, the estimation is stable and the evolution is smooth.
+ * Therefore, only the offset is estimated.
  */
 struct sc_clock {
-    // Circular array
-    struct sc_clock_point points[SC_CLOCK_RANGE];
-
-    // Number of points in the array (count <= SC_CLOCK_RANGE)
-    unsigned count;
-
-    // Index of the next point to write
-    unsigned head;
-
-    // Sum of the first count/2 points
-    struct sc_clock_point left_sum;
-
-    // Sum of the last (count+1)/2 points
-    struct sc_clock_point right_sum;
-
-    // Estimated slope and offset
-    // (computed on sc_clock_update(), used by sc_clock_to_system_time())
-    double slope;
+    unsigned range;
     sc_tick offset;
 };
 

app/src/common.h

@@ -5,8 +5,8 @@
 #include "compat.h"
 
 #define ARRAY_LEN(a) (sizeof(a) / sizeof(a[0]))
-#define MIN(X,Y) (X) < (Y) ? (X) : (Y)
-#define MAX(X,Y) (X) > (Y) ? (X) : (Y)
+#define MIN(X,Y) ((X) < (Y) ? (X) : (Y))
+#define MAX(X,Y) ((X) > (Y) ? (X) : (Y))
 #define CLAMP(V,X,Y) MIN( MAX((V),(X)), (Y) )
 
 #define container_of(ptr, type, member) \

app/src/compat.c

@@ -3,6 +3,9 @@
 #include "config.h"
 
 #include <assert.h>
+#ifndef HAVE_REALLOCARRAY
+# include <errno.h>
+#endif
 #include <stdlib.h>
 #include <stdio.h>
 #include <stdarg.h>
@@ -51,3 +54,57 @@ int vasprintf(char **strp, const char *fmt, va_list ap) {
     return len;
 }
 #endif
+
+#if !defined(HAVE_NRAND48) || !defined(HAVE_JRAND48)
+#define SC_RAND48_MASK UINT64_C(0xFFFFFFFFFFFF) // 48 bits
+#define SC_RAND48_A UINT64_C(0x5DEECE66D)
+#define SC_RAND48_C 0xB
+static inline uint64_t rand_iter48(uint64_t x) {
+    assert((x & ~SC_RAND48_MASK) == 0);
+    return (x * SC_RAND48_A + SC_RAND48_C) & SC_RAND48_MASK;
+}
+
+static uint64_t rand_iter48_xsubi(unsigned short xsubi[3]) {
+    uint64_t x = ((uint64_t) xsubi[0] << 32)
+               | ((uint64_t) xsubi[1] << 16)
+               | xsubi[2];
+
+    x = rand_iter48(x);
+
+    xsubi[0] = (x >> 32) & 0XFFFF;
+    xsubi[1] = (x >> 16) & 0XFFFF;
+    xsubi[2] = x & 0XFFFF;
+
+    return x;
+}
+
+#ifndef HAVE_NRAND48
+long nrand48(unsigned short xsubi[3]) {
+    // range [0, 2^31)
+    return rand_iter48_xsubi(xsubi) >> 17;
+}
+#endif
+
+#ifndef HAVE_JRAND48
+long jrand48(unsigned short xsubi[3]) {
+    // range [-2^31, 2^31)
+    union {
+        uint32_t u;
+        int32_t i;
+    } v;
+    v.u = rand_iter48_xsubi(xsubi) >> 16;
+    return v.i;
+}
+#endif
+#endif
+
+#ifndef HAVE_REALLOCARRAY
+void *reallocarray(void *ptr, size_t nmemb, size_t size) {
+    size_t bytes;
+    if (__builtin_mul_overflow(nmemb, size, &bytes)) {
+      errno = ENOMEM;
+      return NULL;
+    }
+    return realloc(ptr, bytes);
+}
+#endif

app/src/compat.h

@@ -3,7 +3,9 @@
 
 #include "config.h"
 
+#include <libavcodec/version.h>
 #include <libavformat/version.h>
+#include <libavutil/version.h>
 #include <SDL2/SDL_version.h>
 
 #ifndef __WIN32
@@ -25,6 +27,12 @@
 # define SCRCPY_LAVF_REQUIRES_REGISTER_ALL
 #endif
 
+// Not documented in ffmpeg/doc/APIchanges, but AV_CODEC_ID_AV1 has been added
+// by FFmpeg commit d42809f9835a4e9e5c7c63210abb09ad0ef19cfb (included in tag
+// n3.3).
+#if LIBAVFORMAT_VERSION_INT >= AV_VERSION_INT(57, 89, 100)
+# define SCRCPY_LAVC_HAS_AV1
+#endif
 
 // In ffmpeg/doc/APIchanges:
 // 2018-01-28 - ea3672b7d6 - lavf 58.7.100 - avformat.h
@@ -37,6 +45,22 @@
 # define SCRCPY_LAVF_HAS_AVFORMATCONTEXT_URL
 #endif
 
+// Not documented in ffmpeg/doc/APIchanges, but the channel_layout API
+// has been replaced by chlayout in FFmpeg commit
+// f423497b455da06c1337846902c770028760e094.
+#if LIBAVUTIL_VERSION_INT >= AV_VERSION_INT(57, 23, 100)
+# define SCRCPY_LAVU_HAS_CHLAYOUT
+#endif
+
+// In ffmpeg/doc/APIchanges:
+// 2023-10-06 - 5432d2aacad - lavc 60.15.100 - avformat.h
+//   Deprecate AVFormatContext.{nb_,}side_data, av_stream_add_side_data(),
+//   av_stream_new_side_data(), and av_stream_get_side_data(). Side data fields
+//   from AVFormatContext.codecpar should be used from now on.
+#if LIBAVCODEC_VERSION_INT >= AV_VERSION_INT(60, 15, 100)
+# define SCRCPY_LAVC_HAS_CODECPAR_CODEC_SIDEDATA
+#endif
+
 #if SDL_VERSION_ATLEAST(2, 0, 6)
 // <https://github.com/libsdl-org/SDL/commit/d7a318de563125e5bb465b1000d6bc9576fbc6fc>
 # define SCRCPY_SDL_HAS_HINT_TOUCH_MOUSE_EVENTS
@@ -47,6 +71,10 @@
 # define SCRCPY_SDL_HAS_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR
 #endif
 
+#if SDL_VERSION_ATLEAST(2, 0, 16)
+# define SCRCPY_SDL_HAS_THREAD_PRIORITY_TIME_CRITICAL
+#endif
+
 #ifndef HAVE_STRDUP
 char *strdup(const char *s);
 #endif
@@ -59,4 +87,16 @@ int asprintf(char **strp, const char *fmt, ...);
 int vasprintf(char **strp, const char *fmt, va_list ap);
 #endif
 
+#ifndef HAVE_NRAND48
+long nrand48(unsigned short xsubi[3]);
+#endif
+
+#ifndef HAVE_JRAND48
+long jrand48(unsigned short xsubi[3]);
+#endif
+
+#ifndef HAVE_REALLOCARRAY
+void *reallocarray(void *ptr, size_t nmemb, size_t size);
+#endif
+
 #endif

app/src/control_msg.c

@@ -87,7 +87,7 @@ write_position(uint8_t *buf, const struct sc_position *position) {
 
 // write length (4 bytes) + string (non null-terminated)
 static size_t
-write_string(const char *utf8, size_t max_len, unsigned char *buf) {
+write_string(const char *utf8, size_t max_len, uint8_t *buf) {
     size_t len = sc_str_utf8_truncation_index(utf8, max_len);
     sc_write32be(buf, len);
     memcpy(&buf[4], utf8, len);
@@ -95,7 +95,7 @@ write_string(const char *utf8, size_t max_len, unsigned char *buf) {
 }
 
 size_t
-sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
+sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf) {
     buf[0] = msg->type;
     switch (msg->type) {
         case SC_CONTROL_MSG_TYPE_INJECT_KEYCODE:
@@ -117,8 +117,9 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
             uint16_t pressure =
                 sc_float_to_u16fp(msg->inject_touch_event.pressure);
             sc_write16be(&buf[22], pressure);
-            sc_write32be(&buf[24], msg->inject_touch_event.buttons);
-            return 28;
+            sc_write32be(&buf[24], msg->inject_touch_event.action_button);
+            sc_write32be(&buf[28], msg->inject_touch_event.buttons);
+            return 32;
         case SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT:
             write_position(&buf[1], &msg->inject_scroll_event.position);
             int16_t hscroll =
@@ -145,10 +146,22 @@ sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf) {
         case SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE:
             buf[1] = msg->set_screen_power_mode.mode;
             return 2;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            sc_write16be(&buf[1], msg->uhid_create.id);
+            sc_write16be(&buf[3], msg->uhid_create.report_desc_size);
+            memcpy(&buf[5], msg->uhid_create.report_desc,
+                            msg->uhid_create.report_desc_size);
+            return 5 + msg->uhid_create.report_desc_size;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT:
+            sc_write16be(&buf[1], msg->uhid_input.id);
+            sc_write16be(&buf[3], msg->uhid_input.size);
+            memcpy(&buf[5], msg->uhid_input.data, msg->uhid_input.size);
+            return 5 + msg->uhid_input.size;
         case SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL:
         case SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL:
         case SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS:
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
             // no additional data
             return 1;
         default:
@@ -179,22 +192,25 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
             if (pointer_name) {
                 // string pointer id
                 LOG_CMSG("touch [id=%s] %-4s position=%" PRIi32 ",%" PRIi32
-                             " pressure=%f buttons=%06lx",
+                             " pressure=%f action_button=%06lx buttons=%06lx",
                          pointer_name,
                          MOTIONEVENT_ACTION_LABEL(action),
                          msg->inject_touch_event.position.point.x,
                          msg->inject_touch_event.position.point.y,
                          msg->inject_touch_event.pressure,
+                         (long) msg->inject_touch_event.action_button,
                          (long) msg->inject_touch_event.buttons);
             } else {
                 // numeric pointer id
                 LOG_CMSG("touch [id=%" PRIu64_ "] %-4s position=%" PRIi32 ",%"
-                             PRIi32 " pressure=%f buttons=%06lx",
+                             PRIi32 " pressure=%f action_button=%06lx"
+                             " buttons=%06lx",
                          id,
                          MOTIONEVENT_ACTION_LABEL(action),
                          msg->inject_touch_event.position.point.x,
                          msg->inject_touch_event.position.point.y,
                          msg->inject_touch_event.pressure,
+                         (long) msg->inject_touch_event.action_button,
                          (long) msg->inject_touch_event.buttons);
             }
             break;
@@ -238,6 +254,26 @@ sc_control_msg_log(const struct sc_control_msg *msg) {
         case SC_CONTROL_MSG_TYPE_ROTATE_DEVICE:
             LOG_CMSG("rotate device");
             break;
+        case SC_CONTROL_MSG_TYPE_UHID_CREATE:
+            LOG_CMSG("UHID create [%" PRIu16 "] report_desc_size=%" PRIu16,
+                     msg->uhid_create.id, msg->uhid_create.report_desc_size);
+            break;
+        case SC_CONTROL_MSG_TYPE_UHID_INPUT: {
+            char *hex = sc_str_to_hex_string(msg->uhid_input.data,
+                                             msg->uhid_input.size);
+            if (hex) {
+                LOG_CMSG("UHID input [%" PRIu16 "] %s",
+                         msg->uhid_input.id, hex);
+                free(hex);
+            } else {
+                LOG_CMSG("UHID input [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_input.id, msg->uhid_input.size);
+            }
+            break;
+        }
+        case SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS:
+            LOG_CMSG("open hard keyboard settings");
+            break;
         default:
             LOG_CMSG("unknown type: %u", (unsigned) msg->type);
             break;

app/src/control_msg.h

@@ -10,6 +10,7 @@
 #include "android/input.h"
 #include "android/keycodes.h"
 #include "coords.h"
+#include "hid/hid_event.h"
 
 #define SC_CONTROL_MSG_MAX_SIZE (1 << 18) // 256k
 
@@ -37,6 +38,9 @@ enum sc_control_msg_type {
     SC_CONTROL_MSG_TYPE_SET_CLIPBOARD,
     SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE,
     SC_CONTROL_MSG_TYPE_ROTATE_DEVICE,
+    SC_CONTROL_MSG_TYPE_UHID_CREATE,
+    SC_CONTROL_MSG_TYPE_UHID_INPUT,
+    SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS,
 };
 
 enum sc_screen_power_mode {
@@ -65,6 +69,7 @@ struct sc_control_msg {
         } inject_text;
         struct {
             enum android_motionevent_action action;
+            enum android_motionevent_buttons action_button;
             enum android_motionevent_buttons buttons;
             uint64_t pointer_id;
             struct sc_position position;
@@ -91,13 +96,23 @@ struct sc_control_msg {
         struct {
             enum sc_screen_power_mode mode;
         } set_screen_power_mode;
+        struct {
+            uint16_t id;
+            uint16_t report_desc_size;
+            const uint8_t *report_desc; // pointer to static data
+        } uhid_create;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t data[SC_HID_MAX_SIZE];
+        } uhid_input;
     };
 };
 
 // buf size must be at least CONTROL_MSG_MAX_SIZE
 // return the number of bytes written
 size_t
-sc_control_msg_serialize(const struct sc_control_msg *msg, unsigned char *buf);
+sc_control_msg_serialize(const struct sc_control_msg *msg, uint8_t *buf);
 
 void
 sc_control_msg_log(const struct sc_control_msg *msg);

app/src/controller.c

@@ -4,26 +4,35 @@
 
 #include "util/log.h"
 
+#define SC_CONTROL_MSG_QUEUE_MAX 64
+
 bool
-sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync) {
-    cbuf_init(&controller->queue);
+sc_controller_init(struct sc_controller *controller, sc_socket control_socket) {
+    sc_vecdeque_init(&controller->queue);
 
-    bool ok = receiver_init(&controller->receiver, control_socket, acksync);
+    bool ok = sc_vecdeque_reserve(&controller->queue, SC_CONTROL_MSG_QUEUE_MAX);
     if (!ok) {
         return false;
     }
 
+    ok = sc_receiver_init(&controller->receiver, control_socket);
+    if (!ok) {
+        sc_vecdeque_destroy(&controller->queue);
+        return false;
+    }
+
     ok = sc_mutex_init(&controller->mutex);
     if (!ok) {
-        receiver_destroy(&controller->receiver);
+        sc_receiver_destroy(&controller->receiver);
+        sc_vecdeque_destroy(&controller->queue);
         return false;
     }
 
     ok = sc_cond_init(&controller->msg_cond);
     if (!ok) {
-        receiver_destroy(&controller->receiver);
+        sc_receiver_destroy(&controller->receiver);
         sc_mutex_destroy(&controller->mutex);
+        sc_vecdeque_destroy(&controller->queue);
         return false;
     }
 
@@ -33,17 +42,27 @@ sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
     return true;
 }
 
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices) {
+    controller->receiver.acksync = acksync;
+    controller->receiver.uhid_devices = uhid_devices;
+}
+
 void
 sc_controller_destroy(struct sc_controller *controller) {
     sc_cond_destroy(&controller->msg_cond);
     sc_mutex_destroy(&controller->mutex);
 
-    struct sc_control_msg msg;
-    while (cbuf_take(&controller->queue, &msg)) {
-        sc_control_msg_destroy(&msg);
+    while (!sc_vecdeque_is_empty(&controller->queue)) {
+        struct sc_control_msg *msg = sc_vecdeque_popref(&controller->queue);
+        assert(msg);
+        sc_control_msg_destroy(msg);
     }
+    sc_vecdeque_destroy(&controller->queue);
 
-    receiver_destroy(&controller->receiver);
+    sc_receiver_destroy(&controller->receiver);
 }
 
 bool
@@ -54,19 +73,25 @@ sc_controller_push_msg(struct sc_controller *controller,
     }
 
     sc_mutex_lock(&controller->mutex);
-    bool was_empty = cbuf_is_empty(&controller->queue);
-    bool res = cbuf_push(&controller->queue, *msg);
-    if (was_empty) {
-        sc_cond_signal(&controller->msg_cond);
+    bool full = sc_vecdeque_is_full(&controller->queue);
+    if (!full) {
+        bool was_empty = sc_vecdeque_is_empty(&controller->queue);
+        sc_vecdeque_push_noresize(&controller->queue, *msg);
+        if (was_empty) {
+            sc_cond_signal(&controller->msg_cond);
+        }
     }
+    // Otherwise (if the queue is full), the msg is discarded
+
     sc_mutex_unlock(&controller->mutex);
-    return res;
+
+    return !full;
 }
 
 static bool
 process_msg(struct sc_controller *controller,
             const struct sc_control_msg *msg) {
-    static unsigned char serialized_msg[SC_CONTROL_MSG_MAX_SIZE];
+    static uint8_t serialized_msg[SC_CONTROL_MSG_MAX_SIZE];
     size_t length = sc_control_msg_serialize(msg, serialized_msg);
     if (!length) {
         return false;
@@ -82,7 +107,8 @@ run_controller(void *data) {
 
     for (;;) {
         sc_mutex_lock(&controller->mutex);
-        while (!controller->stopped && cbuf_is_empty(&controller->queue)) {
+        while (!controller->stopped
+                && sc_vecdeque_is_empty(&controller->queue)) {
             sc_cond_wait(&controller->msg_cond, &controller->mutex);
         }
         if (controller->stopped) {
@@ -90,10 +116,9 @@ run_controller(void *data) {
             sc_mutex_unlock(&controller->mutex);
             break;
         }
-        struct sc_control_msg msg;
-        bool non_empty = cbuf_take(&controller->queue, &msg);
-        assert(non_empty);
-        (void) non_empty;
+
+        assert(!sc_vecdeque_is_empty(&controller->queue));
+        struct sc_control_msg msg = sc_vecdeque_pop(&controller->queue);
         sc_mutex_unlock(&controller->mutex);
 
         bool ok = process_msg(controller, &msg);
@@ -117,7 +142,7 @@ sc_controller_start(struct sc_controller *controller) {
         return false;
     }
 
-    if (!receiver_start(&controller->receiver)) {
+    if (!sc_receiver_start(&controller->receiver)) {
         sc_controller_stop(controller);
         sc_thread_join(&controller->thread, NULL);
         return false;
@@ -137,5 +162,5 @@ sc_controller_stop(struct sc_controller *controller) {
 void
 sc_controller_join(struct sc_controller *controller) {
     sc_thread_join(&controller->thread, NULL);
-    receiver_join(&controller->receiver);
+    sc_receiver_join(&controller->receiver);
 }

app/src/controller.h

@@ -8,11 +8,11 @@
 #include "control_msg.h"
 #include "receiver.h"
 #include "util/acksync.h"
-#include "util/cbuf.h"
 #include "util/net.h"
 #include "util/thread.h"
+#include "util/vecdeque.h"
 
-struct sc_control_msg_queue CBUF(struct sc_control_msg, 64);
+struct sc_control_msg_queue SC_VECDEQUE(struct sc_control_msg);
 
 struct sc_controller {
     sc_socket control_socket;
@@ -21,12 +21,16 @@ struct sc_controller {
     sc_cond msg_cond;
     bool stopped;
     struct sc_control_msg_queue queue;
-    struct receiver receiver;
+    struct sc_receiver receiver;
 };
 
 bool
-sc_controller_init(struct sc_controller *controller, sc_socket control_socket,
-                   struct sc_acksync *acksync);
+sc_controller_init(struct sc_controller *controller, sc_socket control_socket);
+
+void
+sc_controller_configure(struct sc_controller *controller,
+                        struct sc_acksync *acksync,
+                        struct sc_uhid_devices *uhid_devices);
 
 void
 sc_controller_destroy(struct sc_controller *controller);

app/src/decoder.c

@@ -2,96 +2,37 @@
 
 #include <libavcodec/avcodec.h>
 #include <libavformat/avformat.h>
+#include <libavutil/channel_layout.h>
 
 #include "events.h"
-#include "video_buffer.h"
 #include "trait/frame_sink.h"
 #include "util/log.h"
 
 /** Downcast packet_sink to decoder */
 #define DOWNCAST(SINK) container_of(SINK, struct sc_decoder, packet_sink)
 
-static void
-sc_decoder_close_first_sinks(struct sc_decoder *decoder, unsigned count) {
-    while (count) {
-        struct sc_frame_sink *sink = decoder->sinks[--count];
-        sink->ops->close(sink);
-    }
-}
-
-static inline void
-sc_decoder_close_sinks(struct sc_decoder *decoder) {
-    sc_decoder_close_first_sinks(decoder, decoder->sink_count);
-}
-
 static bool
-sc_decoder_open_sinks(struct sc_decoder *decoder) {
-    for (unsigned i = 0; i < decoder->sink_count; ++i) {
-        struct sc_frame_sink *sink = decoder->sinks[i];
-        if (!sink->ops->open(sink)) {
-            LOGE("Could not open frame sink %d", i);
-            sc_decoder_close_first_sinks(decoder, i);
-            return false;
-        }
-    }
-
-    return true;
-}
-
-static bool
-sc_decoder_open(struct sc_decoder *decoder, const AVCodec *codec) {
-    decoder->codec_ctx = avcodec_alloc_context3(codec);
-    if (!decoder->codec_ctx) {
-        LOG_OOM();
-        return false;
-    }
-
-    decoder->codec_ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
-
-    if (avcodec_open2(decoder->codec_ctx, codec, NULL) < 0) {
-        LOGE("Could not open codec");
-        avcodec_free_context(&decoder->codec_ctx);
-        return false;
-    }
-
+sc_decoder_open(struct sc_decoder *decoder, AVCodecContext *ctx) {
     decoder->frame = av_frame_alloc();
     if (!decoder->frame) {
         LOG_OOM();
-        avcodec_close(decoder->codec_ctx);
-        avcodec_free_context(&decoder->codec_ctx);
         return false;
     }
 
-    if (!sc_decoder_open_sinks(decoder)) {
-        LOGE("Could not open decoder sinks");
+    if (!sc_frame_source_sinks_open(&decoder->frame_source, ctx)) {
         av_frame_free(&decoder->frame);
-        avcodec_close(decoder->codec_ctx);
-        avcodec_free_context(&decoder->codec_ctx);
         return false;
     }
 
+    decoder->ctx = ctx;
+
     return true;
 }
 
 static void
 sc_decoder_close(struct sc_decoder *decoder) {
-    sc_decoder_close_sinks(decoder);
+    sc_frame_source_sinks_close(&decoder->frame_source);
     av_frame_free(&decoder->frame);
-    avcodec_close(decoder->codec_ctx);
-    avcodec_free_context(&decoder->codec_ctx);
-}
-
-static bool
-push_frame_to_sinks(struct sc_decoder *decoder, const AVFrame *frame) {
-    for (unsigned i = 0; i < decoder->sink_count; ++i) {
-        struct sc_frame_sink *sink = decoder->sinks[i];
-        if (!sink->ops->push(sink, frame)) {
-            LOGE("Could not send frame to sink %d", i);
-            return false;
-        }
-    }
-
-    return true;
 }
 
 static bool
@@ -102,31 +43,42 @@ sc_decoder_push(struct sc_decoder *decoder, const AVPacket *packet) {
         return true;
     }
 
-    int ret = avcodec_send_packet(decoder->codec_ctx, packet);
+    int ret = avcodec_send_packet(decoder->ctx, packet);
     if (ret < 0 && ret != AVERROR(EAGAIN)) {
-        LOGE("Could not send video packet: %d", ret);
+        LOGE("Decoder '%s': could not send video packet: %d",
+             decoder->name, ret);
         return false;
     }
-    ret = avcodec_receive_frame(decoder->codec_ctx, decoder->frame);
-    if (!ret) {
-        // a frame was received
-        bool ok = push_frame_to_sinks(decoder, decoder->frame);
-        // A frame lost should not make the whole pipeline fail. The error, if
-        // any, is already logged.
-        (void) ok;
 
+    for (;;) {
+        ret = avcodec_receive_frame(decoder->ctx, decoder->frame);
+        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
+            break;
+        }
+
+        if (ret) {
+            LOGE("Decoder '%s', could not receive video frame: %d",
+                 decoder->name, ret);
+            return false;
+        }
+
+        // a frame was received
+        bool ok = sc_frame_source_sinks_push(&decoder->frame_source,
+                                             decoder->frame);
         av_frame_unref(decoder->frame);
-    } else if (ret != AVERROR(EAGAIN)) {
-        LOGE("Could not receive video frame: %d", ret);
-        return false;
+        if (!ok) {
+            // Error already logged
+            return false;
+        }
     }
+
     return true;
 }
 
 static bool
-sc_decoder_packet_sink_open(struct sc_packet_sink *sink, const AVCodec *codec) {
+sc_decoder_packet_sink_open(struct sc_packet_sink *sink, AVCodecContext *ctx) {
     struct sc_decoder *decoder = DOWNCAST(sink);
-    return sc_decoder_open(decoder, codec);
+    return sc_decoder_open(decoder, ctx);
 }
 
 static void
@@ -143,8 +95,9 @@ sc_decoder_packet_sink_push(struct sc_packet_sink *sink,
 }
 
 void
-sc_decoder_init(struct sc_decoder *decoder) {
-    decoder->sink_count = 0;
+sc_decoder_init(struct sc_decoder *decoder, const char *name) {
+    decoder->name = name; // statically allocated
+    sc_frame_source_init(&decoder->frame_source);
 
     static const struct sc_packet_sink_ops ops = {
         .open = sc_decoder_packet_sink_open,
@@ -154,11 +107,3 @@ sc_decoder_init(struct sc_decoder *decoder) {
 
     decoder->packet_sink.ops = &ops;
 }
-
-void
-sc_decoder_add_sink(struct sc_decoder *decoder, struct sc_frame_sink *sink) {
-    assert(decoder->sink_count < SC_DECODER_MAX_SINKS);
-    assert(sink);
-    assert(sink->ops);
-    decoder->sinks[decoder->sink_count++] = sink;
-}

app/src/decoder.h

@@ -3,28 +3,25 @@
 
 #include "common.h"
 
+#include "trait/frame_source.h"
 #include "trait/packet_sink.h"
 
 #include <stdbool.h>
 #include <libavcodec/avcodec.h>
 #include <libavformat/avformat.h>
 
-#define SC_DECODER_MAX_SINKS 2
-
 struct sc_decoder {
     struct sc_packet_sink packet_sink; // packet sink trait
+    struct sc_frame_source frame_source; // frame source trait
 
-    struct sc_frame_sink *sinks[SC_DECODER_MAX_SINKS];
-    unsigned sink_count;
+    const char *name; // must be statically allocated (e.g. a string literal)
 
-    AVCodecContext *codec_ctx;
+    AVCodecContext *ctx;
     AVFrame *frame;
 };
 
+// The name must be statically allocated (e.g. a string literal)
 void
-sc_decoder_init(struct sc_decoder *decoder);
-
-void
-sc_decoder_add_sink(struct sc_decoder *decoder, struct sc_frame_sink *sink);
+sc_decoder_init(struct sc_decoder *decoder, const char *name);
 
 #endif

app/src/delay_buffer.c

@@ -0,0 +1,244 @@
+#include "delay_buffer.h"
+
+#include <assert.h>
+#include <stdlib.h>
+
+#include <libavutil/avutil.h>
+#include <libavformat/avformat.h>
+
+#include "util/log.h"
+
+#define SC_BUFFERING_NDEBUG // comment to debug
+
+/** Downcast frame_sink to sc_delay_buffer */
+#define DOWNCAST(SINK) container_of(SINK, struct sc_delay_buffer, frame_sink)
+
+static bool
+sc_delayed_frame_init(struct sc_delayed_frame *dframe, const AVFrame *frame) {
+    dframe->frame = av_frame_alloc();
+    if (!dframe->frame) {
+        LOG_OOM();
+        return false;
+    }
+
+    if (av_frame_ref(dframe->frame, frame)) {
+        LOG_OOM();
+        av_frame_free(&dframe->frame);
+        return false;
+    }
+
+    return true;
+}
+
+static void
+sc_delayed_frame_destroy(struct sc_delayed_frame *dframe) {
+    av_frame_unref(dframe->frame);
+    av_frame_free(&dframe->frame);
+}
+
+static int
+run_buffering(void *data) {
+    struct sc_delay_buffer *db = data;
+
+    assert(db->delay > 0);
+
+    for (;;) {
+        sc_mutex_lock(&db->mutex);
+
+        while (!db->stopped && sc_vecdeque_is_empty(&db->queue)) {
+            sc_cond_wait(&db->queue_cond, &db->mutex);
+        }
+
+        if (db->stopped) {
+            sc_mutex_unlock(&db->mutex);
+            goto stopped;
+        }
+
+        struct sc_delayed_frame dframe = sc_vecdeque_pop(&db->queue);
+
+        sc_tick max_deadline = sc_tick_now() + db->delay;
+        // PTS (written by the server) are expressed in microseconds
+        sc_tick pts = SC_TICK_FROM_US(dframe.frame->pts);
+
+        bool timed_out = false;
+        while (!db->stopped && !timed_out) {
+            sc_tick deadline = sc_clock_to_system_time(&db->clock, pts)
+                             + db->delay;
+            if (deadline > max_deadline) {
+                deadline = max_deadline;
+            }
+
+            timed_out =
+                !sc_cond_timedwait(&db->wait_cond, &db->mutex, deadline);
+        }
+
+        bool stopped = db->stopped;
+        sc_mutex_unlock(&db->mutex);
+
+        if (stopped) {
+            sc_delayed_frame_destroy(&dframe);
+            goto stopped;
+        }
+
+#ifndef SC_BUFFERING_NDEBUG
+        LOGD("Buffering: %" PRItick ";%" PRItick ";%" PRItick,
+             pts, dframe.push_date, sc_tick_now());
+#endif
+
+        bool ok = sc_frame_source_sinks_push(&db->frame_source, dframe.frame);
+        sc_delayed_frame_destroy(&dframe);
+        if (!ok) {
+            LOGE("Delayed frame could not be pushed, stopping");
+            sc_mutex_lock(&db->mutex);
+            // Prevent to push any new frame
+            db->stopped = true;
+            sc_mutex_unlock(&db->mutex);
+            goto stopped;
+        }
+    }
+
+stopped:
+    assert(db->stopped);
+
+    // Flush queue
+    while (!sc_vecdeque_is_empty(&db->queue)) {
+        struct sc_delayed_frame *dframe = sc_vecdeque_popref(&db->queue);
+        sc_delayed_frame_destroy(dframe);
+    }
+
+    LOGD("Buffering thread ended");
+
+    return 0;
+}
+
+static bool
+sc_delay_buffer_frame_sink_open(struct sc_frame_sink *sink,
+                                const AVCodecContext *ctx) {
+    struct sc_delay_buffer *db = DOWNCAST(sink);
+    (void) ctx;
+
+    bool ok = sc_mutex_init(&db->mutex);
+    if (!ok) {
+        return false;
+    }
+
+    ok = sc_cond_init(&db->queue_cond);
+    if (!ok) {
+        goto error_destroy_mutex;
+    }
+
+    ok = sc_cond_init(&db->wait_cond);
+    if (!ok) {
+        goto error_destroy_queue_cond;
+    }
+
+    sc_clock_init(&db->clock);
+    sc_vecdeque_init(&db->queue);
+
+    if (!sc_frame_source_sinks_open(&db->frame_source, ctx)) {
+        goto error_destroy_wait_cond;
+    }
+
+    ok = sc_thread_create(&db->thread, run_buffering, "scrcpy-dbuf", db);
+    if (!ok) {
+        LOGE("Could not start buffering thread");
+        goto error_close_sinks;
+    }
+
+    return true;
+
+error_close_sinks:
+    sc_frame_source_sinks_close(&db->frame_source);
+error_destroy_wait_cond:
+    sc_cond_destroy(&db->wait_cond);
+error_destroy_queue_cond:
+    sc_cond_destroy(&db->queue_cond);
+error_destroy_mutex:
+    sc_mutex_destroy(&db->mutex);
+
+    return false;
+}
+
+static void
+sc_delay_buffer_frame_sink_close(struct sc_frame_sink *sink) {
+    struct sc_delay_buffer *db = DOWNCAST(sink);
+
+    sc_mutex_lock(&db->mutex);
+    db->stopped = true;
+    sc_cond_signal(&db->queue_cond);
+    sc_cond_signal(&db->wait_cond);
+    sc_mutex_unlock(&db->mutex);
+
+    sc_thread_join(&db->thread, NULL);
+
+    sc_frame_source_sinks_close(&db->frame_source);
+
+    sc_cond_destroy(&db->wait_cond);
+    sc_cond_destroy(&db->queue_cond);
+    sc_mutex_destroy(&db->mutex);
+}
+
+static bool
+sc_delay_buffer_frame_sink_push(struct sc_frame_sink *sink,
+                                const AVFrame *frame) {
+    struct sc_delay_buffer *db = DOWNCAST(sink);
+
+    sc_mutex_lock(&db->mutex);
+
+    if (db->stopped) {
+        sc_mutex_unlock(&db->mutex);
+        return false;
+    }
+
+    sc_tick pts = SC_TICK_FROM_US(frame->pts);
+    sc_clock_update(&db->clock, sc_tick_now(), pts);
+    sc_cond_signal(&db->wait_cond);
+
+    if (db->first_frame_asap && db->clock.range == 1) {
+        sc_mutex_unlock(&db->mutex);
+        return sc_frame_source_sinks_push(&db->frame_source, frame);
+    }
+
+    struct sc_delayed_frame dframe;
+    bool ok = sc_delayed_frame_init(&dframe, frame);
+    if (!ok) {
+        sc_mutex_unlock(&db->mutex);
+        return false;
+    }
+
+#ifndef SC_BUFFERING_NDEBUG
+    dframe.push_date = sc_tick_now();
+#endif
+
+    ok = sc_vecdeque_push(&db->queue, dframe);
+    if (!ok) {
+        sc_mutex_unlock(&db->mutex);
+        LOG_OOM();
+        return false;
+    }
+
+    sc_cond_signal(&db->queue_cond);
+
+    sc_mutex_unlock(&db->mutex);
+
+    return true;
+}
+
+void
+sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
+                     bool first_frame_asap) {
+    assert(delay > 0);
+
+    db->delay = delay;
+    db->first_frame_asap = first_frame_asap;
+
+    sc_frame_source_init(&db->frame_source);
+
+    static const struct sc_frame_sink_ops ops = {
+        .open = sc_delay_buffer_frame_sink_open,
+        .close = sc_delay_buffer_frame_sink_close,
+        .push = sc_delay_buffer_frame_sink_push,
+    };
+
+    db->frame_sink.ops = &ops;
+}

app/src/delay_buffer.h

@@ -0,0 +1,60 @@
+#ifndef SC_DELAY_BUFFER_H
+#define SC_DELAY_BUFFER_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "clock.h"
+#include "trait/frame_source.h"
+#include "trait/frame_sink.h"
+#include "util/thread.h"
+#include "util/tick.h"
+#include "util/vecdeque.h"
+
+// forward declarations
+typedef struct AVFrame AVFrame;
+
+struct sc_delayed_frame {
+    AVFrame *frame;
+#ifndef NDEBUG
+    sc_tick push_date;
+#endif
+};
+
+struct sc_delayed_frame_queue SC_VECDEQUE(struct sc_delayed_frame);
+
+struct sc_delay_buffer {
+    struct sc_frame_source frame_source; // frame source trait
+    struct sc_frame_sink frame_sink; // frame sink trait
+
+    sc_tick delay;
+    bool first_frame_asap;
+
+    sc_thread thread;
+    sc_mutex mutex;
+    sc_cond queue_cond;
+    sc_cond wait_cond;
+
+    struct sc_clock clock;
+    struct sc_delayed_frame_queue queue;
+    bool stopped;
+};
+
+struct sc_delay_buffer_callbacks {
+    bool (*on_new_frame)(struct sc_delay_buffer *db, const AVFrame *frame,
+                         void *userdata);
+};
+
+/**
+ * Initialize a delay buffer.
+ *
+ * \param delay a (strictly) positive delay
+ * \param first_frame_asap if true, do not delay the first frame (useful for
+                           a video stream).
+ */
+void
+sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
+                     bool first_frame_asap);
+
+#endif

app/src/demuxer.c

@@ -1,11 +1,13 @@
 #include "demuxer.h"
 
 #include <assert.h>
+#include <libavutil/channel_layout.h>
 #include <libavutil/time.h>
 #include <unistd.h>
 
 #include "decoder.h"
 #include "events.h"
+#include "packet_merger.h"
 #include "recorder.h"
 #include "util/binary.h"
 #include "util/log.h"
@@ -17,11 +19,71 @@
 
 #define SC_PACKET_PTS_MASK (SC_PACKET_FLAG_KEY_FRAME - 1)
 
+static enum AVCodecID
+sc_demuxer_to_avcodec_id(uint32_t codec_id) {
+#define SC_CODEC_ID_H264 UINT32_C(0x68323634) // "h264" in ASCII
+#define SC_CODEC_ID_H265 UINT32_C(0x68323635) // "h265" in ASCII
+#define SC_CODEC_ID_AV1 UINT32_C(0x00617631) // "av1" in ASCII
+#define SC_CODEC_ID_OPUS UINT32_C(0x6f707573) // "opus" in ASCII
+#define SC_CODEC_ID_AAC UINT32_C(0x00616163) // "aac" in ASCII
+#define SC_CODEC_ID_FLAC UINT32_C(0x666c6163) // "flac" in ASCII
+#define SC_CODEC_ID_RAW UINT32_C(0x00726177) // "raw" in ASCII
+    switch (codec_id) {
+        case SC_CODEC_ID_H264:
+            return AV_CODEC_ID_H264;
+        case SC_CODEC_ID_H265:
+            return AV_CODEC_ID_HEVC;
+        case SC_CODEC_ID_AV1:
+#ifdef SCRCPY_LAVC_HAS_AV1
+            return AV_CODEC_ID_AV1;
+#else
+            LOGE("AV1 not supported by this FFmpeg version");
+            return AV_CODEC_ID_NONE;
+#endif
+        case SC_CODEC_ID_OPUS:
+            return AV_CODEC_ID_OPUS;
+        case SC_CODEC_ID_AAC:
+            return AV_CODEC_ID_AAC;
+        case SC_CODEC_ID_FLAC:
+            return AV_CODEC_ID_FLAC;
+        case SC_CODEC_ID_RAW:
+            return AV_CODEC_ID_PCM_S16LE;
+        default:
+            LOGE("Unknown codec id 0x%08" PRIx32, codec_id);
+            return AV_CODEC_ID_NONE;
+    }
+}
+
+static bool
+sc_demuxer_recv_codec_id(struct sc_demuxer *demuxer, uint32_t *codec_id) {
+    uint8_t data[4];
+    ssize_t r = net_recv_all(demuxer->socket, data, 4);
+    if (r < 4) {
+        return false;
+    }
+
+    *codec_id = sc_read32be(data);
+    return true;
+}
+
+static bool
+sc_demuxer_recv_video_size(struct sc_demuxer *demuxer, uint32_t *width,
+                           uint32_t *height) {
+    uint8_t data[8];
+    ssize_t r = net_recv_all(demuxer->socket, data, 8);
+    if (r < 8) {
+        return false;
+    }
+
+    *width = sc_read32be(data);
+    *height = sc_read32be(data + 4);
+    return true;
+}
+
 static bool
 sc_demuxer_recv_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
-    // The video stream contains raw packets, without time information. When we
-    // record, we retrieve the timestamps separately, from a "meta" header
-    // added by the server before each raw packet.
+    // The video and audio streams contain a sequence of raw packets (as
+    // provided by MediaCodec), each prefixed with a "meta" header.
     //
     // The "meta" header length is 12 bytes:
     // [. . . . . . . .|. . . .]. . . . . . . . . . . . . . . ...
@@ -75,200 +137,178 @@ sc_demuxer_recv_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
     return true;
 }
 
-static bool
-push_packet_to_sinks(struct sc_demuxer *demuxer, const AVPacket *packet) {
-    for (unsigned i = 0; i < demuxer->sink_count; ++i) {
-        struct sc_packet_sink *sink = demuxer->sinks[i];
-        if (!sink->ops->push(sink, packet)) {
-            LOGE("Could not send config packet to sink %d", i);
-            return false;
-        }
-    }
-
-    return true;
-}
-
-static bool
-sc_demuxer_push_packet(struct sc_demuxer *demuxer, AVPacket *packet) {
-    bool is_config = packet->pts == AV_NOPTS_VALUE;
-
-    // A config packet must not be decoded immediately (it contains no
-    // frame); instead, it must be concatenated with the future data packet.
-    if (demuxer->pending || is_config) {
-        size_t offset;
-        if (demuxer->pending) {
-            offset = demuxer->pending->size;
-            if (av_grow_packet(demuxer->pending, packet->size)) {
-                LOG_OOM();
-                return false;
-            }
-        } else {
-            offset = 0;
-            demuxer->pending = av_packet_alloc();
-            if (!demuxer->pending) {
-                LOG_OOM();
-                return false;
-            }
-            if (av_new_packet(demuxer->pending, packet->size)) {
-                LOG_OOM();
-                av_packet_free(&demuxer->pending);
-                return false;
-            }
-        }
-
-        memcpy(demuxer->pending->data + offset, packet->data, packet->size);
-
-        if (!is_config) {
-            // prepare the concat packet to send to the decoder
-            demuxer->pending->pts = packet->pts;
-            demuxer->pending->dts = packet->dts;
-            demuxer->pending->flags = packet->flags;
-            packet = demuxer->pending;
-        }
-    }
-
-    bool ok = push_packet_to_sinks(demuxer, packet);
+static int
+run_demuxer(void *data) {
+    struct sc_demuxer *demuxer = data;
 
-    if (!is_config && demuxer->pending) {
-        // the pending packet must be discarded (consumed or error)
-        av_packet_free(&demuxer->pending);
-    }
+    // Flag to report end-of-stream (i.e. device disconnected)
+    enum sc_demuxer_status status = SC_DEMUXER_STATUS_ERROR;
 
+    uint32_t raw_codec_id;
+    bool ok = sc_demuxer_recv_codec_id(demuxer, &raw_codec_id);
     if (!ok) {
-        LOGE("Could not process packet");
-        return false;
+        LOGE("Demuxer '%s': stream disabled due to connection error",
+             demuxer->name);
+        goto end;
     }
 
-    return true;
-}
-
-static void
-sc_demuxer_close_first_sinks(struct sc_demuxer *demuxer, unsigned count) {
-    while (count) {
-        struct sc_packet_sink *sink = demuxer->sinks[--count];
-        sink->ops->close(sink);
+    if (raw_codec_id == 0) {
+        LOGW("Demuxer '%s': stream explicitly disabled by the device",
+             demuxer->name);
+        sc_packet_source_sinks_disable(&demuxer->packet_source);
+        status = SC_DEMUXER_STATUS_DISABLED;
+        goto end;
     }
-}
-
-static inline void
-sc_demuxer_close_sinks(struct sc_demuxer *demuxer) {
-    sc_demuxer_close_first_sinks(demuxer, demuxer->sink_count);
-}
 
-static bool
-sc_demuxer_open_sinks(struct sc_demuxer *demuxer, const AVCodec *codec) {
-    for (unsigned i = 0; i < demuxer->sink_count; ++i) {
-        struct sc_packet_sink *sink = demuxer->sinks[i];
-        if (!sink->ops->open(sink, codec)) {
-            LOGE("Could not open packet sink %d", i);
-            sc_demuxer_close_first_sinks(demuxer, i);
-            return false;
-        }
+    if (raw_codec_id == 1) {
+        LOGE("Demuxer '%s': stream configuration error on the device",
+             demuxer->name);
+        goto end;
     }
 
-    return true;
-}
-
-static int
-run_demuxer(void *data) {
-    struct sc_demuxer *demuxer = data;
+    enum AVCodecID codec_id = sc_demuxer_to_avcodec_id(raw_codec_id);
+    if (codec_id == AV_CODEC_ID_NONE) {
+        LOGE("Demuxer '%s': stream disabled due to unsupported codec",
+             demuxer->name);
+        sc_packet_source_sinks_disable(&demuxer->packet_source);
+        goto end;
+    }
 
-    const AVCodec *codec = avcodec_find_decoder(AV_CODEC_ID_H264);
+    const AVCodec *codec = avcodec_find_decoder(codec_id);
     if (!codec) {
-        LOGE("H.264 decoder not found");
+        LOGE("Demuxer '%s': stream disabled due to missing decoder",
+             demuxer->name);
+        sc_packet_source_sinks_disable(&demuxer->packet_source);
         goto end;
     }
 
-    demuxer->codec_ctx = avcodec_alloc_context3(codec);
-    if (!demuxer->codec_ctx) {
+    AVCodecContext *codec_ctx = avcodec_alloc_context3(codec);
+    if (!codec_ctx) {
         LOG_OOM();
         goto end;
     }
 
-    if (!sc_demuxer_open_sinks(demuxer, codec)) {
-        LOGE("Could not open demuxer sinks");
-        goto finally_free_codec_ctx;
+    codec_ctx->flags |= AV_CODEC_FLAG_LOW_DELAY;
+
+    if (codec->type == AVMEDIA_TYPE_VIDEO) {
+        uint32_t width;
+        uint32_t height;
+        ok = sc_demuxer_recv_video_size(demuxer, &width, &height);
+        if (!ok) {
+            goto finally_free_context;
+        }
+
+        codec_ctx->width = width;
+        codec_ctx->height = height;
+        codec_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
+    } else {
+        // Hardcoded audio properties
+#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
+        codec_ctx->ch_layout = (AVChannelLayout) AV_CHANNEL_LAYOUT_STEREO;
+#else
+        codec_ctx->channel_layout = AV_CH_LAYOUT_STEREO;
+        codec_ctx->channels = 2;
+#endif
+        codec_ctx->sample_rate = 48000;
+
+        if (raw_codec_id == SC_CODEC_ID_FLAC) {
+            // The sample_fmt is not set by the FLAC decoder
+            codec_ctx->sample_fmt = AV_SAMPLE_FMT_S16;
+        }
     }
 
-    demuxer->parser = av_parser_init(AV_CODEC_ID_H264);
-    if (!demuxer->parser) {
-        LOGE("Could not initialize parser");
-        goto finally_close_sinks;
+    if (avcodec_open2(codec_ctx, codec, NULL) < 0) {
+        LOGE("Demuxer '%s': could not open codec", demuxer->name);
+        goto finally_free_context;
+    }
+
+    if (!sc_packet_source_sinks_open(&demuxer->packet_source, codec_ctx)) {
+        goto finally_free_context;
     }
 
-    // We must only pass complete frames to av_parser_parse2()!
-    // It's more complicated, but this allows to reduce the latency by 1 frame!
-    demuxer->parser->flags |= PARSER_FLAG_COMPLETE_FRAMES;
+    // Config packets must be merged with the next non-config packet only for
+    // H.26x
+    bool must_merge_config_packet = raw_codec_id == SC_CODEC_ID_H264
+                                 || raw_codec_id == SC_CODEC_ID_H265;
+
+    struct sc_packet_merger merger;
+
+    if (must_merge_config_packet) {
+        sc_packet_merger_init(&merger);
+    }
 
     AVPacket *packet = av_packet_alloc();
     if (!packet) {
         LOG_OOM();
-        goto finally_close_parser;
+        goto finally_close_sinks;
     }
 
     for (;;) {
         bool ok = sc_demuxer_recv_packet(demuxer, packet);
         if (!ok) {
             // end of stream
+            status = SC_DEMUXER_STATUS_EOS;
             break;
         }
 
-        ok = sc_demuxer_push_packet(demuxer, packet);
+        if (must_merge_config_packet) {
+            // Prepend any config packet to the next media packet
+            ok = sc_packet_merger_merge(&merger, packet);
+            if (!ok) {
+                av_packet_unref(packet);
+                break;
+            }
+        }
+
+        ok = sc_packet_source_sinks_push(&demuxer->packet_source, packet);
         av_packet_unref(packet);
         if (!ok) {
-            // cannot process packet (error already logged)
+            // The sink already logged its concrete error
             break;
         }
     }
 
-    LOGD("End of frames");
+    LOGD("Demuxer '%s': end of frames", demuxer->name);
 
-    if (demuxer->pending) {
-        av_packet_free(&demuxer->pending);
+    if (must_merge_config_packet) {
+        sc_packet_merger_destroy(&merger);
     }
 
     av_packet_free(&packet);
-finally_close_parser:
-    av_parser_close(demuxer->parser);
 finally_close_sinks:
-    sc_demuxer_close_sinks(demuxer);
-finally_free_codec_ctx:
-    avcodec_free_context(&demuxer->codec_ctx);
+    sc_packet_source_sinks_close(&demuxer->packet_source);
+finally_free_context:
+    // This also calls avcodec_close() internally
+    avcodec_free_context(&codec_ctx);
 end:
-    demuxer->cbs->on_eos(demuxer, demuxer->cbs_userdata);
+    demuxer->cbs->on_ended(demuxer, status, demuxer->cbs_userdata);
 
     return 0;
 }
 
 void
-sc_demuxer_init(struct sc_demuxer *demuxer, sc_socket socket,
+sc_demuxer_init(struct sc_demuxer *demuxer, const char *name, sc_socket socket,
                 const struct sc_demuxer_callbacks *cbs, void *cbs_userdata) {
+    assert(socket != SC_SOCKET_NONE);
+
+    demuxer->name = name; // statically allocated
     demuxer->socket = socket;
-    demuxer->pending = NULL;
-    demuxer->sink_count = 0;
+    sc_packet_source_init(&demuxer->packet_source);
 
-    assert(cbs && cbs->on_eos);
+    assert(cbs && cbs->on_ended);
 
     demuxer->cbs = cbs;
     demuxer->cbs_userdata = cbs_userdata;
 }
 
-void
-sc_demuxer_add_sink(struct sc_demuxer *demuxer, struct sc_packet_sink *sink) {
-    assert(demuxer->sink_count < SC_DEMUXER_MAX_SINKS);
-    assert(sink);
-    assert(sink->ops);
-    demuxer->sinks[demuxer->sink_count++] = sink;
-}
-
 bool
 sc_demuxer_start(struct sc_demuxer *demuxer) {
-    LOGD("Starting demuxer thread");
+    LOGD("Demuxer '%s': starting thread", demuxer->name);
 
     bool ok = sc_thread_create(&demuxer->thread, run_demuxer, "scrcpy-demuxer",
                                demuxer);
     if (!ok) {
-        LOGE("Could not start demuxer thread");
+        LOGE("Demuxer '%s': could not start thread", demuxer->name);
         return false;
     }
     return true;

app/src/demuxer.h

@@ -8,40 +8,39 @@
 #include <libavcodec/avcodec.h>
 #include <libavformat/avformat.h>
 
+#include "trait/packet_source.h"
 #include "trait/packet_sink.h"
 #include "util/net.h"
 #include "util/thread.h"
 
-#define SC_DEMUXER_MAX_SINKS 2
-
 struct sc_demuxer {
-    sc_socket socket;
-    sc_thread thread;
+    struct sc_packet_source packet_source; // packet source trait
 
-    struct sc_packet_sink *sinks[SC_DEMUXER_MAX_SINKS];
-    unsigned sink_count;
+    const char *name; // must be statically allocated (e.g. a string literal)
 
-    AVCodecContext *codec_ctx;
-    AVCodecParserContext *parser;
-    // successive packets may need to be concatenated, until a non-config
-    // packet is available
-    AVPacket *pending;
+    sc_socket socket;
+    sc_thread thread;
 
     const struct sc_demuxer_callbacks *cbs;
     void *cbs_userdata;
 };
 
+enum sc_demuxer_status {
+    SC_DEMUXER_STATUS_EOS,
+    SC_DEMUXER_STATUS_DISABLED,
+    SC_DEMUXER_STATUS_ERROR,
+};
+
 struct sc_demuxer_callbacks {
-    void (*on_eos)(struct sc_demuxer *demuxer, void *userdata);
+    void (*on_ended)(struct sc_demuxer *demuxer, enum sc_demuxer_status,
+                     void *userdata);
 };
 
+// The name must be statically allocated (e.g. a string literal)
 void
-sc_demuxer_init(struct sc_demuxer *demuxer, sc_socket socket,
+sc_demuxer_init(struct sc_demuxer *demuxer, const char *name, sc_socket socket,
                 const struct sc_demuxer_callbacks *cbs, void *cbs_userdata);
 
-void
-sc_demuxer_add_sink(struct sc_demuxer *demuxer, struct sc_packet_sink *sink);
-
 bool
 sc_demuxer_start(struct sc_demuxer *demuxer);
 

app/src/device_msg.c

@@ -8,19 +8,22 @@
 #include "util/log.h"
 
 ssize_t
-device_msg_deserialize(const unsigned char *buf, size_t len,
-                       struct device_msg *msg) {
-    if (len < 5) {
-        // at least type + empty string length
-        return 0; // not available
+sc_device_msg_deserialize(const uint8_t *buf, size_t len,
+                          struct sc_device_msg *msg) {
+    if (!len) {
+        return 0; // no message
     }
 
     msg->type = buf[0];
     switch (msg->type) {
         case DEVICE_MSG_TYPE_CLIPBOARD: {
+            if (len < 5) {
+                // at least type + empty string length
+                return 0; // no complete message
+            }
             size_t clipboard_len = sc_read32be(&buf[1]);
             if (clipboard_len > len - 5) {
-                return 0; // not available
+                return 0; // no complete message
             }
             char *text = malloc(clipboard_len + 1);
             if (!text) {
@@ -36,10 +39,38 @@ device_msg_deserialize(const unsigned char *buf, size_t len,
             return 5 + clipboard_len;
         }
         case DEVICE_MSG_TYPE_ACK_CLIPBOARD: {
+            if (len < 9) {
+                return 0; // no complete message
+            }
             uint64_t sequence = sc_read64be(&buf[1]);
             msg->ack_clipboard.sequence = sequence;
             return 9;
         }
+        case DEVICE_MSG_TYPE_UHID_OUTPUT: {
+            if (len < 5) {
+                // at least id + size
+                return 0; // not available
+            }
+            uint16_t id = sc_read16be(&buf[1]);
+            size_t size = sc_read16be(&buf[3]);
+            if (size < len - 5) {
+                return 0; // not available
+            }
+            uint8_t *data = malloc(size);
+            if (!data) {
+                LOG_OOM();
+                return -1;
+            }
+            if (size) {
+                memcpy(data, &buf[5], size);
+            }
+
+            msg->uhid_output.id = id;
+            msg->uhid_output.size = size;
+            msg->uhid_output.data = data;
+
+            return 5 + size;
+        }
         default:
             LOGW("Unknown device message type: %d", (int) msg->type);
             return -1; // error, we cannot recover
@@ -47,8 +78,16 @@ device_msg_deserialize(const unsigned char *buf, size_t len,
 }
 
 void
-device_msg_destroy(struct device_msg *msg) {
-    if (msg->type == DEVICE_MSG_TYPE_CLIPBOARD) {
-        free(msg->clipboard.text);
+sc_device_msg_destroy(struct sc_device_msg *msg) {
+    switch (msg->type) {
+        case DEVICE_MSG_TYPE_CLIPBOARD:
+            free(msg->clipboard.text);
+            break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            free(msg->uhid_output.data);
+            break;
+        default:
+            // nothing to do
+            break;
     }
 }

app/src/device_msg.h

@@ -11,13 +11,14 @@
 // type: 1 byte; length: 4 bytes
 #define DEVICE_MSG_TEXT_MAX_LENGTH (DEVICE_MSG_MAX_SIZE - 5)
 
-enum device_msg_type {
+enum sc_device_msg_type {
     DEVICE_MSG_TYPE_CLIPBOARD,
     DEVICE_MSG_TYPE_ACK_CLIPBOARD,
+    DEVICE_MSG_TYPE_UHID_OUTPUT,
 };
 
-struct device_msg {
-    enum device_msg_type type;
+struct sc_device_msg {
+    enum sc_device_msg_type type;
     union {
         struct {
             char *text; // owned, to be freed by free()
@@ -25,15 +26,20 @@ struct device_msg {
         struct {
             uint64_t sequence;
         } ack_clipboard;
+        struct {
+            uint16_t id;
+            uint16_t size;
+            uint8_t *data; // owned, to be freed by free()
+        } uhid_output;
     };
 };
 
 // return the number of bytes consumed (0 for no msg available, -1 on error)
 ssize_t
-device_msg_deserialize(const unsigned char *buf, size_t len,
-                       struct device_msg *msg);
+sc_device_msg_deserialize(const uint8_t *buf, size_t len,
+                          struct sc_device_msg *msg);
 
 void
-device_msg_destroy(struct device_msg *msg);
+sc_device_msg_destroy(struct sc_device_msg *msg);
 
 #endif

app/src/display.c

@@ -0,0 +1,286 @@
+#include "display.h"
+
+#include <assert.h>
+
+#include "util/log.h"
+
+bool
+sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
+    display->renderer =
+        SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
+    if (!display->renderer) {
+        LOGE("Could not create renderer: %s", SDL_GetError());
+        return false;
+    }
+
+    SDL_RendererInfo renderer_info;
+    int r = SDL_GetRendererInfo(display->renderer, &renderer_info);
+    const char *renderer_name = r ? NULL : renderer_info.name;
+    LOGI("Renderer: %s", renderer_name ? renderer_name : "(unknown)");
+
+    display->mipmaps = false;
+
+    // starts with "opengl"
+    bool use_opengl = renderer_name && !strncmp(renderer_name, "opengl", 6);
+    if (use_opengl) {
+
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+        // Persuade macOS to give us something better than OpenGL 2.1.
+        // If we create a Core Profile context, we get the best OpenGL version.
+        SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK,
+                            SDL_GL_CONTEXT_PROFILE_CORE);
+
+        LOGD("Creating OpenGL Core Profile context");
+        display->gl_context = SDL_GL_CreateContext(window);
+        if (!display->gl_context) {
+            LOGE("Could not create OpenGL context: %s", SDL_GetError());
+            SDL_DestroyRenderer(display->renderer);
+            return false;
+        }
+#endif
+
+        struct sc_opengl *gl = &display->gl;
+        sc_opengl_init(gl);
+
+        LOGI("OpenGL version: %s", gl->version);
+
+        if (mipmaps) {
+            bool supports_mipmaps =
+                sc_opengl_version_at_least(gl, 3, 0, /* OpenGL 3.0+ */
+                                               2, 0  /* OpenGL ES 2.0+ */);
+            if (supports_mipmaps) {
+                LOGI("Trilinear filtering enabled");
+                display->mipmaps = true;
+            } else {
+                LOGW("Trilinear filtering disabled "
+                     "(OpenGL 3.0+ or ES 2.0+ required)");
+            }
+        } else {
+            LOGI("Trilinear filtering disabled");
+        }
+    } else if (mipmaps) {
+        LOGD("Trilinear filtering disabled (not an OpenGL renderer)");
+    }
+
+    display->texture = NULL;
+    display->pending.flags = 0;
+    display->pending.frame = NULL;
+
+    return true;
+}
+
+void
+sc_display_destroy(struct sc_display *display) {
+    if (display->pending.frame) {
+        av_frame_free(&display->pending.frame);
+    }
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+    SDL_GL_DeleteContext(display->gl_context);
+#endif
+    if (display->texture) {
+        SDL_DestroyTexture(display->texture);
+    }
+    SDL_DestroyRenderer(display->renderer);
+}
+
+static SDL_Texture *
+sc_display_create_texture(struct sc_display *display,
+                          struct sc_size size) {
+    SDL_Renderer *renderer = display->renderer;
+    SDL_Texture *texture = SDL_CreateTexture(renderer, SDL_PIXELFORMAT_YV12,
+                                             SDL_TEXTUREACCESS_STREAMING,
+                                             size.width, size.height);
+    if (!texture) {
+        LOGD("Could not create texture: %s", SDL_GetError());
+        return NULL;
+    }
+
+    if (display->mipmaps) {
+        struct sc_opengl *gl = &display->gl;
+
+        SDL_GL_BindTexture(texture, NULL, NULL);
+
+        // Enable trilinear filtering for downscaling
+        gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
+                          GL_LINEAR_MIPMAP_LINEAR);
+        gl->TexParameterf(GL_TEXTURE_2D, GL_TEXTURE_LOD_BIAS, -1.f);
+
+        SDL_GL_UnbindTexture(texture);
+    }
+
+    return texture;
+}
+
+static inline void
+sc_display_set_pending_size(struct sc_display *display, struct sc_size size) {
+    assert(!display->texture);
+    display->pending.size = size;
+    display->pending.flags |= SC_DISPLAY_PENDING_FLAG_SIZE;
+}
+
+static bool
+sc_display_set_pending_frame(struct sc_display *display, const AVFrame *frame) {
+    if (!display->pending.frame) {
+        display->pending.frame = av_frame_alloc();
+        if (!display->pending.frame) {
+            LOG_OOM();
+            return false;
+        }
+    }
+
+    int r = av_frame_ref(display->pending.frame, frame);
+    if (r) {
+        LOGE("Could not ref frame: %d", r);
+        return false;
+    }
+
+    display->pending.flags |= SC_DISPLAY_PENDING_FLAG_FRAME;
+
+    return true;
+}
+
+static bool
+sc_display_apply_pending(struct sc_display *display) {
+    if (display->pending.flags & SC_DISPLAY_PENDING_FLAG_SIZE) {
+        assert(!display->texture);
+        display->texture =
+            sc_display_create_texture(display, display->pending.size);
+        if (!display->texture) {
+            return false;
+        }
+
+        display->pending.flags &= ~SC_DISPLAY_PENDING_FLAG_SIZE;
+    }
+
+    if (display->pending.flags & SC_DISPLAY_PENDING_FLAG_FRAME) {
+        assert(display->pending.frame);
+        bool ok = sc_display_update_texture(display, display->pending.frame);
+        if (!ok) {
+            return false;
+        }
+
+        av_frame_unref(display->pending.frame);
+        display->pending.flags &= ~SC_DISPLAY_PENDING_FLAG_FRAME;
+    }
+
+    return true;
+}
+
+static bool
+sc_display_set_texture_size_internal(struct sc_display *display,
+                                     struct sc_size size) {
+    assert(size.width && size.height);
+
+    if (display->texture) {
+        SDL_DestroyTexture(display->texture);
+    }
+
+    display->texture = sc_display_create_texture(display, size);
+    if (!display->texture) {
+        return false;
+    }
+
+    LOGI("Texture: %" PRIu16 "x%" PRIu16, size.width, size.height);
+    return true;
+}
+
+enum sc_display_result
+sc_display_set_texture_size(struct sc_display *display, struct sc_size size) {
+    bool ok = sc_display_set_texture_size_internal(display, size);
+    if (!ok) {
+        sc_display_set_pending_size(display, size);
+        return SC_DISPLAY_RESULT_PENDING;
+
+    }
+
+    return SC_DISPLAY_RESULT_OK;
+}
+
+static bool
+sc_display_update_texture_internal(struct sc_display *display,
+                                   const AVFrame *frame) {
+    int ret = SDL_UpdateYUVTexture(display->texture, NULL,
+                                   frame->data[0], frame->linesize[0],
+                                   frame->data[1], frame->linesize[1],
+                                   frame->data[2], frame->linesize[2]);
+    if (ret) {
+        LOGD("Could not update texture: %s", SDL_GetError());
+        return false;
+    }
+
+    if (display->mipmaps) {
+        SDL_GL_BindTexture(display->texture, NULL, NULL);
+        display->gl.GenerateMipmap(GL_TEXTURE_2D);
+        SDL_GL_UnbindTexture(display->texture);
+    }
+
+    return true;
+}
+
+enum sc_display_result
+sc_display_update_texture(struct sc_display *display, const AVFrame *frame) {
+    bool ok = sc_display_update_texture_internal(display, frame);
+    if (!ok) {
+        ok = sc_display_set_pending_frame(display, frame);
+        if (!ok) {
+            LOGE("Could not set pending frame");
+            return SC_DISPLAY_RESULT_ERROR;
+        }
+
+        return SC_DISPLAY_RESULT_PENDING;
+    }
+
+    return SC_DISPLAY_RESULT_OK;
+}
+
+enum sc_display_result
+sc_display_render(struct sc_display *display, const SDL_Rect *geometry,
+                  enum sc_orientation orientation) {
+    SDL_RenderClear(display->renderer);
+
+    if (display->pending.flags) {
+        bool ok = sc_display_apply_pending(display);
+        if (!ok) {
+            return SC_DISPLAY_RESULT_PENDING;
+        }
+    }
+
+    SDL_Renderer *renderer = display->renderer;
+    SDL_Texture *texture = display->texture;
+
+    if (orientation == SC_ORIENTATION_0) {
+        int ret = SDL_RenderCopy(renderer, texture, NULL, geometry);
+        if (ret) {
+            LOGE("Could not render texture: %s", SDL_GetError());
+            return SC_DISPLAY_RESULT_ERROR;
+        }
+    } else {
+        unsigned cw_rotation = sc_orientation_get_rotation(orientation);
+        double angle = 90 * cw_rotation;
+
+        const SDL_Rect *dstrect = NULL;
+        SDL_Rect rect;
+        if (sc_orientation_is_swap(orientation)) {
+            rect.x = geometry->x + (geometry->w - geometry->h) / 2;
+            rect.y = geometry->y + (geometry->h - geometry->w) / 2;
+            rect.w = geometry->h;
+            rect.h = geometry->w;
+            dstrect = &rect;
+        } else {
+            dstrect = geometry;
+        }
+
+        SDL_RendererFlip flip = sc_orientation_is_mirror(orientation)
+                              ? SDL_FLIP_HORIZONTAL : 0;
+
+        int ret = SDL_RenderCopyEx(renderer, texture, NULL, dstrect, angle,
+                                   NULL, flip);
+        if (ret) {
+            LOGE("Could not render texture: %s", SDL_GetError());
+            return SC_DISPLAY_RESULT_ERROR;
+        }
+    }
+
+    SDL_RenderPresent(display->renderer);
+    return SC_DISPLAY_RESULT_OK;
+}

app/src/display.h

@@ -0,0 +1,60 @@
+#ifndef SC_DISPLAY_H
+#define SC_DISPLAY_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <libavformat/avformat.h>
+#include <SDL2/SDL.h>
+
+#include "coords.h"
+#include "opengl.h"
+#include "options.h"
+
+#ifdef __APPLE__
+# define SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+#endif
+
+struct sc_display {
+    SDL_Renderer *renderer;
+    SDL_Texture *texture;
+
+    struct sc_opengl gl;
+#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
+    SDL_GLContext *gl_context;
+#endif
+
+    bool mipmaps;
+
+    struct {
+#define SC_DISPLAY_PENDING_FLAG_SIZE 1
+#define SC_DISPLAY_PENDING_FLAG_FRAME 2
+        int8_t flags;
+        struct sc_size size;
+        AVFrame *frame;
+    } pending;
+};
+
+enum sc_display_result {
+    SC_DISPLAY_RESULT_OK,
+    SC_DISPLAY_RESULT_PENDING,
+    SC_DISPLAY_RESULT_ERROR,
+};
+
+bool
+sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps);
+
+void
+sc_display_destroy(struct sc_display *display);
+
+enum sc_display_result
+sc_display_set_texture_size(struct sc_display *display, struct sc_size size);
+
+enum sc_display_result
+sc_display_update_texture(struct sc_display *display, const AVFrame *frame);
+
+enum sc_display_result
+sc_display_render(struct sc_display *display, const SDL_Rect *geometry,
+                  enum sc_orientation orientation);
+
+#endif

app/src/events.h

@@ -1,5 +1,9 @@
-#define EVENT_NEW_FRAME                 SDL_USEREVENT
-#define EVENT_STREAM_STOPPED           (SDL_USEREVENT + 1)
-#define EVENT_SERVER_CONNECTION_FAILED (SDL_USEREVENT + 2)
-#define EVENT_SERVER_CONNECTED         (SDL_USEREVENT + 3)
-#define EVENT_USB_DEVICE_DISCONNECTED  (SDL_USEREVENT + 4)
+#define SC_EVENT_NEW_FRAME                 SDL_USEREVENT
+#define SC_EVENT_DEVICE_DISCONNECTED      (SDL_USEREVENT + 1)
+#define SC_EVENT_SERVER_CONNECTION_FAILED (SDL_USEREVENT + 2)
+#define SC_EVENT_SERVER_CONNECTED         (SDL_USEREVENT + 3)
+#define SC_EVENT_USB_DEVICE_DISCONNECTED  (SDL_USEREVENT + 4)
+#define SC_EVENT_DEMUXER_ERROR            (SDL_USEREVENT + 5)
+#define SC_EVENT_RECORDER_ERROR           (SDL_USEREVENT + 6)
+#define SC_EVENT_SCREEN_INIT_SIZE         (SDL_USEREVENT + 7)
+#define SC_EVENT_TIME_LIMIT_REACHED       (SDL_USEREVENT + 8)

app/src/file_pusher.c

@@ -19,7 +19,7 @@ sc_file_pusher_init(struct sc_file_pusher *fp, const char *serial,
                     const char *push_target) {
     assert(serial);
 
-    cbuf_init(&fp->queue);
+    sc_vecdeque_init(&fp->queue);
 
     bool ok = sc_mutex_init(&fp->mutex);
     if (!ok) {
@@ -65,9 +65,10 @@ sc_file_pusher_destroy(struct sc_file_pusher *fp) {
     sc_intr_destroy(&fp->intr);
     free(fp->serial);
 
-    struct sc_file_pusher_request req;
-    while (cbuf_take(&fp->queue, &req)) {
-        sc_file_pusher_request_destroy(&req);
+    while (!sc_vecdeque_is_empty(&fp->queue)) {
+        struct sc_file_pusher_request *req = sc_vecdeque_popref(&fp->queue);
+        assert(req);
+        sc_file_pusher_request_destroy(req);
     }
 }
 
@@ -91,13 +92,20 @@ sc_file_pusher_request(struct sc_file_pusher *fp,
     };
 
     sc_mutex_lock(&fp->mutex);
-    bool was_empty = cbuf_is_empty(&fp->queue);
-    bool res = cbuf_push(&fp->queue, req);
+    bool was_empty = sc_vecdeque_is_empty(&fp->queue);
+    bool res = sc_vecdeque_push(&fp->queue, req);
+    if (!res) {
+        LOG_OOM();
+        sc_mutex_unlock(&fp->mutex);
+        return false;
+    }
+
     if (was_empty) {
         sc_cond_signal(&fp->event_cond);
     }
     sc_mutex_unlock(&fp->mutex);
-    return res;
+
+    return true;
 }
 
 static int
@@ -113,7 +121,7 @@ run_file_pusher(void *data) {
 
     for (;;) {
         sc_mutex_lock(&fp->mutex);
-        while (!fp->stopped && cbuf_is_empty(&fp->queue)) {
+        while (!fp->stopped && sc_vecdeque_is_empty(&fp->queue)) {
             sc_cond_wait(&fp->event_cond, &fp->mutex);
         }
         if (fp->stopped) {
@@ -121,10 +129,9 @@ run_file_pusher(void *data) {
             sc_mutex_unlock(&fp->mutex);
             break;
         }
-        struct sc_file_pusher_request req;
-        bool non_empty = cbuf_take(&fp->queue, &req);
-        assert(non_empty);
-        (void) non_empty;
+
+        assert(!sc_vecdeque_is_empty(&fp->queue));
+        struct sc_file_pusher_request req = sc_vecdeque_pop(&fp->queue);
         sc_mutex_unlock(&fp->mutex);
 
         if (req.action == SC_FILE_PUSHER_ACTION_INSTALL_APK) {
@@ -165,14 +172,18 @@ sc_file_pusher_start(struct sc_file_pusher *fp) {
 
 void
 sc_file_pusher_stop(struct sc_file_pusher *fp) {
-    sc_mutex_lock(&fp->mutex);
-    fp->stopped = true;
-    sc_cond_signal(&fp->event_cond);
-    sc_intr_interrupt(&fp->intr);
-    sc_mutex_unlock(&fp->mutex);
+    if (fp->initialized) {
+        sc_mutex_lock(&fp->mutex);
+        fp->stopped = true;
+        sc_cond_signal(&fp->event_cond);
+        sc_intr_interrupt(&fp->intr);
+        sc_mutex_unlock(&fp->mutex);
+    }
 }
 
 void
 sc_file_pusher_join(struct sc_file_pusher *fp) {
-    sc_thread_join(&fp->thread, NULL);
+    if (fp->initialized) {
+        sc_thread_join(&fp->thread, NULL);
+    }
 }

app/src/file_pusher.h

@@ -5,9 +5,9 @@
 
 #include <stdbool.h>
 
-#include "util/cbuf.h"
-#include "util/thread.h"
 #include "util/intr.h"
+#include "util/thread.h"
+#include "util/vecdeque.h"
 
 enum sc_file_pusher_action {
     SC_FILE_PUSHER_ACTION_INSTALL_APK,
@@ -19,7 +19,7 @@ struct sc_file_pusher_request {
     char *file;
 };
 
-struct sc_file_pusher_request_queue CBUF(struct sc_file_pusher_request, 16);
+struct sc_file_pusher_request_queue SC_VECDEQUE(struct sc_file_pusher_request);
 
 struct sc_file_pusher {
     char *serial;

app/src/fps_counter.c

@@ -96,6 +96,7 @@ run_fps_counter(void *data) {
 bool
 sc_fps_counter_start(struct sc_fps_counter *counter) {
     sc_mutex_lock(&counter->mutex);
+    counter->interrupted = false;
     counter->next_timestamp = sc_tick_now() + SC_FPS_COUNTER_INTERVAL;
     counter->nr_rendered = 0;
     counter->nr_skipped = 0;

app/src/hid/hid_event.h

@@ -0,0 +1,15 @@
+#ifndef SC_HID_EVENT_H
+#define SC_HID_EVENT_H
+
+#include "common.h"
+
+#include <stdint.h>
+
+#define SC_HID_MAX_SIZE 8
+
+struct sc_hid_event {
+    uint8_t data[SC_HID_MAX_SIZE];
+    uint8_t size;
+};
+
+#endif

app/src/hid/hid_keyboard.c

@@ -1,39 +1,34 @@
 #include "hid_keyboard.h"
 
-#include <assert.h>
+#include <string.h>
 
-#include "input_events.h"
 #include "util/log.h"
 
-/** Downcast key processor to hid_keyboard */
-#define DOWNCAST(KP) container_of(KP, struct sc_hid_keyboard, key_processor)
+#define SC_HID_MOD_NONE 0x00
+#define SC_HID_MOD_LEFT_CONTROL (1 << 0)
+#define SC_HID_MOD_LEFT_SHIFT (1 << 1)
+#define SC_HID_MOD_LEFT_ALT (1 << 2)
+#define SC_HID_MOD_LEFT_GUI (1 << 3)
+#define SC_HID_MOD_RIGHT_CONTROL (1 << 4)
+#define SC_HID_MOD_RIGHT_SHIFT (1 << 5)
+#define SC_HID_MOD_RIGHT_ALT (1 << 6)
+#define SC_HID_MOD_RIGHT_GUI (1 << 7)
 
-#define HID_KEYBOARD_ACCESSORY_ID 1
-
-#define HID_MODIFIER_NONE 0x00
-#define HID_MODIFIER_LEFT_CONTROL (1 << 0)
-#define HID_MODIFIER_LEFT_SHIFT (1 << 1)
-#define HID_MODIFIER_LEFT_ALT (1 << 2)
-#define HID_MODIFIER_LEFT_GUI (1 << 3)
-#define HID_MODIFIER_RIGHT_CONTROL (1 << 4)
-#define HID_MODIFIER_RIGHT_SHIFT (1 << 5)
-#define HID_MODIFIER_RIGHT_ALT (1 << 6)
-#define HID_MODIFIER_RIGHT_GUI (1 << 7)
-
-#define HID_KEYBOARD_INDEX_MODIFIER 0
-#define HID_KEYBOARD_INDEX_KEYS 2
+#define SC_HID_KEYBOARD_INDEX_MODS 0
+#define SC_HID_KEYBOARD_INDEX_KEYS 2
 
 // USB HID protocol says 6 keys in an event is the requirement for BIOS
 // keyboard support, though OS could support more keys via modifying the report
 // desc. 6 should be enough for scrcpy.
-#define HID_KEYBOARD_MAX_KEYS 6
-#define HID_KEYBOARD_EVENT_SIZE (2 + HID_KEYBOARD_MAX_KEYS)
+#define SC_HID_KEYBOARD_MAX_KEYS 6
+#define SC_HID_KEYBOARD_EVENT_SIZE \
+    (SC_HID_KEYBOARD_INDEX_KEYS + SC_HID_KEYBOARD_MAX_KEYS)
 
-#define HID_RESERVED 0x00
-#define HID_ERROR_ROLL_OVER 0x01
+#define SC_HID_RESERVED 0x00
+#define SC_HID_ERROR_ROLL_OVER 0x01
 
 /**
- * For HID over AOAv2, only report descriptor is needed.
+ * For HID, only report descriptor is needed.
  *
  * The specification is available here:
  * <https://www.usb.org/sites/default/files/hid1_11.pdf>
@@ -52,7 +47,7 @@
  *
  * (change vid:pid' to your device's vendor ID and product ID).
  */
-static const unsigned char keyboard_report_desc[]  = {
+const uint8_t SC_HID_KEYBOARD_REPORT_DESC[] = {
     // Usage Page (Generic Desktop)
     0x05, 0x01,
     // Usage (Keyboard)
@@ -118,7 +113,7 @@ static const unsigned char keyboard_report_desc[]  = {
     // Report Size (8)
     0x75, 0x08,
     // Report Count (6)
-    0x95, HID_KEYBOARD_MAX_KEYS,
+    0x95, SC_HID_KEYBOARD_MAX_KEYS,
     // Input (Data, Array): Keys
     0x81, 0x00,
 
@@ -126,6 +121,9 @@ static const unsigned char keyboard_report_desc[]  = {
     0xC0
 };
 
+const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN =
+    sizeof(SC_HID_KEYBOARD_REPORT_DESC);
+
 /**
  * A keyboard HID event is 8 bytes long:
  *
@@ -200,51 +198,50 @@ static const unsigned char keyboard_report_desc[]  = {
  *                  +---------------+
  */
 
-static unsigned char
-sdl_keymod_to_hid_modifiers(uint16_t mod) {
-    unsigned char modifiers = HID_MODIFIER_NONE;
+static void
+sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = SC_HID_KEYBOARD_EVENT_SIZE;
+
+    uint8_t *data = hid_event->data;
+
+    data[SC_HID_KEYBOARD_INDEX_MODS] = SC_HID_MOD_NONE;
+    data[1] = SC_HID_RESERVED;
+    memset(&data[SC_HID_KEYBOARD_INDEX_KEYS], 0, SC_HID_KEYBOARD_MAX_KEYS);
+}
+
+static uint16_t
+sc_hid_mod_from_sdl_keymod(uint16_t mod) {
+    uint16_t mods = SC_HID_MOD_NONE;
     if (mod & SC_MOD_LCTRL) {
-        modifiers |= HID_MODIFIER_LEFT_CONTROL;
+        mods |= SC_HID_MOD_LEFT_CONTROL;
     }
     if (mod & SC_MOD_LSHIFT) {
-        modifiers |= HID_MODIFIER_LEFT_SHIFT;
+        mods |= SC_HID_MOD_LEFT_SHIFT;
     }
     if (mod & SC_MOD_LALT) {
-        modifiers |= HID_MODIFIER_LEFT_ALT;
+        mods |= SC_HID_MOD_LEFT_ALT;
     }
     if (mod & SC_MOD_LGUI) {
-        modifiers |= HID_MODIFIER_LEFT_GUI;
+        mods |= SC_HID_MOD_LEFT_GUI;
     }
     if (mod & SC_MOD_RCTRL) {
-        modifiers |= HID_MODIFIER_RIGHT_CONTROL;
+        mods |= SC_HID_MOD_RIGHT_CONTROL;
     }
     if (mod & SC_MOD_RSHIFT) {
-        modifiers |= HID_MODIFIER_RIGHT_SHIFT;
+        mods |= SC_HID_MOD_RIGHT_SHIFT;
     }
     if (mod & SC_MOD_RALT) {
-        modifiers |= HID_MODIFIER_RIGHT_ALT;
+        mods |= SC_HID_MOD_RIGHT_ALT;
     }
     if (mod & SC_MOD_RGUI) {
-        modifiers |= HID_MODIFIER_RIGHT_GUI;
+        mods |= SC_HID_MOD_RIGHT_GUI;
     }
-    return modifiers;
+    return mods;
 }
 
-static bool
-sc_hid_keyboard_event_init(struct sc_hid_event *hid_event) {
-    unsigned char *buffer = malloc(HID_KEYBOARD_EVENT_SIZE);
-    if (!buffer) {
-        LOG_OOM();
-        return false;
-    }
-
-    buffer[HID_KEYBOARD_INDEX_MODIFIER] = HID_MODIFIER_NONE;
-    buffer[1] = HID_RESERVED;
-    memset(&buffer[HID_KEYBOARD_INDEX_KEYS], 0, HID_KEYBOARD_MAX_KEYS);
-
-    sc_hid_event_init(hid_event, HID_KEYBOARD_ACCESSORY_ID, buffer,
-                      HID_KEYBOARD_EVENT_SIZE);
-    return true;
+void
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid) {
+    memset(hid->keys, false, SC_HID_KEYBOARD_KEYS);
 }
 
 static inline bool
@@ -252,10 +249,10 @@ scancode_is_modifier(enum sc_scancode scancode) {
     return scancode >= SC_SCANCODE_LCTRL && scancode <= SC_SCANCODE_RGUI;
 }
 
-static bool
-convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
-                           struct sc_hid_event *hid_event,
-                           const struct sc_key_event *event) {
+bool
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event) {
     enum sc_scancode scancode = event->scancode;
     assert(scancode >= 0);
 
@@ -267,39 +264,37 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
         return false;
     }
 
-    if (!sc_hid_keyboard_event_init(hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
-        return false;
-    }
+    sc_hid_keyboard_event_init(hid_event);
 
-    unsigned char modifiers = sdl_keymod_to_hid_modifiers(event->mods_state);
+    uint16_t mods = sc_hid_mod_from_sdl_keymod(event->mods_state);
 
     if (scancode < SC_HID_KEYBOARD_KEYS) {
         // Pressed is true and released is false
-        kb->keys[scancode] = (event->action == SC_ACTION_DOWN);
+        hid->keys[scancode] = (event->action == SC_ACTION_DOWN);
         LOGV("keys[%02x] = %s", scancode,
-             kb->keys[scancode] ? "true" : "false");
+             hid->keys[scancode] ? "true" : "false");
     }
 
-    hid_event->buffer[HID_KEYBOARD_INDEX_MODIFIER] = modifiers;
+    hid_event->data[SC_HID_KEYBOARD_INDEX_MODS] = mods;
 
-    unsigned char *keys_buffer = &hid_event->buffer[HID_KEYBOARD_INDEX_KEYS];
+    uint8_t *keys_data = &hid_event->data[SC_HID_KEYBOARD_INDEX_KEYS];
     // Re-calculate pressed keys every time
     int keys_pressed_count = 0;
     for (int i = 0; i < SC_HID_KEYBOARD_KEYS; ++i) {
-        if (kb->keys[i]) {
+        if (hid->keys[i]) {
             // USB HID protocol says that if keys exceeds report count, a
             // phantom state should be reported
-            if (keys_pressed_count >= HID_KEYBOARD_MAX_KEYS) {
+            if (keys_pressed_count >= SC_HID_KEYBOARD_MAX_KEYS) {
                 // Phantom state:
                 //  - Modifiers
                 //  - Reserved
                 //  - ErrorRollOver * HID_MAX_KEYS
-                memset(keys_buffer, HID_ERROR_ROLL_OVER, HID_KEYBOARD_MAX_KEYS);
+                memset(keys_data, SC_HID_ERROR_ROLL_OVER,
+                       SC_HID_KEYBOARD_MAX_KEYS);
                 goto end;
             }
 
-            keys_buffer[keys_pressed_count] = i;
+            keys_data[keys_pressed_count] = i;
             ++keys_pressed_count;
         }
     }
@@ -307,124 +302,32 @@ convert_hid_keyboard_event(struct sc_hid_keyboard *kb,
 end:
     LOGV("hid keyboard: key %-4s scancode=%02x (%u) mod=%02x",
          event->action == SC_ACTION_DOWN ? "down" : "up", event->scancode,
-         event->scancode, modifiers);
+         event->scancode, mods);
 
     return true;
 }
 
-
-static bool
-push_mod_lock_state(struct sc_hid_keyboard *kb, uint16_t mods_state) {
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state) {
     bool capslock = mods_state & SC_MOD_CAPS;
     bool numlock = mods_state & SC_MOD_NUM;
     if (!capslock && !numlock) {
         // Nothing to do
-        return true;
-    }
-
-    struct sc_hid_event hid_event;
-    if (!sc_hid_keyboard_event_init(&hid_event)) {
-        LOGW("Could not initialize HID keyboard event");
         return false;
     }
 
+    sc_hid_keyboard_event_init(event);
+
     unsigned i = 0;
     if (capslock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_CAPSLOCK;
         ++i;
     }
     if (numlock) {
-        hid_event.buffer[HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
+        event->data[SC_HID_KEYBOARD_INDEX_KEYS + i] = SC_SCANCODE_NUMLOCK;
         ++i;
     }
 
-    if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-        sc_hid_event_destroy(&hid_event);
-        LOGW("Could not request HID event (mod lock state)");
-        return false;
-    }
-
-    LOGD("HID keyboard state synchronized");
-
-    return true;
-}
-
-static void
-sc_key_processor_process_key(struct sc_key_processor *kp,
-                             const struct sc_key_event *event,
-                             uint64_t ack_to_wait) {
-    if (event->repeat) {
-        // In USB HID protocol, key repeat is handled by the host (Android), so
-        // just ignore key repeat here.
-        return;
-    }
-
-    struct sc_hid_keyboard *kb = DOWNCAST(kp);
-
-    struct sc_hid_event hid_event;
-    // Not all keys are supported, just ignore unsupported keys
-    if (convert_hid_keyboard_event(kb, &hid_event, event)) {
-        if (!kb->mod_lock_synchronized) {
-            // Inject CAPSLOCK and/or NUMLOCK if necessary to synchronize
-            // keyboard state
-            if (push_mod_lock_state(kb, event->mods_state)) {
-                kb->mod_lock_synchronized = true;
-            }
-        }
-
-        if (ack_to_wait) {
-            // Ctrl+v is pressed, so clipboard synchronization has been
-            // requested. Wait until clipboard synchronization is acknowledged
-            // by the server, otherwise it could paste the old clipboard
-            // content.
-            hid_event.ack_to_wait = ack_to_wait;
-        }
-
-        if (!sc_aoa_push_hid_event(kb->aoa, &hid_event)) {
-            sc_hid_event_destroy(&hid_event);
-            LOGW("Could not request HID event (key)");
-        }
-    }
-}
-
-bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa) {
-    kb->aoa = aoa;
-
-    bool ok = sc_aoa_setup_hid(aoa, HID_KEYBOARD_ACCESSORY_ID,
-                               keyboard_report_desc,
-                               ARRAY_LEN(keyboard_report_desc));
-    if (!ok) {
-        LOGW("Register HID keyboard failed");
-        return false;
-    }
-
-    // Reset all states
-    memset(kb->keys, false, SC_HID_KEYBOARD_KEYS);
-
-    kb->mod_lock_synchronized = false;
-
-    static const struct sc_key_processor_ops ops = {
-        .process_key = sc_key_processor_process_key,
-        // Never forward text input via HID (all the keys are injected
-        // separately)
-        .process_text = NULL,
-    };
-
-    // Clipboard synchronization is requested over the control socket, while HID
-    // events are sent over AOA, so it must wait for clipboard synchronization
-    // to be acknowledged by the device before injecting Ctrl+v.
-    kb->key_processor.async_paste = true;
-    kb->key_processor.ops = &ops;
-
     return true;
 }
-
-void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb) {
-    // Unregister HID keyboard so the soft keyboard shows again on Android
-    bool ok = sc_aoa_unregister_hid(kb->aoa, HID_KEYBOARD_ACCESSORY_ID);
-    if (!ok) {
-        LOGW("Could not unregister HID keyboard");
-    }
-}

app/src/hid/hid_keyboard.h

@@ -5,8 +5,8 @@
 
 #include <stdbool.h>
 
-#include "aoa_hid.h"
-#include "trait/key_processor.h"
+#include "hid/hid_event.h"
+#include "input_events.h"
 
 // See "SDL2/SDL_scancode.h".
 // Maybe SDL_Keycode is used by most people, but SDL_Scancode is taken from USB
@@ -14,6 +14,9 @@
 // 0x65 is Application, typically AT-101 Keyboard ends here.
 #define SC_HID_KEYBOARD_KEYS 0x66
 
+extern const uint8_t SC_HID_KEYBOARD_REPORT_DESC[];
+extern const size_t SC_HID_KEYBOARD_REPORT_DESC_LEN;
+
 /**
  * HID keyboard events are sequence-based, every time keyboard state changes
  * it sends an array of currently pressed keys, the host is responsible for
@@ -27,18 +30,19 @@
  * phantom state.
  */
 struct sc_hid_keyboard {
-    struct sc_key_processor key_processor; // key processor trait
-
-    struct sc_aoa *aoa;
     bool keys[SC_HID_KEYBOARD_KEYS];
-
-    bool mod_lock_synchronized;
 };
 
+void
+sc_hid_keyboard_init(struct sc_hid_keyboard *hid);
+
 bool
-sc_hid_keyboard_init(struct sc_hid_keyboard *kb, struct sc_aoa *aoa);
+sc_hid_keyboard_event_from_key(struct sc_hid_keyboard *hid,
+                               struct sc_hid_event *hid_event,
+                               const struct sc_key_event *event);
 
-void
-sc_hid_keyboard_destroy(struct sc_hid_keyboard *kb);
+bool
+sc_hid_keyboard_event_from_mods(struct sc_hid_event *event,
+                                uint16_t mods_state);
 
 #endif

app/src/hid/hid_mouse.c

@@ -0,0 +1,192 @@
+#include "hid_mouse.h"
+
+// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
+// 1 byte for wheel motion
+#define HID_MOUSE_EVENT_SIZE 4
+
+/**
+ * Mouse descriptor from the specification:
+ * <https://www.usb.org/sites/default/files/hid1_11.pdf>
+ *
+ * Appendix E (p71): §E.10 Report Descriptor (Mouse)
+ *
+ * The usage tags (like Wheel) are listed in "HID Usage Tables":
+ * <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+ * §4 Generic Desktop Page (0x01) (p26)
+ */
+const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (Mouse)
+    0x09, 0x02,
+
+    // Collection (Application)
+    0xA1, 0x01,
+
+    // Usage (Pointer)
+    0x09, 0x01,
+
+    // Collection (Physical)
+    0xA1, 0x00,
+
+    // Usage Page (Buttons)
+    0x05, 0x09,
+
+    // Usage Minimum (1)
+    0x19, 0x01,
+     // Usage Maximum (5)
+    0x29, 0x05,
+    // Logical Minimum (0)
+    0x15, 0x00,
+    // Logical Maximum (1)
+    0x25, 0x01,
+    // Report Count (5)
+    0x95, 0x05,
+    // Report Size (1)
+    0x75, 0x01,
+    // Input (Data, Variable, Absolute): 5 buttons bits
+    0x81, 0x02,
+
+    // Report Count (1)
+    0x95, 0x01,
+    // Report Size (3)
+    0x75, 0x03,
+    // Input (Constant): 3 bits padding
+    0x81, 0x01,
+
+    // Usage Page (Generic Desktop)
+    0x05, 0x01,
+    // Usage (X)
+    0x09, 0x30,
+    // Usage (Y)
+    0x09, 0x31,
+    // Usage (Wheel)
+    0x09, 0x38,
+    // Local Minimum (-127)
+    0x15, 0x81,
+    // Local Maximum (127)
+    0x25, 0x7F,
+    // Report Size (8)
+    0x75, 0x08,
+    // Report Count (3)
+    0x95, 0x03,
+    // Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
+    0x81, 0x06,
+
+    // End Collection
+    0xC0,
+
+    // End Collection
+    0xC0,
+};
+
+const size_t SC_HID_MOUSE_REPORT_DESC_LEN =
+    sizeof(SC_HID_MOUSE_REPORT_DESC);
+
+/**
+ * A mouse HID event is 4 bytes long:
+ *
+ *  - byte 0: buttons state
+ *  - byte 1: relative x motion (signed byte from -127 to 127)
+ *  - byte 2: relative y motion (signed byte from -127 to 127)
+ *  - byte 3: wheel motion (-1, 0 or 1)
+ *
+ *                   7 6 5 4 3 2 1 0
+ *                  +---------------+
+ *         byte 0:  |0 0 0 . . . . .| buttons state
+ *                  +---------------+
+ *                         ^ ^ ^ ^ ^
+ *                         | | | | `- left button
+ *                         | | | `--- right button
+ *                         | | `----- middle button
+ *                         | `------- button 4
+ *                         `--------- button 5
+ *
+ *                  +---------------+
+ *         byte 1:  |. . . . . . . .| relative x motion
+ *                  +---------------+
+ *         byte 2:  |. . . . . . . .| relative y motion
+ *                  +---------------+
+ *         byte 3:  |. . . . . . . .| wheel motion
+ *                  +---------------+
+ *
+ * As an example, here is the report for a motion of (x=5, y=-4) with left
+ * button pressed:
+ *
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 1| left button pressed
+ *                  +---------------+
+ *                  |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
+ *                  +---------------+
+ *                  |1 1 1 1 1 1 0 0| relative y motion (y = -4)
+ *                  +---------------+
+ *                  |0 0 0 0 0 0 0 0| wheel motion
+ *                  +---------------+
+ */
+
+static void
+sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
+    hid_event->size = HID_MOUSE_EVENT_SIZE;
+    // Leave hid_event->data uninitialized, it will be fully initialized by
+    // callers
+}
+
+static uint8_t
+sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
+    uint8_t c = 0;
+    if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
+        c |= 1 << 0;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
+        c |= 1 << 1;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
+        c |= 1 << 2;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X1) {
+        c |= 1 << 3;
+    }
+    if (buttons_state & SC_MOUSE_BUTTON_X2) {
+        c |= 1 << 4;
+    }
+    return c;
+}
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = CLAMP(event->xrel, -127, 127);
+    data[2] = CLAMP(event->yrel, -127, 127);
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    data[3] = 0; // wheel coordinates only used for scrolling
+}
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event) {
+    sc_hid_mouse_event_init(hid_event);
+
+    uint8_t *data = hid_event->data;
+    data[0] = 0; // buttons state irrelevant (and unknown)
+    data[1] = 0; // no x motion
+    data[2] = 0; // no y motion
+    // In practice, vscroll is always -1, 0 or 1, but in theory other values
+    // are possible
+    data[3] = CLAMP(event->vscroll, -127, 127);
+    // Horizontal scrolling ignored
+}

app/src/hid/hid_mouse.h

@@ -0,0 +1,26 @@
+#ifndef SC_HID_MOUSE_H
+#define SC_HID_MOUSE_H
+
+#endif
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "hid/hid_event.h"
+#include "input_events.h"
+
+extern const uint8_t SC_HID_MOUSE_REPORT_DESC[];
+extern const size_t SC_HID_MOUSE_REPORT_DESC_LEN;
+
+void
+sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_motion_event *event);
+
+void
+sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
+                              const struct sc_mouse_click_event *event);
+
+void
+sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
+                               const struct sc_mouse_scroll_event *event);

app/src/icon.c

@@ -69,7 +69,7 @@ decode_image(const char *path) {
     }
 
     if (avformat_open_input(&ctx, path, NULL, NULL) < 0) {
-        LOGE("Could not open image codec: %s", path);
+        LOGE("Could not open icon image: %s", path);
         goto free_ctx;
     }
 
@@ -271,7 +271,7 @@ error:
 }
 
 SDL_Surface *
-scrcpy_icon_load() {
+scrcpy_icon_load(void) {
     char *icon_path = get_icon_path();
     if (!icon_path) {
         return NULL;

app/src/input_manager.c

@@ -52,8 +52,11 @@ is_shortcut_mod(struct sc_input_manager *im, uint16_t sdl_mod) {
 void
 sc_input_manager_init(struct sc_input_manager *im,
                       const struct sc_input_manager_params *params) {
-    assert(!params->controller || (params->kp && params->kp->ops));
-    assert(!params->controller || (params->mp && params->mp->ops));
+    // A key/mouse processor may not be present if there is no controller
+    assert((!params->kp && !params->mp) || params->controller);
+    // A processor must have ops initialized
+    assert(!params->kp || params->kp->ops);
+    assert(!params->mp || params->mp->ops);
 
     im->controller = params->controller;
     im->fp = params->fp;
@@ -76,6 +79,8 @@ sc_input_manager_init(struct sc_input_manager *im,
     im->sdl_shortcut_mods.count = shortcut_mods->count;
 
     im->vfinger_down = false;
+    im->vfinger_invert_x = false;
+    im->vfinger_invert_y = false;
 
     im->last_keycode = SDLK_UNKNOWN;
     im->last_mod = 0;
@@ -85,8 +90,10 @@ sc_input_manager_init(struct sc_input_manager *im,
 }
 
 static void
-send_keycode(struct sc_controller *controller, enum android_keycode keycode,
+send_keycode(struct sc_input_manager *im, enum android_keycode keycode,
              enum sc_action action, const char *name) {
+    assert(im->controller && im->kp);
+
     // send DOWN event
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_KEYCODE;
@@ -97,100 +104,109 @@ send_keycode(struct sc_controller *controller, enum android_keycode keycode,
     msg.inject_keycode.metastate = 0;
     msg.inject_keycode.repeat = 0;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'inject %s'", name);
     }
 }
 
 static inline void
-action_home(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_HOME, action, "HOME");
+action_home(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_HOME, action, "HOME");
 }
 
 static inline void
-action_back(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_BACK, action, "BACK");
+action_back(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_BACK, action, "BACK");
 }
 
 static inline void
-action_app_switch(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
+action_app_switch(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_APP_SWITCH, action, "APP_SWITCH");
 }
 
 static inline void
-action_power(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_POWER, action, "POWER");
+action_power(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_POWER, action, "POWER");
 }
 
 static inline void
-action_volume_up(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
+action_volume_up(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_UP, action, "VOLUME_UP");
 }
 
 static inline void
-action_volume_down(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
+action_volume_down(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_VOLUME_DOWN, action, "VOLUME_DOWN");
 }
 
 static inline void
-action_menu(struct sc_controller *controller, enum sc_action action) {
-    send_keycode(controller, AKEYCODE_MENU, action, "MENU");
+action_menu(struct sc_input_manager *im, enum sc_action action) {
+    send_keycode(im, AKEYCODE_MENU, action, "MENU");
 }
 
 // turn the screen on if it was off, press BACK otherwise
 // If the screen is off, it is turned on only on ACTION_DOWN
 static void
-press_back_or_turn_screen_on(struct sc_controller *controller,
+press_back_or_turn_screen_on(struct sc_input_manager *im,
                              enum sc_action action) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_BACK_OR_SCREEN_ON;
     msg.back_or_screen_on.action = action == SC_ACTION_DOWN
                                  ? AKEY_EVENT_ACTION_DOWN
                                  : AKEY_EVENT_ACTION_UP;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'press back or turn screen on'");
     }
 }
 
 static void
-expand_notification_panel(struct sc_controller *controller) {
+expand_notification_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_NOTIFICATION_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand notification panel'");
     }
 }
 
 static void
-expand_settings_panel(struct sc_controller *controller) {
+expand_settings_panel(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_EXPAND_SETTINGS_PANEL;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'expand settings panel'");
     }
 }
 
 static void
-collapse_panels(struct sc_controller *controller) {
+collapse_panels(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_COLLAPSE_PANELS;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'collapse notification panel'");
     }
 }
 
 static bool
-get_device_clipboard(struct sc_controller *controller,
-                     enum sc_copy_key copy_key) {
+get_device_clipboard(struct sc_input_manager *im, enum sc_copy_key copy_key) {
+    assert(im->controller && im->kp);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_GET_CLIPBOARD;
     msg.get_clipboard.copy_key = copy_key;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'get device clipboard'");
         return false;
     }
@@ -199,8 +215,10 @@ get_device_clipboard(struct sc_controller *controller,
 }
 
 static bool
-set_device_clipboard(struct sc_controller *controller, bool paste,
+set_device_clipboard(struct sc_input_manager *im, bool paste,
                      uint64_t sequence) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -220,7 +238,7 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
     msg.set_clipboard.text = text_dup;
     msg.set_clipboard.paste = paste;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'set device clipboard'");
         return false;
@@ -230,19 +248,23 @@ set_device_clipboard(struct sc_controller *controller, bool paste,
 }
 
 static void
-set_screen_power_mode(struct sc_controller *controller,
+set_screen_power_mode(struct sc_input_manager *im,
                       enum sc_screen_power_mode mode) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
     msg.set_screen_power_mode.mode = mode;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request 'set screen power mode'");
     }
 }
 
 static void
-switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
+switch_fps_counter_state(struct sc_input_manager *im) {
+    struct sc_fps_counter *fps_counter = &im->screen->fps_counter;
+
     // the started state can only be written from the current thread, so there
     // is no ToCToU issue
     if (sc_fps_counter_is_started(fps_counter)) {
@@ -254,7 +276,9 @@ switch_fps_counter_state(struct sc_fps_counter *fps_counter) {
 }
 
 static void
-clipboard_paste(struct sc_controller *controller) {
+clipboard_paste(struct sc_input_manager *im) {
+    assert(im->controller && im->kp);
+
     char *text = SDL_GetClipboardText();
     if (!text) {
         LOGW("Could not get clipboard text: %s", SDL_GetError());
@@ -276,32 +300,43 @@ clipboard_paste(struct sc_controller *controller) {
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_INJECT_TEXT;
     msg.inject_text.text = text_dup;
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         free(text_dup);
         LOGW("Could not request 'paste clipboard'");
     }
 }
 
 static void
-rotate_device(struct sc_controller *controller) {
+rotate_device(struct sc_input_manager *im) {
+    assert(im->controller);
+
     struct sc_control_msg msg;
     msg.type = SC_CONTROL_MSG_TYPE_ROTATE_DEVICE;
 
-    if (!sc_controller_push_msg(controller, &msg)) {
+    if (!sc_controller_push_msg(im->controller, &msg)) {
         LOGW("Could not request device rotation");
     }
 }
 
 static void
-rotate_client_left(struct sc_screen *screen) {
-    unsigned new_rotation = (screen->rotation + 1) % 4;
-    sc_screen_set_rotation(screen, new_rotation);
+open_hard_keyboard_settings(struct sc_input_manager *im) {
+    assert(im->controller);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_OPEN_HARD_KEYBOARD_SETTINGS;
+
+    if (!sc_controller_push_msg(im->controller, &msg)) {
+        LOGW("Could not request opening hard keyboard settings");
+    }
 }
 
 static void
-rotate_client_right(struct sc_screen *screen) {
-    unsigned new_rotation = (screen->rotation + 3) % 4;
-    sc_screen_set_rotation(screen, new_rotation);
+apply_orientation_transform(struct sc_input_manager *im,
+                            enum sc_orientation transform) {
+    struct sc_screen *screen = im->screen;
+    enum sc_orientation new_orientation =
+        sc_orientation_apply(screen->orientation, transform);
+    sc_screen_set_orientation(screen, new_orientation);
 }
 
 static void
@@ -339,6 +374,7 @@ simulate_virtual_finger(struct sc_input_manager *im,
         im->forward_all_clicks ? POINTER_ID_VIRTUAL_MOUSE
                                : POINTER_ID_VIRTUAL_FINGER;
     msg.inject_touch_event.pressure = up ? 0.0f : 1.0f;
+    msg.inject_touch_event.action_button = 0;
     msg.inject_touch_event.buttons = 0;
 
     if (!sc_controller_push_msg(im->controller, &msg)) {
@@ -350,9 +386,14 @@ simulate_virtual_finger(struct sc_input_manager *im,
 }
 
 static struct sc_point
-inverse_point(struct sc_point point, struct sc_size size) {
-    point.x = size.width - point.x;
-    point.y = size.height - point.y;
+inverse_point(struct sc_point point, struct sc_size size,
+              bool invert_x, bool invert_y) {
+    if (invert_x) {
+        point.x = size.width - point.x;
+    }
+    if (invert_y) {
+        point.y = size.height - point.y;
+    }
     return point;
 }
 
@@ -360,7 +401,7 @@ static void
 sc_input_manager_process_key(struct sc_input_manager *im,
                              const SDL_KeyboardEvent *event) {
     // controller is NULL if --no-control is requested
-    struct sc_controller *controller = im->controller;
+    bool control = im->controller;
 
     SDL_Keycode keycode = event->keysym.sym;
     uint16_t mod = event->keysym.mod;
@@ -386,81 +427,102 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
         switch (keycode) {
             case SDLK_h:
-                if (controller && !shift && !repeat) {
-                    action_home(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_home(im, action);
                 }
                 return;
             case SDLK_b: // fall-through
             case SDLK_BACKSPACE:
-                if (controller && !shift && !repeat) {
-                    action_back(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_back(im, action);
                 }
                 return;
             case SDLK_s:
-                if (controller && !shift && !repeat) {
-                    action_app_switch(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_app_switch(im, action);
                 }
                 return;
             case SDLK_m:
-                if (controller && !shift && !repeat) {
-                    action_menu(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_menu(im, action);
                 }
                 return;
             case SDLK_p:
-                if (controller && !shift && !repeat) {
-                    action_power(controller, action);
+                if (im->kp && !shift && !repeat) {
+                    action_power(im, action);
                 }
                 return;
             case SDLK_o:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down) {
                     enum sc_screen_power_mode mode = shift
                                                    ? SC_SCREEN_POWER_MODE_NORMAL
                                                    : SC_SCREEN_POWER_MODE_OFF;
-                    set_screen_power_mode(controller, mode);
+                    set_screen_power_mode(im, mode);
                 }
                 return;
             case SDLK_DOWN:
-                if (controller && !shift) {
+                if (shift) {
+                    if (!repeat & down) {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_FLIP_180);
+                    }
+                } else if (im->kp) {
                     // forward repeated events
-                    action_volume_down(controller, action);
+                    action_volume_down(im, action);
                 }
                 return;
             case SDLK_UP:
-                if (controller && !shift) {
+                if (shift) {
+                    if (!repeat & down) {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_FLIP_180);
+                    }
+                } else if (im->kp) {
                     // forward repeated events
-                    action_volume_up(controller, action);
+                    action_volume_up(im, action);
                 }
                 return;
             case SDLK_LEFT:
-                if (!shift && !repeat && down) {
-                    rotate_client_left(im->screen);
+                if (!repeat && down) {
+                    if (shift) {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_FLIP_0);
+                    } else {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_270);
+                    }
                 }
                 return;
             case SDLK_RIGHT:
-                if (!shift && !repeat && down) {
-                    rotate_client_right(im->screen);
+                if (!repeat && down) {
+                    if (shift) {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_FLIP_0);
+                    } else {
+                        apply_orientation_transform(im,
+                                                    SC_ORIENTATION_90);
+                    }
                 }
                 return;
             case SDLK_c:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_COPY);
+                if (im->kp && !shift && !repeat && down) {
+                    get_device_clipboard(im, SC_COPY_KEY_COPY);
                 }
                 return;
             case SDLK_x:
-                if (controller && !shift && !repeat && down) {
-                    get_device_clipboard(controller, SC_COPY_KEY_CUT);
+                if (im->kp && !shift && !repeat && down) {
+                    get_device_clipboard(im, SC_COPY_KEY_CUT);
                 }
                 return;
             case SDLK_v:
-                if (controller && !repeat && down) {
+                if (im->kp && !repeat && down) {
                     if (shift || im->legacy_paste) {
                         // inject the text as input events
-                        clipboard_paste(controller);
+                        clipboard_paste(im);
                     } else {
                         // store the text in the device clipboard and paste,
                         // without requesting an acknowledgment
-                        set_device_clipboard(controller, true,
-                                             SC_SEQUENCE_INVALID);
+                        set_device_clipboard(im, true, SC_SEQUENCE_INVALID);
                     }
                 }
                 return;
@@ -481,23 +543,30 @@ sc_input_manager_process_key(struct sc_input_manager *im,
                 return;
             case SDLK_i:
                 if (!shift && !repeat && down) {
-                    switch_fps_counter_state(&im->screen->fps_counter);
+                    switch_fps_counter_state(im);
                 }
                 return;
             case SDLK_n:
-                if (controller && !repeat && down) {
+                if (control && !repeat && down) {
                     if (shift) {
-                        collapse_panels(controller);
+                        collapse_panels(im);
                     } else if (im->key_repeat == 0) {
-                        expand_notification_panel(controller);
+                        expand_notification_panel(im);
                     } else {
-                        expand_settings_panel(controller);
+                        expand_settings_panel(im);
                     }
                 }
                 return;
             case SDLK_r:
-                if (controller && !shift && !repeat && down) {
-                    rotate_device(controller);
+                if (control && !shift && !repeat && down) {
+                    rotate_device(im);
+                }
+                return;
+            case SDLK_k:
+                if (control && !shift && !repeat && down
+                        && im->kp && im->kp->hid) {
+                    // Only if the current keyboard is hid
+                    open_hard_keyboard_settings(im);
                 }
                 return;
         }
@@ -505,7 +574,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
         return;
     }
 
-    if (!controller) {
+    if (!im->kp) {
         return;
     }
 
@@ -514,7 +583,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
     if (im->clipboard_autosync && is_ctrl_v) {
         if (im->legacy_paste) {
             // inject the text as input events
-            clipboard_paste(controller);
+            clipboard_paste(im);
             return;
         }
 
@@ -524,7 +593,7 @@ sc_input_manager_process_key(struct sc_input_manager *im,
 
         // Synchronize the computer clipboard to the device clipboard before
         // sending Ctrl+v, to allow seamless copy-paste.
-        bool ok = set_device_clipboard(controller, false, sequence);
+        bool ok = set_device_clipboard(im, false, sequence);
         if (!ok) {
             LOGW("Clipboard could not be synchronized, Ctrl+v not injected");
             return;
@@ -586,7 +655,9 @@ sc_input_manager_process_mouse_motion(struct sc_input_manager *im,
         struct sc_point mouse =
            sc_screen_convert_window_to_frame_coords(im->screen, event->x,
                                                     event->y);
-        struct sc_point vfinger = inverse_point(mouse, im->screen->frame_size);
+        struct sc_point vfinger = inverse_point(mouse, im->screen->frame_size,
+                                                im->vfinger_invert_x,
+                                                im->vfinger_invert_y);
         simulate_virtual_finger(im, AMOTION_EVENT_ACTION_MOVE, vfinger);
     }
 }
@@ -624,7 +695,7 @@ sc_input_manager_process_touch(struct sc_input_manager *im,
 static void
 sc_input_manager_process_mouse_button(struct sc_input_manager *im,
                                       const SDL_MouseButtonEvent *event) {
-    struct sc_controller *controller = im->controller;
+    bool control = im->controller;
 
     if (event->which == SDL_TOUCH_MOUSEID) {
         // simulated from touch events, so it's a duplicate
@@ -633,27 +704,27 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
 
     bool down = event->type == SDL_MOUSEBUTTONDOWN;
     if (!im->forward_all_clicks) {
-        if (controller) {
+        if (control) {
             enum sc_action action = down ? SC_ACTION_DOWN : SC_ACTION_UP;
 
-            if (event->button == SDL_BUTTON_X1) {
-                action_app_switch(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_X1) {
+                action_app_switch(im, action);
                 return;
             }
             if (event->button == SDL_BUTTON_X2 && down) {
                 if (event->clicks < 2) {
-                    expand_notification_panel(controller);
+                    expand_notification_panel(im);
                 } else {
-                    expand_settings_panel(controller);
+                    expand_settings_panel(im);
                 }
                 return;
             }
-            if (event->button == SDL_BUTTON_RIGHT) {
-                press_back_or_turn_screen_on(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_RIGHT) {
+                press_back_or_turn_screen_on(im, action);
                 return;
             }
-            if (event->button == SDL_BUTTON_MIDDLE) {
-                action_home(controller, action);
+            if (im->kp && event->button == SDL_BUTTON_MIDDLE) {
+                action_home(im, action);
                 return;
             }
         }
@@ -676,7 +747,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
         // otherwise, send the click event to the device
     }
 
-    if (!controller) {
+    if (!im->mp) {
         return;
     }
 
@@ -707,7 +778,7 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
         return;
     }
 
-    // Pinch-to-zoom simulation.
+    // Pinch-to-zoom, rotate and tilt simulation.
     //
     // If Ctrl is hold when the left-click button is pressed, then
     // pinch-to-zoom mode is enabled: on every mouse event until the left-click
@@ -716,14 +787,29 @@ sc_input_manager_process_mouse_button(struct sc_input_manager *im,
     //
     // In other words, the center of the rotation/scaling is the center of the
     // screen.
-#define CTRL_PRESSED (SDL_GetModState() & (KMOD_LCTRL | KMOD_RCTRL))
+    //
+    // To simulate a tilt gesture (a vertical slide with two fingers), Shift
+    // can be used instead of Ctrl. The "virtual finger" has a position
+    // inverted with respect to the vertical axis of symmetry in the middle of
+    // the screen.
+    const SDL_Keymod keymod = SDL_GetModState();
+    const bool ctrl_pressed = keymod & KMOD_CTRL;
+    const bool shift_pressed = keymod & KMOD_SHIFT;
     if (event->button == SDL_BUTTON_LEFT &&
-            ((down && !im->vfinger_down && CTRL_PRESSED) ||
+            ((down && !im->vfinger_down &&
+              ((ctrl_pressed && !shift_pressed) ||
+               (!ctrl_pressed && shift_pressed))) ||
              (!down && im->vfinger_down))) {
         struct sc_point mouse =
             sc_screen_convert_window_to_frame_coords(im->screen, event->x,
                                                                  event->y);
-        struct sc_point vfinger = inverse_point(mouse, im->screen->frame_size);
+        if (down) {
+            im->vfinger_invert_x = ctrl_pressed || shift_pressed;
+            im->vfinger_invert_y = ctrl_pressed;
+        }
+        struct sc_point vfinger = inverse_point(mouse, im->screen->frame_size,
+                                                im->vfinger_invert_x,
+                                                im->vfinger_invert_y);
         enum android_motionevent_action action = down
                                                ? AMOTION_EVENT_ACTION_DOWN
                                                : AMOTION_EVENT_ACTION_UP;
@@ -796,11 +882,12 @@ sc_input_manager_process_file(struct sc_input_manager *im,
 }
 
 void
-sc_input_manager_handle_event(struct sc_input_manager *im, SDL_Event *event) {
+sc_input_manager_handle_event(struct sc_input_manager *im,
+                              const SDL_Event *event) {
     bool control = im->controller;
     switch (event->type) {
         case SDL_TEXTINPUT:
-            if (!control) {
+            if (!im->kp) {
                 break;
             }
             sc_input_manager_process_text_input(im, &event->text);
@@ -812,13 +899,13 @@ sc_input_manager_handle_event(struct sc_input_manager *im, SDL_Event *event) {
             sc_input_manager_process_key(im, &event->key);
             break;
         case SDL_MOUSEMOTION:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_mouse_motion(im, &event->motion);
             break;
         case SDL_MOUSEWHEEL:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_mouse_wheel(im, &event->wheel);
@@ -832,7 +919,7 @@ sc_input_manager_handle_event(struct sc_input_manager *im, SDL_Event *event) {
         case SDL_FINGERMOTION:
         case SDL_FINGERDOWN:
         case SDL_FINGERUP:
-            if (!control) {
+            if (!im->mp) {
                 break;
             }
             sc_input_manager_process_touch(im, &event->tfinger);

app/src/input_manager.h

@@ -32,6 +32,8 @@ struct sc_input_manager {
     } sdl_shortcut_mods;
 
     bool vfinger_down;
+    bool vfinger_invert_x;
+    bool vfinger_invert_y;
 
     // Tracks the number of identical consecutive shortcut key down events.
     // Not to be confused with event->repeat, which counts the number of
@@ -61,6 +63,7 @@ sc_input_manager_init(struct sc_input_manager *im,
                       const struct sc_input_manager_params *params);
 
 void
-sc_input_manager_handle_event(struct sc_input_manager *im, SDL_Event *event);
+sc_input_manager_handle_event(struct sc_input_manager *im,
+                              const SDL_Event *event);
 
 #endif

app/src/keyboard_sdk.c

@@ -1,4 +1,4 @@
-#include "keyboard_inject.h"
+#include "keyboard_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast key processor to sc_keyboard_inject */
-#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_inject, key_processor)
+/** Downcast key processor to sc_keyboard_sdk */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_sdk, key_processor)
 
 static enum android_keyevent_action
 convert_keycode_action(enum sc_action action) {
@@ -271,20 +271,20 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
     // is set before injecting Ctrl+v.
     (void) ack_to_wait;
 
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
     if (event->repeat) {
-        if (!ki->forward_key_repeat) {
+        if (!kb->forward_key_repeat) {
             return;
         }
-        ++ki->repeat;
+        ++kb->repeat;
     } else {
-        ki->repeat = 0;
+        kb->repeat = 0;
     }
 
     struct sc_control_msg msg;
-    if (convert_input_key(event, &msg, ki->key_inject_mode, ki->repeat)) {
-        if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (convert_input_key(event, &msg, kb->key_inject_mode, kb->repeat)) {
+        if (!sc_controller_push_msg(kb->controller, &msg)) {
             LOGW("Could not request 'inject keycode'");
         }
     }
@@ -293,14 +293,14 @@ sc_key_processor_process_key(struct sc_key_processor *kp,
 static void
 sc_key_processor_process_text(struct sc_key_processor *kp,
                               const struct sc_text_event *event) {
-    struct sc_keyboard_inject *ki = DOWNCAST(kp);
+    struct sc_keyboard_sdk *kb = DOWNCAST(kp);
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_RAW) {
         // Never inject text events
         return;
     }
 
-    if (ki->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
+    if (kb->key_inject_mode == SC_KEY_INJECT_MODE_MIXED) {
         char c = event->text[0];
         if (isalpha(c) || c == ' ') {
             assert(event->text[1] == '\0');
@@ -316,22 +316,22 @@ sc_key_processor_process_text(struct sc_key_processor *kp,
         LOGW("Could not strdup input text");
         return;
     }
-    if (!sc_controller_push_msg(ki->controller, &msg)) {
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
         free(msg.inject_text.text);
         LOGW("Could not request 'inject text'");
     }
 }
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat) {
-    ki->controller = controller;
-    ki->key_inject_mode = key_inject_mode;
-    ki->forward_key_repeat = forward_key_repeat;
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat) {
+    kb->controller = controller;
+    kb->key_inject_mode = key_inject_mode;
+    kb->forward_key_repeat = forward_key_repeat;
 
-    ki->repeat = 0;
+    kb->repeat = 0;
 
     static const struct sc_key_processor_ops ops = {
         .process_key = sc_key_processor_process_key,
@@ -339,6 +339,7 @@ sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
     };
 
     // Key injection and clipboard synchronization are serialized
-    ki->key_processor.async_paste = false;
-    ki->key_processor.ops = &ops;
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = false;
+    kb->key_processor.ops = &ops;
 }

app/src/keyboard_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_KEYBOARD_INJECT_H
-#define SC_KEYBOARD_INJECT_H
+#ifndef SC_KEYBOARD_SDK_H
+#define SC_KEYBOARD_SDK_H
 
 #include "common.h"
 
@@ -9,7 +9,7 @@
 #include "options.h"
 #include "trait/key_processor.h"
 
-struct sc_keyboard_inject {
+struct sc_keyboard_sdk {
     struct sc_key_processor key_processor; // key processor trait
 
     struct sc_controller *controller;
@@ -23,9 +23,9 @@ struct sc_keyboard_inject {
 };
 
 void
-sc_keyboard_inject_init(struct sc_keyboard_inject *ki,
-                        struct sc_controller *controller,
-                        enum sc_key_inject_mode key_inject_mode,
-                        bool forward_key_repeat);
+sc_keyboard_sdk_init(struct sc_keyboard_sdk *kb,
+                     struct sc_controller *controller,
+                     enum sc_key_inject_mode key_inject_mode,
+                     bool forward_key_repeat);
 
 #endif

app/src/main.c

@@ -4,10 +4,6 @@
 #include <stdbool.h>
 #include <unistd.h>
 #include <libavformat/avformat.h>
-#ifdef _WIN32
-#include <windows.h>
-#include "util/str.h"
-#endif
 #ifdef HAVE_V4L2
 # include <libavdevice/avdevice.h>
 #endif
@@ -19,9 +15,15 @@
 #include "scrcpy.h"
 #include "usb/scrcpy_otg.h"
 #include "util/log.h"
+#include "util/net.h"
 #include "version.h"
 
-int
+#ifdef _WIN32
+#include <windows.h>
+#include "util/str.h"
+#endif
+
+static int
 main_scrcpy(int argc, char *argv[]) {
 #ifdef _WIN32
     // disable buffering, we want logs immediately
@@ -37,26 +39,32 @@ main_scrcpy(int argc, char *argv[]) {
         .opts = scrcpy_options_default,
         .help = false,
         .version = false,
+        .pause_on_exit = SC_PAUSE_ON_EXIT_FALSE,
     };
 
 #ifndef NDEBUG
     args.opts.log_level = SC_LOG_LEVEL_DEBUG;
 #endif
 
+    enum scrcpy_exit_code ret;
+
     if (!scrcpy_parse_args(&args, argc, argv)) {
-        return SCRCPY_EXIT_FAILURE;
+        ret = SCRCPY_EXIT_FAILURE;
+        goto end;
     }
 
     sc_set_log_level(args.opts.log_level);
 
     if (args.help) {
         scrcpy_print_usage(argv[0]);
-        return SCRCPY_EXIT_SUCCESS;
+        ret = SCRCPY_EXIT_SUCCESS;
+        goto end;
     }
 
     if (args.version) {
         scrcpy_print_version();
-        return SCRCPY_EXIT_SUCCESS;
+        ret = SCRCPY_EXIT_SUCCESS;
+        goto end;
     }
 
 #ifdef SCRCPY_LAVF_REQUIRES_REGISTER_ALL
@@ -69,18 +77,26 @@ main_scrcpy(int argc, char *argv[]) {
     }
 #endif
 
-    if (avformat_network_init()) {
-        return SCRCPY_EXIT_FAILURE;
+    if (!net_init()) {
+        ret = SCRCPY_EXIT_FAILURE;
+        goto end;
     }
 
+    sc_log_configure();
+
 #ifdef HAVE_USB
-    enum scrcpy_exit_code ret = args.opts.otg ? scrcpy_otg(&args.opts)
-                                              : scrcpy(&args.opts);
+    ret = args.opts.otg ? scrcpy_otg(&args.opts) : scrcpy(&args.opts);
 #else
-    enum scrcpy_exit_code ret = scrcpy(&args.opts);
+    ret = scrcpy(&args.opts);
 #endif
 
-    avformat_network_deinit(); // ignore failure
+end:
+    if (args.pause_on_exit == SC_PAUSE_ON_EXIT_TRUE ||
+            (args.pause_on_exit == SC_PAUSE_ON_EXIT_IF_ERROR &&
+                ret != SCRCPY_EXIT_SUCCESS)) {
+        printf("Press Enter to continue...\n");
+        getchar();
+    }
 
     return ret;
 }

app/src/mouse_sdk.c

@@ -1,4 +1,4 @@
-#include "mouse_inject.h"
+#include "mouse_sdk.h"
 
 #include <assert.h>
 
@@ -9,8 +9,8 @@
 #include "util/intmap.h"
 #include "util/log.h"
 
-/** Downcast mouse processor to sc_mouse_inject */
-#define DOWNCAST(MP) container_of(MP, struct sc_mouse_inject, mouse_processor)
+/** Downcast mouse processor to sc_mouse_sdk */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_sdk, mouse_processor)
 
 static enum android_motionevent_buttons
 convert_mouse_buttons(uint32_t state) {
@@ -63,7 +63,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         return;
     }
 
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -76,7 +76,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse motion event'");
     }
 }
@@ -84,7 +84,7 @@ sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
                                     const struct sc_mouse_click_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -93,11 +93,12 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
             .pointer_id = event->pointer_id,
             .position = event->position,
             .pressure = event->action == SC_ACTION_DOWN ? 1.f : 0.f,
+            .action_button = convert_mouse_buttons(event->button),
             .buttons = convert_mouse_buttons(event->buttons_state),
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse click event'");
     }
 }
@@ -105,7 +106,7 @@ sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
                                    const struct sc_mouse_scroll_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_SCROLL_EVENT,
@@ -117,7 +118,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject mouse scroll event'");
     }
 }
@@ -125,7 +126,7 @@ sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
 static void
 sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
                                  const struct sc_touch_event *event) {
-    struct sc_mouse_inject *mi = DOWNCAST(mp);
+    struct sc_mouse_sdk *m = DOWNCAST(mp);
 
     struct sc_control_msg msg = {
         .type = SC_CONTROL_MSG_TYPE_INJECT_TOUCH_EVENT,
@@ -138,15 +139,14 @@ sc_mouse_processor_process_touch(struct sc_mouse_processor *mp,
         },
     };
 
-    if (!sc_controller_push_msg(mi->controller, &msg)) {
+    if (!sc_controller_push_msg(m->controller, &msg)) {
         LOGW("Could not request 'inject touch event'");
     }
 }
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller) {
-    mi->controller = controller;
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller) {
+    m->controller = controller;
 
     static const struct sc_mouse_processor_ops ops = {
         .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
@@ -155,7 +155,7 @@ sc_mouse_inject_init(struct sc_mouse_inject *mi,
         .process_touch = sc_mouse_processor_process_touch,
     };
 
-    mi->mouse_processor.ops = &ops;
+    m->mouse_processor.ops = &ops;
 
-    mi->mouse_processor.relative_mode = false;
+    m->mouse_processor.relative_mode = false;
 }

app/src/mouse_sdk.h

@@ -1,5 +1,5 @@
-#ifndef SC_MOUSE_INJECT_H
-#define SC_MOUSE_INJECT_H
+#ifndef SC_MOUSE_SDK_H
+#define SC_MOUSE_SDK_H
 
 #include "common.h"
 
@@ -9,14 +9,13 @@
 #include "screen.h"
 #include "trait/mouse_processor.h"
 
-struct sc_mouse_inject {
+struct sc_mouse_sdk {
     struct sc_mouse_processor mouse_processor; // mouse processor trait
 
     struct sc_controller *controller;
 };
 
 void
-sc_mouse_inject_init(struct sc_mouse_inject *mi,
-                     struct sc_controller *controller);
+sc_mouse_sdk_init(struct sc_mouse_sdk *m, struct sc_controller *controller);
 
 #endif

app/src/options.c

@@ -7,14 +7,23 @@ const struct scrcpy_options scrcpy_options_default = {
     .window_title = NULL,
     .push_target = NULL,
     .render_driver = NULL,
-    .codec_options = NULL,
-    .encoder_name = NULL,
-#ifdef HAVE_V4L2
-    .v4l2_device = NULL,
-#endif
+    .video_codec_options = NULL,
+    .audio_codec_options = NULL,
+    .video_encoder = NULL,
+    .audio_encoder = NULL,
+    .camera_id = NULL,
+    .camera_size = NULL,
+    .camera_ar = NULL,
+    .camera_fps = 0,
     .log_level = SC_LOG_LEVEL_INFO,
+    .video_codec = SC_CODEC_H264,
+    .audio_codec = SC_CODEC_OPUS,
+    .video_source = SC_VIDEO_SOURCE_DISPLAY,
+    .audio_source = SC_AUDIO_SOURCE_AUTO,
     .record_format = SC_RECORD_FORMAT_AUTO,
-    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT,
+    .keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_AUTO,
+    .mouse_input_mode = SC_MOUSE_INPUT_MODE_AUTO,
+    .camera_facing = SC_CAMERA_FACING_ANY,
     .port_range = {
         .first = DEFAULT_LOCAL_PORT_RANGE_FIRST,
         .last = DEFAULT_LOCAL_PORT_RANGE_LAST,
@@ -26,17 +35,25 @@ const struct scrcpy_options scrcpy_options_default = {
         .count = 2,
     },
     .max_size = 0,
-    .bit_rate = DEFAULT_BIT_RATE,
+    .video_bit_rate = 0,
+    .audio_bit_rate = 0,
     .max_fps = 0,
     .lock_video_orientation = SC_LOCK_VIDEO_ORIENTATION_UNLOCKED,
-    .rotation = 0,
+    .display_orientation = SC_ORIENTATION_0,
+    .record_orientation = SC_ORIENTATION_0,
     .window_x = SC_WINDOW_POSITION_UNDEFINED,
     .window_y = SC_WINDOW_POSITION_UNDEFINED,
     .window_width = 0,
     .window_height = 0,
     .display_id = 0,
     .display_buffer = 0,
+    .audio_buffer = -1, // depends on the audio format,
+    .audio_output_buffer = SC_TICK_FROM_MS(5),
+    .time_limit = 0,
+#ifdef HAVE_V4L2
+    .v4l2_device = NULL,
     .v4l2_buffer = 0,
+#endif
 #ifdef HAVE_USB
     .otg = false,
 #endif
@@ -44,7 +61,8 @@ const struct scrcpy_options scrcpy_options_default = {
     .fullscreen = false,
     .always_on_top = false,
     .control = true,
-    .display = true,
+    .video_playback = true,
+    .audio_playback = true,
     .turn_screen_off = false,
     .key_inject_mode = SC_KEY_INJECT_MODE_MIXED,
     .window_borderless = false,
@@ -65,4 +83,46 @@ const struct scrcpy_options scrcpy_options_default = {
     .cleanup = true,
     .start_fps_counter = false,
     .power_on = true,
+    .video = true,
+    .audio = true,
+    .require_audio = false,
+    .kill_adb_on_close = false,
+    .camera_high_speed = false,
+    .list = 0,
 };
+
+enum sc_orientation
+sc_orientation_apply(enum sc_orientation src, enum sc_orientation transform) {
+    assert(!(src & ~7));
+    assert(!(transform & ~7));
+
+    unsigned transform_hflip = transform & 4;
+    unsigned transform_rotation = transform & 3;
+    unsigned src_hflip = src & 4;
+    unsigned src_rotation = src & 3;
+    unsigned src_swap = src & 1;
+    if (src_swap && transform_hflip) {
+        // If the src is rotated by 90 or 270 degrees, applying a flipped
+        // transformation requires an additional 180 degrees rotation to
+        // compensate for the inversion of the order of multiplication:
+        //
+        //     hflip1 × rotate1 × hflip2 × rotate2
+        //     `--------------'   `--------------'
+        //           src             transform
+        //
+        // In the final result, we want all the hflips then all the rotations,
+        // so we must move hflip2 to the left:
+        //
+        //     hflip1 × hflip2 × rotate1' × rotate2
+        //
+        // with rotate1' = | rotate1           if src is 0° or 180°
+        //                 | rotate1 + 180°    if src is 90° or 270°
+
+        src_rotation += 2;
+    }
+
+    unsigned result_hflip = src_hflip ^ transform_hflip;
+    unsigned result_rotation = (transform_rotation + src_rotation) % 4;
+    enum sc_orientation result = result_hflip | result_rotation;
+    return result;
+}

app/src/options.h

@@ -3,6 +3,7 @@
 
 #include "common.h"
 
+#include <assert.h>
 #include <stdbool.h>
 #include <stddef.h>
 #include <stdint.h>
@@ -21,26 +22,137 @@ enum sc_record_format {
     SC_RECORD_FORMAT_AUTO,
     SC_RECORD_FORMAT_MP4,
     SC_RECORD_FORMAT_MKV,
+    SC_RECORD_FORMAT_M4A,
+    SC_RECORD_FORMAT_MKA,
+    SC_RECORD_FORMAT_OPUS,
+    SC_RECORD_FORMAT_AAC,
+    SC_RECORD_FORMAT_FLAC,
+    SC_RECORD_FORMAT_WAV,
 };
 
+static inline bool
+sc_record_format_is_audio_only(enum sc_record_format fmt) {
+    return fmt == SC_RECORD_FORMAT_M4A
+        || fmt == SC_RECORD_FORMAT_MKA
+        || fmt == SC_RECORD_FORMAT_OPUS
+        || fmt == SC_RECORD_FORMAT_AAC
+        || fmt == SC_RECORD_FORMAT_FLAC
+        || fmt == SC_RECORD_FORMAT_WAV;
+}
+
+enum sc_codec {
+    SC_CODEC_H264,
+    SC_CODEC_H265,
+    SC_CODEC_AV1,
+    SC_CODEC_OPUS,
+    SC_CODEC_AAC,
+    SC_CODEC_FLAC,
+    SC_CODEC_RAW,
+};
+
+enum sc_video_source {
+    SC_VIDEO_SOURCE_DISPLAY,
+    SC_VIDEO_SOURCE_CAMERA,
+};
+
+enum sc_audio_source {
+    SC_AUDIO_SOURCE_AUTO, // OUTPUT for video DISPLAY, MIC for video CAMERA
+    SC_AUDIO_SOURCE_OUTPUT,
+    SC_AUDIO_SOURCE_MIC,
+};
+
+enum sc_camera_facing {
+    SC_CAMERA_FACING_ANY,
+    SC_CAMERA_FACING_FRONT,
+    SC_CAMERA_FACING_BACK,
+    SC_CAMERA_FACING_EXTERNAL,
+};
+
+                              // ,----- hflip (applied before the rotation)
+                              // | ,--- 180°
+                              // | | ,- 90° clockwise
+                              // | | |
+enum sc_orientation {         // v v v
+    SC_ORIENTATION_0,         // 0 0 0
+    SC_ORIENTATION_90,        // 0 0 1
+    SC_ORIENTATION_180,       // 0 1 0
+    SC_ORIENTATION_270,       // 0 1 1
+    SC_ORIENTATION_FLIP_0,    // 1 0 0
+    SC_ORIENTATION_FLIP_90,   // 1 0 1
+    SC_ORIENTATION_FLIP_180,  // 1 1 0
+    SC_ORIENTATION_FLIP_270,  // 1 1 1
+};
+
+static inline bool
+sc_orientation_is_mirror(enum sc_orientation orientation) {
+    assert(!(orientation & ~7));
+    return orientation & 4;
+}
+
+// Does the orientation swap width and height?
+static inline bool
+sc_orientation_is_swap(enum sc_orientation orientation) {
+    assert(!(orientation & ~7));
+    return orientation & 1;
+}
+
+static inline enum sc_orientation
+sc_orientation_get_rotation(enum sc_orientation orientation) {
+    assert(!(orientation & ~7));
+    return orientation & 3;
+}
+
+enum sc_orientation
+sc_orientation_apply(enum sc_orientation src, enum sc_orientation transform);
+
+static inline const char *
+sc_orientation_get_name(enum sc_orientation orientation) {
+    switch (orientation) {
+        case SC_ORIENTATION_0:
+            return "0";
+        case SC_ORIENTATION_90:
+            return "90";
+        case SC_ORIENTATION_180:
+            return "180";
+        case SC_ORIENTATION_270:
+            return "270";
+        case SC_ORIENTATION_FLIP_0:
+            return "flip0";
+        case SC_ORIENTATION_FLIP_90:
+            return "flip90";
+        case SC_ORIENTATION_FLIP_180:
+            return "flip180";
+        case SC_ORIENTATION_FLIP_270:
+            return "flip270";
+        default:
+            return "(unknown)";
+    }
+}
+
 enum sc_lock_video_orientation {
     SC_LOCK_VIDEO_ORIENTATION_UNLOCKED = -1,
     // lock the current orientation when scrcpy starts
     SC_LOCK_VIDEO_ORIENTATION_INITIAL = -2,
     SC_LOCK_VIDEO_ORIENTATION_0 = 0,
-    SC_LOCK_VIDEO_ORIENTATION_1,
-    SC_LOCK_VIDEO_ORIENTATION_2,
-    SC_LOCK_VIDEO_ORIENTATION_3,
+    SC_LOCK_VIDEO_ORIENTATION_90 = 3,
+    SC_LOCK_VIDEO_ORIENTATION_180 = 2,
+    SC_LOCK_VIDEO_ORIENTATION_270 = 1,
 };
 
 enum sc_keyboard_input_mode {
-    SC_KEYBOARD_INPUT_MODE_INJECT,
-    SC_KEYBOARD_INPUT_MODE_HID,
+    SC_KEYBOARD_INPUT_MODE_AUTO,
+    SC_KEYBOARD_INPUT_MODE_DISABLED,
+    SC_KEYBOARD_INPUT_MODE_SDK,
+    SC_KEYBOARD_INPUT_MODE_UHID,
+    SC_KEYBOARD_INPUT_MODE_AOA,
 };
 
 enum sc_mouse_input_mode {
-    SC_MOUSE_INPUT_MODE_INJECT,
-    SC_MOUSE_INPUT_MODE_HID,
+    SC_MOUSE_INPUT_MODE_AUTO,
+    SC_MOUSE_INPUT_MODE_DISABLED,
+    SC_MOUSE_INPUT_MODE_SDK,
+    SC_MOUSE_INPUT_MODE_UHID,
+    SC_MOUSE_INPUT_MODE_AOA,
 };
 
 enum sc_key_inject_mode {
@@ -87,31 +199,47 @@ struct scrcpy_options {
     const char *window_title;
     const char *push_target;
     const char *render_driver;
-    const char *codec_options;
-    const char *encoder_name;
-#ifdef HAVE_V4L2
-    const char *v4l2_device;
-#endif
+    const char *video_codec_options;
+    const char *audio_codec_options;
+    const char *video_encoder;
+    const char *audio_encoder;
+    const char *camera_id;
+    const char *camera_size;
+    const char *camera_ar;
+    uint16_t camera_fps;
     enum sc_log_level log_level;
+    enum sc_codec video_codec;
+    enum sc_codec audio_codec;
+    enum sc_video_source video_source;
+    enum sc_audio_source audio_source;
     enum sc_record_format record_format;
     enum sc_keyboard_input_mode keyboard_input_mode;
     enum sc_mouse_input_mode mouse_input_mode;
+    enum sc_camera_facing camera_facing;
     struct sc_port_range port_range;
     uint32_t tunnel_host;
     uint16_t tunnel_port;
     struct sc_shortcut_mods shortcut_mods;
     uint16_t max_size;
-    uint32_t bit_rate;
+    uint32_t video_bit_rate;
+    uint32_t audio_bit_rate;
     uint16_t max_fps;
     enum sc_lock_video_orientation lock_video_orientation;
-    uint8_t rotation;
+    enum sc_orientation display_orientation;
+    enum sc_orientation record_orientation;
     int16_t window_x; // SC_WINDOW_POSITION_UNDEFINED for "auto"
     int16_t window_y; // SC_WINDOW_POSITION_UNDEFINED for "auto"
     uint16_t window_width;
     uint16_t window_height;
     uint32_t display_id;
     sc_tick display_buffer;
+    sc_tick audio_buffer;
+    sc_tick audio_output_buffer;
+    sc_tick time_limit;
+#ifdef HAVE_V4L2
+    const char *v4l2_device;
     sc_tick v4l2_buffer;
+#endif
 #ifdef HAVE_USB
     bool otg;
 #endif
@@ -119,7 +247,8 @@ struct scrcpy_options {
     bool fullscreen;
     bool always_on_top;
     bool control;
-    bool display;
+    bool video_playback;
+    bool audio_playback;
     bool turn_screen_off;
     enum sc_key_inject_mode key_inject_mode;
     bool window_borderless;
@@ -140,6 +269,16 @@ struct scrcpy_options {
     bool cleanup;
     bool start_fps_counter;
     bool power_on;
+    bool video;
+    bool audio;
+    bool require_audio;
+    bool kill_adb_on_close;
+    bool camera_high_speed;
+#define SC_OPTION_LIST_ENCODERS 0x1
+#define SC_OPTION_LIST_DISPLAYS 0x2
+#define SC_OPTION_LIST_CAMERAS 0x4
+#define SC_OPTION_LIST_CAMERA_SIZES 0x8
+    uint8_t list;
 };
 
 extern const struct scrcpy_options scrcpy_options_default;

app/src/packet_merger.c

@@ -0,0 +1,48 @@
+#include "packet_merger.h"
+
+#include "util/log.h"
+
+void
+sc_packet_merger_init(struct sc_packet_merger *merger) {
+    merger->config = NULL;
+}
+
+void
+sc_packet_merger_destroy(struct sc_packet_merger *merger) {
+    free(merger->config);
+}
+
+bool
+sc_packet_merger_merge(struct sc_packet_merger *merger, AVPacket *packet) {
+    bool is_config = packet->pts == AV_NOPTS_VALUE;
+
+    if (is_config) {
+        free(merger->config);
+
+        merger->config = malloc(packet->size);
+        if (!merger->config) {
+            LOG_OOM();
+            return false;
+        }
+
+        memcpy(merger->config, packet->data, packet->size);
+        merger->config_size = packet->size;
+    } else if (merger->config) {
+        size_t config_size = merger->config_size;
+        size_t media_size = packet->size;
+
+        if (av_grow_packet(packet, config_size)) {
+            LOG_OOM();
+            return false;
+        }
+
+        memmove(packet->data + config_size, packet->data, media_size);
+        memcpy(packet->data, merger->config, config_size);
+
+        free(merger->config);
+        merger->config = NULL;
+        // merger->size is meaningless when merger->config is NULL
+    }
+
+    return true;
+}

app/src/packet_merger.h

@@ -0,0 +1,43 @@
+#ifndef SC_PACKET_MERGER_H
+#define SC_PACKET_MERGER_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <stdint.h>
+#include <libavcodec/avcodec.h>
+
+/**
+ * Config packets (containing the SPS/PPS) are sent in-band. A new config
+ * packet is sent whenever a new encoding session is started (on start and on
+ * device orientation change).
+ *
+ * Every time a config packet is received, it must be sent alone (for recorder
+ * extradata), then concatenated to the next media packet (for correct decoding
+ * and recording).
+ *
+ * This helper reads every input packet and modifies each media packet which
+ * immediately follows a config packet to prepend the config packet payload.
+ */
+
+struct sc_packet_merger {
+    uint8_t *config;
+    size_t config_size;
+};
+
+void
+sc_packet_merger_init(struct sc_packet_merger *merger);
+
+void
+sc_packet_merger_destroy(struct sc_packet_merger *merger);
+
+/**
+ * If the packet is a config packet, then keep its data for later.
+ * Otherwise (if the packet is a media packet), then if a config packet is
+ * pending, prepend the config packet to this packet (so the packet is
+ * modified!).
+ */
+bool
+sc_packet_merger_merge(struct sc_packet_merger *merger, AVPacket *packet);
+
+#endif

app/src/receiver.c

@@ -1,32 +1,35 @@
 #include "receiver.h"
 
 #include <assert.h>
+#include <inttypes.h>
+#include <stdint.h>
 #include <SDL2/SDL_clipboard.h>
 
 #include "device_msg.h"
 #include "util/log.h"
+#include "util/str.h"
 
 bool
-receiver_init(struct receiver *receiver, sc_socket control_socket,
-              struct sc_acksync *acksync) {
+sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket) {
     bool ok = sc_mutex_init(&receiver->mutex);
     if (!ok) {
         return false;
     }
 
     receiver->control_socket = control_socket;
-    receiver->acksync = acksync;
+    receiver->acksync = NULL;
+    receiver->uhid_devices = NULL;
 
     return true;
 }
 
 void
-receiver_destroy(struct receiver *receiver) {
+sc_receiver_destroy(struct sc_receiver *receiver) {
     sc_mutex_destroy(&receiver->mutex);
 }
 
 static void
-process_msg(struct receiver *receiver, struct device_msg *msg) {
+process_msg(struct sc_receiver *receiver, struct sc_device_msg *msg) {
     switch (msg->type) {
         case DEVICE_MSG_TYPE_CLIPBOARD: {
             char *current = SDL_GetClipboardText();
@@ -42,20 +45,65 @@ process_msg(struct receiver *receiver, struct device_msg *msg) {
             break;
         }
         case DEVICE_MSG_TYPE_ACK_CLIPBOARD:
-            assert(receiver->acksync);
             LOGD("Ack device clipboard sequence=%" PRIu64_,
                  msg->ack_clipboard.sequence);
+
+            // This is a programming error to receive this message if there is
+            // no ACK synchronization mechanism
+            assert(receiver->acksync);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->acksync) {
+                LOGE("Received unexpected ack");
+                return;
+            }
+
             sc_acksync_ack(receiver->acksync, msg->ack_clipboard.sequence);
             break;
+        case DEVICE_MSG_TYPE_UHID_OUTPUT:
+            if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
+                char *hex = sc_str_to_hex_string(msg->uhid_output.data,
+                                                 msg->uhid_output.size);
+                if (hex) {
+                    LOGV("UHID output [%" PRIu16 "] %s",
+                         msg->uhid_output.id, hex);
+                    free(hex);
+                } else {
+                    LOGV("UHID output [%" PRIu16 "] size=%" PRIu16,
+                         msg->uhid_output.id, msg->uhid_output.size);
+                }
+            }
+
+            // This is a programming error to receive this message if there is
+            // no uhid_devices instance
+            assert(receiver->uhid_devices);
+
+            // Also check at runtime (do not trust the server)
+            if (!receiver->uhid_devices) {
+                LOGE("Received unexpected HID output message");
+                return;
+            }
+
+            struct sc_uhid_receiver *uhid_receiver =
+                sc_uhid_devices_get_receiver(receiver->uhid_devices,
+                                             msg->uhid_output.id);
+            if (uhid_receiver) {
+                uhid_receiver->ops->process_output(uhid_receiver,
+                                                   msg->uhid_output.data,
+                                                   msg->uhid_output.size);
+            } else {
+                LOGW("No UHID receiver for id %" PRIu16, msg->uhid_output.id);
+            }
+            break;
     }
 }
 
 static ssize_t
-process_msgs(struct receiver *receiver, const unsigned char *buf, size_t len) {
+process_msgs(struct sc_receiver *receiver, const uint8_t *buf, size_t len) {
     size_t head = 0;
     for (;;) {
-        struct device_msg msg;
-        ssize_t r = device_msg_deserialize(&buf[head], len - head, &msg);
+        struct sc_device_msg msg;
+        ssize_t r = sc_device_msg_deserialize(&buf[head], len - head, &msg);
         if (r == -1) {
             return -1;
         }
@@ -64,7 +112,7 @@ process_msgs(struct receiver *receiver, const unsigned char *buf, size_t len) {
         }
 
         process_msg(receiver, &msg);
-        device_msg_destroy(&msg);
+        sc_device_msg_destroy(&msg);
 
         head += r;
         assert(head <= len);
@@ -76,9 +124,9 @@ process_msgs(struct receiver *receiver, const unsigned char *buf, size_t len) {
 
 static int
 run_receiver(void *data) {
-    struct receiver *receiver = data;
+    struct sc_receiver *receiver = data;
 
-    static unsigned char buf[DEVICE_MSG_MAX_SIZE];
+    static uint8_t buf[DEVICE_MSG_MAX_SIZE];
     size_t head = 0;
 
     for (;;) {
@@ -108,7 +156,7 @@ run_receiver(void *data) {
 }
 
 bool
-receiver_start(struct receiver *receiver) {
+sc_receiver_start(struct sc_receiver *receiver) {
     LOGD("Starting receiver thread");
 
     bool ok = sc_thread_create(&receiver->thread, run_receiver,
@@ -122,6 +170,6 @@ receiver_start(struct receiver *receiver) {
 }
 
 void
-receiver_join(struct receiver *receiver) {
+sc_receiver_join(struct sc_receiver *receiver) {
     sc_thread_join(&receiver->thread, NULL);
 }

app/src/receiver.h

@@ -5,33 +5,34 @@
 
 #include <stdbool.h>
 
+#include "uhid/uhid_output.h"
 #include "util/acksync.h"
 #include "util/net.h"
 #include "util/thread.h"
 
 // receive events from the device
 // managed by the controller
-struct receiver {
+struct sc_receiver {
     sc_socket control_socket;
     sc_thread thread;
     sc_mutex mutex;
 
     struct sc_acksync *acksync;
+    struct sc_uhid_devices *uhid_devices;
 };
 
 bool
-receiver_init(struct receiver *receiver, sc_socket control_socket,
-              struct sc_acksync *acksync);
+sc_receiver_init(struct sc_receiver *receiver, sc_socket control_socket);
 
 void
-receiver_destroy(struct receiver *receiver);
+sc_receiver_destroy(struct sc_receiver *receiver);
 
 bool
-receiver_start(struct receiver *receiver);
+sc_receiver_start(struct sc_receiver *receiver);
 
-// no receiver_stop(), it will automatically stop on control_socket shutdown
+// no sc_receiver_stop(), it will automatically stop on control_socket shutdown
 
 void
-receiver_join(struct receiver *receiver);
+sc_receiver_join(struct sc_receiver *receiver);
 
 #endif

app/src/recorder.h

@@ -9,43 +9,77 @@
 #include "coords.h"
 #include "options.h"
 #include "trait/packet_sink.h"
-#include "util/queue.h"
 #include "util/thread.h"
+#include "util/vecdeque.h"
 
-struct sc_record_packet {
-    AVPacket *packet;
-    struct sc_record_packet *next;
-};
+struct sc_recorder_queue SC_VECDEQUE(AVPacket *);
 
-struct sc_recorder_queue SC_QUEUE(struct sc_record_packet);
+struct sc_recorder_stream {
+    int index;
+    int64_t last_pts;
+};
 
 struct sc_recorder {
-    struct sc_packet_sink packet_sink; // packet sink trait
+    struct sc_packet_sink video_packet_sink;
+    struct sc_packet_sink audio_packet_sink;
+
+    /* The audio flag is unprotected:
+     *  - it is initialized from sc_recorder_init() from the main thread;
+     *  - it may be reset once from the recorder thread if the audio is
+     *    disabled dynamically.
+     *
+     * Therefore, once the recorder thread is started, only the recorder thread
+     * may access it without data races.
+     */
+    bool audio;
+    bool video;
+
+    enum sc_orientation orientation;
 
     char *filename;
     enum sc_record_format format;
     AVFormatContext *ctx;
-    struct sc_size declared_frame_size;
-    bool header_written;
 
     sc_thread thread;
     sc_mutex mutex;
-    sc_cond queue_cond;
-    bool stopped; // set on recorder_close()
-    bool failed; // set on packet write failure
-    struct sc_recorder_queue queue;
-
-    // we can write a packet only once we received the next one so that we can
-    // set its duration (next_pts - current_pts)
-    // "previous" is only accessed from the recorder thread, so it does not
-    // need to be protected by the mutex
-    struct sc_record_packet *previous;
+    sc_cond cond;
+    // set on sc_recorder_stop(), packet_sink close or recording failure
+    bool stopped;
+    struct sc_recorder_queue video_queue;
+    struct sc_recorder_queue audio_queue;
+
+    // wake up the recorder thread once the video or audio codec is known
+    bool video_init;
+    bool audio_init;
+
+    bool audio_expects_config_packet;
+
+    struct sc_recorder_stream video_stream;
+    struct sc_recorder_stream audio_stream;
+
+    const struct sc_recorder_callbacks *cbs;
+    void *cbs_userdata;
+};
+
+struct sc_recorder_callbacks {
+    void (*on_ended)(struct sc_recorder *recorder, bool success,
+                     void *userdata);
 };
 
 bool
 sc_recorder_init(struct sc_recorder *recorder, const char *filename,
-                 enum sc_record_format format,
-                 struct sc_size declared_frame_size);
+                 enum sc_record_format format, bool video, bool audio,
+                 enum sc_orientation orientation,
+                 const struct sc_recorder_callbacks *cbs, void *cbs_userdata);
+
+bool
+sc_recorder_start(struct sc_recorder *recorder);
+
+void
+sc_recorder_stop(struct sc_recorder *recorder);
+
+void
+sc_recorder_join(struct sc_recorder *recorder);
 
 void
 sc_recorder_destroy(struct sc_recorder *recorder);

app/src/scrcpy.c

@@ -13,25 +13,31 @@
 # include <windows.h>
 #endif
 
+#include "audio_player.h"
 #include "controller.h"
 #include "decoder.h"
+#include "delay_buffer.h"
 #include "demuxer.h"
 #include "events.h"
 #include "file_pusher.h"
-#include "keyboard_inject.h"
-#include "mouse_inject.h"
+#include "keyboard_sdk.h"
+#include "mouse_sdk.h"
 #include "recorder.h"
 #include "screen.h"
 #include "server.h"
+#include "uhid/keyboard_uhid.h"
+#include "uhid/mouse_uhid.h"
 #ifdef HAVE_USB
 # include "usb/aoa_hid.h"
-# include "usb/hid_keyboard.h"
-# include "usb/hid_mouse.h"
+# include "usb/keyboard_aoa.h"
+# include "usb/mouse_aoa.h"
 # include "usb/usb.h"
 #endif
 #include "util/acksync.h"
 #include "util/log.h"
 #include "util/net.h"
+#include "util/rand.h"
+#include "util/timeout.h"
 #ifdef HAVE_V4L2
 # include "v4l2_sink.h"
 #endif
@@ -39,11 +45,16 @@
 struct scrcpy {
     struct sc_server server;
     struct sc_screen screen;
-    struct sc_demuxer demuxer;
-    struct sc_decoder decoder;
+    struct sc_audio_player audio_player;
+    struct sc_demuxer video_demuxer;
+    struct sc_demuxer audio_demuxer;
+    struct sc_decoder video_decoder;
+    struct sc_decoder audio_decoder;
     struct sc_recorder recorder;
+    struct sc_delay_buffer display_buffer;
 #ifdef HAVE_V4L2
     struct sc_v4l2_sink v4l2_sink;
+    struct sc_delay_buffer v4l2_buffer;
 #endif
     struct sc_controller controller;
     struct sc_file_pusher file_pusher;
@@ -52,19 +63,23 @@ struct scrcpy {
     struct sc_aoa aoa;
     // sequence/ack helper to synchronize clipboard and Ctrl+v via HID
     struct sc_acksync acksync;
+    struct sc_uhid_devices uhid_devices;
 #endif
     union {
-        struct sc_keyboard_inject keyboard_inject;
+        struct sc_keyboard_sdk keyboard_sdk;
+        struct sc_keyboard_uhid keyboard_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_keyboard keyboard_hid;
+        struct sc_keyboard_aoa keyboard_aoa;
 #endif
     };
     union {
-        struct sc_mouse_inject mouse_inject;
+        struct sc_mouse_sdk mouse_sdk;
+        struct sc_mouse_uhid mouse_uhid;
 #ifdef HAVE_USB
-        struct sc_hid_mouse mouse_hid;
+        struct sc_mouse_aoa mouse_aoa;
 #endif
     };
+    struct sc_timeout timeout;
 };
 
 static inline void
@@ -80,7 +95,7 @@ push_event(uint32_t type, const char *name) {
 #define PUSH_EVENT(TYPE) push_event(TYPE, # TYPE)
 
 #ifdef _WIN32
-BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
+static BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
     if (ctrl_type == CTRL_C_EVENT) {
         PUSH_EVENT(SDL_QUIT);
         return TRUE;
@@ -91,7 +106,6 @@ BOOL WINAPI windows_ctrl_handler(DWORD ctrl_type) {
 
 static void
 sdl_set_hints(const char *render_driver) {
-
     if (render_driver && !SDL_SetHint(SDL_HINT_RENDER_DRIVER, render_driver)) {
         LOGW("Could not set render driver");
     }
@@ -129,7 +143,7 @@ sdl_set_hints(const char *render_driver) {
 }
 
 static void
-sdl_configure(bool display, bool disable_screensaver) {
+sdl_configure(bool video_playback, bool disable_screensaver) {
 #ifdef _WIN32
     // Clean up properly on Ctrl+C on Windows
     bool ok = SetConsoleCtrlHandler(windows_ctrl_handler, TRUE);
@@ -138,7 +152,7 @@ sdl_configure(bool display, bool disable_screensaver) {
     }
 #endif // _WIN32
 
-    if (!display) {
+    if (!video_playback) {
         return;
     }
 
@@ -154,14 +168,25 @@ event_loop(struct scrcpy *s) {
     SDL_Event event;
     while (SDL_WaitEvent(&event)) {
         switch (event.type) {
-            case EVENT_STREAM_STOPPED:
+            case SC_EVENT_DEVICE_DISCONNECTED:
                 LOGW("Device disconnected");
                 return SCRCPY_EXIT_DISCONNECTED;
+            case SC_EVENT_DEMUXER_ERROR:
+                LOGE("Demuxer error");
+                return SCRCPY_EXIT_FAILURE;
+            case SC_EVENT_RECORDER_ERROR:
+                LOGE("Recorder error");
+                return SCRCPY_EXIT_FAILURE;
+            case SC_EVENT_TIME_LIMIT_REACHED:
+                LOGI("Time limit reached");
+                return SCRCPY_EXIT_SUCCESS;
             case SDL_QUIT:
                 LOGD("User requested to quit");
                 return SCRCPY_EXIT_SUCCESS;
             default:
-                sc_screen_handle_event(&s->screen, &event);
+                if (!sc_screen_handle_event(&s->screen, &event)) {
+                    return SCRCPY_EXIT_FAILURE;
+                }
                 break;
         }
     }
@@ -175,15 +200,16 @@ await_for_server(bool *connected) {
     while (SDL_WaitEvent(&event)) {
         switch (event.type) {
             case SDL_QUIT:
-                LOGD("User requested to quit");
-                *connected = false;
+                if (connected) {
+                    *connected = false;
+                }
                 return true;
-            case EVENT_SERVER_CONNECTION_FAILED:
-                LOGE("Server connection failed");
+            case SC_EVENT_SERVER_CONNECTION_FAILED:
                 return false;
-            case EVENT_SERVER_CONNECTED:
-                LOGD("Server connected");
-                *connected = true;
+            case SC_EVENT_SERVER_CONNECTED:
+                if (connected) {
+                    *connected = true;
+                }
                 return true;
             default:
                 break;
@@ -194,49 +220,49 @@ await_for_server(bool *connected) {
     return false;
 }
 
-static SDL_LogPriority
-sdl_priority_from_av_level(int level) {
-    switch (level) {
-        case AV_LOG_PANIC:
-        case AV_LOG_FATAL:
-            return SDL_LOG_PRIORITY_CRITICAL;
-        case AV_LOG_ERROR:
-            return SDL_LOG_PRIORITY_ERROR;
-        case AV_LOG_WARNING:
-            return SDL_LOG_PRIORITY_WARN;
-        case AV_LOG_INFO:
-            return SDL_LOG_PRIORITY_INFO;
-    }
-    // do not forward others, which are too verbose
-    return 0;
+static void
+sc_recorder_on_ended(struct sc_recorder *recorder, bool success,
+                     void *userdata) {
+    (void) recorder;
+    (void) userdata;
+
+    if (!success) {
+        PUSH_EVENT(SC_EVENT_RECORDER_ERROR);
+    }
 }
 
 static void
-av_log_callback(void *avcl, int level, const char *fmt, va_list vl) {
-    (void) avcl;
-    SDL_LogPriority priority = sdl_priority_from_av_level(level);
-    if (priority == 0) {
-        return;
-    }
+sc_video_demuxer_on_ended(struct sc_demuxer *demuxer,
+                          enum sc_demuxer_status status, void *userdata) {
+    (void) demuxer;
+    (void) userdata;
 
-    size_t fmt_len = strlen(fmt);
-    char *local_fmt = malloc(fmt_len + 10);
-    if (!local_fmt) {
-        LOG_OOM();
-        return;
+    // The device may not decide to disable the video
+    assert(status != SC_DEMUXER_STATUS_DISABLED);
+
+    if (status == SC_DEMUXER_STATUS_EOS) {
+        PUSH_EVENT(SC_EVENT_DEVICE_DISCONNECTED);
+    } else {
+        PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
     }
-    memcpy(local_fmt, "[FFmpeg] ", 9); // do not write the final '\0'
-    memcpy(local_fmt + 9, fmt, fmt_len + 1); // include '\0'
-    SDL_LogMessageV(SDL_LOG_CATEGORY_VIDEO, priority, local_fmt, vl);
-    free(local_fmt);
 }
 
 static void
-sc_demuxer_on_eos(struct sc_demuxer *demuxer, void *userdata) {
+sc_audio_demuxer_on_ended(struct sc_demuxer *demuxer,
+                          enum sc_demuxer_status status, void *userdata) {
     (void) demuxer;
-    (void) userdata;
 
-    PUSH_EVENT(EVENT_STREAM_STOPPED);
+    const struct scrcpy_options *options = userdata;
+
+    // Contrary to the video demuxer, keep mirroring if only the audio fails
+    // (unless --require-audio is set).
+    if (status == SC_DEMUXER_STATUS_EOS) {
+        PUSH_EVENT(SC_EVENT_DEVICE_DISCONNECTED);
+    } else if (status == SC_DEMUXER_STATUS_ERROR
+            || (status == SC_DEMUXER_STATUS_DISABLED
+                && options->require_audio)) {
+        PUSH_EVENT(SC_EVENT_DEMUXER_ERROR);
+    }
 }
 
 static void
@@ -244,7 +270,7 @@ sc_server_on_connection_failed(struct sc_server *server, void *userdata) {
     (void) server;
     (void) userdata;
 
-    PUSH_EVENT(EVENT_SERVER_CONNECTION_FAILED);
+    PUSH_EVENT(SC_EVENT_SERVER_CONNECTION_FAILED);
 }
 
 static void
@@ -252,7 +278,7 @@ sc_server_on_connected(struct sc_server *server, void *userdata) {
     (void) server;
     (void) userdata;
 
-    PUSH_EVENT(EVENT_SERVER_CONNECTED);
+    PUSH_EVENT(SC_EVENT_SERVER_CONNECTED);
 }
 
 static void
@@ -265,9 +291,30 @@ sc_server_on_disconnected(struct sc_server *server, void *userdata) {
     // event
 }
 
+static void
+sc_timeout_on_timeout(struct sc_timeout *timeout, void *userdata) {
+    (void) timeout;
+    (void) userdata;
+
+    PUSH_EVENT(SC_EVENT_TIME_LIMIT_REACHED);
+}
+
+// Generate a scrcpy id to differentiate multiple running scrcpy instances
+static uint32_t
+scrcpy_generate_scid(void) {
+    struct sc_rand rand;
+    sc_rand_init(&rand);
+    // Only use 31 bits to avoid issues with signed values on the Java-side
+    return sc_rand_u32(&rand) & 0x7FFFFFFF;
+}
+
 enum scrcpy_exit_code
 scrcpy(struct scrcpy_options *options) {
     static struct scrcpy scrcpy;
+#ifndef NDEBUG
+    // Detect missing initializations
+    memset(&scrcpy, 42, sizeof(scrcpy));
+#endif
     struct scrcpy *s = &scrcpy;
 
     // Minimal SDL initialization
@@ -283,40 +330,62 @@ scrcpy(struct scrcpy_options *options) {
     bool server_started = false;
     bool file_pusher_initialized = false;
     bool recorder_initialized = false;
+    bool recorder_started = false;
 #ifdef HAVE_V4L2
     bool v4l2_sink_initialized = false;
 #endif
-    bool demuxer_started = false;
+    bool video_demuxer_started = false;
+    bool audio_demuxer_started = false;
 #ifdef HAVE_USB
     bool aoa_hid_initialized = false;
-    bool hid_keyboard_initialized = false;
-    bool hid_mouse_initialized = false;
+    bool keyboard_aoa_initialized = false;
+    bool mouse_aoa_initialized = false;
 #endif
     bool controller_initialized = false;
     bool controller_started = false;
     bool screen_initialized = false;
+    bool timeout_initialized = false;
+    bool timeout_started = false;
 
     struct sc_acksync *acksync = NULL;
+    struct sc_uhid_devices *uhid_devices = NULL;
+
+    uint32_t scid = scrcpy_generate_scid();
 
     struct sc_server_params params = {
+        .scid = scid,
         .req_serial = options->serial,
         .select_usb = options->select_usb,
         .select_tcpip = options->select_tcpip,
         .log_level = options->log_level,
+        .video_codec = options->video_codec,
+        .audio_codec = options->audio_codec,
+        .video_source = options->video_source,
+        .audio_source = options->audio_source,
+        .camera_facing = options->camera_facing,
         .crop = options->crop,
         .port_range = options->port_range,
         .tunnel_host = options->tunnel_host,
         .tunnel_port = options->tunnel_port,
         .max_size = options->max_size,
-        .bit_rate = options->bit_rate,
+        .video_bit_rate = options->video_bit_rate,
+        .audio_bit_rate = options->audio_bit_rate,
         .max_fps = options->max_fps,
         .lock_video_orientation = options->lock_video_orientation,
         .control = options->control,
         .display_id = options->display_id,
+        .video = options->video,
+        .audio = options->audio,
         .show_touches = options->show_touches,
         .stay_awake = options->stay_awake,
-        .codec_options = options->codec_options,
-        .encoder_name = options->encoder_name,
+        .video_codec_options = options->video_codec_options,
+        .audio_codec_options = options->audio_codec_options,
+        .video_encoder = options->video_encoder,
+        .audio_encoder = options->audio_encoder,
+        .camera_id = options->camera_id,
+        .camera_size = options->camera_size,
+        .camera_ar = options->camera_ar,
+        .camera_fps = options->camera_fps,
         .force_adb_forward = options->force_adb_forward,
         .power_off_on_close = options->power_off_on_close,
         .clipboard_autosync = options->clipboard_autosync,
@@ -325,6 +394,9 @@ scrcpy(struct scrcpy_options *options) {
         .tcpip_dst = options->tcpip_dst,
         .cleanup = options->cleanup,
         .power_on = options->power_on,
+        .kill_adb_on_close = options->kill_adb_on_close,
+        .camera_high_speed = options->camera_high_speed,
+        .list = options->list,
     };
 
     static const struct sc_server_callbacks cbs = {
@@ -336,36 +408,70 @@ scrcpy(struct scrcpy_options *options) {
         return SCRCPY_EXIT_FAILURE;
     }
 
+    if (options->video_playback) {
+        // Set hints before starting the server thread to avoid race conditions
+        // in SDL
+        sdl_set_hints(options->render_driver);
+    }
+
     if (!sc_server_start(&s->server)) {
         goto end;
     }
 
     server_started = true;
 
-    if (options->display) {
-        sdl_set_hints(options->render_driver);
+    if (options->list) {
+        bool ok = await_for_server(NULL);
+        ret = ok ? SCRCPY_EXIT_SUCCESS : SCRCPY_EXIT_FAILURE;
+        goto end;
     }
 
-    // Initialize SDL video in addition if display is enabled
-    if (options->display && SDL_Init(SDL_INIT_VIDEO)) {
-        LOGE("Could not initialize SDL: %s", SDL_GetError());
-        goto end;
+    // playback implies capture
+    assert(!options->video_playback || options->video);
+    assert(!options->audio_playback || options->audio);
+
+    if (options->video_playback ||
+            (options->control && options->clipboard_autosync)) {
+        // Initialize the video subsystem even if --no-video or
+        // --no-video-playback is passed so that clipboard synchronization
+        // still works.
+        // <https://github.com/Genymobile/scrcpy/issues/4418>
+        if (SDL_Init(SDL_INIT_VIDEO)) {
+            // If it fails, it is an error only if video playback is enabled
+            if (options->video_playback) {
+                LOGE("Could not initialize SDL video: %s", SDL_GetError());
+                goto end;
+            } else {
+                LOGW("Could not initialize SDL video: %s", SDL_GetError());
+            }
+        }
     }
 
-    sdl_configure(options->display, options->disable_screensaver);
+    if (options->audio_playback) {
+        if (SDL_Init(SDL_INIT_AUDIO)) {
+            LOGE("Could not initialize SDL audio: %s", SDL_GetError());
+            goto end;
+        }
+    }
+
+    sdl_configure(options->video_playback, options->disable_screensaver);
 
     // Await for server without blocking Ctrl+C handling
     bool connected;
     if (!await_for_server(&connected)) {
+        LOGE("Server connection failed");
         goto end;
     }
 
     if (!connected) {
         // This is not an error, user requested to quit
+        LOGD("User requested to quit");
         ret = SCRCPY_EXIT_SUCCESS;
         goto end;
     }
 
+    LOGD("Server connected");
+
     // It is necessarily initialized here, since the device is connected
     struct sc_server_info *info = &s->server.info;
 
@@ -374,7 +480,7 @@ scrcpy(struct scrcpy_options *options) {
 
     struct sc_file_pusher *fp = NULL;
 
-    if (options->display && options->control) {
+    if (options->video_playback && options->control) {
         if (!sc_file_pusher_init(&s->file_pusher, serial,
                                  options->push_target)) {
             goto end;
@@ -383,41 +489,63 @@ scrcpy(struct scrcpy_options *options) {
         file_pusher_initialized = true;
     }
 
-    struct sc_decoder *dec = NULL;
-    bool needs_decoder = options->display;
+    if (options->video) {
+        static const struct sc_demuxer_callbacks video_demuxer_cbs = {
+            .on_ended = sc_video_demuxer_on_ended,
+        };
+        sc_demuxer_init(&s->video_demuxer, "video", s->server.video_socket,
+                        &video_demuxer_cbs, NULL);
+    }
+
+    if (options->audio) {
+        static const struct sc_demuxer_callbacks audio_demuxer_cbs = {
+            .on_ended = sc_audio_demuxer_on_ended,
+        };
+        sc_demuxer_init(&s->audio_demuxer, "audio", s->server.audio_socket,
+                        &audio_demuxer_cbs, options);
+    }
+
+    bool needs_video_decoder = options->video_playback;
+    bool needs_audio_decoder = options->audio_playback;
 #ifdef HAVE_V4L2
-    needs_decoder |= !!options->v4l2_device;
+    needs_video_decoder |= !!options->v4l2_device;
 #endif
-    if (needs_decoder) {
-        sc_decoder_init(&s->decoder);
-        dec = &s->decoder;
+    if (needs_video_decoder) {
+        sc_decoder_init(&s->video_decoder, "video");
+        sc_packet_source_add_sink(&s->video_demuxer.packet_source,
+                                  &s->video_decoder.packet_sink);
+    }
+    if (needs_audio_decoder) {
+        sc_decoder_init(&s->audio_decoder, "audio");
+        sc_packet_source_add_sink(&s->audio_demuxer.packet_source,
+                                  &s->audio_decoder.packet_sink);
     }
 
-    struct sc_recorder *rec = NULL;
     if (options->record_filename) {
-        if (!sc_recorder_init(&s->recorder,
-                              options->record_filename,
-                              options->record_format,
-                              info->frame_size)) {
+        static const struct sc_recorder_callbacks recorder_cbs = {
+            .on_ended = sc_recorder_on_ended,
+        };
+        if (!sc_recorder_init(&s->recorder, options->record_filename,
+                              options->record_format, options->video,
+                              options->audio, options->record_orientation,
+                              &recorder_cbs, NULL)) {
             goto end;
         }
-        rec = &s->recorder;
         recorder_initialized = true;
-    }
-
-    av_log_set_callback(av_log_callback);
 
-    static const struct sc_demuxer_callbacks demuxer_cbs = {
-        .on_eos = sc_demuxer_on_eos,
-    };
-    sc_demuxer_init(&s->demuxer, s->server.video_socket, &demuxer_cbs, NULL);
-
-    if (dec) {
-        sc_demuxer_add_sink(&s->demuxer, &dec->packet_sink);
-    }
+        if (!sc_recorder_start(&s->recorder)) {
+            goto end;
+        }
+        recorder_started = true;
 
-    if (rec) {
-        sc_demuxer_add_sink(&s->demuxer, &rec->packet_sink);
+        if (options->video) {
+            sc_packet_source_add_sink(&s->video_demuxer.packet_source,
+                                      &s->recorder.video_packet_sink);
+        }
+        if (options->audio) {
+            sc_packet_source_add_sink(&s->audio_demuxer.packet_source,
+                                      &s->recorder.audio_packet_sink);
+        }
     }
 
     struct sc_controller *controller = NULL;
@@ -425,12 +553,19 @@ scrcpy(struct scrcpy_options *options) {
     struct sc_mouse_processor *mp = NULL;
 
     if (options->control) {
+        if (!sc_controller_init(&s->controller, s->server.control_socket)) {
+            goto end;
+        }
+        controller_initialized = true;
+
+        controller = &s->controller;
+
 #ifdef HAVE_USB
-        bool use_hid_keyboard =
-            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_HID;
-        bool use_hid_mouse =
-            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_HID;
-        if (use_hid_keyboard || use_hid_mouse) {
+        bool use_keyboard_aoa =
+            options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_AOA;
+        bool use_mouse_aoa =
+            options->mouse_input_mode == SC_MOUSE_INPUT_MODE_AOA;
+        if (use_keyboard_aoa || use_mouse_aoa) {
             bool ok = sc_acksync_init(&s->acksync);
             if (!ok) {
                 goto end;
@@ -440,7 +575,7 @@ scrcpy(struct scrcpy_options *options) {
             if (!ok) {
                 LOGE("Failed to initialize USB");
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             assert(serial);
@@ -448,7 +583,7 @@ scrcpy(struct scrcpy_options *options) {
             ok = sc_usb_select_device(&s->usb, serial, &usb_device);
             if (!ok) {
                 sc_usb_destroy(&s->usb);
-                goto aoa_hid_end;
+                goto end;
             }
 
             LOGI("USB device: %s (%04" PRIx16 ":%04" PRIx16 ") %s %s",
@@ -461,7 +596,7 @@ scrcpy(struct scrcpy_options *options) {
                 LOGE("Failed to connect to USB device %s", serial);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
             ok = sc_aoa_init(&s->aoa, &s->usb, &s->acksync);
@@ -470,112 +605,86 @@ scrcpy(struct scrcpy_options *options) {
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_acksync_destroy(&s->acksync);
-                goto aoa_hid_end;
+                goto end;
             }
 
-            if (use_hid_keyboard) {
-                if (sc_hid_keyboard_init(&s->keyboard_hid, &s->aoa)) {
-                    hid_keyboard_initialized = true;
-                    kp = &s->keyboard_hid.key_processor;
+            if (use_keyboard_aoa) {
+                if (sc_keyboard_aoa_init(&s->keyboard_aoa, &s->aoa)) {
+                    keyboard_aoa_initialized = true;
+                    kp = &s->keyboard_aoa.key_processor;
                 } else {
                     LOGE("Could not initialize HID keyboard");
                 }
             }
 
-            if (use_hid_mouse) {
-                if (sc_hid_mouse_init(&s->mouse_hid, &s->aoa)) {
-                    hid_mouse_initialized = true;
-                    mp = &s->mouse_hid.mouse_processor;
+            if (use_mouse_aoa) {
+                if (sc_mouse_aoa_init(&s->mouse_aoa, &s->aoa)) {
+                    mouse_aoa_initialized = true;
+                    mp = &s->mouse_aoa.mouse_processor;
                 } else {
                     LOGE("Could not initialized HID mouse");
                 }
             }
 
-            bool need_aoa = hid_keyboard_initialized || hid_mouse_initialized;
+            bool need_aoa = keyboard_aoa_initialized || mouse_aoa_initialized;
 
             if (!need_aoa || !sc_aoa_start(&s->aoa)) {
                 sc_acksync_destroy(&s->acksync);
                 sc_usb_disconnect(&s->usb);
                 sc_usb_destroy(&s->usb);
                 sc_aoa_destroy(&s->aoa);
-                goto aoa_hid_end;
+                goto end;
             }
 
             acksync = &s->acksync;
 
             aoa_hid_initialized = true;
-
-aoa_hid_end:
-            if (!aoa_hid_initialized) {
-                if (hid_keyboard_initialized) {
-                    sc_hid_keyboard_destroy(&s->keyboard_hid);
-                    hid_keyboard_initialized = false;
-                }
-                if (hid_mouse_initialized) {
-                    sc_hid_mouse_destroy(&s->mouse_hid);
-                    hid_mouse_initialized = false;
-                }
-            }
-
-            if (use_hid_keyboard && !hid_keyboard_initialized) {
-                LOGE("Fallback to default keyboard injection method "
-                     "(-K/--hid-keyboard ignored)");
-                options->keyboard_input_mode = SC_KEYBOARD_INPUT_MODE_INJECT;
-            }
-
-            if (use_hid_mouse && !hid_mouse_initialized) {
-                LOGE("Fallback to default mouse injection method "
-                     "(-M/--hid-mouse ignored)");
-                options->mouse_input_mode = SC_MOUSE_INPUT_MODE_INJECT;
-            }
         }
 #else
-        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_HID);
-        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_HID);
+        assert(options->keyboard_input_mode != SC_KEYBOARD_INPUT_MODE_AOA);
+        assert(options->mouse_input_mode != SC_MOUSE_INPUT_MODE_AOA);
 #endif
 
-        // keyboard_input_mode may have been reset if HID mode failed
-        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_INJECT) {
-            sc_keyboard_inject_init(&s->keyboard_inject, &s->controller,
-                                    options->key_inject_mode,
-                                    options->forward_key_repeat);
-            kp = &s->keyboard_inject.key_processor;
+        if (options->keyboard_input_mode == SC_KEYBOARD_INPUT_MODE_SDK) {
+            sc_keyboard_sdk_init(&s->keyboard_sdk, &s->controller,
+                                 options->key_inject_mode,
+                                 options->forward_key_repeat);
+            kp = &s->keyboard_sdk.key_processor;
+        } else if (options->keyboard_input_mode
+                == SC_KEYBOARD_INPUT_MODE_UHID) {
+            sc_uhid_devices_init(&s->uhid_devices);
+            bool ok = sc_keyboard_uhid_init(&s->keyboard_uhid, &s->controller,
+                                            &s->uhid_devices);
+            if (!ok) {
+                goto end;
+            }
+            uhid_devices = &s->uhid_devices;
+            kp = &s->keyboard_uhid.key_processor;
         }
 
-        // mouse_input_mode may have been reset if HID mode failed
-        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_INJECT) {
-            sc_mouse_inject_init(&s->mouse_inject, &s->controller);
-            mp = &s->mouse_inject.mouse_processor;
+        if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_SDK) {
+            sc_mouse_sdk_init(&s->mouse_sdk, &s->controller);
+            mp = &s->mouse_sdk.mouse_processor;
+        } else if (options->mouse_input_mode == SC_MOUSE_INPUT_MODE_UHID) {
+            bool ok = sc_mouse_uhid_init(&s->mouse_uhid, &s->controller);
+            if (!ok) {
+                goto end;
+            }
+            mp = &s->mouse_uhid.mouse_processor;
         }
 
-        if (!sc_controller_init(&s->controller, s->server.control_socket,
-                                acksync)) {
-            goto end;
-        }
-        controller_initialized = true;
+        sc_controller_configure(&s->controller, acksync, uhid_devices);
 
         if (!sc_controller_start(&s->controller)) {
             goto end;
         }
         controller_started = true;
-        controller = &s->controller;
-
-        if (options->turn_screen_off) {
-            struct sc_control_msg msg;
-            msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
-            msg.set_screen_power_mode.mode = SC_SCREEN_POWER_MODE_OFF;
-
-            if (!sc_controller_push_msg(&s->controller, &msg)) {
-                LOGW("Could not request 'set screen power mode'");
-            }
-        }
-
     }
 
     // There is a controller if and only if control is enabled
     assert(options->control == !!controller);
 
-    if (options->display) {
+    if (options->video_playback) {
         const char *window_title =
             options->window_title ? options->window_title : info->device_name;
 
@@ -589,65 +698,131 @@ aoa_hid_end:
             .clipboard_autosync = options->clipboard_autosync,
             .shortcut_mods = &options->shortcut_mods,
             .window_title = window_title,
-            .frame_size = info->frame_size,
             .always_on_top = options->always_on_top,
             .window_x = options->window_x,
             .window_y = options->window_y,
             .window_width = options->window_width,
             .window_height = options->window_height,
             .window_borderless = options->window_borderless,
-            .rotation = options->rotation,
+            .orientation = options->display_orientation,
             .mipmaps = options->mipmaps,
             .fullscreen = options->fullscreen,
             .start_fps_counter = options->start_fps_counter,
-            .buffering_time = options->display_buffer,
         };
 
+        struct sc_frame_source *src = &s->video_decoder.frame_source;
+        if (options->display_buffer) {
+            sc_delay_buffer_init(&s->display_buffer, options->display_buffer,
+                                 true);
+            sc_frame_source_add_sink(src, &s->display_buffer.frame_sink);
+            src = &s->display_buffer.frame_source;
+        }
+
         if (!sc_screen_init(&s->screen, &screen_params)) {
             goto end;
         }
         screen_initialized = true;
 
-        sc_decoder_add_sink(&s->decoder, &s->screen.frame_sink);
+        sc_frame_source_add_sink(src, &s->screen.frame_sink);
+    }
+
+    if (options->audio_playback) {
+        sc_audio_player_init(&s->audio_player, options->audio_buffer,
+                             options->audio_output_buffer);
+        sc_frame_source_add_sink(&s->audio_decoder.frame_source,
+                                 &s->audio_player.frame_sink);
     }
 
 #ifdef HAVE_V4L2
     if (options->v4l2_device) {
-        if (!sc_v4l2_sink_init(&s->v4l2_sink, options->v4l2_device,
-                               info->frame_size, options->v4l2_buffer)) {
+        if (!sc_v4l2_sink_init(&s->v4l2_sink, options->v4l2_device)) {
             goto end;
         }
 
-        sc_decoder_add_sink(&s->decoder, &s->v4l2_sink.frame_sink);
+        struct sc_frame_source *src = &s->video_decoder.frame_source;
+        if (options->v4l2_buffer) {
+            sc_delay_buffer_init(&s->v4l2_buffer, options->v4l2_buffer, true);
+            sc_frame_source_add_sink(src, &s->v4l2_buffer.frame_sink);
+            src = &s->v4l2_buffer.frame_source;
+        }
+
+        sc_frame_source_add_sink(src, &s->v4l2_sink.frame_sink);
 
         v4l2_sink_initialized = true;
     }
 #endif
 
-    // now we consumed the header values, the socket receives the video stream
-    // start the demuxer
-    if (!sc_demuxer_start(&s->demuxer)) {
-        goto end;
+    // Now that the header values have been consumed, the socket(s) will
+    // receive the stream(s). Start the demuxer(s).
+
+    if (options->video) {
+        if (!sc_demuxer_start(&s->video_demuxer)) {
+            goto end;
+        }
+        video_demuxer_started = true;
+    }
+
+    if (options->audio) {
+        if (!sc_demuxer_start(&s->audio_demuxer)) {
+            goto end;
+        }
+        audio_demuxer_started = true;
+    }
+
+    // If the device screen is to be turned off, send the control message after
+    // everything is set up
+    if (options->control && options->turn_screen_off) {
+        struct sc_control_msg msg;
+        msg.type = SC_CONTROL_MSG_TYPE_SET_SCREEN_POWER_MODE;
+        msg.set_screen_power_mode.mode = SC_SCREEN_POWER_MODE_OFF;
+
+        if (!sc_controller_push_msg(&s->controller, &msg)) {
+            LOGW("Could not request 'set screen power mode'");
+        }
+    }
+
+    if (options->time_limit) {
+        bool ok = sc_timeout_init(&s->timeout);
+        if (!ok) {
+            goto end;
+        }
+
+        timeout_initialized = true;
+
+        sc_tick deadline = sc_tick_now() + options->time_limit;
+        static const struct sc_timeout_callbacks cbs = {
+            .on_timeout = sc_timeout_on_timeout,
+        };
+
+        ok = sc_timeout_start(&s->timeout, deadline, &cbs, NULL);
+        if (!ok) {
+            goto end;
+        }
+
+        timeout_started = true;
     }
-    demuxer_started = true;
 
     ret = event_loop(s);
     LOGD("quit...");
 
     // Close the window immediately on closing, because screen_destroy() may
-    // only be called once the demuxer thread is joined (it may take time)
+    // only be called once the video demuxer thread is joined (it may take time)
     sc_screen_hide_window(&s->screen);
 
 end:
+    if (timeout_started) {
+        sc_timeout_stop(&s->timeout);
+    }
+
     // The demuxer is not stopped explicitly, because it will stop by itself on
     // end-of-stream
 #ifdef HAVE_USB
     if (aoa_hid_initialized) {
-        if (hid_keyboard_initialized) {
-            sc_hid_keyboard_destroy(&s->keyboard_hid);
+        if (keyboard_aoa_initialized) {
+            sc_keyboard_aoa_destroy(&s->keyboard_aoa);
         }
-        if (hid_mouse_initialized) {
-            sc_hid_mouse_destroy(&s->mouse_hid);
+        if (mouse_aoa_initialized) {
+            sc_mouse_aoa_destroy(&s->mouse_aoa);
         }
         sc_aoa_stop(&s->aoa);
         sc_usb_stop(&s->usb);
@@ -662,6 +837,9 @@ end:
     if (file_pusher_initialized) {
         sc_file_pusher_stop(&s->file_pusher);
     }
+    if (recorder_initialized) {
+        sc_recorder_stop(&s->recorder);
+    }
     if (screen_initialized) {
         sc_screen_interrupt(&s->screen);
     }
@@ -671,10 +849,21 @@ end:
         sc_server_stop(&s->server);
     }
 
+    if (timeout_started) {
+        sc_timeout_join(&s->timeout);
+    }
+    if (timeout_initialized) {
+        sc_timeout_destroy(&s->timeout);
+    }
+
     // now that the sockets are shutdown, the demuxer and controller are
     // interrupted, we can join them
-    if (demuxer_started) {
-        sc_demuxer_join(&s->demuxer);
+    if (video_demuxer_started) {
+        sc_demuxer_join(&s->video_demuxer);
+    }
+
+    if (audio_demuxer_started) {
+        sc_demuxer_join(&s->audio_demuxer);
     }
 
 #ifdef HAVE_V4L2
@@ -693,8 +882,9 @@ end:
     }
 #endif
 
-    // Destroy the screen only after the demuxer is guaranteed to be finished,
-    // because otherwise the screen could receive new frames after destruction
+    // Destroy the screen only after the video demuxer is guaranteed to be
+    // finished, because otherwise the screen could receive new frames after
+    // destruction
     if (screen_initialized) {
         sc_screen_join(&s->screen);
         sc_screen_destroy(&s->screen);
@@ -707,6 +897,9 @@ end:
         sc_controller_destroy(&s->controller);
     }
 
+    if (recorder_started) {
+        sc_recorder_join(&s->recorder);
+    }
     if (recorder_initialized) {
         sc_recorder_destroy(&s->recorder);
     }
@@ -716,6 +909,10 @@ end:
         sc_file_pusher_destroy(&s->file_pusher);
     }
 
+    if (server_started) {
+        sc_server_join(&s->server);
+    }
+
     sc_server_destroy(&s->server);
 
     return ret;

app/src/screen.c

@@ -7,7 +7,6 @@
 #include "events.h"
 #include "icon.h"
 #include "options.h"
-#include "video_buffer.h"
 #include "util/log.h"
 
 #define DISPLAY_MARGINS 96
@@ -15,16 +14,16 @@
 #define DOWNCAST(SINK) container_of(SINK, struct sc_screen, frame_sink)
 
 static inline struct sc_size
-get_rotated_size(struct sc_size size, int rotation) {
-    struct sc_size rotated_size;
-    if (rotation & 1) {
-        rotated_size.width = size.height;
-        rotated_size.height = size.width;
+get_oriented_size(struct sc_size size, enum sc_orientation orientation) {
+    struct sc_size oriented_size;
+    if (sc_orientation_is_swap(orientation)) {
+        oriented_size.width = size.height;
+        oriented_size.height = size.width;
     } else {
-        rotated_size.width = size.width;
-        rotated_size.height = size.height;
+        oriented_size.width = size.width;
+        oriented_size.height = size.height;
     }
-    return rotated_size;
+    return oriented_size;
 }
 
 // get the window size in a struct sc_size
@@ -57,6 +56,7 @@ static void
 set_window_size(struct sc_screen *screen, struct sc_size new_size) {
     assert(!screen->fullscreen);
     assert(!screen->maximized);
+    assert(!screen->minimized);
     SDL_SetWindowSize(screen->window, new_size.width, new_size.height);
 }
 
@@ -240,33 +240,6 @@ sc_screen_update_content_rect(struct sc_screen *screen) {
     }
 }
 
-static inline SDL_Texture *
-create_texture(struct sc_screen *screen) {
-    SDL_Renderer *renderer = screen->renderer;
-    struct sc_size size = screen->frame_size;
-    SDL_Texture *texture = SDL_CreateTexture(renderer, SDL_PIXELFORMAT_YV12,
-                                             SDL_TEXTUREACCESS_STREAMING,
-                                             size.width, size.height);
-    if (!texture) {
-        return NULL;
-    }
-
-    if (screen->mipmaps) {
-        struct sc_opengl *gl = &screen->gl;
-
-        SDL_GL_BindTexture(texture, NULL, NULL);
-
-        // Enable trilinear filtering for downscaling
-        gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
-                          GL_LINEAR_MIPMAP_LINEAR);
-        gl->TexParameterf(GL_TEXTURE_2D, GL_TEXTURE_LOD_BIAS, -1.f);
-
-        SDL_GL_UnbindTexture(texture);
-    }
-
-    return texture;
-}
-
 // render the texture to the renderer
 //
 // Set the update_content_rect flag if the window or content size may have
@@ -277,43 +250,18 @@ sc_screen_render(struct sc_screen *screen, bool update_content_rect) {
         sc_screen_update_content_rect(screen);
     }
 
-    SDL_RenderClear(screen->renderer);
-    if (screen->rotation == 0) {
-        SDL_RenderCopy(screen->renderer, screen->texture, NULL, &screen->rect);
-    } else {
-        // rotation in RenderCopyEx() is clockwise, while screen->rotation is
-        // counterclockwise (to be consistent with --lock-video-orientation)
-        int cw_rotation = (4 - screen->rotation) % 4;
-        double angle = 90 * cw_rotation;
-
-        SDL_Rect *dstrect = NULL;
-        SDL_Rect rect;
-        if (screen->rotation & 1) {
-            rect.x = screen->rect.x + (screen->rect.w - screen->rect.h) / 2;
-            rect.y = screen->rect.y + (screen->rect.h - screen->rect.w) / 2;
-            rect.w = screen->rect.h;
-            rect.h = screen->rect.w;
-            dstrect = ▭
-        } else {
-            assert(screen->rotation == 2);
-            dstrect = &screen->rect;
-        }
-
-        SDL_RenderCopyEx(screen->renderer, screen->texture, NULL, dstrect,
-                         angle, NULL, 0);
-    }
-    SDL_RenderPresent(screen->renderer);
+    enum sc_display_result res =
+        sc_display_render(&screen->display, &screen->rect, screen->orientation);
+    (void) res; // any error already logged
 }
 
-
-#if defined(__APPLE__)
+#if defined(__APPLE__) || defined(__WINDOWS__)
 # define CONTINUOUS_RESIZING_WORKAROUND
 #endif
 
 #ifdef CONTINUOUS_RESIZING_WORKAROUND
 // On Windows and MacOS, resizing blocks the event loop, so resizing events are
-// not triggered. On MacOS, as a workaround, handle them in an event handler
-// (it does not work for Windows unfortunately).
+// not triggered. As a workaround, handle them in an event handler.
 //
 // <https://bugzilla.libsdl.org/show_bug.cgi?id=2077>
 // <https://stackoverflow.com/a/40693139/1987178>
@@ -331,9 +279,31 @@ event_watcher(void *data, SDL_Event *event) {
 #endif
 
 static bool
-sc_screen_frame_sink_open(struct sc_frame_sink *sink) {
+sc_screen_frame_sink_open(struct sc_frame_sink *sink,
+                          const AVCodecContext *ctx) {
+    assert(ctx->pix_fmt == AV_PIX_FMT_YUV420P);
+    (void) ctx;
+
     struct sc_screen *screen = DOWNCAST(sink);
-    (void) screen;
+
+    assert(ctx->width > 0 && ctx->width <= 0xFFFF);
+    assert(ctx->height > 0 && ctx->height <= 0xFFFF);
+    // screen->frame_size is never used before the event is pushed, and the
+    // event acts as a memory barrier so it is safe without mutex
+    screen->frame_size.width = ctx->width;
+    screen->frame_size.height = ctx->height;
+
+    static SDL_Event event = {
+        .type = SC_EVENT_SCREEN_INIT_SIZE,
+    };
+
+    // Post the event on the UI thread (the texture must be created from there)
+    int ret = SDL_PushEvent(&event);
+    if (ret < 0) {
+        LOGW("Could not post init size event: %s", SDL_GetError());
+        return false;
+    }
+
 #ifndef NDEBUG
     screen->open = true;
 #endif
@@ -356,43 +326,31 @@ sc_screen_frame_sink_close(struct sc_frame_sink *sink) {
 static bool
 sc_screen_frame_sink_push(struct sc_frame_sink *sink, const AVFrame *frame) {
     struct sc_screen *screen = DOWNCAST(sink);
-    return sc_video_buffer_push(&screen->vb, frame);
-}
 
-static void
-sc_video_buffer_on_new_frame(struct sc_video_buffer *vb, bool previous_skipped,
-                             void *userdata) {
-    (void) vb;
-    struct sc_screen *screen = userdata;
-
-    // event_failed implies previous_skipped (the previous frame may not have
-    // been consumed if the event was not sent)
-    assert(!screen->event_failed || previous_skipped);
+    bool previous_skipped;
+    bool ok = sc_frame_buffer_push(&screen->fb, frame, &previous_skipped);
+    if (!ok) {
+        return false;
+    }
 
-    bool need_new_event;
     if (previous_skipped) {
         sc_fps_counter_add_skipped_frame(&screen->fps_counter);
-        // The EVENT_NEW_FRAME triggered for the previous frame will consume
-        // this new frame instead, unless the previous event failed
-        need_new_event = screen->event_failed;
+        // The SC_EVENT_NEW_FRAME triggered for the previous frame will consume
+        // this new frame instead
     } else {
-        need_new_event = true;
-    }
-
-    if (need_new_event) {
         static SDL_Event new_frame_event = {
-            .type = EVENT_NEW_FRAME,
+            .type = SC_EVENT_NEW_FRAME,
         };
 
         // Post the event on the UI thread
         int ret = SDL_PushEvent(&new_frame_event);
         if (ret < 0) {
             LOGW("Could not post new frame event: %s", SDL_GetError());
-            screen->event_failed = true;
-        } else {
-            screen->event_failed = false;
+            return false;
         }
     }
+
+    return true;
 }
 
 bool
@@ -402,7 +360,7 @@ sc_screen_init(struct sc_screen *screen,
     screen->has_frame = false;
     screen->fullscreen = false;
     screen->maximized = false;
-    screen->event_failed = false;
+    screen->minimized = false;
     screen->mouse_capture_key_pressed = 0;
 
     screen->req.x = params->window_x;
@@ -412,33 +370,20 @@ sc_screen_init(struct sc_screen *screen,
     screen->req.fullscreen = params->fullscreen;
     screen->req.start_fps_counter = params->start_fps_counter;
 
-    static const struct sc_video_buffer_callbacks cbs = {
-        .on_new_frame = sc_video_buffer_on_new_frame,
-    };
-
-    bool ok = sc_video_buffer_init(&screen->vb, params->buffering_time, &cbs,
-                                   screen);
+    bool ok = sc_frame_buffer_init(&screen->fb);
     if (!ok) {
         return false;
     }
 
-    ok = sc_video_buffer_start(&screen->vb);
-    if (!ok) {
-        goto error_destroy_video_buffer;
-    }
-
     if (!sc_fps_counter_init(&screen->fps_counter)) {
-        goto error_stop_and_join_video_buffer;
+        goto error_destroy_frame_buffer;
     }
 
-    screen->frame_size = params->frame_size;
-    screen->rotation = params->rotation;
-    if (screen->rotation) {
-        LOGI("Initial display rotation set to %u", screen->rotation);
+    screen->orientation = params->orientation;
+    if (screen->orientation != SC_ORIENTATION_0) {
+        LOGI("Initial display orientation set to %s",
+             sc_orientation_get_name(screen->orientation));
     }
-    struct sc_size content_size =
-        get_rotated_size(screen->frame_size, screen->rotation);
-    screen->content_size = content_size;
 
     uint32_t window_flags = SDL_WINDOW_HIDDEN
                           | SDL_WINDOW_RESIZABLE
@@ -458,46 +403,11 @@ sc_screen_init(struct sc_screen *screen,
         goto error_destroy_fps_counter;
     }
 
-    screen->renderer = SDL_CreateRenderer(screen->window, -1,
-                                          SDL_RENDERER_ACCELERATED);
-    if (!screen->renderer) {
-        LOGE("Could not create renderer: %s", SDL_GetError());
+    ok = sc_display_init(&screen->display, screen->window, params->mipmaps);
+    if (!ok) {
         goto error_destroy_window;
     }
 
-    SDL_RendererInfo renderer_info;
-    int r = SDL_GetRendererInfo(screen->renderer, &renderer_info);
-    const char *renderer_name = r ? NULL : renderer_info.name;
-    LOGI("Renderer: %s", renderer_name ? renderer_name : "(unknown)");
-
-    screen->mipmaps = false;
-
-    // starts with "opengl"
-    bool use_opengl = renderer_name && !strncmp(renderer_name, "opengl", 6);
-    if (use_opengl) {
-        struct sc_opengl *gl = &screen->gl;
-        sc_opengl_init(gl);
-
-        LOGI("OpenGL version: %s", gl->version);
-
-        if (params->mipmaps) {
-            bool supports_mipmaps =
-                sc_opengl_version_at_least(gl, 3, 0, /* OpenGL 3.0+ */
-                                               2, 0  /* OpenGL ES 2.0+ */);
-            if (supports_mipmaps) {
-                LOGI("Trilinear filtering enabled");
-                screen->mipmaps = true;
-            } else {
-                LOGW("Trilinear filtering disabled "
-                     "(OpenGL 3.0+ or ES 2.0+ required)");
-            }
-        } else {
-            LOGI("Trilinear filtering disabled");
-        }
-    } else if (params->mipmaps) {
-        LOGD("Trilinear filtering disabled (not an OpenGL renderer)");
-    }
-
     SDL_Surface *icon = scrcpy_icon_load();
     if (icon) {
         SDL_SetWindowIcon(screen->window, icon);
@@ -506,18 +416,10 @@ sc_screen_init(struct sc_screen *screen,
         LOGW("Could not load icon");
     }
 
-    LOGI("Initial texture: %" PRIu16 "x%" PRIu16, params->frame_size.width,
-                                                  params->frame_size.height);
-    screen->texture = create_texture(screen);
-    if (!screen->texture) {
-        LOGE("Could not create texture: %s", SDL_GetError());
-        goto error_destroy_renderer;
-    }
-
     screen->frame = av_frame_alloc();
     if (!screen->frame) {
         LOG_OOM();
-        goto error_destroy_texture;
+        goto error_destroy_display;
     }
 
     struct sc_input_manager_params im_params = {
@@ -552,19 +454,14 @@ sc_screen_init(struct sc_screen *screen,
 
     return true;
 
-error_destroy_texture:
-    SDL_DestroyTexture(screen->texture);
-error_destroy_renderer:
-    SDL_DestroyRenderer(screen->renderer);
+error_destroy_display:
+    sc_display_destroy(&screen->display);
 error_destroy_window:
     SDL_DestroyWindow(screen->window);
 error_destroy_fps_counter:
     sc_fps_counter_destroy(&screen->fps_counter);
-error_stop_and_join_video_buffer:
-    sc_video_buffer_stop(&screen->vb);
-    sc_video_buffer_join(&screen->vb);
-error_destroy_video_buffer:
-    sc_video_buffer_destroy(&screen->vb);
+error_destroy_frame_buffer:
+    sc_frame_buffer_destroy(&screen->fb);
 
     return false;
 }
@@ -592,6 +489,7 @@ sc_screen_show_initial_window(struct sc_screen *screen) {
     }
 
     SDL_ShowWindow(screen->window);
+    sc_screen_update_content_rect(screen);
 }
 
 void
@@ -601,13 +499,11 @@ sc_screen_hide_window(struct sc_screen *screen) {
 
 void
 sc_screen_interrupt(struct sc_screen *screen) {
-    sc_video_buffer_stop(&screen->vb);
     sc_fps_counter_interrupt(&screen->fps_counter);
 }
 
 void
 sc_screen_join(struct sc_screen *screen) {
-    sc_video_buffer_join(&screen->vb);
     sc_fps_counter_join(&screen->fps_counter);
 }
 
@@ -616,12 +512,11 @@ sc_screen_destroy(struct sc_screen *screen) {
 #ifndef NDEBUG
     assert(!screen->open);
 #endif
+    sc_display_destroy(&screen->display);
     av_frame_free(&screen->frame);
-    SDL_DestroyTexture(screen->texture);
-    SDL_DestroyRenderer(screen->renderer);
     SDL_DestroyWindow(screen->window);
     sc_fps_counter_destroy(&screen->fps_counter);
-    sc_video_buffer_destroy(&screen->vb);
+    sc_frame_buffer_destroy(&screen->fb);
 }
 
 static void
@@ -640,11 +535,11 @@ resize_for_content(struct sc_screen *screen, struct sc_size old_content_size,
 
 static void
 set_content_size(struct sc_screen *screen, struct sc_size new_content_size) {
-    if (!screen->fullscreen && !screen->maximized) {
+    if (!screen->fullscreen && !screen->maximized && !screen->minimized) {
         resize_for_content(screen, screen->content_size, new_content_size);
     } else if (!screen->resize_pending) {
         // Store the windowed size to be able to compute the optimal size once
-        // fullscreen and maximized are disabled
+        // fullscreen/maximized/minimized are disabled
         screen->windowed_content_size = screen->content_size;
         screen->resize_pending = true;
     }
@@ -656,6 +551,7 @@ static void
 apply_pending_resize(struct sc_screen *screen) {
     assert(!screen->fullscreen);
     assert(!screen->maximized);
+    assert(!screen->minimized);
     if (screen->resize_pending) {
         resize_for_content(screen, screen->windowed_content_size,
                                    screen->content_size);
@@ -664,79 +560,85 @@ apply_pending_resize(struct sc_screen *screen) {
 }
 
 void
-sc_screen_set_rotation(struct sc_screen *screen, unsigned rotation) {
-    assert(rotation < 4);
-    if (rotation == screen->rotation) {
+sc_screen_set_orientation(struct sc_screen *screen,
+                          enum sc_orientation orientation) {
+    if (orientation == screen->orientation) {
         return;
     }
 
     struct sc_size new_content_size =
-        get_rotated_size(screen->frame_size, rotation);
+        get_oriented_size(screen->frame_size, orientation);
 
     set_content_size(screen, new_content_size);
 
-    screen->rotation = rotation;
-    LOGI("Display rotation set to %u", rotation);
+    screen->orientation = orientation;
+    LOGI("Display orientation set to %s", sc_orientation_get_name(orientation));
 
     sc_screen_render(screen, true);
 }
 
-// recreate the texture and resize the window if the frame size has changed
 static bool
-prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
-    if (screen->frame_size.width != new_frame_size.width
-            || screen->frame_size.height != new_frame_size.height) {
-        // frame dimension changed, destroy texture
-        SDL_DestroyTexture(screen->texture);
+sc_screen_init_size(struct sc_screen *screen) {
+    // Before first frame
+    assert(!screen->has_frame);
 
-        screen->frame_size = new_frame_size;
+    // The requested size is passed via screen->frame_size
 
-        struct sc_size new_content_size =
-            get_rotated_size(new_frame_size, screen->rotation);
-        set_content_size(screen, new_content_size);
+    struct sc_size content_size =
+        get_oriented_size(screen->frame_size, screen->orientation);
+    screen->content_size = content_size;
 
-        sc_screen_update_content_rect(screen);
+    enum sc_display_result res =
+        sc_display_set_texture_size(&screen->display, screen->frame_size);
+    return res != SC_DISPLAY_RESULT_ERROR;
+}
 
-        LOGI("New texture: %" PRIu16 "x%" PRIu16,
-                     screen->frame_size.width, screen->frame_size.height);
-        screen->texture = create_texture(screen);
-        if (!screen->texture) {
-            LOGE("Could not create texture: %s", SDL_GetError());
-            return false;
-        }
+// recreate the texture and resize the window if the frame size has changed
+static enum sc_display_result
+prepare_for_frame(struct sc_screen *screen, struct sc_size new_frame_size) {
+    if (screen->frame_size.width == new_frame_size.width
+            && screen->frame_size.height == new_frame_size.height) {
+        return SC_DISPLAY_RESULT_OK;
     }
 
-    return true;
-}
+    // frame dimension changed
+    screen->frame_size = new_frame_size;
 
-// write the frame into the texture
-static void
-update_texture(struct sc_screen *screen, const AVFrame *frame) {
-    SDL_UpdateYUVTexture(screen->texture, NULL,
-            frame->data[0], frame->linesize[0],
-            frame->data[1], frame->linesize[1],
-            frame->data[2], frame->linesize[2]);
+    struct sc_size new_content_size =
+        get_oriented_size(new_frame_size, screen->orientation);
+    set_content_size(screen, new_content_size);
 
-    if (screen->mipmaps) {
-        SDL_GL_BindTexture(screen->texture, NULL, NULL);
-        screen->gl.GenerateMipmap(GL_TEXTURE_2D);
-        SDL_GL_UnbindTexture(screen->texture);
-    }
+    sc_screen_update_content_rect(screen);
+
+    return sc_display_set_texture_size(&screen->display, screen->frame_size);
 }
 
 static bool
 sc_screen_update_frame(struct sc_screen *screen) {
     av_frame_unref(screen->frame);
-    sc_video_buffer_consume(&screen->vb, screen->frame);
+    sc_frame_buffer_consume(&screen->fb, screen->frame);
     AVFrame *frame = screen->frame;
 
     sc_fps_counter_add_rendered_frame(&screen->fps_counter);
 
     struct sc_size new_frame_size = {frame->width, frame->height};
-    if (!prepare_for_frame(screen, new_frame_size)) {
+    enum sc_display_result res = prepare_for_frame(screen, new_frame_size);
+    if (res == SC_DISPLAY_RESULT_ERROR) {
         return false;
     }
-    update_texture(screen, frame);
+    if (res == SC_DISPLAY_RESULT_PENDING) {
+        // Not an error, but do not continue
+        return true;
+    }
+
+    res = sc_display_update_texture(&screen->display, frame);
+    if (res == SC_DISPLAY_RESULT_ERROR) {
+        return false;
+    }
+    if (res == SC_DISPLAY_RESULT_PENDING) {
+        // Not an error, but do not continue
+        return true;
+    }
 
     if (!screen->has_frame) {
         screen->has_frame = true;
@@ -762,7 +664,7 @@ sc_screen_switch_fullscreen(struct sc_screen *screen) {
     }
 
     screen->fullscreen = !screen->fullscreen;
-    if (!screen->fullscreen && !screen->maximized) {
+    if (!screen->fullscreen && !screen->maximized && !screen->minimized) {
         apply_pending_resize(screen);
     }
 
@@ -772,7 +674,7 @@ sc_screen_switch_fullscreen(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_fit(struct sc_screen *screen) {
-    if (screen->fullscreen || screen->maximized) {
+    if (screen->fullscreen || screen->maximized || screen->minimized) {
         return;
     }
 
@@ -796,7 +698,7 @@ sc_screen_resize_to_fit(struct sc_screen *screen) {
 
 void
 sc_screen_resize_to_pixel_perfect(struct sc_screen *screen) {
-    if (screen->fullscreen) {
+    if (screen->fullscreen || screen->minimized) {
         return;
     }
 
@@ -816,22 +718,32 @@ sc_screen_is_mouse_capture_key(SDL_Keycode key) {
     return key == SDLK_LALT || key == SDLK_LGUI || key == SDLK_RGUI;
 }
 
-void
-sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
+bool
+sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event) {
     bool relative_mode = sc_screen_is_relative_mode(screen);
 
     switch (event->type) {
-        case EVENT_NEW_FRAME: {
+        case SC_EVENT_SCREEN_INIT_SIZE: {
+            // The initial size is passed via screen->frame_size
+            bool ok = sc_screen_init_size(screen);
+            if (!ok) {
+                LOGE("Could not initialize screen size");
+                return false;
+            }
+            return true;
+        }
+        case SC_EVENT_NEW_FRAME: {
             bool ok = sc_screen_update_frame(screen);
             if (!ok) {
-                LOGW("Frame update failed\n");
+                LOGE("Frame update failed\n");
+                return false;
             }
-            return;
+            return true;
         }
         case SDL_WINDOWEVENT:
             if (!screen->has_frame) {
                 // Do nothing
-                return;
+                return true;
             }
             switch (event->window.event) {
                 case SDL_WINDOWEVENT_EXPOSED:
@@ -843,6 +755,9 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
                 case SDL_WINDOWEVENT_MAXIMIZED:
                     screen->maximized = true;
                     break;
+                case SDL_WINDOWEVENT_MINIMIZED:
+                    screen->minimized = true;
+                    break;
                 case SDL_WINDOWEVENT_RESTORED:
                     if (screen->fullscreen) {
                         // On Windows, in maximized+fullscreen, disabling
@@ -853,6 +768,7 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
                         break;
                     }
                     screen->maximized = false;
+                    screen->minimized = false;
                     apply_pending_resize(screen);
                     sc_screen_render(screen, true);
                     break;
@@ -862,7 +778,7 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
                     }
                     break;
             }
-            return;
+            return true;
         case SDL_KEYDOWN:
             if (relative_mode) {
                 SDL_Keycode key = event->key.keysym.sym;
@@ -875,7 +791,7 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
                         screen->mouse_capture_key_pressed = 0;
                     }
                     // Mouse capture keys are never forwarded to the device
-                    return;
+                    return true;
                 }
             }
             break;
@@ -891,7 +807,7 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
                         sc_screen_toggle_mouse_capture(screen);
                     }
                     // Mouse capture keys are never forwarded to the device
-                    return;
+                    return true;
                 }
             }
             break;
@@ -901,7 +817,7 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
             if (relative_mode && !sc_screen_get_mouse_capture(screen)) {
                 // Do not forward to input manager, the mouse will be captured
                 // on SDL_MOUSEBUTTONUP
-                return;
+                return true;
             }
             break;
         case SDL_FINGERMOTION:
@@ -910,54 +826,72 @@ sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event) {
             if (relative_mode) {
                 // Touch events are not compatible with relative mode
                 // (coordinates are not relative)
-                return;
+                return true;
             }
             break;
         case SDL_MOUSEBUTTONUP:
             if (relative_mode && !sc_screen_get_mouse_capture(screen)) {
                 sc_screen_set_mouse_capture(screen, true);
-                return;
+                return true;
             }
             break;
     }
 
     sc_input_manager_handle_event(&screen->im, event);
+    return true;
 }
 
 struct sc_point
 sc_screen_convert_drawable_to_frame_coords(struct sc_screen *screen,
                                            int32_t x, int32_t y) {
-    unsigned rotation = screen->rotation;
-    assert(rotation < 4);
+    enum sc_orientation orientation = screen->orientation;
 
     int32_t w = screen->content_size.width;
     int32_t h = screen->content_size.height;
 
+    // screen->rect must be initialized to avoid a division by zero
+    assert(screen->rect.w && screen->rect.h);
 
     x = (int64_t) (x - screen->rect.x) * w / screen->rect.w;
     y = (int64_t) (y - screen->rect.y) * h / screen->rect.h;
 
-    // rotate
     struct sc_point result;
-    switch (rotation) {
-        case 0:
+    switch (orientation) {
+        case SC_ORIENTATION_0:
             result.x = x;
             result.y = y;
             break;
-        case 1:
+        case SC_ORIENTATION_90:
+            result.x = y;
+            result.y = w - x;
+            break;
+        case SC_ORIENTATION_180:
+            result.x = w - x;
+            result.y = h - y;
+            break;
+        case SC_ORIENTATION_270:
             result.x = h - y;
             result.y = x;
             break;
-        case 2:
+        case SC_ORIENTATION_FLIP_0:
             result.x = w - x;
+            result.y = y;
+            break;
+        case SC_ORIENTATION_FLIP_90:
+            result.x = h - y;
+            result.y = w - x;
+            break;
+        case SC_ORIENTATION_FLIP_180:
+            result.x = x;
             result.y = h - y;
             break;
         default:
-            assert(rotation == 3);
+            assert(orientation == SC_ORIENTATION_FLIP_270);
             result.x = y;
-            result.y = w - x;
+            result.y = x;
             break;
     }
+
     return result;
 }
 

app/src/screen.h

@@ -9,13 +9,15 @@
 
 #include "controller.h"
 #include "coords.h"
+#include "display.h"
 #include "fps_counter.h"
+#include "frame_buffer.h"
 #include "input_manager.h"
 #include "opengl.h"
+#include "options.h"
 #include "trait/key_processor.h"
 #include "trait/frame_sink.h"
 #include "trait/mouse_processor.h"
-#include "video_buffer.h"
 
 struct sc_screen {
     struct sc_frame_sink frame_sink; // frame sink trait
@@ -24,8 +26,9 @@ struct sc_screen {
     bool open; // track the open/close state to assert correct behavior
 #endif
 
+    struct sc_display display;
     struct sc_input_manager im;
-    struct sc_video_buffer vb;
+    struct sc_frame_buffer fb;
     struct sc_fps_counter fps_counter;
 
     // The initial requested window properties
@@ -39,9 +42,6 @@ struct sc_screen {
     } req;
 
     SDL_Window *window;
-    SDL_Renderer *renderer;
-    SDL_Texture *texture;
-    struct sc_opengl gl;
     struct sc_size frame_size;
     struct sc_size content_size; // rotated frame_size
 
@@ -50,16 +50,14 @@ struct sc_screen {
     // fullscreen (meaningful only when resize_pending is true)
     struct sc_size windowed_content_size;
 
-    // client rotation: 0, 1, 2 or 3 (x90 degrees counterclockwise)
-    unsigned rotation;
+    // client orientation
+    enum sc_orientation orientation;
     // rectangle of the content (excluding black borders)
     struct SDL_Rect rect;
     bool has_frame;
     bool fullscreen;
     bool maximized;
-    bool mipmaps;
-
-    bool event_failed; // in case SDL_PushEvent() returned an error
+    bool minimized;
 
     // To enable/disable mouse capture, a mouse capture key (LALT, LGUI or
     // RGUI) must be pressed. This variable tracks the pressed capture key.
@@ -80,7 +78,6 @@ struct sc_screen_params {
     const struct sc_shortcut_mods *shortcut_mods;
 
     const char *window_title;
-    struct sc_size frame_size;
     bool always_on_top;
 
     int16_t window_x; // accepts SC_WINDOW_POSITION_UNDEFINED
@@ -90,13 +87,11 @@ struct sc_screen_params {
 
     bool window_borderless;
 
-    uint8_t rotation;
+    enum sc_orientation orientation;
     bool mipmaps;
 
     bool fullscreen;
     bool start_fps_counter;
-
-    sc_tick buffering_time;
 };
 
 // initialize screen, create window, renderer and texture (window is hidden)
@@ -135,13 +130,15 @@ sc_screen_resize_to_fit(struct sc_screen *screen);
 void
 sc_screen_resize_to_pixel_perfect(struct sc_screen *screen);
 
-// set the display rotation (0, 1, 2 or 3, x90 degrees counterclockwise)
+// set the display orientation
 void
-sc_screen_set_rotation(struct sc_screen *screen, unsigned rotation);
+sc_screen_set_orientation(struct sc_screen *screen,
+                          enum sc_orientation orientation);
 
 // react to SDL events
-void
-sc_screen_handle_event(struct sc_screen *screen, SDL_Event *event);
+// If this function returns false, scrcpy must exit with an error.
+bool
+sc_screen_handle_event(struct sc_screen *screen, const SDL_Event *event);
 
 // convert point from window coordinates to frame coordinates
 // x and y are expressed in pixels

app/src/server.c

@@ -8,6 +8,7 @@
 #include <SDL2/SDL_platform.h>
 
 #include "adb/adb.h"
+#include "util/binary.h"
 #include "util/file.h"
 #include "util/log.h"
 #include "util/net_intr.h"
@@ -20,6 +21,7 @@
 #define SC_DEVICE_SERVER_PATH "/data/local/tmp/scrcpy-server.jar"
 
 #define SC_ADB_PORT_DEFAULT 5555
+#define SC_SOCKET_NAME_PREFIX "scrcpy_"
 
 static char *
 get_server_path(void) {
@@ -69,9 +71,13 @@ sc_server_params_destroy(struct sc_server_params *params) {
     // The server stores a copy of the params provided by the user
     free((char *) params->req_serial);
     free((char *) params->crop);
-    free((char *) params->codec_options);
-    free((char *) params->encoder_name);
+    free((char *) params->video_codec_options);
+    free((char *) params->audio_codec_options);
+    free((char *) params->video_encoder);
+    free((char *) params->audio_encoder);
     free((char *) params->tcpip_dst);
+    free((char *) params->camera_id);
+    free((char *) params->camera_ar);
 }
 
 static bool
@@ -82,20 +88,25 @@ sc_server_params_copy(struct sc_server_params *dst,
     // The params reference user-allocated memory, so we must copy them to
     // handle them from another thread
 
-#define COPY(FIELD) \
+#define COPY(FIELD) do { \
     dst->FIELD = NULL; \
     if (src->FIELD) { \
         dst->FIELD = strdup(src->FIELD); \
         if (!dst->FIELD) { \
             goto error; \
         } \
-    }
+    } \
+} while(0)
 
     COPY(req_serial);
     COPY(crop);
-    COPY(codec_options);
-    COPY(encoder_name);
+    COPY(video_codec_options);
+    COPY(audio_codec_options);
+    COPY(video_encoder);
+    COPY(audio_encoder);
     COPY(tcpip_dst);
+    COPY(camera_id);
+    COPY(camera_ar);
 #undef COPY
 
     return true;
@@ -154,6 +165,42 @@ sc_server_sleep(struct sc_server *server, sc_tick deadline) {
     return !stopped;
 }
 
+static const char *
+sc_server_get_codec_name(enum sc_codec codec) {
+    switch (codec) {
+        case SC_CODEC_H264:
+            return "h264";
+        case SC_CODEC_H265:
+            return "h265";
+        case SC_CODEC_AV1:
+            return "av1";
+        case SC_CODEC_OPUS:
+            return "opus";
+        case SC_CODEC_AAC:
+            return "aac";
+        case SC_CODEC_FLAC:
+            return "flac";
+        case SC_CODEC_RAW:
+            return "raw";
+        default:
+            return NULL;
+    }
+}
+
+static const char *
+sc_server_get_camera_facing_name(enum sc_camera_facing camera_facing) {
+    switch (camera_facing) {
+        case SC_CAMERA_FACING_FRONT:
+            return "front";
+        case SC_CAMERA_FACING_BACK:
+            return "back";
+        case SC_CAMERA_FACING_EXTERNAL:
+            return "external";
+        default:
+            return NULL;
+    }
+}
+
 static sc_pid
 execute_server(struct sc_server *server,
                const struct sc_server_params *params) {
@@ -189,17 +236,44 @@ execute_server(struct sc_server *server,
     cmd[count++] = SCRCPY_VERSION;
 
     unsigned dyn_idx = count; // from there, the strings are allocated
-#define ADD_PARAM(fmt, ...) { \
-        char *p = (char *) &cmd[count]; \
+#define ADD_PARAM(fmt, ...) do { \
+        char *p; \
         if (asprintf(&p, fmt, ## __VA_ARGS__) == -1) { \
             goto end; \
         } \
         cmd[count++] = p; \
-    }
+    } while(0)
 
+    ADD_PARAM("scid=%08x", params->scid);
     ADD_PARAM("log_level=%s", log_level_to_server_string(params->log_level));
-    ADD_PARAM("bit_rate=%" PRIu32, params->bit_rate);
 
+    if (!params->video) {
+        ADD_PARAM("video=false");
+    }
+    if (params->video_bit_rate) {
+        ADD_PARAM("video_bit_rate=%" PRIu32, params->video_bit_rate);
+    }
+    if (!params->audio) {
+        ADD_PARAM("audio=false");
+    }
+    if (params->audio_bit_rate) {
+        ADD_PARAM("audio_bit_rate=%" PRIu32, params->audio_bit_rate);
+    }
+    if (params->video_codec != SC_CODEC_H264) {
+        ADD_PARAM("video_codec=%s",
+                  sc_server_get_codec_name(params->video_codec));
+    }
+    if (params->audio_codec != SC_CODEC_OPUS) {
+        ADD_PARAM("audio_codec=%s",
+            sc_server_get_codec_name(params->audio_codec));
+    }
+    if (params->video_source != SC_VIDEO_SOURCE_DISPLAY) {
+        assert(params->video_source == SC_VIDEO_SOURCE_CAMERA);
+        ADD_PARAM("video_source=camera");
+    }
+    if (params->audio_source == SC_AUDIO_SOURCE_MIC) {
+        ADD_PARAM("audio_source=mic");
+    }
     if (params->max_size) {
         ADD_PARAM("max_size=%" PRIu16, params->max_size);
     }
@@ -223,17 +297,42 @@ execute_server(struct sc_server *server,
     if (params->display_id) {
         ADD_PARAM("display_id=%" PRIu32, params->display_id);
     }
+    if (params->camera_id) {
+        ADD_PARAM("camera_id=%s", params->camera_id);
+    }
+    if (params->camera_size) {
+        ADD_PARAM("camera_size=%s", params->camera_size);
+    }
+    if (params->camera_facing != SC_CAMERA_FACING_ANY) {
+        ADD_PARAM("camera_facing=%s",
+            sc_server_get_camera_facing_name(params->camera_facing));
+    }
+    if (params->camera_ar) {
+        ADD_PARAM("camera_ar=%s", params->camera_ar);
+    }
+    if (params->camera_fps) {
+        ADD_PARAM("camera_fps=%" PRIu16, params->camera_fps);
+    }
+    if (params->camera_high_speed) {
+        ADD_PARAM("camera_high_speed=true");
+    }
     if (params->show_touches) {
         ADD_PARAM("show_touches=true");
     }
     if (params->stay_awake) {
         ADD_PARAM("stay_awake=true");
     }
-    if (params->codec_options) {
-        ADD_PARAM("codec_options=%s", params->codec_options);
+    if (params->video_codec_options) {
+        ADD_PARAM("video_codec_options=%s", params->video_codec_options);
+    }
+    if (params->audio_codec_options) {
+        ADD_PARAM("audio_codec_options=%s", params->audio_codec_options);
+    }
+    if (params->video_encoder) {
+        ADD_PARAM("video_encoder=%s", params->video_encoder);
     }
-    if (params->encoder_name) {
-        ADD_PARAM("encoder_name=%s", params->encoder_name);
+    if (params->audio_encoder) {
+        ADD_PARAM("audio_encoder=%s", params->audio_encoder);
     }
     if (params->power_off_on_close) {
         ADD_PARAM("power_off_on_close=true");
@@ -254,6 +353,18 @@ execute_server(struct sc_server *server,
         // By default, power_on is true
         ADD_PARAM("power_on=false");
     }
+    if (params->list & SC_OPTION_LIST_ENCODERS) {
+        ADD_PARAM("list_encoders=true");
+    }
+    if (params->list & SC_OPTION_LIST_DISPLAYS) {
+        ADD_PARAM("list_displays=true");
+    }
+    if (params->list & SC_OPTION_LIST_CAMERAS) {
+        ADD_PARAM("list_cameras=true");
+    }
+    if (params->list & SC_OPTION_LIST_CAMERA_SIZES) {
+        ADD_PARAM("list_camera_sizes=true");
+    }
 
 #undef ADD_PARAM
 
@@ -364,9 +475,11 @@ sc_server_init(struct sc_server *server, const struct sc_server_params *params,
     }
 
     server->serial = NULL;
+    server->device_socket_name = NULL;
     server->stopped = false;
 
     server->video_socket = SC_SOCKET_NONE;
+    server->audio_socket = SC_SOCKET_NONE;
     server->control_socket = SC_SOCKET_NONE;
 
     sc_adb_tunnel_init(&server->tunnel);
@@ -385,9 +498,9 @@ sc_server_init(struct sc_server *server, const struct sc_server_params *params,
 static bool
 device_read_info(struct sc_intr *intr, sc_socket device_socket,
                  struct sc_server_info *info) {
-    unsigned char buf[SC_DEVICE_NAME_FIELD_LENGTH + 4];
+    uint8_t buf[SC_DEVICE_NAME_FIELD_LENGTH];
     ssize_t r = net_recv_all_intr(intr, device_socket, buf, sizeof(buf));
-    if (r < SC_DEVICE_NAME_FIELD_LENGTH + 4) {
+    if (r < SC_DEVICE_NAME_FIELD_LENGTH) {
         LOGE("Could not retrieve device information");
         return false;
     }
@@ -395,10 +508,6 @@ device_read_info(struct sc_intr *intr, sc_socket device_socket,
     buf[SC_DEVICE_NAME_FIELD_LENGTH - 1] = '\0';
     memcpy(info->device_name, (char *) buf, sizeof(info->device_name));
 
-    info->frame_size.width = (buf[SC_DEVICE_NAME_FIELD_LENGTH] << 8)
-                           | buf[SC_DEVICE_NAME_FIELD_LENGTH + 1];
-    info->frame_size.height = (buf[SC_DEVICE_NAME_FIELD_LENGTH + 2] << 8)
-                            | buf[SC_DEVICE_NAME_FIELD_LENGTH + 3];
     return true;
 }
 
@@ -411,14 +520,28 @@ sc_server_connect_to(struct sc_server *server, struct sc_server_info *info) {
     const char *serial = server->serial;
     assert(serial);
 
+    bool video = server->params.video;
+    bool audio = server->params.audio;
     bool control = server->params.control;
 
     sc_socket video_socket = SC_SOCKET_NONE;
+    sc_socket audio_socket = SC_SOCKET_NONE;
     sc_socket control_socket = SC_SOCKET_NONE;
     if (!tunnel->forward) {
-        video_socket = net_accept_intr(&server->intr, tunnel->server_socket);
-        if (video_socket == SC_SOCKET_NONE) {
-            goto fail;
+        if (video) {
+            video_socket =
+                net_accept_intr(&server->intr, tunnel->server_socket);
+            if (video_socket == SC_SOCKET_NONE) {
+                goto fail;
+            }
+        }
+
+        if (audio) {
+            audio_socket =
+                net_accept_intr(&server->intr, tunnel->server_socket);
+            if (audio_socket == SC_SOCKET_NONE) {
+                goto fail;
+            }
         }
 
         if (control) {
@@ -441,40 +564,69 @@ sc_server_connect_to(struct sc_server *server, struct sc_server_info *info) {
 
         unsigned attempts = 100;
         sc_tick delay = SC_TICK_FROM_MS(100);
-        video_socket = connect_to_server(server, attempts, delay, tunnel_host,
-                                         tunnel_port);
-        if (video_socket == SC_SOCKET_NONE) {
+        sc_socket first_socket = connect_to_server(server, attempts, delay,
+                                                   tunnel_host, tunnel_port);
+        if (first_socket == SC_SOCKET_NONE) {
             goto fail;
         }
 
-        if (control) {
-            // we know that the device is listening, we don't need several
-            // attempts
-            control_socket = net_socket();
-            if (control_socket == SC_SOCKET_NONE) {
-                goto fail;
+        if (video) {
+            video_socket = first_socket;
+        }
+
+        if (audio) {
+            if (!video) {
+                audio_socket = first_socket;
+            } else {
+                audio_socket = net_socket();
+                if (audio_socket == SC_SOCKET_NONE) {
+                    goto fail;
+                }
+                bool ok = net_connect_intr(&server->intr, audio_socket,
+                                           tunnel_host, tunnel_port);
+                if (!ok) {
+                    goto fail;
+                }
             }
-            bool ok = net_connect_intr(&server->intr, control_socket,
-                                       tunnel_host, tunnel_port);
-            if (!ok) {
-                goto fail;
+        }
+
+        if (control) {
+            if (!video && !audio) {
+                control_socket = first_socket;
+            } else {
+                control_socket = net_socket();
+                if (control_socket == SC_SOCKET_NONE) {
+                    goto fail;
+                }
+                bool ok = net_connect_intr(&server->intr, control_socket,
+                                           tunnel_host, tunnel_port);
+                if (!ok) {
+                    goto fail;
+                }
             }
         }
     }
 
     // we don't need the adb tunnel anymore
-    sc_adb_tunnel_close(tunnel, &server->intr, serial);
+    sc_adb_tunnel_close(tunnel, &server->intr, serial,
+                        server->device_socket_name);
+
+    sc_socket first_socket = video ? video_socket
+                           : audio ? audio_socket
+                                   : control_socket;
 
     // The sockets will be closed on stop if device_read_info() fails
-    bool ok = device_read_info(&server->intr, video_socket, info);
+    bool ok = device_read_info(&server->intr, first_socket, info);
     if (!ok) {
         goto fail;
     }
 
-    assert(video_socket != SC_SOCKET_NONE);
+    assert(!video || video_socket != SC_SOCKET_NONE);
+    assert(!audio || audio_socket != SC_SOCKET_NONE);
     assert(!control || control_socket != SC_SOCKET_NONE);
 
     server->video_socket = video_socket;
+    server->audio_socket = audio_socket;
     server->control_socket = control_socket;
 
     return true;
@@ -486,6 +638,12 @@ fail:
         }
     }
 
+    if (audio_socket != SC_SOCKET_NONE) {
+        if (!net_close(audio_socket)) {
+            LOGW("Could not close audio socket");
+        }
+    }
+
     if (control_socket != SC_SOCKET_NONE) {
         if (!net_close(control_socket)) {
             LOGW("Could not close control socket");
@@ -494,7 +652,8 @@ fail:
 
     if (tunnel->enabled) {
         // Always leave this function with tunnel disabled
-        sc_adb_tunnel_close(tunnel, &server->intr, serial);
+        sc_adb_tunnel_close(tunnel, &server->intr, serial,
+                            server->device_socket_name);
     }
 
     return false;
@@ -667,6 +826,11 @@ sc_server_configure_tcpip_unknown_address(struct sc_server *server,
     if (is_already_tcpip) {
         // Nothing to do
         LOGI("Device already connected via TCP/IP: %s", serial);
+        server->serial = strdup(serial);
+        if (!server->serial) {
+            LOG_OOM();
+            return false;
+        }
         return true;
     }
 
@@ -679,6 +843,15 @@ sc_server_configure_tcpip_unknown_address(struct sc_server *server,
     return sc_server_connect_to_tcpip(server, ip_port);
 }
 
+static void
+sc_server_kill_adb_if_requested(struct sc_server *server) {
+    if (server->params.kill_adb_on_close) {
+        LOGI("Killing adb server...");
+        unsigned flags = SC_ADB_NO_STDOUT | SC_ADB_NO_STDERR | SC_ADB_NO_LOGERR;
+        sc_adb_kill_server(&server->intr, flags);
+    }
+}
+
 static int
 run_server(void *data) {
     struct sc_server *server = data;
@@ -690,7 +863,7 @@ run_server(void *data) {
     // is parsed, so it is not output)
     bool ok = sc_adb_start_server(&server->intr, 0);
     if (!ok) {
-        LOGE("Could not start adb daemon");
+        LOGE("Could not start adb server");
         goto error_connection_failed;
     }
 
@@ -769,8 +942,32 @@ run_server(void *data) {
         goto error_connection_failed;
     }
 
+    // If --list-* is passed, then the server just prints the requested data
+    // then exits.
+    if (params->list) {
+        sc_pid pid = execute_server(server, params);
+        if (pid == SC_PROCESS_NONE) {
+            goto error_connection_failed;
+        }
+        sc_process_wait(pid, NULL); // ignore exit code
+        sc_process_close(pid);
+        // Wake up await_for_server()
+        server->cbs->on_connected(server, server->cbs_userdata);
+        return 0;
+    }
+
+    int r = asprintf(&server->device_socket_name, SC_SOCKET_NAME_PREFIX "%08x",
+                     params->scid);
+    if (r == -1) {
+        LOG_OOM();
+        goto error_connection_failed;
+    }
+    assert(r == sizeof(SC_SOCKET_NAME_PREFIX) - 1 + 8);
+    assert(server->device_socket_name);
+
     ok = sc_adb_tunnel_open(&server->tunnel, &server->intr, serial,
-                            params->port_range, params->force_adb_forward);
+                            server->device_socket_name, params->port_range,
+                            params->force_adb_forward);
     if (!ok) {
         goto error_connection_failed;
     }
@@ -778,7 +975,8 @@ run_server(void *data) {
     // server will connect to our server socket
     sc_pid pid = execute_server(server, params);
     if (pid == SC_PROCESS_NONE) {
-        sc_adb_tunnel_close(&server->tunnel, &server->intr, serial);
+        sc_adb_tunnel_close(&server->tunnel, &server->intr, serial,
+                            server->device_socket_name);
         goto error_connection_failed;
     }
 
@@ -790,7 +988,8 @@ run_server(void *data) {
     if (!ok) {
         sc_process_terminate(pid);
         sc_process_wait(pid, true); // ignore exit code
-        sc_adb_tunnel_close(&server->tunnel, &server->intr, serial);
+        sc_adb_tunnel_close(&server->tunnel, &server->intr, serial,
+                            server->device_socket_name);
         goto error_connection_failed;
     }
 
@@ -815,8 +1014,16 @@ run_server(void *data) {
     sc_mutex_unlock(&server->mutex);
 
     // Interrupt sockets to wake up socket blocking calls on the server
-    assert(server->video_socket != SC_SOCKET_NONE);
-    net_interrupt(server->video_socket);
+
+    if (server->video_socket != SC_SOCKET_NONE) {
+        // There is no video_socket if --no-video is set
+        net_interrupt(server->video_socket);
+    }
+
+    if (server->audio_socket != SC_SOCKET_NONE) {
+        // There is no audio_socket if --no-audio is set
+        net_interrupt(server->audio_socket);
+    }
 
     if (server->control_socket != SC_SOCKET_NONE) {
         // There is no control_socket if --no-control is set
@@ -844,9 +1051,12 @@ run_server(void *data) {
 
     sc_process_close(pid);
 
+    sc_server_kill_adb_if_requested(server);
+
     return 0;
 
 error_connection_failed:
+    sc_server_kill_adb_if_requested(server);
     server->cbs->on_connection_failed(server, server->cbs_userdata);
     return -1;
 }
@@ -870,7 +1080,10 @@ sc_server_stop(struct sc_server *server) {
     sc_cond_signal(&server->cond_stopped);
     sc_intr_interrupt(&server->intr);
     sc_mutex_unlock(&server->mutex);
+}
 
+void
+sc_server_join(struct sc_server *server) {
     sc_thread_join(&server->thread, NULL);
 }
 
@@ -879,11 +1092,15 @@ sc_server_destroy(struct sc_server *server) {
     if (server->video_socket != SC_SOCKET_NONE) {
         net_close(server->video_socket);
     }
+    if (server->audio_socket != SC_SOCKET_NONE) {
+        net_close(server->audio_socket);
+    }
     if (server->control_socket != SC_SOCKET_NONE) {
         net_close(server->control_socket);
     }
 
     free(server->serial);
+    free(server->device_socket_name);
     sc_server_params_destroy(&server->params);
     sc_intr_destroy(&server->intr);
     sc_cond_destroy(&server->cond_stopped);

app/src/server.h

@@ -18,24 +18,38 @@
 #define SC_DEVICE_NAME_FIELD_LENGTH 64
 struct sc_server_info {
     char device_name[SC_DEVICE_NAME_FIELD_LENGTH];
-    struct sc_size frame_size;
 };
 
 struct sc_server_params {
+    uint32_t scid;
     const char *req_serial;
     enum sc_log_level log_level;
+    enum sc_codec video_codec;
+    enum sc_codec audio_codec;
+    enum sc_video_source video_source;
+    enum sc_audio_source audio_source;
+    enum sc_camera_facing camera_facing;
     const char *crop;
-    const char *codec_options;
-    const char *encoder_name;
+    const char *video_codec_options;
+    const char *audio_codec_options;
+    const char *video_encoder;
+    const char *audio_encoder;
+    const char *camera_id;
+    const char *camera_size;
+    const char *camera_ar;
+    uint16_t camera_fps;
     struct sc_port_range port_range;
     uint32_t tunnel_host;
     uint16_t tunnel_port;
     uint16_t max_size;
-    uint32_t bit_rate;
+    uint32_t video_bit_rate;
+    uint32_t audio_bit_rate;
     uint16_t max_fps;
     int8_t lock_video_orientation;
     bool control;
     uint32_t display_id;
+    bool video;
+    bool audio;
     bool show_touches;
     bool stay_awake;
     bool force_adb_forward;
@@ -48,12 +62,16 @@ struct sc_server_params {
     bool select_tcpip;
     bool cleanup;
     bool power_on;
+    bool kill_adb_on_close;
+    bool camera_high_speed;
+    uint8_t list;
 };
 
 struct sc_server {
     // The internal allocated strings are copies owned by the server
     struct sc_server_params params;
     char *serial;
+    char *device_socket_name;
 
     sc_thread thread;
     struct sc_server_info info; // initialized once connected
@@ -66,6 +84,7 @@ struct sc_server {
     struct sc_adb_tunnel tunnel;
 
     sc_socket video_socket;
+    sc_socket audio_socket;
     sc_socket control_socket;
 
     const struct sc_server_callbacks *cbs;
@@ -105,6 +124,10 @@ sc_server_start(struct sc_server *server);
 void
 sc_server_stop(struct sc_server *server);
 
+// join the server thread
+void
+sc_server_join(struct sc_server *server);
+
 // close and release sockets
 void
 sc_server_destroy(struct sc_server *server);

app/src/trait/frame_sink.h

@@ -5,8 +5,7 @@
 
 #include <assert.h>
 #include <stdbool.h>
-
-typedef struct AVFrame AVFrame;
+#include <libavcodec/avcodec.h>
 
 /**
  * Frame sink trait.
@@ -18,7 +17,8 @@ struct sc_frame_sink {
 };
 
 struct sc_frame_sink_ops {
-    bool (*open)(struct sc_frame_sink *sink);
+    /* The codec context is valid until the sink is closed */
+    bool (*open)(struct sc_frame_sink *sink, const AVCodecContext *ctx);
     void (*close)(struct sc_frame_sink *sink);
     bool (*push)(struct sc_frame_sink *sink, const AVFrame *frame);
 };

app/src/trait/frame_source.c

@@ -0,0 +1,59 @@
+#include "frame_source.h"
+
+void
+sc_frame_source_init(struct sc_frame_source *source) {
+    source->sink_count = 0;
+}
+
+void
+sc_frame_source_add_sink(struct sc_frame_source *source,
+                         struct sc_frame_sink *sink) {
+    assert(source->sink_count < SC_FRAME_SOURCE_MAX_SINKS);
+    assert(sink);
+    assert(sink->ops);
+    source->sinks[source->sink_count++] = sink;
+}
+
+static void
+sc_frame_source_sinks_close_firsts(struct sc_frame_source *source,
+                                    unsigned count) {
+    while (count) {
+        struct sc_frame_sink *sink = source->sinks[--count];
+        sink->ops->close(sink);
+    }
+}
+
+bool
+sc_frame_source_sinks_open(struct sc_frame_source *source,
+                           const AVCodecContext *ctx) {
+    assert(source->sink_count);
+    for (unsigned i = 0; i < source->sink_count; ++i) {
+        struct sc_frame_sink *sink = source->sinks[i];
+        if (!sink->ops->open(sink, ctx)) {
+            sc_frame_source_sinks_close_firsts(source, i);
+            return false;
+        }
+    }
+
+    return true;
+}
+
+void
+sc_frame_source_sinks_close(struct sc_frame_source *source) {
+    assert(source->sink_count);
+    sc_frame_source_sinks_close_firsts(source, source->sink_count);
+}
+
+bool
+sc_frame_source_sinks_push(struct sc_frame_source *source,
+                            const AVFrame *frame) {
+    assert(source->sink_count);
+    for (unsigned i = 0; i < source->sink_count; ++i) {
+        struct sc_frame_sink *sink = source->sinks[i];
+        if (!sink->ops->push(sink, frame)) {
+            return false;
+        }
+    }
+
+    return true;
+}

app/src/trait/frame_source.h

@@ -0,0 +1,38 @@
+#ifndef SC_FRAME_SOURCE_H
+#define SC_FRAME_SOURCE_H
+
+#include "common.h"
+
+#include "frame_sink.h"
+
+#define SC_FRAME_SOURCE_MAX_SINKS 2
+
+/**
+ * Frame source trait
+ *
+ * Component able to send AVFrames should implement this trait.
+ */
+struct sc_frame_source {
+    struct sc_frame_sink *sinks[SC_FRAME_SOURCE_MAX_SINKS];
+    unsigned sink_count;
+};
+
+void
+sc_frame_source_init(struct sc_frame_source *source);
+
+void
+sc_frame_source_add_sink(struct sc_frame_source *source,
+                         struct sc_frame_sink *sink);
+
+bool
+sc_frame_source_sinks_open(struct sc_frame_source *source,
+                           const AVCodecContext *ctx);
+
+void
+sc_frame_source_sinks_close(struct sc_frame_source *source);
+
+bool
+sc_frame_source_sinks_push(struct sc_frame_source *source,
+                           const AVFrame *frame);
+
+#endif

app/src/trait/key_processor.h

@@ -23,6 +23,13 @@ struct sc_key_processor {
      */
     bool async_paste;
 
+    /**
+     * Set by the implementation to indicate that the keyboard is HID. In
+     * practice, it is used to react on a shortcut to open the hard keyboard
+     * settings only if the keyboard is HID.
+     */
+    bool hid;
+
     const struct sc_key_processor_ops *ops;
 };
 

app/src/trait/packet_sink.h

@@ -5,9 +5,7 @@
 
 #include <assert.h>
 #include <stdbool.h>
-
-typedef struct AVCodec AVCodec;
-typedef struct AVPacket AVPacket;
+#include <libavcodec/avcodec.h>
 
 /**
  * Packet sink trait.
@@ -19,9 +17,20 @@ struct sc_packet_sink {
 };
 
 struct sc_packet_sink_ops {
-    bool (*open)(struct sc_packet_sink *sink, const AVCodec *codec);
+    /* The codec context is valid until the sink is closed */
+    bool (*open)(struct sc_packet_sink *sink, AVCodecContext *ctx);
     void (*close)(struct sc_packet_sink *sink);
     bool (*push)(struct sc_packet_sink *sink, const AVPacket *packet);
+
+    /*/
+     * Called when the input stream has been disabled at runtime.
+     *
+     * If it is called, then open(), close() and push() will never be called.
+     *
+     * It is useful to notify the recorder that the requested audio stream has
+     * finally been disabled because the device could not capture it.
+     */
+    void (*disable)(struct sc_packet_sink *sink);
 };
 
 #endif

app/src/trait/packet_source.c

@@ -0,0 +1,70 @@
+#include "packet_source.h"
+
+void
+sc_packet_source_init(struct sc_packet_source *source) {
+    source->sink_count = 0;
+}
+
+void
+sc_packet_source_add_sink(struct sc_packet_source *source,
+                          struct sc_packet_sink *sink) {
+    assert(source->sink_count < SC_PACKET_SOURCE_MAX_SINKS);
+    assert(sink);
+    assert(sink->ops);
+    source->sinks[source->sink_count++] = sink;
+}
+
+static void
+sc_packet_source_sinks_close_firsts(struct sc_packet_source *source,
+                                    unsigned count) {
+    while (count) {
+        struct sc_packet_sink *sink = source->sinks[--count];
+        sink->ops->close(sink);
+    }
+}
+
+bool
+sc_packet_source_sinks_open(struct sc_packet_source *source,
+                            AVCodecContext *ctx) {
+    assert(source->sink_count);
+    for (unsigned i = 0; i < source->sink_count; ++i) {
+        struct sc_packet_sink *sink = source->sinks[i];
+        if (!sink->ops->open(sink, ctx)) {
+            sc_packet_source_sinks_close_firsts(source, i);
+            return false;
+        }
+    }
+
+    return true;
+}
+
+void
+sc_packet_source_sinks_close(struct sc_packet_source *source) {
+    assert(source->sink_count);
+    sc_packet_source_sinks_close_firsts(source, source->sink_count);
+}
+
+bool
+sc_packet_source_sinks_push(struct sc_packet_source *source,
+                            const AVPacket *packet) {
+    assert(source->sink_count);
+    for (unsigned i = 0; i < source->sink_count; ++i) {
+        struct sc_packet_sink *sink = source->sinks[i];
+        if (!sink->ops->push(sink, packet)) {
+            return false;
+        }
+    }
+
+    return true;
+}
+
+void
+sc_packet_source_sinks_disable(struct sc_packet_source *source) {
+    assert(source->sink_count);
+    for (unsigned i = 0; i < source->sink_count; ++i) {
+        struct sc_packet_sink *sink = source->sinks[i];
+        if (sink->ops->disable) {
+            sink->ops->disable(sink);
+        }
+    }
+}

app/src/trait/packet_source.h

@@ -0,0 +1,41 @@
+#ifndef SC_PACKET_SOURCE_H
+#define SC_PACKET_SOURCE_H
+
+#include "common.h"
+
+#include "packet_sink.h"
+
+#define SC_PACKET_SOURCE_MAX_SINKS 2
+
+/**
+ * Packet source trait
+ *
+ * Component able to send AVPackets should implement this trait.
+ */
+struct sc_packet_source {
+    struct sc_packet_sink *sinks[SC_PACKET_SOURCE_MAX_SINKS];
+    unsigned sink_count;
+};
+
+void
+sc_packet_source_init(struct sc_packet_source *source);
+
+void
+sc_packet_source_add_sink(struct sc_packet_source *source,
+                          struct sc_packet_sink *sink);
+
+bool
+sc_packet_source_sinks_open(struct sc_packet_source *source,
+                            AVCodecContext *ctx);
+
+void
+sc_packet_source_sinks_close(struct sc_packet_source *source);
+
+bool
+sc_packet_source_sinks_push(struct sc_packet_source *source,
+                            const AVPacket *packet);
+
+void
+sc_packet_source_sinks_disable(struct sc_packet_source *source);
+
+#endif

app/src/uhid/keyboard_uhid.c

@@ -0,0 +1,162 @@
+#include "keyboard_uhid.h"
+
+#include "util/log.h"
+
+/** Downcast key processor to keyboard_uhid */
+#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
+
+/** Downcast uhid_receiver to keyboard_uhid */
+#define DOWNCAST_RECEIVER(UR) \
+    container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
+
+#define UHID_KEYBOARD_ID 1
+
+static void
+sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
+                            const struct sc_hid_event *event) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_KEYBOARD_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(kb->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (key)");
+    }
+}
+
+static void
+sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
+    SDL_Keymod sdl_mod = SDL_GetModState();
+    uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
+
+    uint16_t device_mod =
+        atomic_load_explicit(&kb->device_mod, memory_order_relaxed);
+    uint16_t diff = mod ^ device_mod;
+
+    if (diff) {
+        // Inherently racy (the HID output reports arrive asynchronously in
+        // response to key presses), but will re-synchronize on next key press
+        // or HID output anyway
+        atomic_store_explicit(&kb->device_mod, mod, memory_order_relaxed);
+
+        struct sc_hid_event hid_event;
+        sc_hid_keyboard_event_from_mods(&hid_event, diff);
+
+        LOGV("HID keyboard state synchronized");
+
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static void
+sc_key_processor_process_key(struct sc_key_processor *kp,
+                             const struct sc_key_event *event,
+                             uint64_t ack_to_wait) {
+    (void) ack_to_wait;
+
+    if (event->repeat) {
+        // In USB HID protocol, key repeat is handled by the host (Android), so
+        // just ignore key repeat here.
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST(kp);
+
+    struct sc_hid_event hid_event;
+
+    // Not all keys are supported, just ignore unsupported keys
+    if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
+        if (event->scancode == SC_SCANCODE_CAPSLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_CAPS,
+                                      memory_order_relaxed);
+        } else if (event->scancode == SC_SCANCODE_NUMLOCK) {
+            atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_NUM,
+                                      memory_order_relaxed);
+        } else {
+            // Synchronize modifiers (only if the scancode itself does not
+            // change the modifiers)
+            sc_keyboard_uhid_synchronize_mod(kb);
+        }
+        sc_keyboard_uhid_send_input(kb, &hid_event);
+    }
+}
+
+static unsigned
+sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
+    // <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
+    // (chapter 11: LED page)
+    unsigned mod = 0;
+    if (hid_led & 0x01) {
+        mod |= SC_MOD_NUM;
+    }
+    if (hid_led & 0x02) {
+        mod |= SC_MOD_CAPS;
+    }
+    return mod;
+}
+
+static void
+sc_uhid_receiver_process_output(struct sc_uhid_receiver *receiver,
+                                const uint8_t *data, size_t len) {
+    // Called from the thread receiving device messages
+
+    assert(len);
+
+    // Also check at runtime (do not trust the server)
+    if (!len) {
+        LOGE("Unexpected empty HID output message");
+        return;
+    }
+
+    struct sc_keyboard_uhid *kb = DOWNCAST_RECEIVER(receiver);
+
+    uint8_t hid_led = data[0];
+    uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
+    atomic_store_explicit(&kb->device_mod, device_mod, memory_order_relaxed);
+}
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices) {
+    sc_hid_keyboard_init(&kb->hid);
+
+    kb->controller = controller;
+    atomic_init(&kb->device_mod, 0);
+
+    static const struct sc_key_processor_ops ops = {
+        .process_key = sc_key_processor_process_key,
+        // Never forward text input via HID (all the keys are injected
+        // separately)
+        .process_text = NULL,
+    };
+
+    // Clipboard synchronization is requested over the same control socket, so
+    // there is no need for a specific synchronization mechanism
+    kb->key_processor.async_paste = false;
+    kb->key_processor.hid = true;
+    kb->key_processor.ops = &ops;
+
+    static const struct sc_uhid_receiver_ops uhid_receiver_ops = {
+        .process_output = sc_uhid_receiver_process_output,
+    };
+
+    kb->uhid_receiver.id = UHID_KEYBOARD_ID;
+    kb->uhid_receiver.ops = &uhid_receiver_ops;
+    sc_uhid_devices_add_receiver(uhid_devices, &kb->uhid_receiver);
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_KEYBOARD_ID;
+    msg.uhid_create.report_desc = SC_HID_KEYBOARD_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_KEYBOARD_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (keyboard)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/keyboard_uhid.h

@@ -0,0 +1,27 @@
+#ifndef SC_KEYBOARD_UHID_H
+#define SC_KEYBOARD_UHID_H
+
+#include "common.h"
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "hid/hid_keyboard.h"
+#include "uhid/uhid_output.h"
+#include "trait/key_processor.h"
+
+struct sc_keyboard_uhid {
+    struct sc_key_processor key_processor; // key processor trait
+    struct sc_uhid_receiver uhid_receiver;
+
+    struct sc_hid_keyboard hid;
+    struct sc_controller *controller;
+    atomic_uint_least16_t device_mod;
+};
+
+bool
+sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
+                      struct sc_controller *controller,
+                      struct sc_uhid_devices *uhid_devices);
+
+#endif

app/src/uhid/mouse_uhid.c

@@ -0,0 +1,89 @@
+#include "mouse_uhid.h"
+
+#include "hid/hid_mouse.h"
+#include "input_events.h"
+#include "util/log.h"
+
+/** Downcast mouse processor to mouse_uhid */
+#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
+
+#define UHID_MOUSE_ID 2
+
+static void
+sc_mouse_uhid_send_input(struct sc_mouse_uhid *mouse,
+                         const struct sc_hid_event *event, const char *name) {
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
+    msg.uhid_input.id = UHID_MOUSE_ID;
+
+    assert(event->size <= SC_HID_MAX_SIZE);
+    memcpy(msg.uhid_input.data, event->data, event->size);
+    msg.uhid_input.size = event->size;
+
+    if (!sc_controller_push_msg(mouse->controller, &msg)) {
+        LOGE("Could not send UHID_INPUT message (%s)", name);
+    }
+}
+
+static void
+sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_motion_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_motion(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse motion");
+}
+
+static void
+sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
+                                   const struct sc_mouse_click_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_click(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse click");
+}
+
+static void
+sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
+                                    const struct sc_mouse_scroll_event *event) {
+    struct sc_mouse_uhid *mouse = DOWNCAST(mp);
+
+    struct sc_hid_event hid_event;
+    sc_hid_mouse_event_from_scroll(&hid_event, event);
+
+    sc_mouse_uhid_send_input(mouse, &hid_event, "mouse scroll");
+}
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller) {
+    mouse->controller = controller;
+
+    static const struct sc_mouse_processor_ops ops = {
+        .process_mouse_motion = sc_mouse_processor_process_mouse_motion,
+        .process_mouse_click = sc_mouse_processor_process_mouse_click,
+        .process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
+        // Touch events not supported (coordinates are not relative)
+        .process_touch = NULL,
+    };
+
+    mouse->mouse_processor.ops = &ops;
+
+    mouse->mouse_processor.relative_mode = true;
+
+    struct sc_control_msg msg;
+    msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
+    msg.uhid_create.id = UHID_MOUSE_ID;
+    msg.uhid_create.report_desc = SC_HID_MOUSE_REPORT_DESC;
+    msg.uhid_create.report_desc_size = SC_HID_MOUSE_REPORT_DESC_LEN;
+    if (!sc_controller_push_msg(controller, &msg)) {
+        LOGE("Could not send UHID_CREATE message (mouse)");
+        return false;
+    }
+
+    return true;
+}

app/src/uhid/mouse_uhid.h

@@ -0,0 +1,19 @@
+#ifndef SC_MOUSE_UHID_H
+#define SC_MOUSE_UHID_H
+
+#include <stdbool.h>
+
+#include "controller.h"
+#include "trait/mouse_processor.h"
+
+struct sc_mouse_uhid {
+    struct sc_mouse_processor mouse_processor; // mouse processor trait
+
+    struct sc_controller *controller;
+};
+
+bool
+sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
+                   struct sc_controller *controller);
+
+#endif

app/src/uhid/uhid_output.c

@@ -0,0 +1,25 @@
+#include "uhid_output.h"
+
+#include <assert.h>
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices) {
+    devices->count = 0;
+}
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver) {
+    assert(devices->count < SC_UHID_MAX_RECEIVERS);
+    devices->receivers[devices->count++] = receiver;
+}
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id) {
+    for (size_t i = 0; i < devices->count; ++i) {
+        if (devices->receivers[i]->id == id) {
+            return devices->receivers[i];
+        }
+    }
+    return NULL;
+}

app/src/uhid/uhid_output.h

@@ -0,0 +1,45 @@
+#ifndef SC_UHID_OUTPUT_H
+#define SC_UHID_OUTPUT_H
+
+#include "common.h"
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/**
+ * The communication with UHID devices is bidirectional.
+ *
+ * This component manages the registration of receivers to handle UHID output
+ * messages (sent from the device to the computer).
+ */
+
+struct sc_uhid_receiver {
+    uint16_t id;
+
+    const struct sc_uhid_receiver_ops *ops;
+};
+
+struct sc_uhid_receiver_ops {
+    void
+    (*process_output)(struct sc_uhid_receiver *receiver,
+                      const uint8_t *data, size_t len);
+};
+
+#define SC_UHID_MAX_RECEIVERS 1
+
+struct sc_uhid_devices {
+    struct sc_uhid_receiver *receivers[SC_UHID_MAX_RECEIVERS];
+    unsigned count;
+};
+
+void
+sc_uhid_devices_init(struct sc_uhid_devices *devices);
+
+void
+sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
+                             struct sc_uhid_receiver *receiver);
+
+struct sc_uhid_receiver *
+sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id);
+
+#endif

app/src/usb/aoa_hid.c

@@ -5,6 +5,7 @@
 
 #include "aoa_hid.h"
 #include "util/log.h"
+#include "util/str.h"
 
 // See <https://source.android.com/devices/accessories/aoa2#hid-support>.
 #define ACCESSORY_REGISTER_HID 54
@@ -14,48 +15,37 @@
 
 #define DEFAULT_TIMEOUT 1000
 
+#define SC_AOA_EVENT_QUEUE_MAX 64
+
 static void
-sc_hid_event_log(const struct sc_hid_event *event) {
+sc_hid_event_log(uint16_t accessory_id, const struct sc_hid_event *event) {
     // HID Event: [00] FF FF FF FF...
     assert(event->size);
-    unsigned buffer_size = event->size * 3 + 1;
-    char *buffer = malloc(buffer_size);
-    if (!buffer) {
-        LOG_OOM();
+    char *hex = sc_str_to_hex_string(event->data, event->size);
+    if (!hex) {
         return;
     }
-    for (unsigned i = 0; i < event->size; ++i) {
-        snprintf(buffer + i * 3, 4, " %02x", event->buffer[i]);
-    }
-    LOGV("HID Event: [%d]%s", event->accessory_id, buffer);
-    free(buffer);
-}
-
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size) {
-    hid_event->accessory_id = accessory_id;
-    hid_event->buffer = buffer;
-    hid_event->size = buffer_size;
-    hid_event->ack_to_wait = SC_SEQUENCE_INVALID;
-}
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event) {
-    free(hid_event->buffer);
+    LOGV("HID Event: [%d] %s", accessory_id, hex);
+    free(hex);
 }
 
 bool
 sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
             struct sc_acksync *acksync) {
-    cbuf_init(&aoa->queue);
+    sc_vecdeque_init(&aoa->queue);
+
+    if (!sc_vecdeque_reserve(&aoa->queue, SC_AOA_EVENT_QUEUE_MAX)) {
+        return false;
+    }
 
     if (!sc_mutex_init(&aoa->mutex)) {
+        sc_vecdeque_destroy(&aoa->queue);
         return false;
     }
 
     if (!sc_cond_init(&aoa->event_cond)) {
         sc_mutex_destroy(&aoa->mutex);
+        sc_vecdeque_destroy(&aoa->queue);
         return false;
     }
 
@@ -68,11 +58,7 @@ sc_aoa_init(struct sc_aoa *aoa, struct sc_usb *usb,
 
 void
 sc_aoa_destroy(struct sc_aoa *aoa) {
-    // Destroy remaining events
-    struct sc_hid_event event;
-    while (cbuf_take(&aoa->queue, &event)) {
-        sc_hid_event_destroy(&event);
-    }
+    sc_vecdeque_destroy(&aoa->queue);
 
     sc_cond_destroy(&aoa->event_cond);
     sc_mutex_destroy(&aoa->mutex);
@@ -88,10 +74,10 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
     // index (arg1): total length of the HID report descriptor
     uint16_t value = accessory_id;
     uint16_t index = report_desc_size;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("REGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -104,7 +90,7 @@ sc_aoa_register_hid(struct sc_aoa *aoa, uint16_t accessory_id,
 
 static bool
 sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
-                           const unsigned char *report_desc,
+                           const uint8_t *report_desc,
                            uint16_t report_desc_size) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_SET_HID_REPORT_DESC;
@@ -121,14 +107,14 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
      * See <https://libusb.sourceforge.io/api-1.0/libusb_packetoverflow.html>
      */
     // value (arg0): accessory assigned ID for the HID device
-    // index (arg1): offset of data (buffer) in descriptor
+    // index (arg1): offset of data in descriptor
     uint16_t value = accessory_id;
     uint16_t index = 0;
     // libusb_control_transfer expects a pointer to non-const
-    unsigned char *buffer = (unsigned char *) report_desc;
+    unsigned char *data = (unsigned char *) report_desc;
     uint16_t length = report_desc_size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SET_HID_REPORT_DESC: libusb error: %s", libusb_strerror(result));
@@ -141,7 +127,7 @@ sc_aoa_set_hid_report_desc(struct sc_aoa *aoa, uint16_t accessory_id,
 
 bool
 sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
-                 const unsigned char *report_desc, uint16_t report_desc_size) {
+                 const uint8_t *report_desc, uint16_t report_desc_size) {
     bool ok = sc_aoa_register_hid(aoa, accessory_id, report_desc_size);
     if (!ok) {
         return false;
@@ -160,18 +146,19 @@ sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
 }
 
 static bool
-sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_send_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_SEND_HID_EVENT;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
     // value (arg0): accessory assigned ID for the HID device
     // index (arg1): 0 (unused)
-    uint16_t value = event->accessory_id;
+    uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = event->buffer;
+    unsigned char *data = (uint8_t *) event->data; // discard const
     uint16_t length = event->size;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("SEND_HID_EVENT: libusb error: %s", libusb_strerror(result));
@@ -183,7 +170,7 @@ sc_aoa_send_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
 }
 
 bool
-sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
+sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id) {
     uint8_t request_type = LIBUSB_ENDPOINT_OUT | LIBUSB_REQUEST_TYPE_VENDOR;
     uint8_t request = ACCESSORY_UNREGISTER_HID;
     // <https://source.android.com/devices/accessories/aoa2.html#hid-support>
@@ -191,10 +178,10 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
     // index (arg1): 0
     uint16_t value = accessory_id;
     uint16_t index = 0;
-    unsigned char *buffer = NULL;
+    unsigned char *data = NULL;
     uint16_t length = 0;
     int result = libusb_control_transfer(aoa->usb->handle, request_type,
-                                         request, value, index, buffer, length,
+                                         request, value, index, data, length,
                                          DEFAULT_TIMEOUT);
     if (result < 0) {
         LOGE("UNREGISTER_HID: libusb error: %s", libusb_strerror(result));
@@ -206,19 +193,34 @@ sc_aoa_unregister_hid(struct sc_aoa *aoa, const uint16_t accessory_id) {
 }
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event) {
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait) {
     if (sc_get_log_level() <= SC_LOG_LEVEL_VERBOSE) {
-        sc_hid_event_log(event);
+        sc_hid_event_log(accessory_id, event);
     }
 
     sc_mutex_lock(&aoa->mutex);
-    bool was_empty = cbuf_is_empty(&aoa->queue);
-    bool res = cbuf_push(&aoa->queue, *event);
-    if (was_empty) {
-        sc_cond_signal(&aoa->event_cond);
+    bool full = sc_vecdeque_is_full(&aoa->queue);
+    if (!full) {
+        bool was_empty = sc_vecdeque_is_empty(&aoa->queue);
+
+        struct sc_aoa_event *aoa_event =
+            sc_vecdeque_push_hole_noresize(&aoa->queue);
+        aoa_event->hid = *event;
+        aoa_event->accessory_id = accessory_id;
+        aoa_event->ack_to_wait = ack_to_wait;
+
+        if (was_empty) {
+            sc_cond_signal(&aoa->event_cond);
+        }
     }
+    // Otherwise (if the queue is full), the event is discarded
+
     sc_mutex_unlock(&aoa->mutex);
-    return res;
+
+    return !full;
 }
 
 static int
@@ -227,7 +229,7 @@ run_aoa_thread(void *data) {
 
     for (;;) {
         sc_mutex_lock(&aoa->mutex);
-        while (!aoa->stopped && cbuf_is_empty(&aoa->queue)) {
+        while (!aoa->stopped && sc_vecdeque_is_empty(&aoa->queue)) {
             sc_cond_wait(&aoa->event_cond, &aoa->mutex);
         }
         if (aoa->stopped) {
@@ -235,11 +237,9 @@ run_aoa_thread(void *data) {
             sc_mutex_unlock(&aoa->mutex);
             break;
         }
-        struct sc_hid_event event;
-        bool non_empty = cbuf_take(&aoa->queue, &event);
-        assert(non_empty);
-        (void) non_empty;
 
+        assert(!sc_vecdeque_is_empty(&aoa->queue));
+        struct sc_aoa_event event = sc_vecdeque_pop(&aoa->queue);
         uint64_t ack_to_wait = event.ack_to_wait;
         sc_mutex_unlock(&aoa->mutex);
 
@@ -258,17 +258,14 @@ run_aoa_thread(void *data) {
 
             if (result == SC_ACKSYNC_WAIT_TIMEOUT) {
                 LOGW("Ack not received after 500ms, discarding HID event");
-                sc_hid_event_destroy(&event);
                 continue;
             } else if (result == SC_ACKSYNC_WAIT_INTR) {
                 // stopped
-                sc_hid_event_destroy(&event);
                 break;
             }
         }
 
-        bool ok = sc_aoa_send_hid_event(aoa, &event);
-        sc_hid_event_destroy(&event);
+        bool ok = sc_aoa_send_hid_event(aoa, event.accessory_id, &event.hid);
         if (!ok) {
             LOGW("Could not send HID event to USB device");
         }

app/src/usb/aoa_hid.h

@@ -6,28 +6,22 @@
 
 #include <libusb-1.0/libusb.h>
 
+#include "hid/hid_event.h"
 #include "usb.h"
 #include "util/acksync.h"
-#include "util/cbuf.h"
 #include "util/thread.h"
 #include "util/tick.h"
+#include "util/vecdeque.h"
 
-struct sc_hid_event {
+#define SC_HID_MAX_SIZE 8
+
+struct sc_aoa_event {
+    struct sc_hid_event hid;
     uint16_t accessory_id;
-    unsigned char *buffer;
-    uint16_t size;
     uint64_t ack_to_wait;
 };
 
-// Takes ownership of buffer
-void
-sc_hid_event_init(struct sc_hid_event *hid_event, uint16_t accessory_id,
-                  unsigned char *buffer, uint16_t buffer_size);
-
-void
-sc_hid_event_destroy(struct sc_hid_event *hid_event);
-
-struct sc_hid_event_queue CBUF(struct sc_hid_event, 64);
+struct sc_aoa_event_queue SC_VECDEQUE(struct sc_aoa_event);
 
 struct sc_aoa {
     struct sc_usb *usb;
@@ -35,7 +29,7 @@ struct sc_aoa {
     sc_mutex mutex;
     sc_cond event_cond;
     bool stopped;
-    struct sc_hid_event_queue queue;
+    struct sc_aoa_event_queue queue;
 
     struct sc_acksync *acksync;
 };
@@ -57,12 +51,22 @@ sc_aoa_join(struct sc_aoa *aoa);
 
 bool
 sc_aoa_setup_hid(struct sc_aoa *aoa, uint16_t accessory_id,
-              const unsigned char *report_desc, uint16_t report_desc_size);
+              const uint8_t *report_desc, uint16_t report_desc_size);
 
 bool
 sc_aoa_unregister_hid(struct sc_aoa *aoa, uint16_t accessory_id);
 
 bool
-sc_aoa_push_hid_event(struct sc_aoa *aoa, const struct sc_hid_event *event);
+sc_aoa_push_hid_event_with_ack_to_wait(struct sc_aoa *aoa,
+                                       uint16_t accessory_id,
+                                       const struct sc_hid_event *event,
+                                       uint64_t ack_to_wait);
+
+static inline bool
+sc_aoa_push_hid_event(struct sc_aoa *aoa, uint16_t accessory_id,
+                      const struct sc_hid_event *event) {
+    return sc_aoa_push_hid_event_with_ack_to_wait(aoa, accessory_id, event,
+                                                  SC_SEQUENCE_INVALID);
+}
 
 #endif