mirror of https://github.com/Genymobile/scrcpy
Compare commits
No commits in common. 'master' and 'v1.25' have entirely different histories.
@ -0,0 +1,309 @@
|
||||
# scrcpy for developers
|
||||
|
||||
## Overview
|
||||
|
||||
This application is composed of two parts:
|
||||
- the server (`scrcpy-server`), to be executed on the device,
|
||||
- the client (the `scrcpy` binary), executed on the host computer.
|
||||
|
||||
The client is responsible to push the server to the device and start its
|
||||
execution.
|
||||
|
||||
Once the client and the server are connected to each other, the server initially
|
||||
sends device information (name and initial screen dimensions), then starts to
|
||||
send a raw H.264 video stream of the device screen. The client decodes the video
|
||||
frames, and display them as soon as possible, without buffering, to minimize
|
||||
latency. The client is not aware of the device rotation (which is handled by the
|
||||
server), it just knows the dimensions of the video frames.
|
||||
|
||||
The client captures relevant keyboard and mouse events, that it transmits to the
|
||||
server, which injects them to the device.
|
||||
|
||||
|
||||
|
||||
## Server
|
||||
|
||||
|
||||
### Privileges
|
||||
|
||||
Capturing the screen requires some privileges, which are granted to `shell`.
|
||||
|
||||
The server is a Java application (with a [`public static void main(String...
|
||||
args)`][main] method), compiled against the Android framework, and executed as
|
||||
`shell` on the Android device.
|
||||
|
||||
[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123
|
||||
|
||||
To run such a Java application, the classes must be [_dexed_][dex] (typically,
|
||||
to `classes.dex`). If `my.package.MainClass` is the main class, compiled to
|
||||
`classes.dex`, pushed to the device in `/data/local/tmp`, then it can be run
|
||||
with:
|
||||
|
||||
adb shell CLASSPATH=/data/local/tmp/classes.dex \
|
||||
app_process / my.package.MainClass
|
||||
|
||||
_The path `/data/local/tmp` is a good candidate to push the server, since it's
|
||||
readable and writable by `shell`, but not world-writable, so a malicious
|
||||
application may not replace the server just before the client executes it._
|
||||
|
||||
Instead of a raw _dex_ file, `app_process` accepts a _jar_ containing
|
||||
`classes.dex` (e.g. an [APK]). For simplicity, and to benefit from the gradle
|
||||
build system, the server is built to an (unsigned) APK (renamed to
|
||||
`scrcpy-server`).
|
||||
|
||||
[dex]: https://en.wikipedia.org/wiki/Dalvik_(software)
|
||||
[apk]: https://en.wikipedia.org/wiki/Android_application_package
|
||||
|
||||
|
||||
### Hidden methods
|
||||
|
||||
Although compiled against the Android framework, [hidden] methods and classes are
|
||||
not directly accessible (and they may differ from one Android version to
|
||||
another).
|
||||
|
||||
They can be called using reflection though. The communication with hidden
|
||||
components is provided by [_wrappers_ classes][wrappers] and [aidl].
|
||||
|
||||
[hidden]: https://stackoverflow.com/a/31908373/1987178
|
||||
[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers
|
||||
[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view
|
||||
|
||||
|
||||
### Threading
|
||||
|
||||
The server uses 3 threads:
|
||||
|
||||
- the **main** thread, encoding and streaming the video to the client;
|
||||
- the **controller** thread, listening for _control messages_ (typically,
|
||||
keyboard and mouse events) from the client;
|
||||
- the **receiver** thread (managed by the controller), sending _device messages_
|
||||
to the clients (currently, it is only used to send the device clipboard
|
||||
content).
|
||||
|
||||
Since the video encoding is typically hardware, there would be no benefit in
|
||||
encoding and streaming in two different threads.
|
||||
|
||||
|
||||
### Screen video encoding
|
||||
|
||||
The encoding is managed by [`ScreenEncoder`].
|
||||
|
||||
The video is encoded using the [`MediaCodec`] API. The codec takes its input
|
||||
from a [surface] associated to the display, and writes the resulting H.264
|
||||
stream to the provided output stream (the socket connected to the client).
|
||||
|
||||
[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
|
||||
[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html
|
||||
[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69
|
||||
|
||||
On device [rotation], the codec, surface and display are reinitialized, and a
|
||||
new video stream is produced.
|
||||
|
||||
New frames are produced only when changes occur on the surface. This is good
|
||||
because it avoids to send unnecessary frames, but there are drawbacks:
|
||||
|
||||
- it does not send any frame on start if the device screen does not change,
|
||||
- after fast motion changes, the last frame may have poor quality.
|
||||
|
||||
Both problems are [solved][repeat] by the flag
|
||||
[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].
|
||||
|
||||
[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90
|
||||
[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148
|
||||
[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER
|
||||
|
||||
|
||||
### Input events injection
|
||||
|
||||
_Control messages_ are received from the client by the [`Controller`] (run in a
|
||||
separate thread). There are several types of input events:
|
||||
- keycode (cf [`KeyEvent`]),
|
||||
- text (special characters may not be handled by keycodes directly),
|
||||
- mouse motion/click,
|
||||
- mouse scroll,
|
||||
- other commands (e.g. to switch the screen on or to copy the clipboard).
|
||||
|
||||
Some of them need to inject input events to the system. To do so, they use the
|
||||
_hidden_ method [`InputManager.injectInputEvent`] (exposed by our
|
||||
[`InputManager` wrapper][inject-wrapper]).
|
||||
|
||||
[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81
|
||||
[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html
|
||||
[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html
|
||||
[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857
|
||||
[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27
|
||||
|
||||
|
||||
|
||||
## Client
|
||||
|
||||
The client relies on [SDL], which provides cross-platform API for UI, input
|
||||
events, threading, etc.
|
||||
|
||||
The video stream is decoded by [libav] (FFmpeg).
|
||||
|
||||
[SDL]: https://www.libsdl.org
|
||||
[libav]: https://www.libav.org/
|
||||
|
||||
### Initialization
|
||||
|
||||
On startup, in addition to _libav_ and _SDL_ initialization, the client must
|
||||
push and start the server on the device, and open two sockets (one for the video
|
||||
stream, one for control) so that they may communicate.
|
||||
|
||||
Note that the client-server roles are expressed at the application level:
|
||||
|
||||
- the server _serves_ video stream and handle requests from the client,
|
||||
- the client _controls_ the device through the server.
|
||||
|
||||
However, the roles are reversed at the network level:
|
||||
|
||||
- the client opens a server socket and listen on a port before starting the
|
||||
server,
|
||||
- the server connects to the client.
|
||||
|
||||
This role inversion guarantees that the connection will not fail due to race
|
||||
conditions, and avoids polling.
|
||||
|
||||
_(Note that over TCP/IP, the roles are not reversed, due to a bug in `adb
|
||||
reverse`. See commit [1038bad] and [issue #5].)_
|
||||
|
||||
Once the server is connected, it sends the device information (name and initial
|
||||
screen dimensions). Thus, the client may init the window and renderer, before
|
||||
the first frame is available.
|
||||
|
||||
To minimize startup time, SDL initialization is performed while listening for
|
||||
the connection from the server (see commit [90a46b4]).
|
||||
|
||||
[1038bad]: https://github.com/Genymobile/scrcpy/commit/1038bad3850f18717a048a4d5c0f8110e54ee172
|
||||
[issue #5]: https://github.com/Genymobile/scrcpy/issues/5
|
||||
[90a46b4]: https://github.com/Genymobile/scrcpy/commit/90a46b4c45637d083e877020d85ade52a9a5fa8e
|
||||
|
||||
|
||||
### Threading
|
||||
|
||||
The client uses 4 threads:
|
||||
|
||||
- the **main** thread, executing the SDL event loop,
|
||||
- the **stream** thread, receiving the video and used for decoding and
|
||||
recording,
|
||||
- the **controller** thread, sending _control messages_ to the server,
|
||||
- the **receiver** thread (managed by the controller), receiving _device
|
||||
messages_ from the server.
|
||||
|
||||
In addition, another thread can be started if necessary to handle APK
|
||||
installation or file push requests (via drag&drop on the main window) or to
|
||||
print the framerate regularly in the console.
|
||||
|
||||
|
||||
|
||||
### Stream
|
||||
|
||||
The video [stream] is received from the socket (connected to the server on the
|
||||
device) in a separate thread.
|
||||
|
||||
If a [decoder] is present (i.e. `--no-display` is not set), then it uses _libav_
|
||||
to decode the H.264 stream from the socket, and notifies the main thread when a
|
||||
new frame is available.
|
||||
|
||||
There are two [frames][video_buffer] simultaneously in memory:
|
||||
- the **decoding** frame, written by the decoder from the decoder thread,
|
||||
- the **rendering** frame, rendered in a texture from the main thread.
|
||||
|
||||
When a new decoded frame is available, the decoder _swaps_ the decoding and
|
||||
rendering frame (with proper synchronization). Thus, it immediately starts
|
||||
to decode a new frame while the main thread renders the last one.
|
||||
|
||||
If a [recorder] is present (i.e. `--record` is enabled), then it muxes the raw
|
||||
H.264 packet to the output video file.
|
||||
|
||||
[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h
|
||||
[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h
|
||||
[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h
|
||||
[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h
|
||||
|
||||
```
|
||||
+----------+ +----------+
|
||||
---> | decoder | ---> | screen |
|
||||
+---------+ / +----------+ +----------+
|
||||
socket ---> | stream | ----
|
||||
+---------+ \ +----------+
|
||||
---> | recorder |
|
||||
+----------+
|
||||
```
|
||||
|
||||
### Controller
|
||||
|
||||
The [controller] is responsible to send _control messages_ to the device. It
|
||||
runs in a separate thread, to avoid I/O on the main thread.
|
||||
|
||||
On SDL event, received on the main thread, the [input manager][inputmanager]
|
||||
creates appropriate [_control messages_][controlmsg]. It is responsible to
|
||||
convert SDL events to Android events (using [convert]). It pushes the _control
|
||||
messages_ to a queue hold by the controller. On its own thread, the controller
|
||||
takes messages from the queue, that it serializes and sends to the client.
|
||||
|
||||
[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h
|
||||
[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h
|
||||
[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h
|
||||
[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h
|
||||
|
||||
|
||||
### UI and event loop
|
||||
|
||||
Initialization, input events and rendering are all [managed][scrcpy] in the main
|
||||
thread.
|
||||
|
||||
Events are handled in the [event loop], which either updates the [screen] or
|
||||
delegates to the [input manager][inputmanager].
|
||||
|
||||
[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c
|
||||
[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201
|
||||
[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h
|
||||
|
||||
|
||||
## Hack
|
||||
|
||||
For more details, go read the code!
|
||||
|
||||
If you find a bug, or have an awesome idea to implement, please discuss and
|
||||
contribute ;-)
|
||||
|
||||
|
||||
### Debug the server
|
||||
|
||||
The server is pushed to the device by the client on startup.
|
||||
|
||||
To debug it, enable the server debugger during configuration:
|
||||
|
||||
```bash
|
||||
meson setup x -Dserver_debugger=true
|
||||
# or, if x is already configured
|
||||
meson configure x -Dserver_debugger=true
|
||||
```
|
||||
|
||||
If your device runs Android 8 or below, set the `server_debugger_method` to
|
||||
`old` in addition:
|
||||
|
||||
```bash
|
||||
meson setup x -Dserver_debugger=true -Dserver_debugger_method=old
|
||||
# or, if x is already configured
|
||||
meson configure x -Dserver_debugger=true -Dserver_debugger_method=old
|
||||
```
|
||||
|
||||
Then recompile.
|
||||
|
||||
When you start scrcpy, it will start a debugger on port 5005 on the device.
|
||||
Redirect that port to the computer:
|
||||
|
||||
```bash
|
||||
adb forward tcp:5005 tcp:5005
|
||||
```
|
||||
|
||||
In Android Studio, _Run_ > _Debug_ > _Edit configurations..._ On the left, click on
|
||||
`+`, _Remote_, and fill the form:
|
||||
|
||||
- Host: `localhost`
|
||||
- Port: `5005`
|
||||
|
||||
Then click on _Debug_.
|
||||
@ -1,2 +1,4 @@
|
||||
@echo off
|
||||
scrcpy.exe --pause-on-exit=if-error %*
|
||||
scrcpy.exe %*
|
||||
:: if the exit code is >= 1, then pause
|
||||
if errorlevel 1 pause
|
||||
|
||||
@ -1 +0,0 @@
|
||||
/work
|
||||
@ -1,27 +0,0 @@
|
||||
This directory (app/deps/) contains:
|
||||
|
||||
*.sh : shell scripts to download and build dependencies
|
||||
|
||||
patches/ : patches to fix dependencies (used by scripts)
|
||||
|
||||
work/sources/ : downloaded tarballs and extracted folders
|
||||
ffmpeg-6.1.1.tar.xz
|
||||
ffmpeg-6.1.1/
|
||||
libusb-1.0.27.tar.gz
|
||||
libusb-1.0.27/
|
||||
...
|
||||
work/build/ : build dirs for each dependency/version/architecture
|
||||
ffmpeg-6.1.1/win32/
|
||||
ffmpeg-6.1.1/win64/
|
||||
libusb-1.0.27/win32/
|
||||
libusb-1.0.27/win64/
|
||||
...
|
||||
work/install/ : install dirs for each architexture
|
||||
win32/bin/
|
||||
win32/include/
|
||||
win32/lib/
|
||||
win32/share/
|
||||
win64/bin/
|
||||
win64/include/
|
||||
win64/lib/
|
||||
win64/share/
|
||||
@ -1,32 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DEPS_DIR"
|
||||
. common
|
||||
|
||||
VERSION=34.0.5
|
||||
FILENAME=platform-tools_r$VERSION-windows.zip
|
||||
PROJECT_DIR=platform-tools-$VERSION
|
||||
SHA256SUM=3f8320152704377de150418a3c4c9d07d16d80a6c0d0d8f7289c22c499e33571
|
||||
|
||||
cd "$SOURCES_DIR"
|
||||
|
||||
if [[ -d "$PROJECT_DIR" ]]
|
||||
then
|
||||
echo "$PWD/$PROJECT_DIR" found
|
||||
else
|
||||
get_file "https://dl.google.com/android/repository/$FILENAME" "$FILENAME" "$SHA256SUM"
|
||||
mkdir -p "$PROJECT_DIR"
|
||||
cd "$PROJECT_DIR"
|
||||
ZIP_PREFIX=platform-tools
|
||||
unzip "../$FILENAME" \
|
||||
"$ZIP_PREFIX"/AdbWinApi.dll \
|
||||
"$ZIP_PREFIX"/AdbWinUsbApi.dll \
|
||||
"$ZIP_PREFIX"/adb.exe
|
||||
mv "$ZIP_PREFIX"/* .
|
||||
rmdir "$ZIP_PREFIX"
|
||||
fi
|
||||
|
||||
mkdir -p "$INSTALL_DIR/$HOST/bin"
|
||||
cd "$INSTALL_DIR/$HOST/bin"
|
||||
cp -r "$SOURCES_DIR/$PROJECT_DIR"/. "$INSTALL_DIR/$HOST/bin/"
|
||||
@ -1,55 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# This file is intended to be sourced by other scripts, not executed
|
||||
|
||||
if [[ $# != 1 ]]
|
||||
then
|
||||
# <host>: win32 or win64
|
||||
echo "Syntax: $0 <host>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
HOST="$1"
|
||||
|
||||
if [[ "$HOST" = win32 ]]
|
||||
then
|
||||
HOST_TRIPLET=i686-w64-mingw32
|
||||
elif [[ "$HOST" = win64 ]]
|
||||
then
|
||||
HOST_TRIPLET=x86_64-w64-mingw32
|
||||
else
|
||||
echo "Unsupported host: $HOST" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DEPS_DIR"
|
||||
|
||||
PATCHES_DIR="$PWD/patches"
|
||||
|
||||
WORK_DIR="$PWD/work"
|
||||
SOURCES_DIR="$WORK_DIR/sources"
|
||||
BUILD_DIR="$WORK_DIR/build"
|
||||
INSTALL_DIR="$WORK_DIR/install"
|
||||
|
||||
mkdir -p "$INSTALL_DIR" "$SOURCES_DIR" "$WORK_DIR"
|
||||
|
||||
checksum() {
|
||||
local file="$1"
|
||||
local sum="$2"
|
||||
echo "$file: verifying checksum..."
|
||||
echo "$sum $file" | sha256sum -c
|
||||
}
|
||||
|
||||
get_file() {
|
||||
local url="$1"
|
||||
local file="$2"
|
||||
local sum="$3"
|
||||
if [[ -f "$file" ]]
|
||||
then
|
||||
echo "$file: found"
|
||||
else
|
||||
echo "$file: not found, downloading..."
|
||||
wget "$url" -O "$file"
|
||||
fi
|
||||
checksum "$file" "$sum"
|
||||
}
|
||||
@ -1,91 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DEPS_DIR"
|
||||
. common
|
||||
|
||||
VERSION=6.1.1
|
||||
FILENAME=ffmpeg-$VERSION.tar.xz
|
||||
PROJECT_DIR=ffmpeg-$VERSION
|
||||
SHA256SUM=8684f4b00f94b85461884c3719382f1261f0d9eb3d59640a1f4ac0873616f968
|
||||
|
||||
cd "$SOURCES_DIR"
|
||||
|
||||
if [[ -d "$PROJECT_DIR" ]]
|
||||
then
|
||||
echo "$PWD/$PROJECT_DIR" found
|
||||
else
|
||||
get_file "https://ffmpeg.org/releases/$FILENAME" "$FILENAME" "$SHA256SUM"
|
||||
tar xf "$FILENAME" # First level directory is "$PROJECT_DIR"
|
||||
patch -d "$PROJECT_DIR" -p1 < "$PATCHES_DIR"/ffmpeg-6.1-fix-build.patch
|
||||
fi
|
||||
|
||||
mkdir -p "$BUILD_DIR/$PROJECT_DIR"
|
||||
cd "$BUILD_DIR/$PROJECT_DIR"
|
||||
|
||||
if [[ "$HOST" = win32 ]]
|
||||
then
|
||||
ARCH=x86
|
||||
elif [[ "$HOST" = win64 ]]
|
||||
then
|
||||
ARCH=x86_64
|
||||
else
|
||||
echo "Unsupported host: $HOST" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# -static-libgcc to avoid missing libgcc_s_dw2-1.dll
|
||||
# -static to avoid dynamic dependency to zlib
|
||||
export CFLAGS='-static-libgcc -static'
|
||||
export CXXFLAGS="$CFLAGS"
|
||||
export LDFLAGS='-static-libgcc -static'
|
||||
|
||||
if [[ -d "$HOST" ]]
|
||||
then
|
||||
echo "'$PWD/$HOST' already exists, not reconfigured"
|
||||
cd "$HOST"
|
||||
else
|
||||
mkdir "$HOST"
|
||||
cd "$HOST"
|
||||
|
||||
"$SOURCES_DIR/$PROJECT_DIR"/configure \
|
||||
--prefix="$INSTALL_DIR/$HOST" \
|
||||
--enable-cross-compile \
|
||||
--target-os=mingw32 \
|
||||
--arch="$ARCH" \
|
||||
--cross-prefix="${HOST_TRIPLET}-" \
|
||||
--cc="${HOST_TRIPLET}-gcc" \
|
||||
--extra-cflags="-O2 -fPIC" \
|
||||
--enable-shared \
|
||||
--disable-static \
|
||||
--disable-programs \
|
||||
--disable-doc \
|
||||
--disable-swscale \
|
||||
--disable-postproc \
|
||||
--disable-avfilter \
|
||||
--disable-avdevice \
|
||||
--disable-network \
|
||||
--disable-everything \
|
||||
--enable-swresample \
|
||||
--enable-decoder=h264 \
|
||||
--enable-decoder=hevc \
|
||||
--enable-decoder=av1 \
|
||||
--enable-decoder=pcm_s16le \
|
||||
--enable-decoder=opus \
|
||||
--enable-decoder=aac \
|
||||
--enable-decoder=flac \
|
||||
--enable-decoder=png \
|
||||
--enable-protocol=file \
|
||||
--enable-demuxer=image2 \
|
||||
--enable-parser=png \
|
||||
--enable-zlib \
|
||||
--enable-muxer=matroska \
|
||||
--enable-muxer=mp4 \
|
||||
--enable-muxer=opus \
|
||||
--enable-muxer=flac \
|
||||
--enable-muxer=wav \
|
||||
--disable-vulkan
|
||||
fi
|
||||
|
||||
make -j
|
||||
make install
|
||||
@ -1,44 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DEPS_DIR"
|
||||
. common
|
||||
|
||||
VERSION=1.0.27
|
||||
FILENAME=libusb-$VERSION.tar.bz2
|
||||
PROJECT_DIR=libusb-$VERSION
|
||||
SHA256SUM=ffaa41d741a8a3bee244ac8e54a72ea05bf2879663c098c82fc5757853441575
|
||||
|
||||
cd "$SOURCES_DIR"
|
||||
|
||||
if [[ -d "$PROJECT_DIR" ]]
|
||||
then
|
||||
echo "$PWD/$PROJECT_DIR" found
|
||||
else
|
||||
get_file "https://github.com/libusb/libusb/releases/download/v$VERSION/libusb-$VERSION.tar.bz2" "$FILENAME" "$SHA256SUM"
|
||||
tar xf "$FILENAME" # First level directory is "$PROJECT_DIR"
|
||||
fi
|
||||
|
||||
mkdir -p "$BUILD_DIR/$PROJECT_DIR"
|
||||
cd "$BUILD_DIR/$PROJECT_DIR"
|
||||
|
||||
export CFLAGS='-O2'
|
||||
export CXXFLAGS="$CFLAGS"
|
||||
|
||||
if [[ -d "$HOST" ]]
|
||||
then
|
||||
echo "'$PWD/$HOST' already exists, not reconfigured"
|
||||
cd "$HOST"
|
||||
else
|
||||
mkdir "$HOST"
|
||||
cd "$HOST"
|
||||
|
||||
"$SOURCES_DIR/$PROJECT_DIR"/configure \
|
||||
--prefix="$INSTALL_DIR/$HOST" \
|
||||
--host="$HOST_TRIPLET" \
|
||||
--enable-shared \
|
||||
--disable-static
|
||||
fi
|
||||
|
||||
make -j
|
||||
make install-strip
|
||||
@ -1,27 +0,0 @@
|
||||
From 03c80197afb324da38c9b70254231e3fdcfa68fc Mon Sep 17 00:00:00 2001
|
||||
From: Romain Vimont <rom@rom1v.com>
|
||||
Date: Sun, 12 Nov 2023 17:58:50 +0100
|
||||
Subject: [PATCH] Fix FFmpeg 6.1 build
|
||||
|
||||
Build failed on tag n6.1 With --enable-decoder=av1 but without
|
||||
--enable-muxer=av1.
|
||||
---
|
||||
libavcodec/Makefile | 2 +-
|
||||
1 file changed, 1 insertion(+), 1 deletion(-)
|
||||
|
||||
diff --git a/libavcodec/Makefile b/libavcodec/Makefile
|
||||
index 580a8d6b54..aff19b670c 100644
|
||||
--- a/libavcodec/Makefile
|
||||
+++ b/libavcodec/Makefile
|
||||
@@ -249,7 +249,7 @@ OBJS-$(CONFIG_ATRAC3PAL_DECODER) += atrac3plusdec.o atrac3plus.o \
|
||||
OBJS-$(CONFIG_ATRAC9_DECODER) += atrac9dec.o
|
||||
OBJS-$(CONFIG_AURA_DECODER) += cyuv.o
|
||||
OBJS-$(CONFIG_AURA2_DECODER) += aura.o
|
||||
-OBJS-$(CONFIG_AV1_DECODER) += av1dec.o
|
||||
+OBJS-$(CONFIG_AV1_DECODER) += av1dec.o av1_parse.o
|
||||
OBJS-$(CONFIG_AV1_CUVID_DECODER) += cuviddec.o
|
||||
OBJS-$(CONFIG_AV1_MEDIACODEC_DECODER) += mediacodecdec.o
|
||||
OBJS-$(CONFIG_AV1_MEDIACODEC_ENCODER) += mediacodecenc.o
|
||||
--
|
||||
2.42.0
|
||||
|
||||
@ -1,47 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -ex
|
||||
DEPS_DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DEPS_DIR"
|
||||
. common
|
||||
|
||||
VERSION=2.28.5
|
||||
FILENAME=SDL-$VERSION.tar.gz
|
||||
PROJECT_DIR=SDL-release-$VERSION
|
||||
SHA256SUM=9f0556e4a24ef5b267010038ad9e9948b62f236d5bcc4b22179f95ef62d84023
|
||||
|
||||
cd "$SOURCES_DIR"
|
||||
|
||||
if [[ -d "$PROJECT_DIR" ]]
|
||||
then
|
||||
echo "$PWD/$PROJECT_DIR" found
|
||||
else
|
||||
get_file "https://github.com/libsdl-org/SDL/archive/refs/tags/release-$VERSION.tar.gz" "$FILENAME" "$SHA256SUM"
|
||||
tar xf "$FILENAME" # First level directory is "$PROJECT_DIR"
|
||||
fi
|
||||
|
||||
mkdir -p "$BUILD_DIR/$PROJECT_DIR"
|
||||
cd "$BUILD_DIR/$PROJECT_DIR"
|
||||
|
||||
export CFLAGS='-O2'
|
||||
export CXXFLAGS="$CFLAGS"
|
||||
|
||||
if [[ -d "$HOST" ]]
|
||||
then
|
||||
echo "'$PWD/$HOST' already exists, not reconfigured"
|
||||
cd "$HOST"
|
||||
else
|
||||
mkdir "$HOST"
|
||||
cd "$HOST"
|
||||
|
||||
"$SOURCES_DIR/$PROJECT_DIR"/configure \
|
||||
--prefix="$INSTALL_DIR/$HOST" \
|
||||
--host="$HOST_TRIPLET" \
|
||||
--enable-shared \
|
||||
--disable-static
|
||||
fi
|
||||
|
||||
make -j
|
||||
# There is no "make install-strip"
|
||||
make install
|
||||
# Strip manually
|
||||
${HOST_TRIPLET}-strip "$INSTALL_DIR/$HOST/bin/SDL2.dll"
|
||||
@ -0,0 +1 @@
|
||||
/data
|
||||
@ -0,0 +1,22 @@
|
||||
PREBUILT_DATA_DIR=data
|
||||
|
||||
checksum() {
|
||||
local file="$1"
|
||||
local sum="$2"
|
||||
echo "$file: verifying checksum..."
|
||||
echo "$sum $file" | sha256sum -c
|
||||
}
|
||||
|
||||
get_file() {
|
||||
local url="$1"
|
||||
local file="$2"
|
||||
local sum="$3"
|
||||
if [[ -f "$file" ]]
|
||||
then
|
||||
echo "$file: found"
|
||||
else
|
||||
echo "$file: not found, downloading..."
|
||||
wget "$url" -O "$file"
|
||||
fi
|
||||
checksum "$file" "$sum"
|
||||
}
|
||||
@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DIR"
|
||||
. common
|
||||
mkdir -p "$PREBUILT_DATA_DIR"
|
||||
cd "$PREBUILT_DATA_DIR"
|
||||
|
||||
DEP_DIR=platform-tools-33.0.3
|
||||
|
||||
FILENAME=platform-tools_r33.0.3-windows.zip
|
||||
SHA256SUM=1e59afd40a74c5c0eab0a9fad3f0faf8a674267106e0b19921be9f67081808c2
|
||||
|
||||
if [[ -d "$DEP_DIR" ]]
|
||||
then
|
||||
echo "$DEP_DIR" found
|
||||
exit 0
|
||||
fi
|
||||
|
||||
get_file "https://dl.google.com/android/repository/$FILENAME" \
|
||||
"$FILENAME" "$SHA256SUM"
|
||||
|
||||
mkdir "$DEP_DIR"
|
||||
cd "$DEP_DIR"
|
||||
|
||||
ZIP_PREFIX=platform-tools
|
||||
unzip "../$FILENAME" \
|
||||
"$ZIP_PREFIX"/AdbWinApi.dll \
|
||||
"$ZIP_PREFIX"/AdbWinUsbApi.dll \
|
||||
"$ZIP_PREFIX"/adb.exe
|
||||
mv "$ZIP_PREFIX"/* .
|
||||
rmdir "$ZIP_PREFIX"
|
||||
@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DIR"
|
||||
. common
|
||||
mkdir -p "$PREBUILT_DATA_DIR"
|
||||
cd "$PREBUILT_DATA_DIR"
|
||||
|
||||
DEP_DIR=ffmpeg-win32-4.3.1
|
||||
|
||||
FILENAME_SHARED=ffmpeg-4.3.1-win32-shared.zip
|
||||
SHA256SUM_SHARED=357af9901a456f4dcbacd107e83a934d344c9cb07ddad8aaf80612eeab7d26d2
|
||||
|
||||
FILENAME_DEV=ffmpeg-4.3.1-win32-dev.zip
|
||||
SHA256SUM_DEV=230efb08e9bcf225bd474da29676c70e591fc94d8790a740ca801408fddcb78b
|
||||
|
||||
if [[ -d "$DEP_DIR" ]]
|
||||
then
|
||||
echo "$DEP_DIR" found
|
||||
exit 0
|
||||
fi
|
||||
|
||||
get_file "https://github.com/Genymobile/scrcpy/releases/download/v1.16/$FILENAME_SHARED" \
|
||||
"$FILENAME_SHARED" "$SHA256SUM_SHARED"
|
||||
get_file "https://github.com/Genymobile/scrcpy/releases/download/v1.16/$FILENAME_DEV" \
|
||||
"$FILENAME_DEV" "$SHA256SUM_DEV"
|
||||
|
||||
mkdir "$DEP_DIR"
|
||||
cd "$DEP_DIR"
|
||||
|
||||
ZIP_PREFIX_SHARED=ffmpeg-4.3.1-win32-shared
|
||||
unzip "../$FILENAME_SHARED" \
|
||||
"$ZIP_PREFIX_SHARED"/bin/avutil-56.dll \
|
||||
"$ZIP_PREFIX_SHARED"/bin/avcodec-58.dll \
|
||||
"$ZIP_PREFIX_SHARED"/bin/avformat-58.dll \
|
||||
"$ZIP_PREFIX_SHARED"/bin/swresample-3.dll \
|
||||
"$ZIP_PREFIX_SHARED"/bin/swscale-5.dll
|
||||
|
||||
ZIP_PREFIX_DEV=ffmpeg-4.3.1-win32-dev
|
||||
unzip "../$FILENAME_DEV" \
|
||||
"$ZIP_PREFIX_DEV/include/*"
|
||||
|
||||
mv "$ZIP_PREFIX_SHARED"/* .
|
||||
mv "$ZIP_PREFIX_DEV"/* .
|
||||
rmdir "$ZIP_PREFIX_SHARED" "$ZIP_PREFIX_DEV"
|
||||
@ -0,0 +1,36 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DIR"
|
||||
. common
|
||||
mkdir -p "$PREBUILT_DATA_DIR"
|
||||
cd "$PREBUILT_DATA_DIR"
|
||||
|
||||
VERSION=5.1.2
|
||||
DEP_DIR=ffmpeg-win64-$VERSION
|
||||
|
||||
FILENAME=ffmpeg-$VERSION-full_build-shared.7z
|
||||
SHA256SUM=d9eb97b72d7cfdae4d0f7eaea59ccffb8c364d67d88018ea715d5e2e193f00e9
|
||||
|
||||
if [[ -d "$DEP_DIR" ]]
|
||||
then
|
||||
echo "$DEP_DIR" found
|
||||
exit 0
|
||||
fi
|
||||
|
||||
get_file "https://github.com/GyanD/codexffmpeg/releases/download/$VERSION/$FILENAME" \
|
||||
"$FILENAME" "$SHA256SUM"
|
||||
|
||||
mkdir "$DEP_DIR"
|
||||
cd "$DEP_DIR"
|
||||
|
||||
ZIP_PREFIX=ffmpeg-$VERSION-full_build-shared
|
||||
7z x "../$FILENAME" \
|
||||
"$ZIP_PREFIX"/bin/avutil-57.dll \
|
||||
"$ZIP_PREFIX"/bin/avcodec-59.dll \
|
||||
"$ZIP_PREFIX"/bin/avformat-59.dll \
|
||||
"$ZIP_PREFIX"/bin/swresample-4.dll \
|
||||
"$ZIP_PREFIX"/bin/swscale-6.dll \
|
||||
"$ZIP_PREFIX"/include
|
||||
mv "$ZIP_PREFIX"/* .
|
||||
rmdir "$ZIP_PREFIX"
|
||||
@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DIR"
|
||||
. common
|
||||
mkdir -p "$PREBUILT_DATA_DIR"
|
||||
cd "$PREBUILT_DATA_DIR"
|
||||
|
||||
DEP_DIR=libusb-1.0.26
|
||||
|
||||
FILENAME=libusb-1.0.26-binaries.7z
|
||||
SHA256SUM=9c242696342dbde9cdc47239391f71833939bf9f7aa2bbb28cdaabe890465ec5
|
||||
|
||||
if [[ -d "$DEP_DIR" ]]
|
||||
then
|
||||
echo "$DEP_DIR" found
|
||||
exit 0
|
||||
fi
|
||||
|
||||
get_file "https://github.com/libusb/libusb/releases/download/v1.0.26/$FILENAME" "$FILENAME" "$SHA256SUM"
|
||||
|
||||
mkdir "$DEP_DIR"
|
||||
cd "$DEP_DIR"
|
||||
|
||||
# include/ is the same in all folders of the archive
|
||||
7z x "../$FILENAME" \
|
||||
libusb-1.0.26-binaries/libusb-MinGW-Win32/bin/msys-usb-1.0.dll \
|
||||
libusb-1.0.26-binaries/libusb-MinGW-x64/bin/msys-usb-1.0.dll \
|
||||
libusb-1.0.26-binaries/libusb-MinGW-x64/include/
|
||||
|
||||
mv libusb-1.0.26-binaries/libusb-MinGW-Win32/bin MinGW-Win32
|
||||
mv libusb-1.0.26-binaries/libusb-MinGW-x64/bin MinGW-x64
|
||||
mv libusb-1.0.26-binaries/libusb-MinGW-x64/include .
|
||||
rm -rf libusb-1.0.26-binaries
|
||||
@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
DIR=$(dirname ${BASH_SOURCE[0]})
|
||||
cd "$DIR"
|
||||
. common
|
||||
mkdir -p "$PREBUILT_DATA_DIR"
|
||||
cd "$PREBUILT_DATA_DIR"
|
||||
|
||||
DEP_DIR=SDL2-2.26.1
|
||||
|
||||
FILENAME=SDL2-devel-2.26.1-mingw.tar.gz
|
||||
SHA256SUM=aa43e1531a89551f9f9e14b27953a81d4ac946a9e574b5813cd0f2b36e83cc1c
|
||||
|
||||
if [[ -d "$DEP_DIR" ]]
|
||||
then
|
||||
echo "$DEP_DIR" found
|
||||
exit 0
|
||||
fi
|
||||
|
||||
get_file "https://libsdl.org/release/$FILENAME" "$FILENAME" "$SHA256SUM"
|
||||
|
||||
mkdir "$DEP_DIR"
|
||||
cd "$DEP_DIR"
|
||||
|
||||
TAR_PREFIX="$DEP_DIR" # root directory inside the tar has the same name
|
||||
tar xf "../$FILENAME" --strip-components=1 \
|
||||
"$TAR_PREFIX"/i686-w64-mingw32/bin/SDL2.dll \
|
||||
"$TAR_PREFIX"/i686-w64-mingw32/include/ \
|
||||
"$TAR_PREFIX"/i686-w64-mingw32/lib/ \
|
||||
"$TAR_PREFIX"/x86_64-w64-mingw32/bin/SDL2.dll \
|
||||
"$TAR_PREFIX"/x86_64-w64-mingw32/include/ \
|
||||
"$TAR_PREFIX"/x86_64-w64-mingw32/lib/ \
|
||||
@ -1,487 +0,0 @@
|
||||
#include "audio_player.h"
|
||||
|
||||
#include <libavcodec/avcodec.h>
|
||||
#include <libavutil/opt.h>
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
#define SC_AUDIO_PLAYER_NDEBUG // comment to debug
|
||||
|
||||
/**
|
||||
* Real-time audio player with configurable latency
|
||||
*
|
||||
* As input, the player regularly receives AVFrames of decoded audio samples.
|
||||
* As output, an SDL callback regularly requests audio samples to be played.
|
||||
* In the middle, an audio buffer stores the samples produced but not consumed
|
||||
* yet.
|
||||
*
|
||||
* The goal of the player is to feed the audio output with a latency as low as
|
||||
* possible while avoiding buffer underrun (i.e. not being able to provide
|
||||
* samples when requested).
|
||||
*
|
||||
* The player aims to feed the audio output with as little latency as possible
|
||||
* while avoiding buffer underrun. To achieve this, it attempts to maintain the
|
||||
* average buffering (the number of samples present in the buffer) around a
|
||||
* target value. If this target buffering is too low, then buffer underrun will
|
||||
* occur frequently. If it is too high, then latency will become unacceptable.
|
||||
* This target value is configured using the scrcpy option --audio-buffer.
|
||||
*
|
||||
* The player cannot adjust the sample input rate (it receives samples produced
|
||||
* in real-time) or the sample output rate (it must provide samples as
|
||||
* requested by the audio output callback). Therefore, it may only apply
|
||||
* compensation by resampling (converting _m_ input samples to _n_ output
|
||||
* samples).
|
||||
*
|
||||
* The compensation itself is applied by libswresample (FFmpeg). It is
|
||||
* configured using swr_set_compensation(). An important work for the player
|
||||
* is to estimate the compensation value regularly and apply it.
|
||||
*
|
||||
* The estimated buffering level is the result of averaging the "natural"
|
||||
* buffering (samples are produced and consumed by blocks, so it must be
|
||||
* smoothed), and making instant adjustments resulting of its own actions
|
||||
* (explicit compensation and silence insertion on underflow), which are not
|
||||
* smoothed.
|
||||
*
|
||||
* Buffer underflow events can occur when packets arrive too late. In that case,
|
||||
* the player inserts silence. Once the packets finally arrive (late), one
|
||||
* strategy could be to drop the samples that were replaced by silence, in
|
||||
* order to keep a minimal latency. However, dropping samples in case of buffer
|
||||
* underflow is inadvisable, as it would temporarily increase the underflow
|
||||
* even more and cause very noticeable audio glitches.
|
||||
*
|
||||
* Therefore, the player doesn't drop any sample on underflow. The compensation
|
||||
* mechanism will absorb the delay introduced by the inserted silence.
|
||||
*/
|
||||
|
||||
/** Downcast frame_sink to sc_audio_player */
|
||||
#define DOWNCAST(SINK) container_of(SINK, struct sc_audio_player, frame_sink)
|
||||
|
||||
#define SC_AV_SAMPLE_FMT AV_SAMPLE_FMT_FLT
|
||||
#define SC_SDL_SAMPLE_FMT AUDIO_F32
|
||||
|
||||
#define TO_BYTES(SAMPLES) sc_audiobuf_to_bytes(&ap->buf, (SAMPLES))
|
||||
#define TO_SAMPLES(BYTES) sc_audiobuf_to_samples(&ap->buf, (BYTES))
|
||||
|
||||
static void SDLCALL
|
||||
sc_audio_player_sdl_callback(void *userdata, uint8_t *stream, int len_int) {
|
||||
struct sc_audio_player *ap = userdata;
|
||||
|
||||
// This callback is called with the lock used by SDL_LockAudioDevice()
|
||||
|
||||
assert(len_int > 0);
|
||||
size_t len = len_int;
|
||||
uint32_t count = TO_SAMPLES(len);
|
||||
|
||||
#ifndef SC_AUDIO_PLAYER_NDEBUG
|
||||
LOGD("[Audio] SDL callback requests %" PRIu32 " samples", count);
|
||||
#endif
|
||||
|
||||
bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
|
||||
if (!played) {
|
||||
uint32_t buffered_samples = sc_audiobuf_can_read(&ap->buf);
|
||||
// Wait until the buffer is filled up to at least target_buffering
|
||||
// before playing
|
||||
if (buffered_samples < ap->target_buffering) {
|
||||
LOGV("[Audio] Inserting initial buffering silence: %" PRIu32
|
||||
" samples", count);
|
||||
// Delay playback starting to reach the target buffering. Fill the
|
||||
// whole buffer with silence (len is small compared to the
|
||||
// arbitrary margin value).
|
||||
memset(stream, 0, len);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
uint32_t read = sc_audiobuf_read(&ap->buf, stream, count);
|
||||
|
||||
if (read < count) {
|
||||
uint32_t silence = count - read;
|
||||
// Insert silence. In theory, the inserted silent samples replace the
|
||||
// missing real samples, which will arrive later, so they should be
|
||||
// dropped to keep the latency minimal. However, this would cause very
|
||||
// audible glitches, so let the clock compensation restore the target
|
||||
// latency.
|
||||
LOGD("[Audio] Buffer underflow, inserting silence: %" PRIu32 " samples",
|
||||
silence);
|
||||
memset(stream + TO_BYTES(read), 0, TO_BYTES(silence));
|
||||
|
||||
bool received = atomic_load_explicit(&ap->received,
|
||||
memory_order_relaxed);
|
||||
if (received) {
|
||||
// Inserting additional samples immediately increases buffering
|
||||
atomic_fetch_add_explicit(&ap->underflow, silence,
|
||||
memory_order_relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
atomic_store_explicit(&ap->played, true, memory_order_relaxed);
|
||||
}
|
||||
|
||||
static uint8_t *
|
||||
sc_audio_player_get_swr_buf(struct sc_audio_player *ap, uint32_t min_samples) {
|
||||
size_t min_buf_size = TO_BYTES(min_samples);
|
||||
if (min_buf_size > ap->swr_buf_alloc_size) {
|
||||
size_t new_size = min_buf_size + 4096;
|
||||
uint8_t *buf = realloc(ap->swr_buf, new_size);
|
||||
if (!buf) {
|
||||
LOG_OOM();
|
||||
// Could not realloc to the requested size
|
||||
return NULL;
|
||||
}
|
||||
ap->swr_buf = buf;
|
||||
ap->swr_buf_alloc_size = new_size;
|
||||
}
|
||||
|
||||
return ap->swr_buf;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_audio_player_frame_sink_push(struct sc_frame_sink *sink,
|
||||
const AVFrame *frame) {
|
||||
struct sc_audio_player *ap = DOWNCAST(sink);
|
||||
|
||||
SwrContext *swr_ctx = ap->swr_ctx;
|
||||
|
||||
int64_t swr_delay = swr_get_delay(swr_ctx, ap->sample_rate);
|
||||
// No need to av_rescale_rnd(), input and output sample rates are the same.
|
||||
// Add more space (256) for clock compensation.
|
||||
int dst_nb_samples = swr_delay + frame->nb_samples + 256;
|
||||
|
||||
uint8_t *swr_buf = sc_audio_player_get_swr_buf(ap, dst_nb_samples);
|
||||
if (!swr_buf) {
|
||||
return false;
|
||||
}
|
||||
|
||||
int ret = swr_convert(swr_ctx, &swr_buf, dst_nb_samples,
|
||||
(const uint8_t **) frame->data, frame->nb_samples);
|
||||
if (ret < 0) {
|
||||
LOGE("Resampling failed: %d", ret);
|
||||
return false;
|
||||
}
|
||||
|
||||
// swr_convert() returns the number of samples which would have been
|
||||
// written if the buffer was big enough.
|
||||
uint32_t samples = MIN(ret, dst_nb_samples);
|
||||
#ifndef SC_AUDIO_PLAYER_NDEBUG
|
||||
LOGD("[Audio] %" PRIu32 " samples written to buffer", samples);
|
||||
#endif
|
||||
|
||||
uint32_t cap = sc_audiobuf_capacity(&ap->buf);
|
||||
if (samples > cap) {
|
||||
// Very very unlikely: a single resampled frame should never
|
||||
// exceed the audio buffer size (or something is very wrong).
|
||||
// Ignore the first bytes in swr_buf to avoid memory corruption anyway.
|
||||
swr_buf += TO_BYTES(samples - cap);
|
||||
samples = cap;
|
||||
}
|
||||
|
||||
uint32_t skipped_samples = 0;
|
||||
|
||||
uint32_t written = sc_audiobuf_write(&ap->buf, swr_buf, samples);
|
||||
if (written < samples) {
|
||||
uint32_t remaining = samples - written;
|
||||
|
||||
// All samples that could be written without locking have been written,
|
||||
// now we need to lock to drop/consume old samples
|
||||
SDL_LockAudioDevice(ap->device);
|
||||
|
||||
// Retry with the lock
|
||||
written += sc_audiobuf_write(&ap->buf,
|
||||
swr_buf + TO_BYTES(written),
|
||||
remaining);
|
||||
if (written < samples) {
|
||||
remaining = samples - written;
|
||||
// Still insufficient, drop old samples to make space
|
||||
skipped_samples = sc_audiobuf_read(&ap->buf, NULL, remaining);
|
||||
assert(skipped_samples == remaining);
|
||||
|
||||
// Now there is enough space
|
||||
uint32_t w = sc_audiobuf_write(&ap->buf,
|
||||
swr_buf + TO_BYTES(written),
|
||||
remaining);
|
||||
assert(w == remaining);
|
||||
(void) w;
|
||||
}
|
||||
|
||||
SDL_UnlockAudioDevice(ap->device);
|
||||
}
|
||||
|
||||
uint32_t underflow = 0;
|
||||
uint32_t max_buffered_samples;
|
||||
bool played = atomic_load_explicit(&ap->played, memory_order_relaxed);
|
||||
if (played) {
|
||||
underflow = atomic_exchange_explicit(&ap->underflow, 0,
|
||||
memory_order_relaxed);
|
||||
|
||||
max_buffered_samples = ap->target_buffering
|
||||
+ 12 * ap->output_buffer
|
||||
+ ap->target_buffering / 10;
|
||||
} else {
|
||||
// SDL playback not started yet, do not accumulate more than
|
||||
// max_initial_buffering samples, this would cause unnecessary delay
|
||||
// (and glitches to compensate) on start.
|
||||
max_buffered_samples = ap->target_buffering + 2 * ap->output_buffer;
|
||||
}
|
||||
|
||||
uint32_t can_read = sc_audiobuf_can_read(&ap->buf);
|
||||
if (can_read > max_buffered_samples) {
|
||||
uint32_t skip_samples = 0;
|
||||
|
||||
SDL_LockAudioDevice(ap->device);
|
||||
can_read = sc_audiobuf_can_read(&ap->buf);
|
||||
if (can_read > max_buffered_samples) {
|
||||
skip_samples = can_read - max_buffered_samples;
|
||||
uint32_t r = sc_audiobuf_read(&ap->buf, NULL, skip_samples);
|
||||
assert(r == skip_samples);
|
||||
(void) r;
|
||||
skipped_samples += skip_samples;
|
||||
}
|
||||
SDL_UnlockAudioDevice(ap->device);
|
||||
|
||||
if (skip_samples) {
|
||||
if (played) {
|
||||
LOGD("[Audio] Buffering threshold exceeded, skipping %" PRIu32
|
||||
" samples", skip_samples);
|
||||
#ifndef SC_AUDIO_PLAYER_NDEBUG
|
||||
} else {
|
||||
LOGD("[Audio] Playback not started, skipping %" PRIu32
|
||||
" samples", skip_samples);
|
||||
#endif
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
atomic_store_explicit(&ap->received, true, memory_order_relaxed);
|
||||
if (!played) {
|
||||
// Nothing more to do
|
||||
return true;
|
||||
}
|
||||
|
||||
// Number of samples added (or removed, if negative) for compensation
|
||||
int32_t instant_compensation = (int32_t) written - frame->nb_samples;
|
||||
// Inserting silence instantly increases buffering
|
||||
int32_t inserted_silence = (int32_t) underflow;
|
||||
// Dropping input samples instantly decreases buffering
|
||||
int32_t dropped = (int32_t) skipped_samples;
|
||||
|
||||
// The compensation must apply instantly, it must not be smoothed
|
||||
ap->avg_buffering.avg += instant_compensation + inserted_silence - dropped;
|
||||
if (ap->avg_buffering.avg < 0) {
|
||||
// Since dropping samples instantly reduces buffering, the difference
|
||||
// is applied immediately to the average value, assuming that the delay
|
||||
// between the producer and the consumer will be caught up.
|
||||
//
|
||||
// However, when this assumption is not valid, the average buffering
|
||||
// may decrease indefinitely. Prevent it to become negative to limit
|
||||
// the consequences.
|
||||
ap->avg_buffering.avg = 0;
|
||||
}
|
||||
|
||||
// However, the buffering level must be smoothed
|
||||
sc_average_push(&ap->avg_buffering, can_read);
|
||||
|
||||
#ifndef SC_AUDIO_PLAYER_NDEBUG
|
||||
LOGD("[Audio] can_read=%" PRIu32 " avg_buffering=%f",
|
||||
can_read, sc_average_get(&ap->avg_buffering));
|
||||
#endif
|
||||
|
||||
ap->samples_since_resync += written;
|
||||
if (ap->samples_since_resync >= ap->sample_rate) {
|
||||
// Recompute compensation every second
|
||||
ap->samples_since_resync = 0;
|
||||
|
||||
float avg = sc_average_get(&ap->avg_buffering);
|
||||
int diff = ap->target_buffering - avg;
|
||||
|
||||
// Enable compensation when the difference exceeds +/- 4ms.
|
||||
// Disable compensation when the difference is lower than +/- 1ms.
|
||||
int threshold = ap->compensation != 0
|
||||
? ap->sample_rate / 1000 /* 1ms */
|
||||
: ap->sample_rate * 4 / 1000; /* 4ms */
|
||||
|
||||
if (abs(diff) < threshold) {
|
||||
// Do not compensate for small values, the error is just noise
|
||||
diff = 0;
|
||||
} else if (diff < 0 && can_read < ap->target_buffering) {
|
||||
// Do not accelerate if the instant buffering level is below the
|
||||
// target, this would increase underflow
|
||||
diff = 0;
|
||||
}
|
||||
// Compensate the diff over 4 seconds (but will be recomputed after 1
|
||||
// second)
|
||||
int distance = 4 * ap->sample_rate;
|
||||
// Limit compensation rate to 2%
|
||||
int abs_max_diff = distance / 50;
|
||||
diff = CLAMP(diff, -abs_max_diff, abs_max_diff);
|
||||
LOGV("[Audio] Buffering: target=%" PRIu32 " avg=%f cur=%" PRIu32
|
||||
" compensation=%d", ap->target_buffering, avg, can_read, diff);
|
||||
|
||||
if (diff != ap->compensation) {
|
||||
int ret = swr_set_compensation(swr_ctx, diff, distance);
|
||||
if (ret < 0) {
|
||||
LOGW("Resampling compensation failed: %d", ret);
|
||||
// not fatal
|
||||
} else {
|
||||
ap->compensation = diff;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_audio_player_frame_sink_open(struct sc_frame_sink *sink,
|
||||
const AVCodecContext *ctx) {
|
||||
struct sc_audio_player *ap = DOWNCAST(sink);
|
||||
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
|
||||
assert(ctx->ch_layout.nb_channels > 0);
|
||||
unsigned nb_channels = ctx->ch_layout.nb_channels;
|
||||
#else
|
||||
int tmp = av_get_channel_layout_nb_channels(ctx->channel_layout);
|
||||
assert(tmp > 0);
|
||||
unsigned nb_channels = tmp;
|
||||
#endif
|
||||
|
||||
assert(ctx->sample_rate > 0);
|
||||
assert(!av_sample_fmt_is_planar(SC_AV_SAMPLE_FMT));
|
||||
int out_bytes_per_sample = av_get_bytes_per_sample(SC_AV_SAMPLE_FMT);
|
||||
assert(out_bytes_per_sample > 0);
|
||||
|
||||
ap->sample_rate = ctx->sample_rate;
|
||||
ap->nb_channels = nb_channels;
|
||||
ap->out_bytes_per_sample = out_bytes_per_sample;
|
||||
|
||||
ap->target_buffering = ap->target_buffering_delay * ap->sample_rate
|
||||
/ SC_TICK_FREQ;
|
||||
|
||||
uint64_t aout_samples = ap->output_buffer_duration * ap->sample_rate
|
||||
/ SC_TICK_FREQ;
|
||||
assert(aout_samples <= 0xFFFF);
|
||||
ap->output_buffer = (uint16_t) aout_samples;
|
||||
|
||||
SDL_AudioSpec desired = {
|
||||
.freq = ctx->sample_rate,
|
||||
.format = SC_SDL_SAMPLE_FMT,
|
||||
.channels = nb_channels,
|
||||
.samples = aout_samples,
|
||||
.callback = sc_audio_player_sdl_callback,
|
||||
.userdata = ap,
|
||||
};
|
||||
SDL_AudioSpec obtained;
|
||||
|
||||
ap->device = SDL_OpenAudioDevice(NULL, 0, &desired, &obtained, 0);
|
||||
if (!ap->device) {
|
||||
LOGE("Could not open audio device: %s", SDL_GetError());
|
||||
return false;
|
||||
}
|
||||
|
||||
SwrContext *swr_ctx = swr_alloc();
|
||||
if (!swr_ctx) {
|
||||
LOG_OOM();
|
||||
goto error_close_audio_device;
|
||||
}
|
||||
ap->swr_ctx = swr_ctx;
|
||||
|
||||
#ifdef SCRCPY_LAVU_HAS_CHLAYOUT
|
||||
av_opt_set_chlayout(swr_ctx, "in_chlayout", &ctx->ch_layout, 0);
|
||||
av_opt_set_chlayout(swr_ctx, "out_chlayout", &ctx->ch_layout, 0);
|
||||
#else
|
||||
av_opt_set_channel_layout(swr_ctx, "in_channel_layout",
|
||||
ctx->channel_layout, 0);
|
||||
av_opt_set_channel_layout(swr_ctx, "out_channel_layout",
|
||||
ctx->channel_layout, 0);
|
||||
#endif
|
||||
|
||||
av_opt_set_int(swr_ctx, "in_sample_rate", ctx->sample_rate, 0);
|
||||
av_opt_set_int(swr_ctx, "out_sample_rate", ctx->sample_rate, 0);
|
||||
|
||||
av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", ctx->sample_fmt, 0);
|
||||
av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", SC_AV_SAMPLE_FMT, 0);
|
||||
|
||||
int ret = swr_init(swr_ctx);
|
||||
if (ret) {
|
||||
LOGE("Failed to initialize the resampling context");
|
||||
goto error_free_swr_ctx;
|
||||
}
|
||||
|
||||
// Use a ring-buffer of the target buffering size plus 1 second between the
|
||||
// producer and the consumer. It's too big on purpose, to guarantee that
|
||||
// the producer and the consumer will be able to access it in parallel
|
||||
// without locking.
|
||||
uint32_t audiobuf_samples = ap->target_buffering + ap->sample_rate;
|
||||
|
||||
size_t sample_size = ap->nb_channels * ap->out_bytes_per_sample;
|
||||
bool ok = sc_audiobuf_init(&ap->buf, sample_size, audiobuf_samples);
|
||||
if (!ok) {
|
||||
goto error_free_swr_ctx;
|
||||
}
|
||||
|
||||
size_t initial_swr_buf_size = TO_BYTES(4096);
|
||||
ap->swr_buf = malloc(initial_swr_buf_size);
|
||||
if (!ap->swr_buf) {
|
||||
LOG_OOM();
|
||||
goto error_destroy_audiobuf;
|
||||
}
|
||||
ap->swr_buf_alloc_size = initial_swr_buf_size;
|
||||
|
||||
// Samples are produced and consumed by blocks, so the buffering must be
|
||||
// smoothed to get a relatively stable value.
|
||||
sc_average_init(&ap->avg_buffering, 128);
|
||||
ap->samples_since_resync = 0;
|
||||
|
||||
ap->received = false;
|
||||
atomic_init(&ap->played, false);
|
||||
atomic_init(&ap->received, false);
|
||||
atomic_init(&ap->underflow, 0);
|
||||
ap->compensation = 0;
|
||||
|
||||
// The thread calling open() is the thread calling push(), which fills the
|
||||
// audio buffer consumed by the SDL audio thread.
|
||||
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_TIME_CRITICAL);
|
||||
if (!ok) {
|
||||
ok = sc_thread_set_priority(SC_THREAD_PRIORITY_HIGH);
|
||||
(void) ok; // We don't care if it worked, at least we tried
|
||||
}
|
||||
|
||||
SDL_PauseAudioDevice(ap->device, 0);
|
||||
|
||||
return true;
|
||||
|
||||
error_destroy_audiobuf:
|
||||
sc_audiobuf_destroy(&ap->buf);
|
||||
error_free_swr_ctx:
|
||||
swr_free(&ap->swr_ctx);
|
||||
error_close_audio_device:
|
||||
SDL_CloseAudioDevice(ap->device);
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_audio_player_frame_sink_close(struct sc_frame_sink *sink) {
|
||||
struct sc_audio_player *ap = DOWNCAST(sink);
|
||||
|
||||
assert(ap->device);
|
||||
SDL_PauseAudioDevice(ap->device, 1);
|
||||
SDL_CloseAudioDevice(ap->device);
|
||||
|
||||
free(ap->swr_buf);
|
||||
sc_audiobuf_destroy(&ap->buf);
|
||||
swr_free(&ap->swr_ctx);
|
||||
}
|
||||
|
||||
void
|
||||
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering,
|
||||
sc_tick output_buffer_duration) {
|
||||
ap->target_buffering_delay = target_buffering;
|
||||
ap->output_buffer_duration = output_buffer_duration;
|
||||
|
||||
static const struct sc_frame_sink_ops ops = {
|
||||
.open = sc_audio_player_frame_sink_open,
|
||||
.close = sc_audio_player_frame_sink_close,
|
||||
.push = sc_audio_player_frame_sink_push,
|
||||
};
|
||||
|
||||
ap->frame_sink.ops = &ops;
|
||||
}
|
||||
@ -1,84 +0,0 @@
|
||||
#ifndef SC_AUDIO_PLAYER_H
|
||||
#define SC_AUDIO_PLAYER_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdatomic.h>
|
||||
#include <stdbool.h>
|
||||
#include <libavformat/avformat.h>
|
||||
#include <libswresample/swresample.h>
|
||||
#include <SDL2/SDL.h>
|
||||
|
||||
#include "trait/frame_sink.h"
|
||||
#include "util/audiobuf.h"
|
||||
#include "util/average.h"
|
||||
#include "util/thread.h"
|
||||
#include "util/tick.h"
|
||||
|
||||
struct sc_audio_player {
|
||||
struct sc_frame_sink frame_sink;
|
||||
|
||||
SDL_AudioDeviceID device;
|
||||
|
||||
// The target buffering between the producer and the consumer. This value
|
||||
// is directly use for compensation.
|
||||
// Since audio capture and/or encoding on the device typically produce
|
||||
// blocks of 960 samples (20ms) or 1024 samples (~21.3ms), this target
|
||||
// value should be higher.
|
||||
sc_tick target_buffering_delay;
|
||||
uint32_t target_buffering; // in samples
|
||||
|
||||
// SDL audio output buffer size.
|
||||
sc_tick output_buffer_duration;
|
||||
uint16_t output_buffer;
|
||||
|
||||
// Audio buffer to communicate between the receiver and the SDL audio
|
||||
// callback
|
||||
struct sc_audiobuf buf;
|
||||
|
||||
// Resampler (only used from the receiver thread)
|
||||
struct SwrContext *swr_ctx;
|
||||
|
||||
// The sample rate is the same for input and output
|
||||
unsigned sample_rate;
|
||||
// The number of channels is the same for input and output
|
||||
unsigned nb_channels;
|
||||
// The number of bytes per sample for a single channel
|
||||
size_t out_bytes_per_sample;
|
||||
|
||||
// Target buffer for resampling (only used by the receiver thread)
|
||||
uint8_t *swr_buf;
|
||||
size_t swr_buf_alloc_size;
|
||||
|
||||
// Number of buffered samples (may be negative on underflow) (only used by
|
||||
// the receiver thread)
|
||||
struct sc_average avg_buffering;
|
||||
// Count the number of samples to trigger a compensation update regularly
|
||||
// (only used by the receiver thread)
|
||||
uint32_t samples_since_resync;
|
||||
|
||||
// Number of silence samples inserted since the last received packet
|
||||
atomic_uint_least32_t underflow;
|
||||
|
||||
// Current applied compensation value (only used by the receiver thread)
|
||||
int compensation;
|
||||
|
||||
// Set to true the first time a sample is received
|
||||
atomic_bool received;
|
||||
|
||||
// Set to true the first time the SDL callback is called
|
||||
atomic_bool played;
|
||||
|
||||
const struct sc_audio_player_callbacks *cbs;
|
||||
void *cbs_userdata;
|
||||
};
|
||||
|
||||
struct sc_audio_player_callbacks {
|
||||
void (*on_ended)(struct sc_audio_player *ap, bool success, void *userdata);
|
||||
};
|
||||
|
||||
void
|
||||
sc_audio_player_init(struct sc_audio_player *ap, sc_tick target_buffering,
|
||||
sc_tick audio_output_buffer);
|
||||
|
||||
#endif
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,36 +1,111 @@
|
||||
#include "clock.h"
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
#define SC_CLOCK_NDEBUG // comment to debug
|
||||
|
||||
#define SC_CLOCK_RANGE 32
|
||||
|
||||
void
|
||||
sc_clock_init(struct sc_clock *clock) {
|
||||
clock->range = 0;
|
||||
clock->offset = 0;
|
||||
clock->count = 0;
|
||||
clock->head = 0;
|
||||
clock->left_sum.system = 0;
|
||||
clock->left_sum.stream = 0;
|
||||
clock->right_sum.system = 0;
|
||||
clock->right_sum.stream = 0;
|
||||
}
|
||||
|
||||
// Estimate the affine function f(stream) = slope * stream + offset
|
||||
static void
|
||||
sc_clock_estimate(struct sc_clock *clock,
|
||||
double *out_slope, sc_tick *out_offset) {
|
||||
assert(clock->count > 1); // two points are necessary
|
||||
|
||||
struct sc_clock_point left_avg = {
|
||||
.system = clock->left_sum.system / (clock->count / 2),
|
||||
.stream = clock->left_sum.stream / (clock->count / 2),
|
||||
};
|
||||
struct sc_clock_point right_avg = {
|
||||
.system = clock->right_sum.system / ((clock->count + 1) / 2),
|
||||
.stream = clock->right_sum.stream / ((clock->count + 1) / 2),
|
||||
};
|
||||
|
||||
double slope = (double) (right_avg.system - left_avg.system)
|
||||
/ (right_avg.stream - left_avg.stream);
|
||||
|
||||
if (clock->count < SC_CLOCK_RANGE) {
|
||||
/* The first frames are typically received and decoded with more delay
|
||||
* than the others, causing a wrong slope estimation on start. To
|
||||
* compensate, assume an initial slope of 1, then progressively use the
|
||||
* estimated slope. */
|
||||
slope = (clock->count * slope + (SC_CLOCK_RANGE - clock->count))
|
||||
/ SC_CLOCK_RANGE;
|
||||
}
|
||||
|
||||
struct sc_clock_point global_avg = {
|
||||
.system = (clock->left_sum.system + clock->right_sum.system)
|
||||
/ clock->count,
|
||||
.stream = (clock->left_sum.stream + clock->right_sum.stream)
|
||||
/ clock->count,
|
||||
};
|
||||
|
||||
sc_tick offset = global_avg.system - (sc_tick) (global_avg.stream * slope);
|
||||
|
||||
*out_slope = slope;
|
||||
*out_offset = offset;
|
||||
}
|
||||
|
||||
void
|
||||
sc_clock_update(struct sc_clock *clock, sc_tick system, sc_tick stream) {
|
||||
if (clock->range < SC_CLOCK_RANGE) {
|
||||
++clock->range;
|
||||
struct sc_clock_point *point = &clock->points[clock->head];
|
||||
|
||||
if (clock->count == SC_CLOCK_RANGE || clock->count & 1) {
|
||||
// One point passes from the right sum to the left sum
|
||||
|
||||
unsigned mid;
|
||||
if (clock->count == SC_CLOCK_RANGE) {
|
||||
mid = (clock->head + SC_CLOCK_RANGE / 2) % SC_CLOCK_RANGE;
|
||||
} else {
|
||||
// Only for the first frames
|
||||
mid = clock->count / 2;
|
||||
}
|
||||
|
||||
struct sc_clock_point *mid_point = &clock->points[mid];
|
||||
clock->left_sum.system += mid_point->system;
|
||||
clock->left_sum.stream += mid_point->stream;
|
||||
clock->right_sum.system -= mid_point->system;
|
||||
clock->right_sum.stream -= mid_point->stream;
|
||||
}
|
||||
|
||||
if (clock->count == SC_CLOCK_RANGE) {
|
||||
// The current point overwrites the previous value in the circular
|
||||
// array, update the left sum accordingly
|
||||
clock->left_sum.system -= point->system;
|
||||
clock->left_sum.stream -= point->stream;
|
||||
} else {
|
||||
++clock->count;
|
||||
}
|
||||
|
||||
sc_tick offset = system - stream;
|
||||
clock->offset = ((clock->range - 1) * clock->offset + offset)
|
||||
/ clock->range;
|
||||
point->system = system;
|
||||
point->stream = stream;
|
||||
|
||||
clock->right_sum.system += system;
|
||||
clock->right_sum.stream += stream;
|
||||
|
||||
clock->head = (clock->head + 1) % SC_CLOCK_RANGE;
|
||||
|
||||
if (clock->count > 1) {
|
||||
// Update estimation
|
||||
sc_clock_estimate(clock, &clock->slope, &clock->offset);
|
||||
|
||||
#ifndef SC_CLOCK_NDEBUG
|
||||
LOGD("Clock estimation: pts + %" PRItick, clock->offset);
|
||||
LOGD("Clock estimation: %f * pts + %" PRItick,
|
||||
clock->slope, clock->offset);
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
sc_tick
|
||||
sc_clock_to_system_time(struct sc_clock *clock, sc_tick stream) {
|
||||
assert(clock->range); // sc_clock_update() must have been called
|
||||
return stream + clock->offset;
|
||||
assert(clock->count > 1); // sc_clock_update() must have been called
|
||||
return (sc_tick) (stream * clock->slope) + clock->offset;
|
||||
}
|
||||
|
||||
@ -1,244 +0,0 @@
|
||||
#include "delay_buffer.h"
|
||||
|
||||
#include <assert.h>
|
||||
#include <stdlib.h>
|
||||
|
||||
#include <libavutil/avutil.h>
|
||||
#include <libavformat/avformat.h>
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
#define SC_BUFFERING_NDEBUG // comment to debug
|
||||
|
||||
/** Downcast frame_sink to sc_delay_buffer */
|
||||
#define DOWNCAST(SINK) container_of(SINK, struct sc_delay_buffer, frame_sink)
|
||||
|
||||
static bool
|
||||
sc_delayed_frame_init(struct sc_delayed_frame *dframe, const AVFrame *frame) {
|
||||
dframe->frame = av_frame_alloc();
|
||||
if (!dframe->frame) {
|
||||
LOG_OOM();
|
||||
return false;
|
||||
}
|
||||
|
||||
if (av_frame_ref(dframe->frame, frame)) {
|
||||
LOG_OOM();
|
||||
av_frame_free(&dframe->frame);
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_delayed_frame_destroy(struct sc_delayed_frame *dframe) {
|
||||
av_frame_unref(dframe->frame);
|
||||
av_frame_free(&dframe->frame);
|
||||
}
|
||||
|
||||
static int
|
||||
run_buffering(void *data) {
|
||||
struct sc_delay_buffer *db = data;
|
||||
|
||||
assert(db->delay > 0);
|
||||
|
||||
for (;;) {
|
||||
sc_mutex_lock(&db->mutex);
|
||||
|
||||
while (!db->stopped && sc_vecdeque_is_empty(&db->queue)) {
|
||||
sc_cond_wait(&db->queue_cond, &db->mutex);
|
||||
}
|
||||
|
||||
if (db->stopped) {
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
goto stopped;
|
||||
}
|
||||
|
||||
struct sc_delayed_frame dframe = sc_vecdeque_pop(&db->queue);
|
||||
|
||||
sc_tick max_deadline = sc_tick_now() + db->delay;
|
||||
// PTS (written by the server) are expressed in microseconds
|
||||
sc_tick pts = SC_TICK_FROM_US(dframe.frame->pts);
|
||||
|
||||
bool timed_out = false;
|
||||
while (!db->stopped && !timed_out) {
|
||||
sc_tick deadline = sc_clock_to_system_time(&db->clock, pts)
|
||||
+ db->delay;
|
||||
if (deadline > max_deadline) {
|
||||
deadline = max_deadline;
|
||||
}
|
||||
|
||||
timed_out =
|
||||
!sc_cond_timedwait(&db->wait_cond, &db->mutex, deadline);
|
||||
}
|
||||
|
||||
bool stopped = db->stopped;
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
|
||||
if (stopped) {
|
||||
sc_delayed_frame_destroy(&dframe);
|
||||
goto stopped;
|
||||
}
|
||||
|
||||
#ifndef SC_BUFFERING_NDEBUG
|
||||
LOGD("Buffering: %" PRItick ";%" PRItick ";%" PRItick,
|
||||
pts, dframe.push_date, sc_tick_now());
|
||||
#endif
|
||||
|
||||
bool ok = sc_frame_source_sinks_push(&db->frame_source, dframe.frame);
|
||||
sc_delayed_frame_destroy(&dframe);
|
||||
if (!ok) {
|
||||
LOGE("Delayed frame could not be pushed, stopping");
|
||||
sc_mutex_lock(&db->mutex);
|
||||
// Prevent to push any new frame
|
||||
db->stopped = true;
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
goto stopped;
|
||||
}
|
||||
}
|
||||
|
||||
stopped:
|
||||
assert(db->stopped);
|
||||
|
||||
// Flush queue
|
||||
while (!sc_vecdeque_is_empty(&db->queue)) {
|
||||
struct sc_delayed_frame *dframe = sc_vecdeque_popref(&db->queue);
|
||||
sc_delayed_frame_destroy(dframe);
|
||||
}
|
||||
|
||||
LOGD("Buffering thread ended");
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_delay_buffer_frame_sink_open(struct sc_frame_sink *sink,
|
||||
const AVCodecContext *ctx) {
|
||||
struct sc_delay_buffer *db = DOWNCAST(sink);
|
||||
(void) ctx;
|
||||
|
||||
bool ok = sc_mutex_init(&db->mutex);
|
||||
if (!ok) {
|
||||
return false;
|
||||
}
|
||||
|
||||
ok = sc_cond_init(&db->queue_cond);
|
||||
if (!ok) {
|
||||
goto error_destroy_mutex;
|
||||
}
|
||||
|
||||
ok = sc_cond_init(&db->wait_cond);
|
||||
if (!ok) {
|
||||
goto error_destroy_queue_cond;
|
||||
}
|
||||
|
||||
sc_clock_init(&db->clock);
|
||||
sc_vecdeque_init(&db->queue);
|
||||
|
||||
if (!sc_frame_source_sinks_open(&db->frame_source, ctx)) {
|
||||
goto error_destroy_wait_cond;
|
||||
}
|
||||
|
||||
ok = sc_thread_create(&db->thread, run_buffering, "scrcpy-dbuf", db);
|
||||
if (!ok) {
|
||||
LOGE("Could not start buffering thread");
|
||||
goto error_close_sinks;
|
||||
}
|
||||
|
||||
return true;
|
||||
|
||||
error_close_sinks:
|
||||
sc_frame_source_sinks_close(&db->frame_source);
|
||||
error_destroy_wait_cond:
|
||||
sc_cond_destroy(&db->wait_cond);
|
||||
error_destroy_queue_cond:
|
||||
sc_cond_destroy(&db->queue_cond);
|
||||
error_destroy_mutex:
|
||||
sc_mutex_destroy(&db->mutex);
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_delay_buffer_frame_sink_close(struct sc_frame_sink *sink) {
|
||||
struct sc_delay_buffer *db = DOWNCAST(sink);
|
||||
|
||||
sc_mutex_lock(&db->mutex);
|
||||
db->stopped = true;
|
||||
sc_cond_signal(&db->queue_cond);
|
||||
sc_cond_signal(&db->wait_cond);
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
|
||||
sc_thread_join(&db->thread, NULL);
|
||||
|
||||
sc_frame_source_sinks_close(&db->frame_source);
|
||||
|
||||
sc_cond_destroy(&db->wait_cond);
|
||||
sc_cond_destroy(&db->queue_cond);
|
||||
sc_mutex_destroy(&db->mutex);
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_delay_buffer_frame_sink_push(struct sc_frame_sink *sink,
|
||||
const AVFrame *frame) {
|
||||
struct sc_delay_buffer *db = DOWNCAST(sink);
|
||||
|
||||
sc_mutex_lock(&db->mutex);
|
||||
|
||||
if (db->stopped) {
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
return false;
|
||||
}
|
||||
|
||||
sc_tick pts = SC_TICK_FROM_US(frame->pts);
|
||||
sc_clock_update(&db->clock, sc_tick_now(), pts);
|
||||
sc_cond_signal(&db->wait_cond);
|
||||
|
||||
if (db->first_frame_asap && db->clock.range == 1) {
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
return sc_frame_source_sinks_push(&db->frame_source, frame);
|
||||
}
|
||||
|
||||
struct sc_delayed_frame dframe;
|
||||
bool ok = sc_delayed_frame_init(&dframe, frame);
|
||||
if (!ok) {
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
return false;
|
||||
}
|
||||
|
||||
#ifndef SC_BUFFERING_NDEBUG
|
||||
dframe.push_date = sc_tick_now();
|
||||
#endif
|
||||
|
||||
ok = sc_vecdeque_push(&db->queue, dframe);
|
||||
if (!ok) {
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
LOG_OOM();
|
||||
return false;
|
||||
}
|
||||
|
||||
sc_cond_signal(&db->queue_cond);
|
||||
|
||||
sc_mutex_unlock(&db->mutex);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
|
||||
bool first_frame_asap) {
|
||||
assert(delay > 0);
|
||||
|
||||
db->delay = delay;
|
||||
db->first_frame_asap = first_frame_asap;
|
||||
|
||||
sc_frame_source_init(&db->frame_source);
|
||||
|
||||
static const struct sc_frame_sink_ops ops = {
|
||||
.open = sc_delay_buffer_frame_sink_open,
|
||||
.close = sc_delay_buffer_frame_sink_close,
|
||||
.push = sc_delay_buffer_frame_sink_push,
|
||||
};
|
||||
|
||||
db->frame_sink.ops = &ops;
|
||||
}
|
||||
@ -1,60 +0,0 @@
|
||||
#ifndef SC_DELAY_BUFFER_H
|
||||
#define SC_DELAY_BUFFER_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#include "clock.h"
|
||||
#include "trait/frame_source.h"
|
||||
#include "trait/frame_sink.h"
|
||||
#include "util/thread.h"
|
||||
#include "util/tick.h"
|
||||
#include "util/vecdeque.h"
|
||||
|
||||
// forward declarations
|
||||
typedef struct AVFrame AVFrame;
|
||||
|
||||
struct sc_delayed_frame {
|
||||
AVFrame *frame;
|
||||
#ifndef NDEBUG
|
||||
sc_tick push_date;
|
||||
#endif
|
||||
};
|
||||
|
||||
struct sc_delayed_frame_queue SC_VECDEQUE(struct sc_delayed_frame);
|
||||
|
||||
struct sc_delay_buffer {
|
||||
struct sc_frame_source frame_source; // frame source trait
|
||||
struct sc_frame_sink frame_sink; // frame sink trait
|
||||
|
||||
sc_tick delay;
|
||||
bool first_frame_asap;
|
||||
|
||||
sc_thread thread;
|
||||
sc_mutex mutex;
|
||||
sc_cond queue_cond;
|
||||
sc_cond wait_cond;
|
||||
|
||||
struct sc_clock clock;
|
||||
struct sc_delayed_frame_queue queue;
|
||||
bool stopped;
|
||||
};
|
||||
|
||||
struct sc_delay_buffer_callbacks {
|
||||
bool (*on_new_frame)(struct sc_delay_buffer *db, const AVFrame *frame,
|
||||
void *userdata);
|
||||
};
|
||||
|
||||
/**
|
||||
* Initialize a delay buffer.
|
||||
*
|
||||
* \param delay a (strictly) positive delay
|
||||
* \param first_frame_asap if true, do not delay the first frame (useful for
|
||||
a video stream).
|
||||
*/
|
||||
void
|
||||
sc_delay_buffer_init(struct sc_delay_buffer *db, sc_tick delay,
|
||||
bool first_frame_asap);
|
||||
|
||||
#endif
|
||||
@ -1,286 +0,0 @@
|
||||
#include "display.h"
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
bool
|
||||
sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps) {
|
||||
display->renderer =
|
||||
SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
|
||||
if (!display->renderer) {
|
||||
LOGE("Could not create renderer: %s", SDL_GetError());
|
||||
return false;
|
||||
}
|
||||
|
||||
SDL_RendererInfo renderer_info;
|
||||
int r = SDL_GetRendererInfo(display->renderer, &renderer_info);
|
||||
const char *renderer_name = r ? NULL : renderer_info.name;
|
||||
LOGI("Renderer: %s", renderer_name ? renderer_name : "(unknown)");
|
||||
|
||||
display->mipmaps = false;
|
||||
|
||||
// starts with "opengl"
|
||||
bool use_opengl = renderer_name && !strncmp(renderer_name, "opengl", 6);
|
||||
if (use_opengl) {
|
||||
|
||||
#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
|
||||
// Persuade macOS to give us something better than OpenGL 2.1.
|
||||
// If we create a Core Profile context, we get the best OpenGL version.
|
||||
SDL_GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK,
|
||||
SDL_GL_CONTEXT_PROFILE_CORE);
|
||||
|
||||
LOGD("Creating OpenGL Core Profile context");
|
||||
display->gl_context = SDL_GL_CreateContext(window);
|
||||
if (!display->gl_context) {
|
||||
LOGE("Could not create OpenGL context: %s", SDL_GetError());
|
||||
SDL_DestroyRenderer(display->renderer);
|
||||
return false;
|
||||
}
|
||||
#endif
|
||||
|
||||
struct sc_opengl *gl = &display->gl;
|
||||
sc_opengl_init(gl);
|
||||
|
||||
LOGI("OpenGL version: %s", gl->version);
|
||||
|
||||
if (mipmaps) {
|
||||
bool supports_mipmaps =
|
||||
sc_opengl_version_at_least(gl, 3, 0, /* OpenGL 3.0+ */
|
||||
2, 0 /* OpenGL ES 2.0+ */);
|
||||
if (supports_mipmaps) {
|
||||
LOGI("Trilinear filtering enabled");
|
||||
display->mipmaps = true;
|
||||
} else {
|
||||
LOGW("Trilinear filtering disabled "
|
||||
"(OpenGL 3.0+ or ES 2.0+ required)");
|
||||
}
|
||||
} else {
|
||||
LOGI("Trilinear filtering disabled");
|
||||
}
|
||||
} else if (mipmaps) {
|
||||
LOGD("Trilinear filtering disabled (not an OpenGL renderer)");
|
||||
}
|
||||
|
||||
display->texture = NULL;
|
||||
display->pending.flags = 0;
|
||||
display->pending.frame = NULL;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
sc_display_destroy(struct sc_display *display) {
|
||||
if (display->pending.frame) {
|
||||
av_frame_free(&display->pending.frame);
|
||||
}
|
||||
#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
|
||||
SDL_GL_DeleteContext(display->gl_context);
|
||||
#endif
|
||||
if (display->texture) {
|
||||
SDL_DestroyTexture(display->texture);
|
||||
}
|
||||
SDL_DestroyRenderer(display->renderer);
|
||||
}
|
||||
|
||||
static SDL_Texture *
|
||||
sc_display_create_texture(struct sc_display *display,
|
||||
struct sc_size size) {
|
||||
SDL_Renderer *renderer = display->renderer;
|
||||
SDL_Texture *texture = SDL_CreateTexture(renderer, SDL_PIXELFORMAT_YV12,
|
||||
SDL_TEXTUREACCESS_STREAMING,
|
||||
size.width, size.height);
|
||||
if (!texture) {
|
||||
LOGD("Could not create texture: %s", SDL_GetError());
|
||||
return NULL;
|
||||
}
|
||||
|
||||
if (display->mipmaps) {
|
||||
struct sc_opengl *gl = &display->gl;
|
||||
|
||||
SDL_GL_BindTexture(texture, NULL, NULL);
|
||||
|
||||
// Enable trilinear filtering for downscaling
|
||||
gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
|
||||
GL_LINEAR_MIPMAP_LINEAR);
|
||||
gl->TexParameterf(GL_TEXTURE_2D, GL_TEXTURE_LOD_BIAS, -1.f);
|
||||
|
||||
SDL_GL_UnbindTexture(texture);
|
||||
}
|
||||
|
||||
return texture;
|
||||
}
|
||||
|
||||
static inline void
|
||||
sc_display_set_pending_size(struct sc_display *display, struct sc_size size) {
|
||||
assert(!display->texture);
|
||||
display->pending.size = size;
|
||||
display->pending.flags |= SC_DISPLAY_PENDING_FLAG_SIZE;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_display_set_pending_frame(struct sc_display *display, const AVFrame *frame) {
|
||||
if (!display->pending.frame) {
|
||||
display->pending.frame = av_frame_alloc();
|
||||
if (!display->pending.frame) {
|
||||
LOG_OOM();
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
int r = av_frame_ref(display->pending.frame, frame);
|
||||
if (r) {
|
||||
LOGE("Could not ref frame: %d", r);
|
||||
return false;
|
||||
}
|
||||
|
||||
display->pending.flags |= SC_DISPLAY_PENDING_FLAG_FRAME;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_display_apply_pending(struct sc_display *display) {
|
||||
if (display->pending.flags & SC_DISPLAY_PENDING_FLAG_SIZE) {
|
||||
assert(!display->texture);
|
||||
display->texture =
|
||||
sc_display_create_texture(display, display->pending.size);
|
||||
if (!display->texture) {
|
||||
return false;
|
||||
}
|
||||
|
||||
display->pending.flags &= ~SC_DISPLAY_PENDING_FLAG_SIZE;
|
||||
}
|
||||
|
||||
if (display->pending.flags & SC_DISPLAY_PENDING_FLAG_FRAME) {
|
||||
assert(display->pending.frame);
|
||||
bool ok = sc_display_update_texture(display, display->pending.frame);
|
||||
if (!ok) {
|
||||
return false;
|
||||
}
|
||||
|
||||
av_frame_unref(display->pending.frame);
|
||||
display->pending.flags &= ~SC_DISPLAY_PENDING_FLAG_FRAME;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_display_set_texture_size_internal(struct sc_display *display,
|
||||
struct sc_size size) {
|
||||
assert(size.width && size.height);
|
||||
|
||||
if (display->texture) {
|
||||
SDL_DestroyTexture(display->texture);
|
||||
}
|
||||
|
||||
display->texture = sc_display_create_texture(display, size);
|
||||
if (!display->texture) {
|
||||
return false;
|
||||
}
|
||||
|
||||
LOGI("Texture: %" PRIu16 "x%" PRIu16, size.width, size.height);
|
||||
return true;
|
||||
}
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_set_texture_size(struct sc_display *display, struct sc_size size) {
|
||||
bool ok = sc_display_set_texture_size_internal(display, size);
|
||||
if (!ok) {
|
||||
sc_display_set_pending_size(display, size);
|
||||
return SC_DISPLAY_RESULT_PENDING;
|
||||
|
||||
}
|
||||
|
||||
return SC_DISPLAY_RESULT_OK;
|
||||
}
|
||||
|
||||
static bool
|
||||
sc_display_update_texture_internal(struct sc_display *display,
|
||||
const AVFrame *frame) {
|
||||
int ret = SDL_UpdateYUVTexture(display->texture, NULL,
|
||||
frame->data[0], frame->linesize[0],
|
||||
frame->data[1], frame->linesize[1],
|
||||
frame->data[2], frame->linesize[2]);
|
||||
if (ret) {
|
||||
LOGD("Could not update texture: %s", SDL_GetError());
|
||||
return false;
|
||||
}
|
||||
|
||||
if (display->mipmaps) {
|
||||
SDL_GL_BindTexture(display->texture, NULL, NULL);
|
||||
display->gl.GenerateMipmap(GL_TEXTURE_2D);
|
||||
SDL_GL_UnbindTexture(display->texture);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_update_texture(struct sc_display *display, const AVFrame *frame) {
|
||||
bool ok = sc_display_update_texture_internal(display, frame);
|
||||
if (!ok) {
|
||||
ok = sc_display_set_pending_frame(display, frame);
|
||||
if (!ok) {
|
||||
LOGE("Could not set pending frame");
|
||||
return SC_DISPLAY_RESULT_ERROR;
|
||||
}
|
||||
|
||||
return SC_DISPLAY_RESULT_PENDING;
|
||||
}
|
||||
|
||||
return SC_DISPLAY_RESULT_OK;
|
||||
}
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_render(struct sc_display *display, const SDL_Rect *geometry,
|
||||
enum sc_orientation orientation) {
|
||||
SDL_RenderClear(display->renderer);
|
||||
|
||||
if (display->pending.flags) {
|
||||
bool ok = sc_display_apply_pending(display);
|
||||
if (!ok) {
|
||||
return SC_DISPLAY_RESULT_PENDING;
|
||||
}
|
||||
}
|
||||
|
||||
SDL_Renderer *renderer = display->renderer;
|
||||
SDL_Texture *texture = display->texture;
|
||||
|
||||
if (orientation == SC_ORIENTATION_0) {
|
||||
int ret = SDL_RenderCopy(renderer, texture, NULL, geometry);
|
||||
if (ret) {
|
||||
LOGE("Could not render texture: %s", SDL_GetError());
|
||||
return SC_DISPLAY_RESULT_ERROR;
|
||||
}
|
||||
} else {
|
||||
unsigned cw_rotation = sc_orientation_get_rotation(orientation);
|
||||
double angle = 90 * cw_rotation;
|
||||
|
||||
const SDL_Rect *dstrect = NULL;
|
||||
SDL_Rect rect;
|
||||
if (sc_orientation_is_swap(orientation)) {
|
||||
rect.x = geometry->x + (geometry->w - geometry->h) / 2;
|
||||
rect.y = geometry->y + (geometry->h - geometry->w) / 2;
|
||||
rect.w = geometry->h;
|
||||
rect.h = geometry->w;
|
||||
dstrect = ▭
|
||||
} else {
|
||||
dstrect = geometry;
|
||||
}
|
||||
|
||||
SDL_RendererFlip flip = sc_orientation_is_mirror(orientation)
|
||||
? SDL_FLIP_HORIZONTAL : 0;
|
||||
|
||||
int ret = SDL_RenderCopyEx(renderer, texture, NULL, dstrect, angle,
|
||||
NULL, flip);
|
||||
if (ret) {
|
||||
LOGE("Could not render texture: %s", SDL_GetError());
|
||||
return SC_DISPLAY_RESULT_ERROR;
|
||||
}
|
||||
}
|
||||
|
||||
SDL_RenderPresent(display->renderer);
|
||||
return SC_DISPLAY_RESULT_OK;
|
||||
}
|
||||
@ -1,60 +0,0 @@
|
||||
#ifndef SC_DISPLAY_H
|
||||
#define SC_DISPLAY_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <libavformat/avformat.h>
|
||||
#include <SDL2/SDL.h>
|
||||
|
||||
#include "coords.h"
|
||||
#include "opengl.h"
|
||||
#include "options.h"
|
||||
|
||||
#ifdef __APPLE__
|
||||
# define SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
|
||||
#endif
|
||||
|
||||
struct sc_display {
|
||||
SDL_Renderer *renderer;
|
||||
SDL_Texture *texture;
|
||||
|
||||
struct sc_opengl gl;
|
||||
#ifdef SC_DISPLAY_FORCE_OPENGL_CORE_PROFILE
|
||||
SDL_GLContext *gl_context;
|
||||
#endif
|
||||
|
||||
bool mipmaps;
|
||||
|
||||
struct {
|
||||
#define SC_DISPLAY_PENDING_FLAG_SIZE 1
|
||||
#define SC_DISPLAY_PENDING_FLAG_FRAME 2
|
||||
int8_t flags;
|
||||
struct sc_size size;
|
||||
AVFrame *frame;
|
||||
} pending;
|
||||
};
|
||||
|
||||
enum sc_display_result {
|
||||
SC_DISPLAY_RESULT_OK,
|
||||
SC_DISPLAY_RESULT_PENDING,
|
||||
SC_DISPLAY_RESULT_ERROR,
|
||||
};
|
||||
|
||||
bool
|
||||
sc_display_init(struct sc_display *display, SDL_Window *window, bool mipmaps);
|
||||
|
||||
void
|
||||
sc_display_destroy(struct sc_display *display);
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_set_texture_size(struct sc_display *display, struct sc_size size);
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_update_texture(struct sc_display *display, const AVFrame *frame);
|
||||
|
||||
enum sc_display_result
|
||||
sc_display_render(struct sc_display *display, const SDL_Rect *geometry,
|
||||
enum sc_orientation orientation);
|
||||
|
||||
#endif
|
||||
@ -1,9 +1,5 @@
|
||||
#define SC_EVENT_NEW_FRAME SDL_USEREVENT
|
||||
#define SC_EVENT_DEVICE_DISCONNECTED (SDL_USEREVENT + 1)
|
||||
#define SC_EVENT_SERVER_CONNECTION_FAILED (SDL_USEREVENT + 2)
|
||||
#define SC_EVENT_SERVER_CONNECTED (SDL_USEREVENT + 3)
|
||||
#define SC_EVENT_USB_DEVICE_DISCONNECTED (SDL_USEREVENT + 4)
|
||||
#define SC_EVENT_DEMUXER_ERROR (SDL_USEREVENT + 5)
|
||||
#define SC_EVENT_RECORDER_ERROR (SDL_USEREVENT + 6)
|
||||
#define SC_EVENT_SCREEN_INIT_SIZE (SDL_USEREVENT + 7)
|
||||
#define SC_EVENT_TIME_LIMIT_REACHED (SDL_USEREVENT + 8)
|
||||
#define EVENT_NEW_FRAME SDL_USEREVENT
|
||||
#define EVENT_STREAM_STOPPED (SDL_USEREVENT + 1)
|
||||
#define EVENT_SERVER_CONNECTION_FAILED (SDL_USEREVENT + 2)
|
||||
#define EVENT_SERVER_CONNECTED (SDL_USEREVENT + 3)
|
||||
#define EVENT_USB_DEVICE_DISCONNECTED (SDL_USEREVENT + 4)
|
||||
|
||||
@ -1,15 +0,0 @@
|
||||
#ifndef SC_HID_EVENT_H
|
||||
#define SC_HID_EVENT_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#define SC_HID_MAX_SIZE 8
|
||||
|
||||
struct sc_hid_event {
|
||||
uint8_t data[SC_HID_MAX_SIZE];
|
||||
uint8_t size;
|
||||
};
|
||||
|
||||
#endif
|
||||
@ -1,192 +0,0 @@
|
||||
#include "hid_mouse.h"
|
||||
|
||||
// 1 byte for buttons + padding, 1 byte for X position, 1 byte for Y position,
|
||||
// 1 byte for wheel motion
|
||||
#define HID_MOUSE_EVENT_SIZE 4
|
||||
|
||||
/**
|
||||
* Mouse descriptor from the specification:
|
||||
* <https://www.usb.org/sites/default/files/hid1_11.pdf>
|
||||
*
|
||||
* Appendix E (p71): §E.10 Report Descriptor (Mouse)
|
||||
*
|
||||
* The usage tags (like Wheel) are listed in "HID Usage Tables":
|
||||
* <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
|
||||
* §4 Generic Desktop Page (0x01) (p26)
|
||||
*/
|
||||
const uint8_t SC_HID_MOUSE_REPORT_DESC[] = {
|
||||
// Usage Page (Generic Desktop)
|
||||
0x05, 0x01,
|
||||
// Usage (Mouse)
|
||||
0x09, 0x02,
|
||||
|
||||
// Collection (Application)
|
||||
0xA1, 0x01,
|
||||
|
||||
// Usage (Pointer)
|
||||
0x09, 0x01,
|
||||
|
||||
// Collection (Physical)
|
||||
0xA1, 0x00,
|
||||
|
||||
// Usage Page (Buttons)
|
||||
0x05, 0x09,
|
||||
|
||||
// Usage Minimum (1)
|
||||
0x19, 0x01,
|
||||
// Usage Maximum (5)
|
||||
0x29, 0x05,
|
||||
// Logical Minimum (0)
|
||||
0x15, 0x00,
|
||||
// Logical Maximum (1)
|
||||
0x25, 0x01,
|
||||
// Report Count (5)
|
||||
0x95, 0x05,
|
||||
// Report Size (1)
|
||||
0x75, 0x01,
|
||||
// Input (Data, Variable, Absolute): 5 buttons bits
|
||||
0x81, 0x02,
|
||||
|
||||
// Report Count (1)
|
||||
0x95, 0x01,
|
||||
// Report Size (3)
|
||||
0x75, 0x03,
|
||||
// Input (Constant): 3 bits padding
|
||||
0x81, 0x01,
|
||||
|
||||
// Usage Page (Generic Desktop)
|
||||
0x05, 0x01,
|
||||
// Usage (X)
|
||||
0x09, 0x30,
|
||||
// Usage (Y)
|
||||
0x09, 0x31,
|
||||
// Usage (Wheel)
|
||||
0x09, 0x38,
|
||||
// Local Minimum (-127)
|
||||
0x15, 0x81,
|
||||
// Local Maximum (127)
|
||||
0x25, 0x7F,
|
||||
// Report Size (8)
|
||||
0x75, 0x08,
|
||||
// Report Count (3)
|
||||
0x95, 0x03,
|
||||
// Input (Data, Variable, Relative): 3 position bytes (X, Y, Wheel)
|
||||
0x81, 0x06,
|
||||
|
||||
// End Collection
|
||||
0xC0,
|
||||
|
||||
// End Collection
|
||||
0xC0,
|
||||
};
|
||||
|
||||
const size_t SC_HID_MOUSE_REPORT_DESC_LEN =
|
||||
sizeof(SC_HID_MOUSE_REPORT_DESC);
|
||||
|
||||
/**
|
||||
* A mouse HID event is 4 bytes long:
|
||||
*
|
||||
* - byte 0: buttons state
|
||||
* - byte 1: relative x motion (signed byte from -127 to 127)
|
||||
* - byte 2: relative y motion (signed byte from -127 to 127)
|
||||
* - byte 3: wheel motion (-1, 0 or 1)
|
||||
*
|
||||
* 7 6 5 4 3 2 1 0
|
||||
* +---------------+
|
||||
* byte 0: |0 0 0 . . . . .| buttons state
|
||||
* +---------------+
|
||||
* ^ ^ ^ ^ ^
|
||||
* | | | | `- left button
|
||||
* | | | `--- right button
|
||||
* | | `----- middle button
|
||||
* | `------- button 4
|
||||
* `--------- button 5
|
||||
*
|
||||
* +---------------+
|
||||
* byte 1: |. . . . . . . .| relative x motion
|
||||
* +---------------+
|
||||
* byte 2: |. . . . . . . .| relative y motion
|
||||
* +---------------+
|
||||
* byte 3: |. . . . . . . .| wheel motion
|
||||
* +---------------+
|
||||
*
|
||||
* As an example, here is the report for a motion of (x=5, y=-4) with left
|
||||
* button pressed:
|
||||
*
|
||||
* +---------------+
|
||||
* |0 0 0 0 0 0 0 1| left button pressed
|
||||
* +---------------+
|
||||
* |0 0 0 0 0 1 0 1| horizontal motion (x = 5)
|
||||
* +---------------+
|
||||
* |1 1 1 1 1 1 0 0| relative y motion (y = -4)
|
||||
* +---------------+
|
||||
* |0 0 0 0 0 0 0 0| wheel motion
|
||||
* +---------------+
|
||||
*/
|
||||
|
||||
static void
|
||||
sc_hid_mouse_event_init(struct sc_hid_event *hid_event) {
|
||||
hid_event->size = HID_MOUSE_EVENT_SIZE;
|
||||
// Leave hid_event->data uninitialized, it will be fully initialized by
|
||||
// callers
|
||||
}
|
||||
|
||||
static uint8_t
|
||||
sc_hid_buttons_from_buttons_state(uint8_t buttons_state) {
|
||||
uint8_t c = 0;
|
||||
if (buttons_state & SC_MOUSE_BUTTON_LEFT) {
|
||||
c |= 1 << 0;
|
||||
}
|
||||
if (buttons_state & SC_MOUSE_BUTTON_RIGHT) {
|
||||
c |= 1 << 1;
|
||||
}
|
||||
if (buttons_state & SC_MOUSE_BUTTON_MIDDLE) {
|
||||
c |= 1 << 2;
|
||||
}
|
||||
if (buttons_state & SC_MOUSE_BUTTON_X1) {
|
||||
c |= 1 << 3;
|
||||
}
|
||||
if (buttons_state & SC_MOUSE_BUTTON_X2) {
|
||||
c |= 1 << 4;
|
||||
}
|
||||
return c;
|
||||
}
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_motion_event *event) {
|
||||
sc_hid_mouse_event_init(hid_event);
|
||||
|
||||
uint8_t *data = hid_event->data;
|
||||
data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
|
||||
data[1] = CLAMP(event->xrel, -127, 127);
|
||||
data[2] = CLAMP(event->yrel, -127, 127);
|
||||
data[3] = 0; // wheel coordinates only used for scrolling
|
||||
}
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_click_event *event) {
|
||||
sc_hid_mouse_event_init(hid_event);
|
||||
|
||||
uint8_t *data = hid_event->data;
|
||||
data[0] = sc_hid_buttons_from_buttons_state(event->buttons_state);
|
||||
data[1] = 0; // no x motion
|
||||
data[2] = 0; // no y motion
|
||||
data[3] = 0; // wheel coordinates only used for scrolling
|
||||
}
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_scroll_event *event) {
|
||||
sc_hid_mouse_event_init(hid_event);
|
||||
|
||||
uint8_t *data = hid_event->data;
|
||||
data[0] = 0; // buttons state irrelevant (and unknown)
|
||||
data[1] = 0; // no x motion
|
||||
data[2] = 0; // no y motion
|
||||
// In practice, vscroll is always -1, 0 or 1, but in theory other values
|
||||
// are possible
|
||||
data[3] = CLAMP(event->vscroll, -127, 127);
|
||||
// Horizontal scrolling ignored
|
||||
}
|
||||
@ -1,26 +0,0 @@
|
||||
#ifndef SC_HID_MOUSE_H
|
||||
#define SC_HID_MOUSE_H
|
||||
|
||||
#endif
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#include "hid/hid_event.h"
|
||||
#include "input_events.h"
|
||||
|
||||
extern const uint8_t SC_HID_MOUSE_REPORT_DESC[];
|
||||
extern const size_t SC_HID_MOUSE_REPORT_DESC_LEN;
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_motion(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_motion_event *event);
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_click(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_click_event *event);
|
||||
|
||||
void
|
||||
sc_hid_mouse_event_from_scroll(struct sc_hid_event *hid_event,
|
||||
const struct sc_mouse_scroll_event *event);
|
||||
@ -1,48 +0,0 @@
|
||||
#include "packet_merger.h"
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
void
|
||||
sc_packet_merger_init(struct sc_packet_merger *merger) {
|
||||
merger->config = NULL;
|
||||
}
|
||||
|
||||
void
|
||||
sc_packet_merger_destroy(struct sc_packet_merger *merger) {
|
||||
free(merger->config);
|
||||
}
|
||||
|
||||
bool
|
||||
sc_packet_merger_merge(struct sc_packet_merger *merger, AVPacket *packet) {
|
||||
bool is_config = packet->pts == AV_NOPTS_VALUE;
|
||||
|
||||
if (is_config) {
|
||||
free(merger->config);
|
||||
|
||||
merger->config = malloc(packet->size);
|
||||
if (!merger->config) {
|
||||
LOG_OOM();
|
||||
return false;
|
||||
}
|
||||
|
||||
memcpy(merger->config, packet->data, packet->size);
|
||||
merger->config_size = packet->size;
|
||||
} else if (merger->config) {
|
||||
size_t config_size = merger->config_size;
|
||||
size_t media_size = packet->size;
|
||||
|
||||
if (av_grow_packet(packet, config_size)) {
|
||||
LOG_OOM();
|
||||
return false;
|
||||
}
|
||||
|
||||
memmove(packet->data + config_size, packet->data, media_size);
|
||||
memcpy(packet->data, merger->config, config_size);
|
||||
|
||||
free(merger->config);
|
||||
merger->config = NULL;
|
||||
// merger->size is meaningless when merger->config is NULL
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -1,43 +0,0 @@
|
||||
#ifndef SC_PACKET_MERGER_H
|
||||
#define SC_PACKET_MERGER_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
#include <libavcodec/avcodec.h>
|
||||
|
||||
/**
|
||||
* Config packets (containing the SPS/PPS) are sent in-band. A new config
|
||||
* packet is sent whenever a new encoding session is started (on start and on
|
||||
* device orientation change).
|
||||
*
|
||||
* Every time a config packet is received, it must be sent alone (for recorder
|
||||
* extradata), then concatenated to the next media packet (for correct decoding
|
||||
* and recording).
|
||||
*
|
||||
* This helper reads every input packet and modifies each media packet which
|
||||
* immediately follows a config packet to prepend the config packet payload.
|
||||
*/
|
||||
|
||||
struct sc_packet_merger {
|
||||
uint8_t *config;
|
||||
size_t config_size;
|
||||
};
|
||||
|
||||
void
|
||||
sc_packet_merger_init(struct sc_packet_merger *merger);
|
||||
|
||||
void
|
||||
sc_packet_merger_destroy(struct sc_packet_merger *merger);
|
||||
|
||||
/**
|
||||
* If the packet is a config packet, then keep its data for later.
|
||||
* Otherwise (if the packet is a media packet), then if a config packet is
|
||||
* pending, prepend the config packet to this packet (so the packet is
|
||||
* modified!).
|
||||
*/
|
||||
bool
|
||||
sc_packet_merger_merge(struct sc_packet_merger *merger, AVPacket *packet);
|
||||
|
||||
#endif
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,59 +0,0 @@
|
||||
#include "frame_source.h"
|
||||
|
||||
void
|
||||
sc_frame_source_init(struct sc_frame_source *source) {
|
||||
source->sink_count = 0;
|
||||
}
|
||||
|
||||
void
|
||||
sc_frame_source_add_sink(struct sc_frame_source *source,
|
||||
struct sc_frame_sink *sink) {
|
||||
assert(source->sink_count < SC_FRAME_SOURCE_MAX_SINKS);
|
||||
assert(sink);
|
||||
assert(sink->ops);
|
||||
source->sinks[source->sink_count++] = sink;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_frame_source_sinks_close_firsts(struct sc_frame_source *source,
|
||||
unsigned count) {
|
||||
while (count) {
|
||||
struct sc_frame_sink *sink = source->sinks[--count];
|
||||
sink->ops->close(sink);
|
||||
}
|
||||
}
|
||||
|
||||
bool
|
||||
sc_frame_source_sinks_open(struct sc_frame_source *source,
|
||||
const AVCodecContext *ctx) {
|
||||
assert(source->sink_count);
|
||||
for (unsigned i = 0; i < source->sink_count; ++i) {
|
||||
struct sc_frame_sink *sink = source->sinks[i];
|
||||
if (!sink->ops->open(sink, ctx)) {
|
||||
sc_frame_source_sinks_close_firsts(source, i);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
sc_frame_source_sinks_close(struct sc_frame_source *source) {
|
||||
assert(source->sink_count);
|
||||
sc_frame_source_sinks_close_firsts(source, source->sink_count);
|
||||
}
|
||||
|
||||
bool
|
||||
sc_frame_source_sinks_push(struct sc_frame_source *source,
|
||||
const AVFrame *frame) {
|
||||
assert(source->sink_count);
|
||||
for (unsigned i = 0; i < source->sink_count; ++i) {
|
||||
struct sc_frame_sink *sink = source->sinks[i];
|
||||
if (!sink->ops->push(sink, frame)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -1,38 +0,0 @@
|
||||
#ifndef SC_FRAME_SOURCE_H
|
||||
#define SC_FRAME_SOURCE_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include "frame_sink.h"
|
||||
|
||||
#define SC_FRAME_SOURCE_MAX_SINKS 2
|
||||
|
||||
/**
|
||||
* Frame source trait
|
||||
*
|
||||
* Component able to send AVFrames should implement this trait.
|
||||
*/
|
||||
struct sc_frame_source {
|
||||
struct sc_frame_sink *sinks[SC_FRAME_SOURCE_MAX_SINKS];
|
||||
unsigned sink_count;
|
||||
};
|
||||
|
||||
void
|
||||
sc_frame_source_init(struct sc_frame_source *source);
|
||||
|
||||
void
|
||||
sc_frame_source_add_sink(struct sc_frame_source *source,
|
||||
struct sc_frame_sink *sink);
|
||||
|
||||
bool
|
||||
sc_frame_source_sinks_open(struct sc_frame_source *source,
|
||||
const AVCodecContext *ctx);
|
||||
|
||||
void
|
||||
sc_frame_source_sinks_close(struct sc_frame_source *source);
|
||||
|
||||
bool
|
||||
sc_frame_source_sinks_push(struct sc_frame_source *source,
|
||||
const AVFrame *frame);
|
||||
|
||||
#endif
|
||||
@ -1,70 +0,0 @@
|
||||
#include "packet_source.h"
|
||||
|
||||
void
|
||||
sc_packet_source_init(struct sc_packet_source *source) {
|
||||
source->sink_count = 0;
|
||||
}
|
||||
|
||||
void
|
||||
sc_packet_source_add_sink(struct sc_packet_source *source,
|
||||
struct sc_packet_sink *sink) {
|
||||
assert(source->sink_count < SC_PACKET_SOURCE_MAX_SINKS);
|
||||
assert(sink);
|
||||
assert(sink->ops);
|
||||
source->sinks[source->sink_count++] = sink;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_packet_source_sinks_close_firsts(struct sc_packet_source *source,
|
||||
unsigned count) {
|
||||
while (count) {
|
||||
struct sc_packet_sink *sink = source->sinks[--count];
|
||||
sink->ops->close(sink);
|
||||
}
|
||||
}
|
||||
|
||||
bool
|
||||
sc_packet_source_sinks_open(struct sc_packet_source *source,
|
||||
AVCodecContext *ctx) {
|
||||
assert(source->sink_count);
|
||||
for (unsigned i = 0; i < source->sink_count; ++i) {
|
||||
struct sc_packet_sink *sink = source->sinks[i];
|
||||
if (!sink->ops->open(sink, ctx)) {
|
||||
sc_packet_source_sinks_close_firsts(source, i);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
sc_packet_source_sinks_close(struct sc_packet_source *source) {
|
||||
assert(source->sink_count);
|
||||
sc_packet_source_sinks_close_firsts(source, source->sink_count);
|
||||
}
|
||||
|
||||
bool
|
||||
sc_packet_source_sinks_push(struct sc_packet_source *source,
|
||||
const AVPacket *packet) {
|
||||
assert(source->sink_count);
|
||||
for (unsigned i = 0; i < source->sink_count; ++i) {
|
||||
struct sc_packet_sink *sink = source->sinks[i];
|
||||
if (!sink->ops->push(sink, packet)) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
void
|
||||
sc_packet_source_sinks_disable(struct sc_packet_source *source) {
|
||||
assert(source->sink_count);
|
||||
for (unsigned i = 0; i < source->sink_count; ++i) {
|
||||
struct sc_packet_sink *sink = source->sinks[i];
|
||||
if (sink->ops->disable) {
|
||||
sink->ops->disable(sink);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,41 +0,0 @@
|
||||
#ifndef SC_PACKET_SOURCE_H
|
||||
#define SC_PACKET_SOURCE_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include "packet_sink.h"
|
||||
|
||||
#define SC_PACKET_SOURCE_MAX_SINKS 2
|
||||
|
||||
/**
|
||||
* Packet source trait
|
||||
*
|
||||
* Component able to send AVPackets should implement this trait.
|
||||
*/
|
||||
struct sc_packet_source {
|
||||
struct sc_packet_sink *sinks[SC_PACKET_SOURCE_MAX_SINKS];
|
||||
unsigned sink_count;
|
||||
};
|
||||
|
||||
void
|
||||
sc_packet_source_init(struct sc_packet_source *source);
|
||||
|
||||
void
|
||||
sc_packet_source_add_sink(struct sc_packet_source *source,
|
||||
struct sc_packet_sink *sink);
|
||||
|
||||
bool
|
||||
sc_packet_source_sinks_open(struct sc_packet_source *source,
|
||||
AVCodecContext *ctx);
|
||||
|
||||
void
|
||||
sc_packet_source_sinks_close(struct sc_packet_source *source);
|
||||
|
||||
bool
|
||||
sc_packet_source_sinks_push(struct sc_packet_source *source,
|
||||
const AVPacket *packet);
|
||||
|
||||
void
|
||||
sc_packet_source_sinks_disable(struct sc_packet_source *source);
|
||||
|
||||
#endif
|
||||
@ -1,162 +0,0 @@
|
||||
#include "keyboard_uhid.h"
|
||||
|
||||
#include "util/log.h"
|
||||
|
||||
/** Downcast key processor to keyboard_uhid */
|
||||
#define DOWNCAST(KP) container_of(KP, struct sc_keyboard_uhid, key_processor)
|
||||
|
||||
/** Downcast uhid_receiver to keyboard_uhid */
|
||||
#define DOWNCAST_RECEIVER(UR) \
|
||||
container_of(UR, struct sc_keyboard_uhid, uhid_receiver)
|
||||
|
||||
#define UHID_KEYBOARD_ID 1
|
||||
|
||||
static void
|
||||
sc_keyboard_uhid_send_input(struct sc_keyboard_uhid *kb,
|
||||
const struct sc_hid_event *event) {
|
||||
struct sc_control_msg msg;
|
||||
msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
|
||||
msg.uhid_input.id = UHID_KEYBOARD_ID;
|
||||
|
||||
assert(event->size <= SC_HID_MAX_SIZE);
|
||||
memcpy(msg.uhid_input.data, event->data, event->size);
|
||||
msg.uhid_input.size = event->size;
|
||||
|
||||
if (!sc_controller_push_msg(kb->controller, &msg)) {
|
||||
LOGE("Could not send UHID_INPUT message (key)");
|
||||
}
|
||||
}
|
||||
|
||||
static void
|
||||
sc_keyboard_uhid_synchronize_mod(struct sc_keyboard_uhid *kb) {
|
||||
SDL_Keymod sdl_mod = SDL_GetModState();
|
||||
uint16_t mod = sc_mods_state_from_sdl(sdl_mod) & (SC_MOD_CAPS | SC_MOD_NUM);
|
||||
|
||||
uint16_t device_mod =
|
||||
atomic_load_explicit(&kb->device_mod, memory_order_relaxed);
|
||||
uint16_t diff = mod ^ device_mod;
|
||||
|
||||
if (diff) {
|
||||
// Inherently racy (the HID output reports arrive asynchronously in
|
||||
// response to key presses), but will re-synchronize on next key press
|
||||
// or HID output anyway
|
||||
atomic_store_explicit(&kb->device_mod, mod, memory_order_relaxed);
|
||||
|
||||
struct sc_hid_event hid_event;
|
||||
sc_hid_keyboard_event_from_mods(&hid_event, diff);
|
||||
|
||||
LOGV("HID keyboard state synchronized");
|
||||
|
||||
sc_keyboard_uhid_send_input(kb, &hid_event);
|
||||
}
|
||||
}
|
||||
|
||||
static void
|
||||
sc_key_processor_process_key(struct sc_key_processor *kp,
|
||||
const struct sc_key_event *event,
|
||||
uint64_t ack_to_wait) {
|
||||
(void) ack_to_wait;
|
||||
|
||||
if (event->repeat) {
|
||||
// In USB HID protocol, key repeat is handled by the host (Android), so
|
||||
// just ignore key repeat here.
|
||||
return;
|
||||
}
|
||||
|
||||
struct sc_keyboard_uhid *kb = DOWNCAST(kp);
|
||||
|
||||
struct sc_hid_event hid_event;
|
||||
|
||||
// Not all keys are supported, just ignore unsupported keys
|
||||
if (sc_hid_keyboard_event_from_key(&kb->hid, &hid_event, event)) {
|
||||
if (event->scancode == SC_SCANCODE_CAPSLOCK) {
|
||||
atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_CAPS,
|
||||
memory_order_relaxed);
|
||||
} else if (event->scancode == SC_SCANCODE_NUMLOCK) {
|
||||
atomic_fetch_xor_explicit(&kb->device_mod, SC_MOD_NUM,
|
||||
memory_order_relaxed);
|
||||
} else {
|
||||
// Synchronize modifiers (only if the scancode itself does not
|
||||
// change the modifiers)
|
||||
sc_keyboard_uhid_synchronize_mod(kb);
|
||||
}
|
||||
sc_keyboard_uhid_send_input(kb, &hid_event);
|
||||
}
|
||||
}
|
||||
|
||||
static unsigned
|
||||
sc_keyboard_uhid_to_sc_mod(uint8_t hid_led) {
|
||||
// <https://www.usb.org/sites/default/files/documents/hut1_12v2.pdf>
|
||||
// (chapter 11: LED page)
|
||||
unsigned mod = 0;
|
||||
if (hid_led & 0x01) {
|
||||
mod |= SC_MOD_NUM;
|
||||
}
|
||||
if (hid_led & 0x02) {
|
||||
mod |= SC_MOD_CAPS;
|
||||
}
|
||||
return mod;
|
||||
}
|
||||
|
||||
static void
|
||||
sc_uhid_receiver_process_output(struct sc_uhid_receiver *receiver,
|
||||
const uint8_t *data, size_t len) {
|
||||
// Called from the thread receiving device messages
|
||||
|
||||
assert(len);
|
||||
|
||||
// Also check at runtime (do not trust the server)
|
||||
if (!len) {
|
||||
LOGE("Unexpected empty HID output message");
|
||||
return;
|
||||
}
|
||||
|
||||
struct sc_keyboard_uhid *kb = DOWNCAST_RECEIVER(receiver);
|
||||
|
||||
uint8_t hid_led = data[0];
|
||||
uint16_t device_mod = sc_keyboard_uhid_to_sc_mod(hid_led);
|
||||
atomic_store_explicit(&kb->device_mod, device_mod, memory_order_relaxed);
|
||||
}
|
||||
|
||||
bool
|
||||
sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
|
||||
struct sc_controller *controller,
|
||||
struct sc_uhid_devices *uhid_devices) {
|
||||
sc_hid_keyboard_init(&kb->hid);
|
||||
|
||||
kb->controller = controller;
|
||||
atomic_init(&kb->device_mod, 0);
|
||||
|
||||
static const struct sc_key_processor_ops ops = {
|
||||
.process_key = sc_key_processor_process_key,
|
||||
// Never forward text input via HID (all the keys are injected
|
||||
// separately)
|
||||
.process_text = NULL,
|
||||
};
|
||||
|
||||
// Clipboard synchronization is requested over the same control socket, so
|
||||
// there is no need for a specific synchronization mechanism
|
||||
kb->key_processor.async_paste = false;
|
||||
kb->key_processor.hid = true;
|
||||
kb->key_processor.ops = &ops;
|
||||
|
||||
static const struct sc_uhid_receiver_ops uhid_receiver_ops = {
|
||||
.process_output = sc_uhid_receiver_process_output,
|
||||
};
|
||||
|
||||
kb->uhid_receiver.id = UHID_KEYBOARD_ID;
|
||||
kb->uhid_receiver.ops = &uhid_receiver_ops;
|
||||
sc_uhid_devices_add_receiver(uhid_devices, &kb->uhid_receiver);
|
||||
|
||||
struct sc_control_msg msg;
|
||||
msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
|
||||
msg.uhid_create.id = UHID_KEYBOARD_ID;
|
||||
msg.uhid_create.report_desc = SC_HID_KEYBOARD_REPORT_DESC;
|
||||
msg.uhid_create.report_desc_size = SC_HID_KEYBOARD_REPORT_DESC_LEN;
|
||||
if (!sc_controller_push_msg(controller, &msg)) {
|
||||
LOGE("Could not send UHID_CREATE message (keyboard)");
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -1,27 +0,0 @@
|
||||
#ifndef SC_KEYBOARD_UHID_H
|
||||
#define SC_KEYBOARD_UHID_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#include "controller.h"
|
||||
#include "hid/hid_keyboard.h"
|
||||
#include "uhid/uhid_output.h"
|
||||
#include "trait/key_processor.h"
|
||||
|
||||
struct sc_keyboard_uhid {
|
||||
struct sc_key_processor key_processor; // key processor trait
|
||||
struct sc_uhid_receiver uhid_receiver;
|
||||
|
||||
struct sc_hid_keyboard hid;
|
||||
struct sc_controller *controller;
|
||||
atomic_uint_least16_t device_mod;
|
||||
};
|
||||
|
||||
bool
|
||||
sc_keyboard_uhid_init(struct sc_keyboard_uhid *kb,
|
||||
struct sc_controller *controller,
|
||||
struct sc_uhid_devices *uhid_devices);
|
||||
|
||||
#endif
|
||||
@ -1,89 +0,0 @@
|
||||
#include "mouse_uhid.h"
|
||||
|
||||
#include "hid/hid_mouse.h"
|
||||
#include "input_events.h"
|
||||
#include "util/log.h"
|
||||
|
||||
/** Downcast mouse processor to mouse_uhid */
|
||||
#define DOWNCAST(MP) container_of(MP, struct sc_mouse_uhid, mouse_processor)
|
||||
|
||||
#define UHID_MOUSE_ID 2
|
||||
|
||||
static void
|
||||
sc_mouse_uhid_send_input(struct sc_mouse_uhid *mouse,
|
||||
const struct sc_hid_event *event, const char *name) {
|
||||
struct sc_control_msg msg;
|
||||
msg.type = SC_CONTROL_MSG_TYPE_UHID_INPUT;
|
||||
msg.uhid_input.id = UHID_MOUSE_ID;
|
||||
|
||||
assert(event->size <= SC_HID_MAX_SIZE);
|
||||
memcpy(msg.uhid_input.data, event->data, event->size);
|
||||
msg.uhid_input.size = event->size;
|
||||
|
||||
if (!sc_controller_push_msg(mouse->controller, &msg)) {
|
||||
LOGE("Could not send UHID_INPUT message (%s)", name);
|
||||
}
|
||||
}
|
||||
|
||||
static void
|
||||
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
|
||||
const struct sc_mouse_motion_event *event) {
|
||||
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
|
||||
|
||||
struct sc_hid_event hid_event;
|
||||
sc_hid_mouse_event_from_motion(&hid_event, event);
|
||||
|
||||
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse motion");
|
||||
}
|
||||
|
||||
static void
|
||||
sc_mouse_processor_process_mouse_click(struct sc_mouse_processor *mp,
|
||||
const struct sc_mouse_click_event *event) {
|
||||
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
|
||||
|
||||
struct sc_hid_event hid_event;
|
||||
sc_hid_mouse_event_from_click(&hid_event, event);
|
||||
|
||||
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse click");
|
||||
}
|
||||
|
||||
static void
|
||||
sc_mouse_processor_process_mouse_scroll(struct sc_mouse_processor *mp,
|
||||
const struct sc_mouse_scroll_event *event) {
|
||||
struct sc_mouse_uhid *mouse = DOWNCAST(mp);
|
||||
|
||||
struct sc_hid_event hid_event;
|
||||
sc_hid_mouse_event_from_scroll(&hid_event, event);
|
||||
|
||||
sc_mouse_uhid_send_input(mouse, &hid_event, "mouse scroll");
|
||||
}
|
||||
|
||||
bool
|
||||
sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
|
||||
struct sc_controller *controller) {
|
||||
mouse->controller = controller;
|
||||
|
||||
static const struct sc_mouse_processor_ops ops = {
|
||||
.process_mouse_motion = sc_mouse_processor_process_mouse_motion,
|
||||
.process_mouse_click = sc_mouse_processor_process_mouse_click,
|
||||
.process_mouse_scroll = sc_mouse_processor_process_mouse_scroll,
|
||||
// Touch events not supported (coordinates are not relative)
|
||||
.process_touch = NULL,
|
||||
};
|
||||
|
||||
mouse->mouse_processor.ops = &ops;
|
||||
|
||||
mouse->mouse_processor.relative_mode = true;
|
||||
|
||||
struct sc_control_msg msg;
|
||||
msg.type = SC_CONTROL_MSG_TYPE_UHID_CREATE;
|
||||
msg.uhid_create.id = UHID_MOUSE_ID;
|
||||
msg.uhid_create.report_desc = SC_HID_MOUSE_REPORT_DESC;
|
||||
msg.uhid_create.report_desc_size = SC_HID_MOUSE_REPORT_DESC_LEN;
|
||||
if (!sc_controller_push_msg(controller, &msg)) {
|
||||
LOGE("Could not send UHID_CREATE message (mouse)");
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
@ -1,19 +0,0 @@
|
||||
#ifndef SC_MOUSE_UHID_H
|
||||
#define SC_MOUSE_UHID_H
|
||||
|
||||
#include <stdbool.h>
|
||||
|
||||
#include "controller.h"
|
||||
#include "trait/mouse_processor.h"
|
||||
|
||||
struct sc_mouse_uhid {
|
||||
struct sc_mouse_processor mouse_processor; // mouse processor trait
|
||||
|
||||
struct sc_controller *controller;
|
||||
};
|
||||
|
||||
bool
|
||||
sc_mouse_uhid_init(struct sc_mouse_uhid *mouse,
|
||||
struct sc_controller *controller);
|
||||
|
||||
#endif
|
||||
@ -1,25 +0,0 @@
|
||||
#include "uhid_output.h"
|
||||
|
||||
#include <assert.h>
|
||||
|
||||
void
|
||||
sc_uhid_devices_init(struct sc_uhid_devices *devices) {
|
||||
devices->count = 0;
|
||||
}
|
||||
|
||||
void
|
||||
sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
|
||||
struct sc_uhid_receiver *receiver) {
|
||||
assert(devices->count < SC_UHID_MAX_RECEIVERS);
|
||||
devices->receivers[devices->count++] = receiver;
|
||||
}
|
||||
|
||||
struct sc_uhid_receiver *
|
||||
sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id) {
|
||||
for (size_t i = 0; i < devices->count; ++i) {
|
||||
if (devices->receivers[i]->id == id) {
|
||||
return devices->receivers[i];
|
||||
}
|
||||
}
|
||||
return NULL;
|
||||
}
|
||||
@ -1,45 +0,0 @@
|
||||
#ifndef SC_UHID_OUTPUT_H
|
||||
#define SC_UHID_OUTPUT_H
|
||||
|
||||
#include "common.h"
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stdint.h>
|
||||
|
||||
/**
|
||||
* The communication with UHID devices is bidirectional.
|
||||
*
|
||||
* This component manages the registration of receivers to handle UHID output
|
||||
* messages (sent from the device to the computer).
|
||||
*/
|
||||
|
||||
struct sc_uhid_receiver {
|
||||
uint16_t id;
|
||||
|
||||
const struct sc_uhid_receiver_ops *ops;
|
||||
};
|
||||
|
||||
struct sc_uhid_receiver_ops {
|
||||
void
|
||||
(*process_output)(struct sc_uhid_receiver *receiver,
|
||||
const uint8_t *data, size_t len);
|
||||
};
|
||||
|
||||
#define SC_UHID_MAX_RECEIVERS 1
|
||||
|
||||
struct sc_uhid_devices {
|
||||
struct sc_uhid_receiver *receivers[SC_UHID_MAX_RECEIVERS];
|
||||
unsigned count;
|
||||
};
|
||||
|
||||
void
|
||||
sc_uhid_devices_init(struct sc_uhid_devices *devices);
|
||||
|
||||
void
|
||||
sc_uhid_devices_add_receiver(struct sc_uhid_devices *devices,
|
||||
struct sc_uhid_receiver *receiver);
|
||||
|
||||
struct sc_uhid_receiver *
|
||||
sc_uhid_devices_get_receiver(struct sc_uhid_devices *devices, uint16_t id);
|
||||
|
||||
#endif
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue