The code to start audio capture is more complicated for Android 11
(launch a fake popup, wait, make several attempts, close the shell
package).
Use a distinct code path specific to Android 11.
On Android 11, a fake popup must be briefly opened to make the system
think that the shell app is in the foreground so that audio may be
recorded.
Making the shell app foreground may take some time depending on the
device, so make 3 attempts, waiting 100ms before each.
Fixes#3796 <https://github.com/Genymobile/scrcpy/issues/3796>
There were several workarounds applied in a single method. Some of them
are specific to Meizu phones, but cause issues on other devices.
Split the method to be able to only fill the app context for audio
capture without applying the Meizu workarounds.
Fixes#3801 <https://github.com/Genymobile/scrcpy/issues/3801>
On initial connection, scrcpy sent some device metadata:
- the device name (to be used as window title)
- the initial video size (before any frame or even SPS/PPS)
But it is better to provide the initial video size as part as the video
stream, so that it can be demuxed and exposed via AVCodecContext to
sinks.
This avoids to pass an explicit "initial frame size" for the screen, the
recorder and the v4l2 sink.
All server logs were printed to stdout, while all client logs were
printed to stderr.
Instead, use stderr for warnings and errors, stdout for the others:
- stdout: verbose, debug, info
- stderr: warn, error
System.out.println() first prints the message, then the new line.
Between these two calls, the client might print a message, breaking
formatting.
Instead, call System.out.print() with '\n' appended to the message.
On the server side, several components are started, stopped and joined.
Extract an interface to handle them generically.
This will help to support both encoded and raw audio stream, because
they will be two different concrete components, but implementing the
same interface.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
On Android 11, it is possible to start the capture only when the running
app is in foreground. But scrcpy is not an app, it's a Java application
started from shell.
As a workaround, start an existing Android shell existing activity just
to start the capture, then close it immediately.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
When audio capture fails on the device, scrcpy continues mirroring the
video stream. This allows to enable audio by default only when
supported.
However, if an audio configuration occurs (for example the user
explicitly selected an unknown audio encoder), this must be treated as
an error and scrcpy must exit.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
The permission "android.permission.RECORD_AUDIO" has been added for
shell in Android 11.
Moreover, on lower versions, it may make the server segfault on the
device (happened on a Nexus 5 with Android 6.0.1).
Refs <4feeee8891%5E%21/>
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
By default, audio is enabled (--no-audio must be explicitly passed to
disable it).
However, some devices may not support audio capture (typically devices
below Android 11, or Android 11 when the shell application is not
foreground on start).
In that case, make the server notify the client to dynamically disable
audio forwarding so that it does not wait indefinitely for an audio
stream.
Also disable audio on unknown codec or missing decoder on the
client-side, for the same reasons.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
For OPUS codec, FFmpeg expects the raw extradata, but MediaCodec wraps
it in some structure.
Fix the config packet to send only the raw extradata.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Create an AudioRecorder to capture the audio source REMOTE_SUBMIX.
For now, the captured packets are just logged into the console.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
When audio is enabled, open a new socket to send the audio stream from
the device to the client.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
Audio will be enabled by default (when supported). Add an option to
disable it.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
Remove USER_ID from ServiceManager, and replace it by a constant in
FakeContext.
This is the same as android.os.Process.ROOT_UID, but this constant has
been introduced in API 29.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Since scrcpy-server is not an Android application (it's a java
executable), it has no Context.
Some features will require a Context instance to get the package name
and the UID. Add a FakeContext for this purpose.
PR #3757 <https://github.com/Genymobile/scrcpy/pull/3757>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
The Callbacks interface notifies new packets. But in addition, the
screen encoder will need to write headers on start.
We could add a function onStart(), but for simplicity, just remove the
interface, which brings no value, and call the streamer directly.
Refs 87972e2022
On any error, all previously opened sockets must be closed.
Handle these errors in a single catch-block. Currently, there are only 2
sockets, but this will simplify even more with more sockets.
Note: this commit is better displayed with --ignore-space-change (-b).
User-friendly error messages are printed on specific configuration
exceptions. In that case, do not print the stacktrace.
Also handle the user-friendly error message directly where the error
occurs, and print multiline messages in a single log call, to avoid
confusing interleaving.
As reported by gradle:
> Setting the namespace via a source AndroidManifest.xml's package
> attribute is deprecated.
>
> Please instead set the namespace (or testNamespace) in the module's
> build.gradle file, as described here:
> https://developer.android.com/studio/build/configure-app-module#set-namespace
DesktopConnection implements Closeable, so it is implicitly closed after
its try-with-resources block. Closing the DesktopConnection shutdowns
the sockets, so it is necessary in particular to wake up blocking read()
calls from the controller.
But the controller thread was joined before the DesktopConnection was
closed, causing a deadlock. To fix the problem, join the controller
thread only after the DesktopConnection is closed.
Refs 400a1c69b1
On close, the client closes the socket. This wakes up socket blocking
calls on the server-side, by throwing an exception. Since this exception
is expected, it was not logged.
However, other IOExceptions might occur, which must not be ignored. For
that purpose, log only IOException when they are not caused by an EPIPE
error.
The PTS received from MediaCodec are expressed relative to an arbitrary
clock origin. We consider the PTS of the first frame to be 0, and the
PTS of every other frame is relative to this first PTS (note that the
PTS is only used for recording, it is ignored for mirroring).
For simplicity, this relative PTS was computed on the server-side.
To prepare support for multiple stream (video and audio), send the
packet with its original PTS, and handle the PTS offset on the
client-side (by the recorder).
Since we can't know in advance which stream will produce the first
packet with the lowest PTS (a packet received later on one stream may
have a PTS lower than a packet received earlier on another stream),
computing the PTS on the server-side would require unnecessary waiting.
On mouse click events:
- the first button pressed must first generate ACTION_DOWN;
- all button pressed (including the first one) must generate
ACTION_BUTTON_PRESS;
- all button released (including the last one) must generate
ACTION_BUTTON_RELEASE;
- the last button released must in addition generate ACTION_UP.
Otherwise, Chrome does not work properly.
Fixes#3635 <https://github.com/Genymobile/scrcpy/issues/3635>
Signed-off-by: Romain Vimont <rom@rom1v.com>
If the pointer is a mouse, the pointer is UP only when no buttons are
pressed (not when a button is released, because there might be other
buttons still pressed).
Refs #3635 <https://github.com/Genymobile/scrcpy/issues/3635>
Signed-off-by: Romain Vimont <rom@rom1v.com>
On click event, only the whole buttons state was passed to the device.
In addition, on ACTION_DOWN and ACTION_UP, pass the button associated to
the action.
Refs #3635 <https://github.com/Genymobile/scrcpy/issues/3635>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
MediaCodec may fail spuriously, typically when stopping an encoding and
starting a new one immediately (for example on device rotation).
In that case, retry a few times, in many cases it should work.
Refs #3693 <https://github.com/Genymobile/scrcpy/issues/3693>
For the initial connection between the device and the computer, an adb
tunnel is established (with "adb reverse" or "adb forward").
The device-side of the tunnel is a local socket having the hard-coded
name "scrcpy". This may cause issues when several scrcpy instances are
started in a few seconds for the same device, since they will try to
bind the same name.
To avoid conflicts, make the client generate a random UID, and append
this UID to the local socket name ("scrcpy_01234567").
Right click and middle click require the source device to be a mouse,
not a touchscreen. Therefore, the source device was changed only when a
button other than the primary button was pressed (see
adc547fa6e).
However, this led to inconsistencies between the ACTION_DOWN when a
secondary button is pressed (with a mouse as source device) and the
matching ACTION_UP when the secondary button is released (with a
touchscreen as source device, because then there is no button pressed).
To avoid the problem in all cases, force a mouse as source device when
--forward-all-clicks is set.
Concretely, for mouse events in --forward-all-clicks mode:
- device source is set to InputDevice.SOURCE_MOUSE;
- motion event toolType is set to MotionEvent.TOOL_TYPE_MOUSE;
Otherwise (when --forward-all-clicks is unset, or for real touch
events), finger events are injected:
- device source is set to InputDevice.SOURCE_TOUCHSCREEN;
- motion event toolType is set to MotionEvent.TOOL_TYPE_FINGER.
Fixes#3568 <https://github.com/Genymobile/scrcpy/issues/3568>
PR #3579 <https://github.com/Genymobile/scrcpy/pull/3579>
Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
There were exactly one instance of ServiceManager and Settings, stored
in Device.
Since a Device instance is not created by the CleanUp executable, it was
not straightforward to call wrapper methods on cleanup.
Remove this artificial restriction and expose them publicly via static
methods (this is equivalent to expose a singleton, but less verbose).
The previous commit replaced the IInterface instance (the "input"
service) by the InputManager instance (retrieved by
InputManager.getInstance()).
Both define an "injectInputEvent" method, but the alternate version
(probably) does not concern the InputManager.
This reverts commit b7a06278fe.
PR #3190 <https://github.com/Genymobile/scrcpy/pull/3190>
A special PTS value was used to encode a config packet.
To prepare for adding more flags, use the most significant bits of the
PTS field to store flags.
The new retry mechanism with a lower definition only worked if the error
occurred during encode(). For example:
java.lang.IllegalStateException
at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:3452)
at com.genymobile.scrcpy.ScreenEncoder.encode(ScreenEncoder.java:114)
at com.genymobile.scrcpy.ScreenEncoder.internalStreamScreen(ScreenEncoder.java:95)
at com.genymobile.scrcpy.ScreenEncoder.streamScreen(ScreenEncoder.java:61)
at com.genymobile.scrcpy.Server.scrcpy(Server.java:80)
at com.genymobile.scrcpy.Server.main(Server.java:255)
However, MediaCodec may also fail before encoding, during configure() or
start(). For example:
android.media.MediaCodec$CodecException: Error 0xfffffc0e
at android.media.MediaCodec.native_configure(Native Method)
at android.media.MediaCodec.configure(MediaCodec.java:1956)
at android.media.MediaCodec.configure(MediaCodec.java:1885)
at com.genymobile.scrcpy.ScreenEncoder.configure(ScreenEncoder.java:158)
at com.genymobile.scrcpy.ScreenEncoder.streamScreen(ScreenEncoder.java:68)
at com.genymobile.scrcpy.Server.scrcpy(Server.java:28)
at com.genymobile.scrcpy.Server.main(Server.java:110)
Also downscale and retry in these cases.
Refs #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
Refs #2988 <https://github.com/Genymobile/scrcpy/issues/2988>
PR #2990 <https://github.com/Genymobile/scrcpy/pull/2990>
For convenience, this new option forces the 3 following options:
- send_device_meta=false
- send_frame_meta=false
- send_dummy_byte=false
This allows to send a raw H.264 stream on the video socket.
Concretely:
adb push scrcpy-server /data/local/tmp/scrcpy-server.jar
adb forward tcp:1234 localabstract:scrcpy
adb shell CLASSPATH=/data/local/tmp/scrcpy-server.jar \
app_process / com.genymobile.scrcpy.Server 1.21 \
raw_video_stream=true tunnel_forward=true control=false
As soon as a client connects via TCP to localhost:1234, it will receive
the raw H.264 stream.
Refs #1419 comment <https://github.com/Genymobile/scrcpy/pull/1419#issuecomment-1013964650>
PR #2971 <https://github.com/Genymobile/scrcpy/pull/2971>
Similar to send_device_frame, this option allows to disable sending the
device name and size on start.
This is only useful when using the scrcpy-server alone to get a raw
H.264 stream, without using the scrcpy client.
PR #2971 <https://github.com/Genymobile/scrcpy/pull/2971>
Move the options unused by the scrcpy client at the end.
These options may be useful to use scrcpy-server directly (to get a raw
H.264 stream for example).
PR #2971 <https://github.com/Genymobile/scrcpy/pull/2971>
Retry with a lower definition if MediaCodec fails before the first
frame, not the first packet.
In practice, the first packet is a config packet without any frame, and
MediaCodec might fail just after.
Refs 2eb6fe7d81
Refs #2963 <https://github.com/Genymobile/scrcpy/issues/2963>
MediaCodec errors always trigger IllegalStateException or a subtype
(like MediaCodec.CodecException).
In practice, this avoids to retry if the error is caused by an
IOException when writing the video packet to the socket.
The purpose of automatic downscaling on error is to make mirroring work
by just starting scrcpy without an explicit -m value, even if the
encoder could not encode at the screen definition.
It is only useful when we detect an encoding failure before the first
frame. Downsizing later could be surprising, so disable it.
PR #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
Now that scrcpy attempts with a lower definition on any MediaCodec
error (or the user explicitly requests to disable auto-downsizing), the
suggestion is unnecessary.
PR #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
Some devices are not able to encode at the device screen definition.
Instead of just failing, try with a lower definition on any MediaCodec
error.
PR #2947 <https://github.com/Genymobile/scrcpy/pull/2947>
A scroll event might be produced when a mouse button is pressed (for
example when scrolling while selecting a text). For consistency, pass
the actual buttons state (instead of 0).
In practice, it seems that this use case does not work properly with
Android event injection, but it will work with HID mouse.
If --no-control is enabled, then it is not necessary to create a second
communication socket between the client and the server.
This also facilitates the use of the server alone (without the client)
to receive only the raw video stream.
Expose the inject input event mode so that it is possible to wait for
the events to be "finished". This will be necessary to read the
clipboard content only after the COPY or CUT key event is handled.
PR #2834 <https://github.com/Genymobile/scrcpy/pull/2834>
