Commit Graph

444 Commits

Author SHA1 Message Date
Romain Vimont
6220456def Merge mouse and touch events
Both are handled the very same way on the device.
2019-10-03 20:37:49 +02:00
Romain Vimont
7e1d52c119 Rename "touch pointer" to "pointer"
There are only touch pointers now, mouse pointers have been removed.
2019-10-03 20:27:28 +02:00
Romain Vimont
280d5b718c Use common pointers for mouse and touch
The mouse is a pointer like any other.
2019-10-03 20:05:29 +02:00
Romain Vimont
f765aae352 Inject touch events on the server
On receiving an "inject touch" control message, update the local
pointers state and inject touches.
2019-10-03 20:05:29 +02:00
Romain Vimont
77f876e29c Add "inject touch" control message
Add a control message type in the protocol to forward touch events to
the device.
2019-10-03 20:05:27 +02:00
Romain Vimont
d90549d1e6 Rename "pointer" to "mouse pointer"
This will help to distinguish them from "touch pointers".
2019-10-02 21:40:26 +02:00
Romain Vimont
6e38e0cbfe Rename variable names "event" to "msg"
Some variable names had not been renamed when "event" was renamed to
"message" (28980bbc90).
2019-09-15 17:29:03 +02:00
toddsierens
27eacc3c11
Update WindowManager.java 2019-08-08 17:17:48 -04:00
Romain Vimont
056e47e752 Replace "cannot" by "could not" 2019-06-23 20:52:03 +02:00
Romain Vimont
439b009a79 Fix expected parameters count in error message 2019-06-23 20:47:21 +02:00
Romain Vimont
4940746bcb Remove useless else
The if-block ends with a return.
2019-06-14 10:15:53 +02:00
Romain Vimont
fe758e6e15 Improve comment
Rephrase to simplify and add a link to the issue.
2019-06-14 10:11:15 +02:00
Arne Schwabe
b769083a5b Use getPhysicalDisplayToken on Anroid Q+ instead of getBuiltInDisplay
This makes the -S (screen off) parameter work on Android Q beta 4

Closes #586
2019-06-13 13:30:54 +02:00
Romain Vimont
8604f16b30 Truncate device name at UTF-8 code point boundary
Just in case.
2019-06-07 17:45:03 +02:00
Romain Vimont
8e66b33000 Add option to turn device screen off
In addition to the shortcut (Ctrl+o) to turn the device screen off, add
a command-line argument to turn it off on start.
2019-06-05 00:55:46 +02:00
Romain Vimont
acc4dcd520 Disable server controller if --no-control
If --no-control is disabled, there is no need for a controller.

It also avoids to power on the device on start if control is disabled.
2019-06-05 00:25:57 +02:00
Romain Vimont
e572d81fa2 Rename function to "power on"
This will reduce confusion between "power on" when the device is off and
"turn device screen off" while mirroring.
2019-06-02 20:34:52 +02:00
Romain Vimont
0792998cc2 Remove unused import
Introduced by the previous commit.
2019-05-31 23:31:53 +02:00
Romain Vimont
12a3bb25d3 Implement device screen off while mirroring
Add two shortcuts:
 - Ctrl+o to turn the device screen off while mirroring
 - Ctrl+Shift+o to turn it back on

On power on (either via the POWER key or BACK while screen is off), both
the device screen and the mirror are turned on.

<https://github.com/Genymobile/scrcpy/issues/175>
2019-05-31 23:07:23 +02:00
Romain Vimont
6537c2ef01 Add clipboard logs
Synchronizing local and device clipboards in invisible. Add INFO logs
on success.
2019-05-31 16:18:00 +02:00
Romain Vimont
ad55a9addc Prefix server logs
Sometimes, it is not obvious whether a log is generated by the server or
by the client. Prefix server logs for clarity.
2019-05-31 16:18:00 +02:00
Romain Vimont
28980bbc90 Rename "event" to "message"
After the recent refactorings, a "control event" is not necessarily an
"event" (it may be a "command"). Similarly, the unique "device event"
used to send the device clipboard content is more a "reponse" to the
request from the client than an "event".

Rename both to "message", and rename the message types to better
describe their intent.
2019-05-31 16:18:00 +02:00
Romain Vimont
c13a24389c Implement computer-to-device clipboard copy
It was already possible to _paste_ (with Ctrl+v) the content of the
computer clipboard on the device. Technically, it injects a sequence of
events to generate the text.

Add a new feature (Ctrl+Shift+v) to copy to the device clipboard
instead, without injecting the content. Contrary to events injection,
this preserves the UTF-8 content exactly, so the text is not broken by
special characters.

<https://github.com/Genymobile/scrcpy/issues/413>
2019-05-30 22:46:52 +02:00
Romain Vimont
2322069656 Extract control event String parsing
Parsing a String from a serialized control event, encoded as length (2
bytes) + data, will be necessary in several events.

Extract it to a separate method.
2019-05-30 22:46:52 +02:00
Romain Vimont
61f5f96b42 Fix control event String parsing
At least 2 bytes must be available to read the length of the String.
2019-05-30 22:46:52 +02:00
Romain Vimont
63c078ee6c Implement device-to-computer clipboard copy
On Ctrl+C:
 - the client sends a GET_CLIPBOARD command to the device;
 - the device retrieve its current clipboard text and sends it in a
   GET_CLIPBOARD device event;
 - the client sets this text as the system clipboard text, so that it
   can be pasted in another application.

Fixes <https://github.com/Genymobile/scrcpy/issues/145>
2019-05-30 22:46:52 +02:00
Romain Vimont
3149e2cf4a Add device event sender
Create a separate component to send device events, managed by the
controller.
2019-05-30 22:46:52 +02:00
Romain Vimont
f9d2d99166 Add GET_CLIPBOARD device event
Add the first device event, used to forward the device clipboard to the
computer.
2019-05-30 22:36:22 +02:00
Romain Vimont
ec71a3f66a Use two sockets for video and control
The socket used the device-to-computer direction to stream the video and
the computer-to-device direction to send control events.

Some features, like copy-paste from device to computer, require to send
non-video data from the device to the computer.

To make them possible, use two sockets:
 - one for streaming the video from the device to the client;
 - one for control/events in both directions.
2019-05-30 22:35:41 +02:00
Romain Vimont
0a7fe7ad57 Add helpers to truncate UTF-8 at code points
This will help to avoid truncating a UTF-8 string in the middle of a
code point, producing an invalid UTF-8 result.
2019-05-30 22:30:18 +02:00
Romain Vimont
ad4c061cd2 Use custom class Point
The framework class android.graphics.Point cannot be used in unit tests.
Implement our own Point.
2019-05-30 22:30:18 +02:00
Romain Vimont
63909fd10d Merge commands with other control events
Several commands were grouped under the same event type "command", with
a separate field to indicate the actual command.

Move these commands at the same level as other control events. It will
allow to implement commands with arguments.
2019-05-30 22:30:18 +02:00
Romain Vimont
7ed976967f Fix checkstyle warning
Checkstyle wants a specific order of imports.
2019-05-30 00:22:05 +02:00
Romain Vimont
3068457b90 Log characters failed to be injected
Some characters may not be injected (e.g. '\r`). Log them instead of
ignoring them silently.
2019-05-20 08:40:10 +02:00
Yu-Chen Lin
1630f923ef Return success count in injectText
It will insert as many text as possible now.
Fix #509, tested on Windows 10 and Arch Linux.

Signed-off-by: Yu-Chen Lin <npes87184@gmail.com>
2019-05-20 08:36:32 +02:00
Romain Vimont
c8338b2918 Recover if expand/collapse panels is not available
Some devices don't have the required method. Recover gracefully without
crashing the server.

Fixes <https://github.com/Genymobile/scrcpy/issues/506>.
2019-05-04 14:49:48 +02:00
Romain Vimont
2837c6eaab Add method to log error without throwable
Add Ln.e(message) in addition to Ln.e(message, error).
2019-05-04 14:49:04 +02:00
Romain Vimont
01664777c8 Merge branch 'master' into dev 2019-05-04 14:48:54 +02:00
Romain Vimont
f11b0ec204 Fix server checkstyle errors
Fix errors reported by:

    gradle -p server check
2019-03-27 21:47:54 +01:00
Romain Vimont
3eda38e5fc Do not call codec.stop() on exception
On exception, the codec is not in a state were .stop() can be called.
2019-03-21 18:46:22 +01:00
Romain Vimont
b8ff35efe6 Remove empty line 2019-03-07 19:03:13 +01:00
Romain Vimont
7d10ec2b5a Add shortcut to expand/collapse notification panel
Use Ctrl+n to expand, Ctrl+Shift+n to collapse.

Fixes <https://github.com/Genymobile/scrcpy/issues/392>
2019-02-26 20:35:37 +01:00
Romain Vimont
b35733edb6 Fix expected mouse event sizes
Commit fefb9816a9 modified mouse events
serialization. The server-side parsing was updated to correctly read the
position, but the expected size of these events was not updated.

As a result, the server might try to parse incomplete events, leading
to BufferUnderflowException.

Fixes
<https://github.com/Genymobile/scrcpy/issues/350#issuecomment-456298816>.
2019-01-22 09:47:26 +01:00
Romain Vimont
1fdde490fd Mirror "secure" content
Some applications, like Silence, prevent the content of a window from
being viewed on non-secure displays:
<https://developer.android.com/reference/android/view/WindowManager.LayoutParams.html#FLAG_SECURE>

We can mirror it by just creating a "secure" display:
<https://developer.android.com/reference/android/view/Display#FLAG_SECURE>
2019-01-16 14:21:18 +01:00
Romain Vimont
39c5e71605 Make the server unlink itself
To clean up the device, the client executed "adb shell rm" once the
server was guaranteed to be started (after the connection succeeded).

This implied to track whether the installation state, and failed if an
additional tunnel was used in "forward" mode:
<https://github.com/Genymobile/scrcpy/issues/386#issuecomment-453936034>

Instead, make the server unlink itself on start.
2019-01-14 21:12:23 +01:00
Romain Vimont
fefb9816a9 Handle mouse events outside device screen
Mouse events position were unsigned (so negative values could not be
handled properly).

To avoid issues with negative values, mouse events outside the device
screen were ignored (commit a7fe9ad779).

But as a consequence, drag&drop were "broken" if the "drop" occurred
outside the device screen.

Instead, use signed 32-bits to store the position, and forward events
outside the device screen.

Fixes <https://github.com/Genymobile/scrcpy/issues/357>.
2018-11-27 08:59:22 +01:00
Romain Vimont
1e22ebcac2 Always use non-empty arguments
The client passes parameters to the server via "adb shell" arguments.

Use "-" instead of "" when no crop is specified to avoid empty
arguments, which are not handled the same way on all devices.

Fixed <https://github.com/Genymobile/scrcpy/issues/337>.
2018-11-16 18:39:41 +01:00
Romain Vimont
d81729ba39 Always expect 5 parameters for the server
The client always sends all the arguments, so there is no need to check.
2018-11-16 18:36:01 +01:00
Romain Vimont
22ff03f2f7 Do not queue invalid PTS
Configuration packets produced by MediaCodec have no valid PTS, and do
not produce frame. Do not queue their (invalid) PTS not to break the
matching between frames and their PTS.
2018-11-11 21:37:31 +01:00
Romain Vimont
345f8858d3 Send frame meta only if recording is enabled
The client needs the PTS for each frame only if recording is enabled.
Otherwise, the PTS are not necessary, and the protocol is more
straighforward.
2018-11-11 21:37:31 +01:00
Romain Vimont
475912a39c Do not transmit MediaCodec flags
Since PTS handling has been fixed, the recorder do not associate a PTS
to a wrong frame anymore, so PTS of "configuration packets" (which never
produce a frame), are never read by the recorder. Therefore, there is no
need to ignore them explicitly, so we can remove the MediaCodec flags
completely.
2018-11-11 21:37:31 +01:00
Romain Vimont
2cd99e7205 Only set valid PTS/DTS
When the PTS is valid, set both PTS and DTS to avoid FFmpeg warnings.

Since configuration packets have no PTS, do not record these packets.
2018-11-11 21:37:26 +01:00
Ivan Gorinov
d706c5df39 Enable video output file, with pts set by server 2018-11-11 16:30:23 +01:00
Romain Vimont
b882322f73 Work around Os.write() not updating position
ByteBuffer position is not updated as expected by Os.write() on old
Android versions. Count the remaining bytes manually.

Fixes <https://github.com/Genymobile/scrcpy/issues/291>.
2018-10-09 08:43:17 +02:00
Romain Vimont
66def38b73 Avoid additional buffer copy in userspace
Directly send the data from MediaCodec buffers to the LocalSocket,
without an intermediate copy in userspace.
2018-09-09 15:05:06 +02:00
Romain Vimont
caa9e30004 Add crop feature
Add an option to crop the screen on the server. This allows to mirror
only part of the device screen.
2018-08-09 19:12:27 +02:00
Romain Vimont
e85010fbc2 Move annotation comment
This confused the Android Studio code formatter.
2018-08-09 18:23:38 +02:00
Romain Vimont
820cd2bb54 Extract video size computation
One method, one thing.
2018-08-09 18:22:52 +02:00
Romain Vimont
c530d95881 Immediately close the server socket on the device
In "adb forward" mode, close the server socket as soon as the client is
connected.

Even if unlikely to be useful, it allows to run several instances of
scrcpy also in "adb forward" mode.
2018-03-14 09:28:25 +01:00
Romain Vimont
1038bad385 Make it work over tcpip
"adb reverse" currently does not work over tcpip (i.e. on a device
connected by "adb connect"):
<https://issuetracker.google.com/issues/37066218>

To work around the problem, if the call to "adb reverse" fails, then
fallback to "adb forward", and reverse the client/server roles.

Keep the "adb reverse" mode as the default because it does not involve
connection retries: when using "adb forward", the client must try to
connect successively until the server listens.

Due to the tunnel, every connect() will succeed, so the client must
attempt to read() to detect a connection failure. For this purpose, when
using the "adb forward" mode, the server initially writes a dummy byte,
read by the client.

Fixes <https://github.com/Genymobile/scrcpy/issues/5>.
2018-03-12 14:10:32 +01:00
Romain Vimont
c075ad0a1e Fix mouse clicks on LG devices
Use default values (0) for some fields of PointerCoords so that mouse
clicks work correctly on LG devices.

Fixes <https://github.com/Genymobile/scrcpy/issues/18>.
2018-03-11 14:11:43 +01:00
Romain Vimont
dac7196bd6 Support screens with dimensions not divisible by 8
The codec only supports dimensions which are multiple of 8.

Thus, when --max-size is specified, the value is always rounded down to
the nearest multiple of 8.

However, it was wrongly assumed that the physical size is always a
multiple of 8. To support such devices, also round down the physical
screen dimensions.

Fixes <https://github.com/Genymobile/scrcpy/issues/39>.
2018-03-11 14:09:25 +01:00
Romain Vimont
675704c71c Map right-click to BACK if screen is on
Right-click was used to turn the screen on. It did nothing when the
screen was already on.

Instead, in that case, press BACK (like Vysor).

Suggested by: <https://www.reddit.com/r/Android/comments/834zmr/introducing_scrcpy_an_app_to_display_and_control/dvfueft/>
2018-03-10 00:16:29 +01:00
Romain Vimont
9396ea6d42 Fix text input event segfault
The text input control_event was initially designed for mapping
SDL_TextInputEvent, limited to 32 characters.

For simplicity, the copy/paste feature was implemented using the same
control_event: it just sends the text to paste.

However, the pasted text might have a length breaking some assumptions:
 - on the client, the event max-size was smaller than the text
   max-length,
 - on the server, the raw buffer storing the events was smaller than the
   max event size.

Fix these inconsistencies, and encode the length on 2 bytes, to accept
more than 256 characters.

Fixes <https://github.com/Genymobile/scrcpy/issues/10>.
2018-03-09 22:30:10 +01:00
Romain Vimont
e2a7abcd53 Implement clipboard paste
Paste computer clipboard to the device on Ctrl+v.

The other direction (pasting the device clipboard to the computer) is
not implemented. It would require a communication channel from the
device to the computer, other than the socket used by the video stream.
2018-03-07 18:07:02 +01:00
Romain Vimont
633b18d786 Provide a better URL to document key decomposition
Directly link to the relevant subsection.
2018-03-07 18:07:02 +01:00
Romain Vimont
e3f5d3b49b Double the default bitrate
Set the default video bitrate to 8Mbps. This greatly increase quality on
fast motion, without negative side effects.
2018-03-07 18:04:38 +01:00
Romain Vimont
a7979e4e74 Rename rotation detection method name
The old name checkRotationChanged() did not suggest that the flag was
reset.
2018-03-07 18:04:38 +01:00
Romain Vimont
b61310a257 Initialize eof flag from BufferInfo
Stop encoding if the codec signaled the end of stream.

The eof flag was read, but never written.
2018-02-28 15:40:52 +01:00
Romain Vimont
487cb10cf0 Rename ScrCpyServer to Server
The Server class is in package scrcpy, there is no need to repeat the
name in the classname.
2018-02-28 14:57:18 +01:00
Romain Vimont
37af0c8076 Terminate event controller thread on EOF
No exception was thrown on EOF, so the event controller did not
terminate. This leaded to a further InvocationTargetException.

Instead, terminate the event controller on EOF, so that the process
terminates properly.
2018-02-16 14:13:13 +01:00
Romain Vimont
ad6209f6ff Terminate the device process gracefully
Call shutdown() on the device socket to make the device process finish
its execution quickly and gracefully, without killing it.
2018-02-16 11:47:42 +01:00
Romain Vimont
a5b6c6cc2d Enable debug logs only for debug builds on server
Print the logs only if their level is not under the threshold, which is
defined to INFO in release mode and DEBUG in debug mode.
2018-02-12 17:19:52 +01:00
Romain Vimont
000c70a644 Decompose accented characters
Characters like 'é' or 'î' are not resolved by getEvents(). For example,
getEvents("é") returns null.

However, it is possible to decompose them. For example,
getEvents("\u0301e") returns the events generating "é".

Thank you Philippe! ;)
2018-02-12 14:17:10 +01:00
Romain Vimont
063cfd1326 Turn screen on in control()
Turning the screen on is semantically associated to control(): only
creating the EventController object should not turn the screen on.
2018-02-09 14:03:25 +01:00
Romain Vimont
c683872bbc Avoid server stacktraces on close
On close, the socket is closed by the client, and the server process is
killed.

This leads to (expected) exceptions, that should not be printed.
2018-02-09 10:59:15 +01:00
Romain Vimont
e55e42a442 Apply Genymobile rules for Android projects
Apply Genymobile checkstyle and gradle build files organization.
2018-02-07 20:58:18 +01:00
Romain Vimont
285fc97d02 Fix horizontal scrolling constant
Use AXIS_HSCROLL (available since API 12) instead of AXIS_SCROLL (since
API 26).
2018-02-07 20:50:58 +01:00
Romain Vimont
c6d01331ed Simplify EventController
In handleEvent(), connection.receiveControlEvent() may never return
null: either it returns a valid ControlEvent, either it throws an
Exception.

Therefore, there is no need to propagate a flag to indicate whether it
returned a valid ControlEvent.
2018-02-07 17:48:21 +01:00
Romain Vimont
03c5f97e3f Extract control event parsing to separate methods
Use one parse method per control event type.
2018-02-07 17:45:28 +01:00
Romain Vimont
5f51d605c0 Recreate codec and display on rotation changed
On some devices, we can reuse the same codec and display, but on some
others (e.g. Nexus 5X with Android 7.1.2), it crashes on codec.stop()
with an IllegalStateException.

Therefore, always recreate the codec and display, so that it works on
all devices.
2018-02-07 16:32:16 +01:00
Romain Vimont
d5acc8adc5 Always release output buffer
If rotation changed, the dequeued output buffer was never released. Move
it to a finally block to avoid the leak.
2018-02-07 15:50:38 +01:00
Romain Vimont
1ac37c4291 Hide services in Device
Expose useful methods directly in Device, without exposing the service
managers.
2018-02-02 17:06:57 +01:00
Romain Vimont
06333e1e08 Cache managers in ServiceManager
To guarantee that we instantiate services only one, cache the value the
first time.
2018-02-02 16:25:37 +01:00
Romain Vimont
3365460658 Turn screen on on start
When starting scrcpy, immediately turn the screen on.
2018-02-02 14:57:33 +01:00
Romain Vimont
a139509f11 Turn screen on on right-click
The right-click is almost useless on Android, so use it to turn the
screen on.

Add a new control event type (command) to request the server to turn the
screen on.
2018-02-02 14:56:03 +01:00
Romain Vimont
228545cefd Reformat EventController
Reformated by Android Studio.
2018-02-02 14:52:00 +01:00
Romain Vimont
6b546a87ab Add bit-rate command-line option
Add a command-line option (-b/--bit-rate) to customize the video
bit-rate.
2018-02-01 17:11:40 +01:00
Romain Vimont
ee93f3f23a Rename maximum_size to max_size
The long option is --max-size, so for consistency, adapt the code
accordingly.
2018-02-01 17:11:40 +01:00
Romain Vimont
865ebb3862 Encode video using MediaCodec API
Replace screenrecord execution by manual screen encoding using the
MediaCodec API.

The "screenrecord" solution had several drawbacks:
 - screenrecord output is buffered, so tiny frames may not be accessible
   immediately;
 - it did not output a frame until the surface changed, leading to a
   black screen on start;
 - it is limited to 3 minutes recording, so it needed to be restarted;
 - screenrecord added black borders in the video when the requested
   dimensions did not preserve aspect-ratio exactly (sometimes
   unavoidable since video dimensions must be multiple of 8);
 - rotation handling was hacky (killing the process and starting a new
   one).

Handling the encoding manually allows to solve all these problems.
2018-01-31 19:51:43 +01:00
Romain Vimont
7ed334915e Register uncaught exception handler
Never miss an exception by using an uncaught exception handler.
2018-01-31 19:50:45 +01:00
Romain Vimont
8c4a454d68 Extract argument parsing
Move argument parsing to a separate method.
2018-01-31 19:49:49 +01:00
Romain Vimont
73831a0837 Use the Point class from the framework
Replace our custom Point class by android.graphics.Point.
2018-01-31 19:42:08 +01:00
Romain Vimont
b67907e24e Convert server to an Android project
To simplify the device server-side build, use gradle to create an APK,
even if we use it as a simple jar, by running its main() method.
2018-01-30 12:01:36 +01:00