Commit Graph

111 Commits

Author SHA1 Message Date
Alexander Borzunov
26ebbfe8f0
Support macOS (#477)
This PR makes both clients and servers work on macOS. Specifically, it:

- Follows https://github.com/learning-at-home/hivemind/pull/586 to run a macOS-compatible `p2pd` binary (both x86-64 and ARM64 are supported)
- Fixes forking issues and tests on macOS, Python 3.10+
- Introduces basic support for serving model blocks on Apple M1/M2 GPUs (torch.mps)
- Increases max number of open files by default (it's not enough on Linux and is really small on macOS)
2023-08-29 07:49:27 +04:00
Alexander Borzunov
75e516a8c1
Refactor readme (#482) 2023-08-28 19:09:13 +04:00
Alexander Borzunov
6967904590
Bump version to 2.1.0 (#474)
* Bump version to 2.1.0
* Suggest using resharded repo
* LLaMA -> Llama in readme
2023-08-24 19:42:19 +04:00
Alexander Borzunov
00d48dcbe1
Override float32 in config to bfloat16 (#431) 2023-08-07 19:47:22 +04:00
Alexander Borzunov
b58141ef66
Remove distracting links from readme (#441) 2023-08-06 18:55:22 +04:00
Alexander Borzunov
679397df0c
Update Discord links from channels to forums (#440)
As our Discord community growths, we found it difficult to look for open and resolved issues in **#running-a-client** and **#running-a-server** channels, as well as navigate through interleaving conversations happening there. That's why we recreated these channels as Discord forums, where different discussions are separated into different posts.
2023-08-06 17:11:49 +04:00
Alexander Borzunov
6a1b8a6a90
Add Stable Beluga 2 to readme (#424) 2023-07-31 01:23:56 +02:00
Alexander Borzunov
cdc0f70653
Add Discord badge and more Discord links to readme (#422) 2023-07-30 16:07:38 +02:00
Guocheng
8072cd9d1b
Fix stale link (#418) 2023-07-25 16:21:15 +02:00
Alexander Borzunov
ffb20b585c
Update commands for hosting Llama 2 in readme (#409) 2023-07-23 13:08:07 +04:00
Alexander Borzunov
48c6b6d963
Update README.md (#407) 2023-07-23 00:41:41 +04:00
Alexander Borzunov
c153cba1fa
Add Llama 2, WSL instructions to readme (#406) 2023-07-23 00:35:19 +04:00
Alexander Borzunov
8666653cf5
Fix routing through relay, default network RPS, --token, logging, readme (#399)
* Hide GeneratorExit in _iterate_inference_steps()
* Update README.md about `--public_name`
* Use .from_pretrained(..., use_auth_token=token) instead of token=token
until it's fully supported across HF libs
* Use default network speed 25 Mbit/s
* Apply relay penalty in max-throughput routing
* Replace RPS with "tokens/sec per block" in logs
* Increase default expiration
2023-07-22 18:27:58 +04:00
justheuristic
e51e84631d
Update to petals.dev (#390)
Since `petals.ml` DNS record is still unavailable, we're switching everything to https://petals.dev

Co-authored-by: Aleksandr Borzunov <hxrussia@gmail.com>
2023-07-20 20:59:28 +04:00
Alexander Borzunov
b1ff8bdd6c
Bump version to 2.0.0.post1 (#384) 2023-07-19 21:13:24 +04:00
Alexander Borzunov
895327a0ae
Fix readme code example, require Python < 3.11 until supported (#374)
* Fix readme code example

* Require Python < 3.11 until it's supported
2023-07-19 12:45:14 +04:00
Alexander Borzunov
c735dd7ba3
Update transformers to 4.31.0 and peft to 0.4.0 (#371) 2023-07-19 05:15:30 +04:00
Alexander Borzunov
3b300c32e4
Update readme to show new models (#365) 2023-07-18 19:57:39 +04:00
Alexander Borzunov
9517dd1e3d
Update readme and "Getting started" link (#360)
This updates readme with the latest updates and fixes an old Colab link, as pointed out in #359.
2023-07-17 05:02:08 +04:00
Alexander Borzunov
294970fe18
Update Colab link 2023-07-12 17:00:15 +04:00
Alexander Borzunov
515a5120cb
Mention LLaMA in readme (#344) 2023-07-12 16:58:58 +04:00
Alexander Borzunov
8f6342a861
Refactor RemoteSequenceManager (#309)
This PR:

1. **Extracts `SequenceManagerConfig` and `SequenceManagerState` subclasses.**

    The config is provided by caller and never changed from inside `RemoteSequenceManager`. The state is a part of the `RemoteSequenceManager`'s state shared between the main manager and its slices. We fix some slicing bugs along the way.

2. **Removes `dht_prefix` and `p2p` arguments, makes `dht` argument optional.**

    `dht_prefix` can always be overridden using `config.dht_prefix`. `p2p` actually needed only under the hood of `RemoteSequenceManager`, so it can extract it by itself without exposing this low-level class to callers. If strictly necessary, a caller can provide `p2p` as a part of `SequenceManagerState`. `dht` is also needed only by `RemoteSequenceManager`, so we can make it optional in the parent classes and create it automatically when it's not provided.

3. **Simplifies retry logic.**

    Previously, we could have "nested" retry loops: one in `._update()`, another in inference/forward/backward steps. The loop in `._update()` could introduce issues to concurrent inference/forward/backward calls, since it blocks the entire class if its delay period becomes too high. Now this logic is simplified: `._update()` performs only one attempt to fetch the DHT info, any retries are triggered by the inference/forward/backward steps.

4. **Removes deprecated `RemoteTransformerBlock`.**

    `RemoteTransformerBlock` was deprecated a long time ago, before Petals 1.0.0. Its removal is long due.

5. **Removes `dht_utils.get_remote_module()`, `dht_utils.get_remote_sequence()`.**

    This functions duplicate the functionality of the `RemoteSequential` constructor.

6. (minor) **Removes `RemoteSequential.is_subsequence` flag.**

    This flag worked incorrectly and was never used. I am removing it for the sake of simplicity.
2023-05-07 13:41:13 +04:00
Alexander Borzunov
5c0b4286b2
Suggest commands for Docker first (#304) 2023-04-13 00:00:35 +04:00
Alexander Borzunov
a7d3d02194
Fix invalid author email in setup.cfg (#287) 2023-03-13 06:21:09 +04:00
Alexander Borzunov
8dab37c1a9
Add benchmarks to readme (#284) 2023-03-13 05:55:27 +04:00
Alexander Borzunov
42594e5173
Link FAQ in readme (#260) 2023-02-17 07:54:02 +04:00
Alexander Borzunov
5d7395e1b5
Prompt-tuning notebooks: suggest to use a smaller model for faster prototyping (#234) 2023-01-24 10:01:31 +04:00
Muhtasham Oblokulov
0ebf6de117
Add citation to readme (#219)
Co-authored-by: Alexander Borzunov <borzunov.alexander@gmail.com>
2023-01-21 07:05:41 +04:00
Alexander Borzunov
fa5ac6e3b4
Mention BLOOMZ in readme (#221) 2023-01-18 03:23:21 +04:00
Alexander Borzunov
e651d73f11
Add one more link to the "Getting started" tutorial (#218)
Some people miss the "Try now in Colab" link or don't understand that it leads to the comprehensive tutorial, so I added one more explicit link.
2023-01-16 04:35:06 +04:00
Alexander Borzunov
37373a66c3 Update Anaconda installation commands (#205) 2023-01-11 21:45:54 +00:00
Alexander Borzunov
391c855208
Add readme subsections (#185) 2023-01-08 09:59:07 +04:00
Alexander Borzunov
f344c7801b
Add link to health.petals.ml to readme (#184) 2023-01-08 08:41:47 +04:00
Alexander Borzunov
27406a9377
Add more links to BLOOM to readme (#183) 2023-01-07 10:48:51 +04:00
Alexander Borzunov
e27706358c
Use slightly less memory in .generate() (#177) 2023-01-05 09:34:03 +04:00
Alexander Borzunov
356e099c3d
Make Docker command more visible (#175) 2023-01-04 02:18:45 +04:00
Alexander Borzunov
779959bc70
Add link to PyPI (#173) 2022-12-31 02:51:52 +04:00
Alexander Borzunov
cdc3b6a25a
Add PyPI badge, update instructions and links in readme (#172) 2022-12-31 02:22:40 +04:00
justheuristic
4014442a0f
Fix instruction for developers (#170) 2022-12-30 23:42:07 +04:00
Alexander Borzunov
26e6120288
Fix code example in readme (#169)
Makes it closer to runnable code, except for imports and defining tokenizer & data loader.
2022-12-30 19:50:38 +04:00
Alexander Borzunov
0b0277ed6f
Add link to chat.petals.ml (#168) 2022-12-30 07:58:33 +04:00
Vadim Peretokin
50fb8205de
Correct grammar in readme (#166) 2022-12-29 16:46:23 +03:00
Alexander Borzunov
714da529e6
Update wording in readme (#165) 2022-12-27 02:40:36 +04:00
Alexander Borzunov
523a7cad33
Fix issues related to petals as a module (#159)
1. Added `from petals.client import *` to `petals/__init__.py`, so you can write just that:

    ```python
    from petals import DistributedBloomForCausalLM
    ```

    I didn't do the same with server, since its classes are supposed to by used by `petals.cli.run_server`, not end-users. Though it's still possible to do `from petals.server.smth import smth` if necessary.

2. Fixed one more logging issue: log lines from hivemind were shown twice due to a bug in #156.

3. Removed unused `runtime.py`, since the server actually uses `hivemind.moe.Runtime`, and `runtime.py` has no significant changes comparing to it.
2022-12-16 09:09:06 +04:00
Aleksandr Borzunov
f42e559c77 Update README.md 2022-12-09 17:54:00 +00:00
Alexander Borzunov
6beb686909
Add link to privacy & security Wiki (#144) 2022-12-09 21:40:29 +04:00
Alexander Borzunov
e99bf36647
Use common folder for all caches, make it a volume in Dockerfile (#141) 2022-12-09 03:54:57 +04:00
Alexander Borzunov
5f50ea9c79
Update Anaconda instructions (#140) 2022-12-09 01:16:33 +04:00
Alexander Borzunov
4cb0ac4718
Update texts in "Terms of use" and "Privacy and security" sections (#138) 2022-12-08 04:41:02 +04:00
Alexander Borzunov
a94c91d870
Add Docker commands, use permanent Discord links (#137) 2022-12-08 01:54:22 +04:00