163 Commits (722c4dc49651f4d3f22d3c790e502507e7527a12)

Author	SHA1	Message	Date
Alexander Borzunov	1ea44b0d3c	Measure throughput for different configs, devices, and dtypes separately (#114)	2 years ago
justheuristic	01838f9a99	Fix Linear8bitlt state config, update tests (#112) ... * fix state initializer * update tests to actually use new code * keep bias during quantization	2 years ago
Aleksandr Borzunov	96033de921	Fix script for running servers robustly	2 years ago
Aleksandr Borzunov	85cf32d2a4	Add script to run servers robustly	2 years ago
justheuristic	088713912d	Patch Linear8bit to enable CxB backward (#111) ... A patch to bitsandbytes 0.34.0 that introduces an option to run backward pass in default (fast) matrix layout. Authors: cxb inversion by @borzunov, original 8bit code by @timdettmers * optimized layout inversion code by @borzunov ([original code](https://colab.research.google.com/drive/1EJ0MKifajXSSVq7O2_QGwtb0l6gRAGrh?usp=sharing)) to use less forward calls * implemented CustomLinear8bitLt, a child of Linear8bitLt that can do backward without CB * added exact match tests for layouts and linear layers: see tests/test_linear8bitlt.py * switched petals to the new layer type Core idea: layouts apply the same permutation to every tile in the matrix. We can treat this as (batched) gather ops. Reshape input tensor so that ij-th gather operation op will apply to ij-th elements in each tile. Prototype: Layout info: https://github.com/TimDettmers/bitsandbytes/blob/main/csrc/kernels.cu#L2130-L2136 Co-authored-by: Alexander Borzunov <hxrussia@gmail.com> Co-authored-by: Aleksandr Borzunov <borzunov.alexander@gmail.com> Co-authored-by: Tim Dettmers <tim.dettmers@gmail.com>	2 years ago
justheuristic	8dc0f513ba	Hotfix span selection (#110) ... Fix an issue in span selection that was introduced in #106	2 years ago
justheuristic	a2066a4096	Optimize RemoteSequenceManager (#106) ... - [x] made RemoteSequenceManager into a background thread that pre-fetches information instead of running just in time - [x] moved routing-related stuff to petals.client.routing - [x] extract remote peer routing information to RemoteSequenceInfo - [x] made sure that the code survives continued use (e.g. one hour) - [x] updated every spot where update_ is called manually - [x] modified get_sequence to check that the thread is alive, warn if not - [x] removed max_retries, switched rpc_info to exponential backoff - [x] fixed a bg that causes RemoteSeq* to lose user-defined hyperparameters (e.g. timeout) upon subsequencing (sequential[3:5]) - [x] moved client-side points strategy to client.routing - [x] ensured that RemoteSequenceManager thread created in get_remote_module properly shuts down when the module is destroyed - [x] resolved minor affected todos - [x] modified tests to no longer use PYTHONPATH - [x] worked around protocol error in rpc_info Co-authored-by: Aleksandr Borzunov <borzunov.alexander@gmail.com> Co-authored-by: Artem Chumachenko <artek.chumak@gmail.com>	2 years ago
Artem Chumachenko	7d859a947b	Expose request_timeout to DistributedBloomConfig (#105) ... Co-authored-by: Alexander Borzunov <borzunov.alexander@gmail.com>	2 years ago
Max Ryabinin	9faf08b898	Remove unused imports, add missing arguments to docstrings (#108) ... * Remove unused imports, add missing arguments to docstrings	2 years ago
justheuristic	b3115dac58	Update throughput.py	2 years ago
Alexander Borzunov	0a1cd3b9ba	Fix ptune with `low_cpu_mem_usage=True` (as in Colab) (#103) ... Fixes: - An exception while creating a model with `ptune/deep_ptune` and `low_cpu_mem_usage=True` (which is currently default). - dtype mismatch between the prompts and the rest of the model in `.forward()`.	2 years ago
Alexander Borzunov	43ac6016ac	Fix dtypes in backend schemas (#99) ... Currently, the schemas use `torch.float32`, so all inputs and outputs converted to float32 before sending and after receiving on both servers and clients. This creates a huge slowdown for the system. * This PR makes the schemas use the server's `--torch_dtype` argument (default is `torch.bloat16` for BLOOM-176B) * an option for client to request a specific output compression. Use case 1: client sends quantized inputs and expects quantized inputs in return. Use case 2: client uses quantization for gradients w.r.t. activations, but keeps grads w.r.t. __prompts__ as is for greater precision. * a comment explaining the purpose of NoSpendingPolicy - since we likely won't have it for the workshop * a test with custom compression (janky implementation for testing purposes) Co-authored-by: justheuristic <justheuristic@gmail.com>	2 years ago
Alexander Borzunov	7bd5916744	Make Petals a pip-installable package (attempt 2) (#102) ... 1. Petals can be now installed using `pip install git+https://github.com/bigscience-workshop/petals` - In case if you already cloned the repo, you can do `pip install .` or `pip install .[dev]` 2. Moved `src` => `src/petals` - Replaced `from src.smth import smth` with `from petals.smth import smth` 3. Moved `cli` => `src/petals/cli` - Replaced `python -m cli.run_smth` with `python -m petals.cli.run_smth` (all utilities are now available right after pip installation) 4. Moved the `requirements*.txt` contents to `setup.cfg` (`requirements.txt` for packages is not supported well by modern packaging utils) 5. Increased the package version from `0.2` to `1.0alpha1`	2 years ago