Archives
/
petals
mirror of https://github.com/bigscience-workshop/petals
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
224 Commits
131 Branches
15 Tags
6.4 MiB
main
test_branch
peft_update
forward_backward
fix-docker
forward_kwargs
bump
test_main
fix-inference-retry
lora_from_hub
payload-size
partial_rollback
qkv_merge
no_qkv_merge
wip_triton
hivemind-dht-fork-process
repetition-penalty
amd-gpus
bnb-0-41-1
lru
beat-docker-into-submission
measurements
debug-leak
fix-nf4-and-dtypes
declare_adapters
empty-weights
download_8bit_weights
no-cpufeature
versions
test_opt_serving
borzunov-patch-2
borzunov-patch-1
processing_attention
yozh-dev-branch
server-increase-startup-timeout
vectorized_beam_search
friendly-timeout-errors
hivemind-1.1.4
fix3
hotfix_bnb
fix-ptune
server-dtypes
pip-installable-v2
pip-installable
diff-compression
client-convenience
server-timeouts
server-logging
beamsearch
fix-protobuf
fix-requirements
fix-joining-announce
bootstrap-peers
fault-tolerant-inference
examples_fix_hivemind
forward-backward-timeouts
fix-rebalancing-issues
add-sst2-example
enable-rebalancing
update_example_1
fix-too-many-open-files
update-hivemind
extract-module-container
instruction-readability-style
readme-clarifications
justheuristic-patch-5
fix-readme
ptune-example-personachat
rtfd
fix-pb2
investigate-segfault
upd-deps
priority-tasks
justheuristic-patch-4
cache
justheuristic-patch-3
generation-inference
deep_prompt_inference
warn-about-6b-instructions
update-readme-disclaimers-faq
justheuristic-patch-2
update-bullet-points
update-readme-pics
readme-release
remove-remote-block
prompt-inference
fix-cache
optimize_seq
fix-seq-backward-recovery
fix-distr-seq-cls
justheuristic-patch-1
fix-convert-8bit
memory_savings
distributed-deep-ptune
ptune-wip
pytest-verbose
rename-test-model
8bit_backward
8bit-model
8bit_model_inference
petals-readme-title
support-backend-dtypes
deep-prompt-tuning
mockup
efficient-forward-backward
fix-branch-name
dbaranchuk-patch-1
get_sequence
generation
fix-ci
fix-master-ci
test-push
facelift
CI
prompt-tuning
client-attempt2
measure-throughput
lm_head
load-balancing
sequence
demo-1
standardize
diff
rpc
update-model
client
fix-auth-token
multiple-experts
8bit_blocks
inference_chain
main_fix
v1.0.0
v1.1.0
v1.1.1
v1.1.2
v1.1.3
v1.1.4
v1.1.5
v2.0.0.post1
v2.0.0.post2
v2.0.0.post3
v2.0.1
v2.0.1.post1
v2.0.1.post2
v2.1.0
v2.2.0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0c3781a89c'
${ noResults }
Commit Graph

25 Commits (0c3781a89c9ca0dd3ecde981d3899765f343d4cd)

Author SHA1 Message Date
Alexander Borzunov c6e1b5a8e5
Add various server timeouts, lower --max_batch_size and --inference_max_length defaults (#97) 2 years ago
Alexander Borzunov 8a73b41a42
Make ServerState announcements work better (#93) 
- Before this PR, `ServerState.JOINING` was announced only once. This announcement quickly expires in case of the full-size BLOOM, since loading blocks takes several minutes. This PR fixes it, so `ServerState.JOINING` is announced periodically in a thread until blocks are loaded.

- This PR also makes the `Server` class a non-thread, so it runs in the main thread and can catch `KeyboardInterrupt`. This is important, since if we are downloading blocks right now, we need to stop it and send the `ServerState.OFFLINE` message. Note that `ModuleContainer` is still a thread.

- (minor) For the sake of readability, I moved the `ModuleContainer.create()` definition, so it is now defined before `Server.__init__()` (this is because `.create()` is invoked first).
 2 years ago
Alexander Borzunov dc71574a63
Use public swarm by default (#92) 
This PR makes servers and clients use public swarm's bootstrap peers if no other initial peers are specified.

If you'd like a server to start a new swarm, provide the `--new_swarm` CLI argument.
 2 years ago
Alexander Borzunov ee4e69c254
Enable rebalancing by default (#84) 2 years ago
Alexander Borzunov f64eb3a665
Update hivemind to 1.1.2, mark `model` argument as required (#81) 2 years ago
Alexander Borzunov 149f433763
Rebalance swarm when necessary (#34) 2 years ago
justheuristic 8caf1145a8
Quality of life changes: update readme, simplify run_server interface (#75) 
- run_server now accepts model name as both positional and keyword argument
- changed names in README to account for interface updates
- moved model conversion from README to a separate wiki page
- updated requirements.txt
 2 years ago
justheuristic e92487e5d2
Update dependency versions (#71) 
* update dependency versions
* install bitsandbytes cpuonly from pip
* remove deprecated API from task pool
* clearer startup logs

Co-authored-by: Tim Dettmers <dettmers@cs.washington.edu>
 2 years ago
Pavel Samygin 50535a8435
Priority tasks (#47) 
* priority in handlers and backend pools
* simple points system on server side
* priortize task in handler before submit task
* fix tests
* s/expert/block/g

Co-authored-by: justheuristic <justheuristic@gmail.com>
 2 years ago
justheuristic d271b75dd4
Let users specify sequence length instead of assuming 2048 (#52) 
- Maximum length is now provided in `.inference_session(max_length=100)`
   - previously, we would always assume max length = 2048
- added a generic way to forward **kwargs to inference session
  - for compatibility with #47 
  - Note to @borzunov : it does *not* pass them arbitrarily, but instead checks for kwarg names at the bottom level
- run_server can be started with a custom max_length for inference
- renamed --cache_size_bytes to --attention_cache_bytes (to avoid collision with --cache_dir)
- --attn_cache_bytes can now support humane file sizes (e.g. 300MB instead of 314572800)
- made some server-side errors more human-readable to user (e.g. when max length is exceeded)

Co-authored-by: Aleksandr Borzunov <borzunov.alexander@gmail.com>
Co-authored-by: Alexander Borzunov <hxrussia@gmail.com>
 2 years ago
Dmitry Baranchuk 11a424837f
integrate mixed-8bit model (#39) 
* integrate mixed-8bit model
* Fix bug with model duplication in RAM
* set throughput=1.0 to fix zero throughput problem
* add revision support
* update hivemind and bitsandbytes
* update deploy scripts
* update installation instructions
 2 years ago
Dmitry Baranchuk f114a6d417
set default num_handlers=16 2 years ago
Alexander Borzunov 75856e4769
Measure and cache network & compute throughput (#21) 2 years ago
Alexander Borzunov aba43f1308
Implement block selection on servers (#20) 2 years ago
justheuristic 9c492bbe8c Infer prefix by defaukt 2 years ago
justheuristic 6e3db6bed6 black-isort 2 years ago
justheuristic d5c410bb1f fix auth token 2 years ago
justheuristic 1ab5fb1630 fetch a specific bloom block without downloading the entire model 2 years ago
justheuristic 3b9351de1c isort 2 years ago
justheuristic 20497f81d1 switch to hivemind-master 2 years ago
Pavel Samygin 57f4e0a899 add identity path 2 years ago
justheuristic a798ea04a6 add minimalistic benchmarks 2 years ago
justheuristic 14e316b52a black-isort 2 years ago
justheuristic 7ce7cd7a97 basic backend 2 years ago
justheuristic 1c49bcb741 basic backend 2 years ago