Archives
/
petals
mirror of https://github.com/bigscience-workshop/petals
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix code example in readme (#169)

Browse Source 
Makes it closer to runnable code, except for imports and defining tokenizer & data loader.
pull/170/head
Alexander Borzunov 1 year ago committed by GitHub
parent 0b0277ed6f
commit 26e6120288
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 9 additions and 9 deletions
Show Stats Download Patch File Download Diff File

18
README.md
Unescape Escape View File

@ -9,14 +9,14 @@ Generate text using distributed BLOOM and fine-tune it for your own tasks:
```python ```python
from petals import DistributedBloomForCausalLM from petals import DistributedBloomForCausalLM
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune", pre_seq_len=16)
# Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet # Embeddings & prompts are on your device, BLOOM blocks are distributed across the Internet
model = DistributedBloomForCausalLM.from_pretrained("bigscience/bloom-petals", tuning_mode="ptune")
inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"] inputs = tokenizer("A cat sat", return_tensors="pt")["input_ids"]
outputs = model.generate(inputs, max_new_tokens=5) outputs = model.generate(inputs, max_new_tokens=5)
print(tokenizer.decode(remote_outputs[0])) # A cat sat on a mat... print(tokenizer.decode(outputs[0])) # A cat sat on a mat...
# Training (updates only prompts or adapters hosted locally) # Fine-tuning (updates only prompts or adapters hosted locally)
optimizer = torch.optim.AdamW(model.parameters()) optimizer = torch.optim.AdamW(model.parameters())
for input_ids, labels in data_loader: for input_ids, labels in data_loader:
outputs = model.forward(input_ids) outputs = model.forward(input_ids)
@ -34,13 +34,13 @@ Connect your own GPU and increase Petals capacity:
```bash ```bash
# In an Anaconda env # In an Anaconda env
(conda) $ conda install pytorch cudatoolkit=11.3 -c pytorch conda install pytorch cudatoolkit=11.3 -c pytorch
(conda) $ pip install git+https://github.com/bigscience-workshop/petals pip install git+https://github.com/bigscience-workshop/petals
(conda) $ python -m petals.cli.run_server bigscience/bloom-petals python -m petals.cli.run_server bigscience/bloom-petals
# Or using a GPU-enabled Docker image # Or using our GPU-enabled Docker image
sudo docker run --net host --ipc host --gpus all --volume petals-cache:/cache --rm learningathome/petals:main \ sudo docker run --net host --ipc host --gpus all --volume petals-cache:/cache --rm \
python -m petals.cli.run_server bigscience/bloom-petals learningathome/petals:main python -m petals.cli.run_server bigscience/bloom-petals
``` ```
💬 If you have any issues or feedback, please join [our Discord server](https://discord.gg/D9MwApKgWa)! 💬 If you have any issues or feedback, please join [our Discord server](https://discord.gg/D9MwApKgWa)!

Write Preview
Loading…
Cancel
Save