Update README.md

8bit_blocks
justheuristic 2 years ago committed by GitHub
parent aaaf0c2dad
commit f3722d52cf
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -8,7 +8,7 @@ if you.read(this) and you.name not in '@timdettmers @borzunov @mryab @greenfatgu
# install
### install
```bash
@ -24,7 +24,7 @@ pip install https://github.com/huggingface/transformers/archive/6589e510fa4e6c44
```
### Test local inference:
### run local inference:
No networking whatsoever, used to verify architecture optimizations
```bash
@ -32,7 +32,7 @@ No networking whatsoever, used to verify architecture optimizations
python -m cli.inference_one_block --config cli/config.json # see other args
```
### Test distributed inference / training
### run distributed inference / training
First, run one or more servers like this:
```bash
@ -71,7 +71,7 @@ with layer3.begin_inference_session() as sess:
```
### Convert regular bloom to distributed
### convert regular bloom to distributed
```bash
# convert model from HF hub to a distributed format (can take hours depending on your connection!)
@ -82,7 +82,7 @@ python -m cli.convert_model --model bigscience/bloom-6b3 \
```
### Test local vs remote model
### test local vs remote block (allclose)
To test distributed inference, run one or more servers, then open a new shell and run pytest with environment variables:
```bash
@ -97,4 +97,4 @@ BLOCK_UID=bloom6b3.4 pytest tests/test_block_exact_match.py
# the test below will fail because server only has layers [3:5)
# BLOCK_UID=bloom6b3.7 pytest tests/test_block_exact_match.py
```
```

Loading…
Cancel
Save