4 Commits (repetition-penalty)

Author	SHA1	Message	Date
Alexander Borzunov	0e7189b3ed	benchmarks: Aggregate speed among workers, set default dtype torch32 (#454)	10 months ago
Alexander Borzunov	8c546d988a	Test Llama, rebalancing, throughput eval, and all CLI scripts (#452) ... This PR extends CI to: 1. Test Llama code using [TinyLlama-v0](https://huggingface.co/Maykeye/TinyLLama-v0). 2. Test rebalancing (sets up a situation where the 1st server needs to change its original position). 3. Check if benchmark scripts run (in case someone breaks its code). Note that the benchmark results are meaningless here (since they're measured on a tiny swarm of CPU servers, with low `--n_steps`). 4. Test `petals.cli.run_dht`. 5. Increase swap space and watch free RAM (a common issue is that actions are cancelled without explanation if there's not enough RAM - so it's a useful reminder + debug tool). 6. Fix flapping tests for bloom-560m by increasing tolerance. Other minor changes: fix `--help` messages to show defaults, fix docs, tune rebalancing constants.	10 months ago
Alexander Borzunov	10c72acdf4	Fix warmup steps and minor issues in benchmarks (#334) ... The previous code was incorrect for the case of `warmup_steps != 1` (this mode was never used, but can be used in future).	11 months ago
Alexander Borzunov	d126ee3053	Add benchmark scripts (#319) ... This PR: - Adds benchmark scripts for inference, forward pass, and full training step (e.g. used for experiments in our paper). - Fixes bug with dtypes in `petals.DistributedBloomForSequenceClassification`. - (minor refactor) Moves `DTYPE_MAP` to `petals.constants` as a useful constant.	11 months ago