petals

Commit Graph

Author	SHA1	Message	Date
Alexander Borzunov	d40eb6c701	Fix prompt tuning after #464 (#501 ) Unfortunately, running inference in models with `"ptune" in config.tuning_mode` was broken after #464.	9 months ago
Alexander Borzunov	dd4a3230bc	Add Falcon support (#499 ) This PR adds: - Support for models based on `transformers.FalconModel` (the in-library format for Falcon). Tested on Falcon-40B. - CI tests for Falcon-RW-1B. - `--throughput dry_run` option to evaluate throughput and exit right away (implemented by @mryab). Limitations: - Backward pass support is broken for now, will be fixed in #500. Co-authored-by: Max Ryabinin <mryabinin0@gmail.com>	9 months ago

Author

SHA1

Message

Date

Alexander Borzunov

d40eb6c701

Fix prompt tuning after #464 (#501 )

Unfortunately, running inference in models with `"ptune" in config.tuning_mode` was broken after #464.

Alexander Borzunov

dd4a3230bc

Add Falcon support (#499 )

This PR adds:

- Support for models based on `transformers.FalconModel` (the in-library format for Falcon). Tested on Falcon-40B.
- CI tests for Falcon-RW-1B.
- `--throughput dry_run` option to evaluate throughput and exit right away (implemented by @mryab).

Limitations:

- Backward pass support is broken for now, will be fixed in #500.

Co-authored-by: Max Ryabinin <mryabinin0@gmail.com>

2 Commits (d40eb6c7015c0de2914cb013601bdd47544d16ef)