petals

Commit Graph

Author	SHA1	Message	Date
Max Ryabinin	ae19b65095	Add position_ids argument to DistributedFalconModel (#525 )	8 months ago
Alexander Borzunov	d40eb6c701	Fix prompt tuning after #464 (#501 ) Unfortunately, running inference in models with `"ptune" in config.tuning_mode` was broken after #464.	9 months ago
Alexander Borzunov	dd4a3230bc	Add Falcon support (#499 ) This PR adds: - Support for models based on `transformers.FalconModel` (the in-library format for Falcon). Tested on Falcon-40B. - CI tests for Falcon-RW-1B. - `--throughput dry_run` option to evaluate throughput and exit right away (implemented by @mryab). Limitations: - Backward pass support is broken for now, will be fixed in #500. Co-authored-by: Max Ryabinin <mryabinin0@gmail.com>	9 months ago

Author

SHA1

Message

Date

Max Ryabinin

ae19b65095

Add position_ids argument to DistributedFalconModel (#525 )

Alexander Borzunov

d40eb6c701

Fix prompt tuning after #464 (#501 )

Unfortunately, running inference in models with `"ptune" in config.tuning_mode` was broken after #464.

Alexander Borzunov

dd4a3230bc

Add Falcon support (#499 )

This PR adds:

- Support for models based on `transformers.FalconModel` (the in-library format for Falcon). Tested on Falcon-40B.
- CI tests for Falcon-RW-1B.
- `--throughput dry_run` option to evaluate throughput and exit right away (implemented by @mryab).

Limitations:

- Backward pass support is broken for now, will be fixed in #500.

Co-authored-by: Max Ryabinin <mryabinin0@gmail.com>

3 Commits (3195579620cd95414cc66cf66f904f37509d9456)