petals

Author	SHA1	Message	Date
Alexander Borzunov	057a2fb5de	Support Llama 2 (#379 )	11 months ago
Alexander Borzunov	3218534745	Fix --token arg (#378 )	11 months ago
justheuristic	398a384075	Inherit bitsandbytes compute dtype correctly (override peft quirk) (#377 )	11 months ago
Alexander Borzunov	c735dd7ba3	Update transformers to 4.31.0 and peft to 0.4.0 (#371 )	11 months ago
justheuristic	37fdcb3fe0	Switch adapters slightly faster (#353 ) Currently, each `TransformerBackend.inference_step` looks for adapters and sets the correct adapter type for each block. This is not very expensive, but it can measurably affect inference time. This pull request uses faster adapter switching with just one variable assignment, without iterating over block.modules().	11 months ago
Alexander Borzunov	9703358df0	Fix bugs in _choose_num_blocks() added in #346 (#354 )	11 months ago
Alexander Borzunov	e12d4c666b	Spam less in server logs (#350 )	11 months ago
justheuristic	010857a834	Estimate adapter memory overhead in choose_num_blocks() (#346 ) * estimate adapter memory overhead * reduce number of heads based on that --------- Co-authored-by: Alexander Borzunov <borzunov.alexander@gmail.com>	11 months ago
Artem Chumachenko	b9f0a5467f	Support peft LoRA adapters (#335 ) Implement an option to deploy PEFT adapters to a server. Clients can set active_adapter=... to use these adapters. --------- Co-authored-by: Aleksandr Borzunov <borzunov.alexander@gmail.com> Co-authored-by: justheuristic <justheuristic@gmail.com>	11 months ago