Commit Graph

Select branches

Hide Pull Requests

8bit-model

8bit_backward

8bit_blocks

8bit_model_inference

CI

add-sst2-example

amd-gpus

beamsearch

beat-docker-into-submission

bnb-0-41-1

bootstrap-peers

borzunov-patch-1

borzunov-patch-2

bump

cache

client

client-attempt2

client-convenience

dbaranchuk-patch-1

debug-leak

declare_adapters

deep-prompt-tuning

deep_prompt_inference

demo-1

diff

diff-compression

distributed-deep-ptune

download_8bit_weights

efficient-forward-backward

empty-weights

enable-rebalancing

examples_fix_hivemind

extract-module-container

facelift

fault-tolerant-inference

fix-auth-token

fix-branch-name

fix-cache

fix-ci

fix-convert-8bit

fix-distr-seq-cls

fix-docker

fix-inference-retry

fix-joining-announce

fix-master-ci

fix-nf4-and-dtypes

fix-pb2

fix-protobuf

fix-ptune

fix-readme

fix-rebalancing-issues

fix-requirements

fix-seq-backward-recovery

fix-too-many-open-files

fix3

forward-backward-timeouts

forward_backward

forward_kwargs

friendly-timeout-errors

generation

generation-inference

get_sequence

hivemind-1.1.4

hivemind-dht-fork-process

hotfix_bnb

inference_chain

instruction-readability-style

investigate-segfault

justheuristic-patch-1

justheuristic-patch-2

justheuristic-patch-3

justheuristic-patch-4

justheuristic-patch-5

lm_head

load-balancing

lora_from_hub

lru

main

main_fix

measure-throughput

measurements

memory_savings

mockup

multiple-experts

no-cpufeature

no_qkv_merge

optimize_seq

partial_rollback

payload-size

peft_update

petals-readme-title

pip-installable

pip-installable-v2

priority-tasks

processing_attention

prompt-inference

prompt-tuning

ptune-example-personachat

ptune-wip

pytest-verbose

qkv_merge

readme-clarifications

readme-release

remove-remote-block

rename-test-model

repetition-penalty

rpc

rtfd

sequence

server-dtypes

server-increase-startup-timeout

server-logging

server-timeouts

standardize

support-backend-dtypes

test-push

test_main

test_opt_serving

upd-deps

update-bullet-points

update-hivemind

update-model

update-readme-disclaimers-faq

update-readme-pics

update_example_1

vectorized_beam_search

versions

warn-about-6b-instructions

wip_triton

yozh-dev-branch

#100

#101

#102

#103

#104

#105

#106

#107

#108

#109

#110

#111

#112

#113

#114

#115

#116

#117

#118

#119

#120

#121

#122

#123

#124

#125

#126

#127

#128

#129

#130

#131

#132

#133

#134

#135

#136

#137

#138

#139

#140

#141

#142

#143

#144

#145

#148

#149

#150

#151

#152

#153

#154

#155

#156

#157

#159

#160

#161

#162

#163

#165

#166

#168

#169

#17

#170

#171

#172

#173

#175

#176

#177

#179

#18

#180

#181

#182

#183

#184

#185

#186

#187

#188

#189

#19

#190

#191

#194

#195

#196

#197

#198

#199

#20

#200

#204

#205

#206

#209

#21

#210

#211

#212

#213

#214

#215

#217

#218

#219

#22

#221

#224

#225

#227

#229

#23

#230

#231

#232

#234

#235

#24

#243

#244

#245

#247

#248

#249

#25

#251

#252

#253

#257

#259

#26

#260

#263

#264

#265

#266

#267

#27

#273

#273

#275

#276

#277

#278

#28

#284

#285

#286

#287

#29

#290

#292

#297

#298

#299

#30

#300

#301

#302

#304

#305

#306

#307

#309

#31

#310

#311

#312

#313

#314

#315

#316

#319

#32

#323

#324

#325

#329

#33

#330

#331

#332

#333

#334

#335

#336

#337

#339

#34

#340

#341

#342

#343

#344

#345

#346

#348

#349

#35

#350

#351

#352

#353

#354

#355

#356

#357

#358

#36

#360

#362

#364

#365

#369

#37

#370

#371

#373

#374

#377

#378

#379

#38

#380

#383

#384

#385

#39

#390

#391

#392

#393

#396

#397

#399

#40

#402

#403

#404

#406

#407

#409

#41

#410

#411

#418

#42

#422

#423

#424

#428

#43

#430

#431

#434

#435

#436

#437

#438

#438

#44

#440

#441

#442

#443

#445

#448

#45

#451

#452

#454

#456

#458

#459

#46

#461

#462

#463

#464

#467

#467

#47

#470

#471

#472

#473

#474

#475

#476

#477

#478

#479

#48

#480

#482

#484

#485

#486

#487

#489

#49

#490

#491

#493

#493

#496

#497

#498

#498

#499

#50

#500

#501

#502

#504

#505

#505

#506

#506

#510

#513

#516

#517

#52

#520

#523

#523

#525

#53

#530

#531

#535

#537

#538

#539

#54

#543

#544

#544

#545

#545

#547

#547

#55

#552

#553

#554

#555

#555

#557

#557

#558

#559

#560

#560

#563

#567

#57

#570

#572

#574

#577

#579

#58

#580

#59

#60

#61

#62

#63

#64

#65

#66

#69

#70

#71

#72

#73

#74

#75

#76

#77

#80

#81

#82

#83

#84

#85

#86

#87

#88

#89

#9

#90

#91

#92

#93

#94

#95

#96

#97

#98

#99

v1.0.0

v1.1.0

v1.1.1

v1.1.2

v1.1.3

v1.1.4

v1.1.5

v2.0.0.post1

v2.0.0.post2

v2.0.0.post3

v2.0.1

v2.0.1.post1

v2.0.1.post2

v2.1.0

v2.2.0

Mono Color

• c7255ef547
  Update README.md Alexander Borzunov 2023-09-22 03:20:18 +0400
• a2484b3053
  Fix file locks in NFS-mounted directories (#517) FYY 2023-09-19 20:01:23 -0400
• d57917b1de #515 update file open mode to avoid acquire file lock fail on NFS shared mount tonywang16 2023-09-19 18:51:01 -0400
• a14c0f3bc9 Fix #515 unable to acquire shared lock on NFS mounted directory tonywang16 2023-09-19 16:01:22 -0400
• 5c8bb55d2a Merge branch 'main' of https://github.com/tonywang16/petals j879@huggingface.co 2023-09-19 15:39:24 -0400
• 172f593084 #515 fix file lock failed to acquire on NFS mounted directory j879@huggingface.co 2023-09-19 15:38:59 -0400
• 5ce4f1a159
  Store (start_block, end_block) in each DHT record for reliability (#510) Alexander Borzunov 2023-09-15 23:53:57 +0400
• 533f9c8a34 Remove excess line Aleksandr Borzunov 2023-09-15 18:32:11 +0000
• 4a302d0bd0 Refactor compute_spans() in petals.client.routing.sequence_info and petals.server.block_selection Aleksandr Borzunov 2023-09-15 17:10:23 +0000
• dcd4641b82 List[Optional[RemoteModuleInfo]] -> List[RemoteModuleInfo] everywhere Aleksandr Borzunov 2023-09-15 16:28:24 +0000
• 2608b5cf8d Fix (2) Aleksandr Borzunov 2023-09-15 15:34:55 +0000
• 22da72aeeb Remove unused imports, methods Aleksandr Borzunov 2023-09-15 15:12:18 +0000
• f20d75ca69 Fix block offset in SequenceManager slices Aleksandr Borzunov 2023-09-15 14:58:49 +0000
• 58634df833 Use stricter DHT validation Aleksandr Borzunov 2023-09-15 14:32:02 +0000
• 7e584448f1 Fix mixed JOINING/ONLINE records Aleksandr Borzunov 2023-09-15 14:22:02 +0000
• 0f527a0788 Use start_block, end_block if present Aleksandr Borzunov 2023-09-15 13:58:49 +0000
• 145377c4cc Announce start_block and end_block Aleksandr Borzunov 2023-09-15 13:49:50 +0000
• 51af621c51
  Merge 1b21dd3217 into 158621677b Artem Chumachenko 2023-09-06 20:20:47 +0300
• 158621677b
  Bump version to 2.2.0 (#502) Alexander Borzunov 2023-09-06 19:43:30 +0400
• 53a9222f0c Add Falcon to readme v2.2.0 Aleksandr Borzunov 2023-09-06 15:26:29 +0000
• 4883923a66 Show license link for Falcon-180B Aleksandr Borzunov 2023-09-06 15:23:48 +0000
• 1b21dd3217 Add memory cache usage lora_from_hub Artem Chumachenko 2023-09-06 12:10:32 +0400
• 01c3cf8d15 Add first version Artem Chumachenko 2023-09-06 10:46:10 +0400
• c665c42cf2 reduce diff Your Name 2023-09-06 06:24:59 +0300
• 3f06b53b1d temporary rollback: allow kwargs only at first inference step Your Name 2023-09-06 05:53:07 +0300
• 3048c3b3ad rollback Your Name 2023-09-06 04:29:28 +0300
• 721f7d2db3 unbreak everything Your Name 2023-09-06 03:52:40 +0300
• 3bffcde0fe black+isort Your Name 2023-09-06 03:37:59 +0300
• 8eb1722f1e standardize: s/backend_kwargs/block_kwargs/g everywhere Your Name 2023-09-06 03:37:45 +0300
• 68b8cea246 note Your Name 2023-09-06 03:35:39 +0300
• a23bd73f3b probably break everyting Your Name 2023-09-06 03:30:26 +0300
• 056cd77f11 standardize checking block_kwargs Your Name 2023-09-06 03:16:15 +0300
• aacd8b2f9d pass args/kwargs via forward Your Name 2023-09-06 03:16:03 +0300
• 62e780c054 check num block kwargs Your Name 2023-09-06 03:13:08 +0300
• 17d278e88a black-isort-clarify Your Name 2023-09-06 02:38:14 +0300
• 9e29140bb0 mention reference issue Your Name 2023-09-06 01:37:48 +0300
• b7bd4770d7 black-isort Your Name 2023-09-06 01:28:12 +0300
• f2049658b6 make it work for fwd, bwd Your Name 2023-09-06 01:28:01 +0300
• 465fd93147 more WIP Your Name 2023-09-05 23:29:16 +0300
• 4393d99e78 1isort Your Name 2023-09-05 22:14:38 +0300
• 49474e5477 wip some more Your Name 2023-09-05 22:07:48 +0300
• e5c2d8eca4 WIP BEFORE MEETING NEED BACKWARD UPDATE Your Name 2023-09-05 15:32:47 +0300
• b945e388e5
  Remove smaller limit for legacy bfloat16 serialization payload-size Alexander Borzunov 2023-09-05 16:23:26 +0400
• 4ac91cf5dd delete duplicate tests Your Name 2023-09-05 14:33:16 +0300
• 2e760319ab add docstr Your Name 2023-09-05 14:28:48 +0300
• cc4fe17a99 minimize diff partial_rollback Your Name 2023-09-05 14:05:41 +0300
• 6c7f762379 rollback: only generic kwarg Your Name 2023-09-05 05:21:12 +0300
• 6256995bb1 Merge remote-tracking branch 'origin/main' into forward_kwargs Your Name 2023-09-05 05:12:19 +0300
• 39d0b31dcf
  Bump version to 2.2.0 Alexander Borzunov 2023-09-04 19:01:51 +0400
• 1ebd88ae7b
  Optimize the Falcon block for inference (#500) Max Ryabinin 2023-09-04 14:38:32 +0200
• 52baffb056 Update test_optimized_layers Max Ryabinin 2023-09-04 14:43:37 +0300
• 2c27c19df4 Do not fuse split_heads with qkv This is most likely due to bitsandbytes performing work not captured in the graph Max Ryabinin 2023-09-04 14:42:13 +0300
• ea6c037c8b Enable CUDA graphs only on CUDA Max Ryabinin 2023-09-04 12:01:10 +0300
• 91f6248535 Run tests on CUDA and CPU, Max Ryabinin 2023-09-04 12:00:59 +0300
• 177669e97f Rollback CUDA graphs Max Ryabinin 2023-09-04 11:25:49 +0300
• b941df5d2f Fix formatting Max Ryabinin 2023-09-04 04:38:00 +0300
• 1f2ef79da3 WIP disable graphs Max Ryabinin 2023-09-04 04:30:31 +0300
• cfaf6c1975 Fix rotary embeddings Max Ryabinin 2023-09-04 02:51:49 +0300
• d56f57acd2 Fix rotary embeddings Max Ryabinin 2023-09-04 02:47:58 +0300
• 841a0d5262 Improve test and compatibility Max Ryabinin 2023-09-04 01:56:29 +0300
• ae30427276 Make the block compatible with other architectures Max Ryabinin 2023-09-04 01:20:33 +0300
• 2c1452de5c Fix buffer registration Max Ryabinin 2023-09-04 01:05:27 +0300
• ce401f1163 Make cos_cached/sin_cached buffers Max Ryabinin 2023-09-04 00:47:27 +0300
• 111bf7e125 Fix the test Max Ryabinin 2023-09-04 00:47:11 +0300
• 67764fea9e Fix formatting, reduce diff Max Ryabinin 2023-09-04 00:16:10 +0300
• 1fc22bd69f Post-rebase changes Max Ryabinin 2023-09-04 00:09:22 +0300
• 1f006c59a1 Fix class names Max Ryabinin 2023-09-03 23:58:13 +0300
• ca4d091a3f Optimize Falcon block for inference Max Ryabinin 2023-09-03 23:50:55 +0300
• d40eb6c701
  Fix prompt tuning after #464 (#501) Alexander Borzunov 2023-09-04 12:25:29 +0400
• db46bf3ac1 Account for pre_seq_len in new session's max_length Aleksandr Borzunov 2023-09-04 08:05:13 +0000
• 8cb6b37e2f Fix prompt tuning after #464 Aleksandr Borzunov 2023-09-04 07:37:35 +0000
• dd4a3230bc
  Add Falcon support (#499) Alexander Borzunov 2023-09-04 01:45:37 +0400
• 1d865d1704 Fix dim order Aleksandr Borzunov 2023-09-03 20:39:13 +0000
• f6553ad4cb Fix cache reordering with seq_len = 0 Aleksandr Borzunov 2023-09-03 19:57:24 +0000
• 4537c77004 Expand/collapse KV caches when config.new_decoder_architecture is True Aleksandr Borzunov 2023-09-03 18:50:29 +0000
• cda4fe8cef Fix num_key_value_groups Aleksandr Borzunov 2023-09-03 16:42:06 +0000
• 4159e557bf Create dummy data when materializing qkv_proj qkv_merge Max Ryabinin 2023-09-03 19:20:07 +0300
• cac654a16f Set default revision for tiiuae/* repos to models in the in-library format Aleksandr Borzunov 2023-09-03 14:29:09 +0000
• 97dd3764d9 Set default pad_token_id Aleksandr Borzunov 2023-09-03 13:37:12 +0000
• 8d78cf2357 Use safetensors float32 model Aleksandr Borzunov 2023-09-03 13:03:23 +0000
• 012ae0e56a Use NF4 for all models on CUDA Aleksandr Borzunov 2023-09-03 13:02:32 +0000
• ff83e7c8ad Fix DistrtibutedFalconModel.word_embeddings_layernorm Aleksandr Borzunov 2023-09-03 06:16:40 +0000
• cc234c57ad Fix comment Aleksandr Borzunov 2023-09-02 23:33:00 +0000
• bd6aa2399c Support --throughput dry_run Aleksandr Borzunov 2023-09-02 23:28:59 +0000
• 15fed1e88b Draft Falcon support Aleksandr Borzunov 2023-09-02 23:24:42 +0000
• 9cb4c721e7 Fix checking for nonexistent keys Max Ryabinin 2023-09-03 01:55:50 +0300
• 16fb547960 Fix removal of nonexistent keys Max Ryabinin 2023-09-03 01:35:00 +0300
• f100915641 Reformat code with black Max Ryabinin 2023-09-03 01:09:56 +0300
• 4644131086 Ignore missing qkv_proj.weight when loading a checkpoint Max Ryabinin 2023-09-03 01:08:55 +0300
• a7f87b636b Disable the optimization no_qkv_merge Max Ryabinin 2023-09-03 00:49:23 +0300
• b2ab84cc33 Add dry_run option to --throughput Max Ryabinin 2023-09-03 00:41:03 +0300
• c666a975d0 Remove unused import in throughput.py Max Ryabinin 2023-09-03 00:38:35 +0300
• 57119bb201 Merge query/key/value projection layers Max Ryabinin 2023-09-03 00:11:03 +0300
• fa464dfc99 WIP Triton+QKV merge wip_triton Max Ryabinin 2023-09-03 00:11:03 +0300
• b4d822afb2
  Force use_cache=True in config only (#497) Alexander Borzunov 2023-09-03 01:16:00 +0400
• 9cd25d95e6 Force use_cache=True in config only Aleksandr Borzunov 2023-09-02 21:03:56 +0000
• abd547735f
  Force use_cache=True (#496) Alexander Borzunov 2023-09-02 22:57:18 +0400
• 4d5a21ca31
  Update setup.cfg Alexander Borzunov 2023-09-02 21:36:34 +0400
• 13f66c4e5e Force use_cache=True Aleksandr Borzunov 2023-09-02 17:32:39 +0000
• ce89b649b5
  Merge branch 'main' into forward_kwargs justheuristic 2023-09-01 16:42:30 +0300