"There is a problem with the VRAM here despite `del model` and emptying the VRAM. You probably need to restart the notebook, re-execute the three first cells, and then execute this one. Please contact me if you have a fix!"
],
"metadata": {
"id": "6WjzALHtSfdb"
}
},
{
"cell_type": "code",
"source": [
@ -270,11 +279,13 @@
" torch_dtype=torch.float16,\n",
" device_map=device_map,\n",
" )\n",
" model = PeftModel.from_pretrained(base_model, \"./adapter\", offload_folder=\"/content/sample_data\")\n",
" model = PeftModel.from_pretrained(base_model, output_dir, offload_folder=\"/content/sample_data\")\n",