langchain/docs/docs/integrations/llms/koboldai.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FPF4vhdZyJ7S"
   },
   "source": [
    "# KoboldAI API\n",
    "\n",
    "[KoboldAI](https://github.com/KoboldAI/KoboldAI-Client) is a \"a browser-based front-end for AI-assisted writing with multiple local & remote AI models...\". It has a public and local API that is able to be used in langchain.\n",
    "\n",
    "This example goes over how to use LangChain with that API.\n",
    "\n",
    "Documentation can be found in the browser adding /api to the end of your endpoint (i.e http://127.0.0.1/:5000/api).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "lyzOsRRTf_Vr"
   },
   "outputs": [],
   "source": [
    "from langchain_community.llms import KoboldApiLLM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1a_H7mvfy51O"
   },
   "source": [
    "Replace the endpoint seen below with the one shown in the output after starting the webui with --api or --public-api\n",
    "\n",
    "Optionally, you can pass in parameters like temperature or max_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "g3vGebq8f_Vr"
   },
   "outputs": [],
   "source": [
    "llm = KoboldApiLLM(endpoint=\"http://192.168.1.144:5000\", max_length=80)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "sPxNGGiDf_Vr",
    "outputId": "024a1d62-3cd7-49a8-c6a8-5278224d02ef"
   },
   "outputs": [],
   "source": [
    "response = llm.invoke(\n",
    "    \"### Instruction:\\nWhat is the first book of the bible?\\n### Response:\"\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "venv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}