langchain/docs/extras/use_cases/code/code-analysis-deeplake.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Use LangChain, GPT and Activeloop's Deep Lake to work with code base\n",
    "In this tutorial, we are going to use Langchain + Activeloop's Deep Lake with GPT to analyze the code base of the LangChain itself. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Design"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Prepare data:\n",
    "   1. Upload all python project files using the `langchain.document_loaders.TextLoader`. We will call these files the **documents**.\n",
    "   2. Split all documents to chunks using the `langchain.text_splitter.CharacterTextSplitter`.\n",
    "   3. Embed chunks and upload them into the DeepLake using `langchain.embeddings.openai.OpenAIEmbeddings` and `langchain.vectorstores.DeepLake`\n",
    "2. Question-Answering:\n",
    "   1. Build a chain from `langchain.chat_models.ChatOpenAI` and `langchain.chains.ConversationalRetrievalChain`\n",
    "   2. Prepare questions.\n",
    "   3. Get answers running the chain.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Integration preparations"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to set up keys for external services and install necessary python libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#!python3 -m pip install --upgrade langchain deeplake openai"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set up OpenAI embeddings, Deep Lake multi-modal vector store api and authenticate. \n",
    "\n",
    "For full documentation of Deep Lake please follow https://docs.activeloop.ai/ and API reference https://docs.deeplake.ai/en/latest/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = getpass()\n",
    "# Please manually enter OpenAI Key"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Authenticate into Deep Lake if you want to create your own dataset and publish it. You can get an API key from the platform at [app.activeloop.ai](https://app.activeloop.ai)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "activeloop_token = getpass(\"Activeloop Token:\")\n",
    "os.environ[\"ACTIVELOOP_TOKEN\"] = activeloop_token"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare data "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load all repository files. Here we assume this notebook is downloaded as the part of the langchain fork and we work with the python files of the `langchain` repo.\n",
    "\n",
    "If you want to use files from different repo, change `root_dir` to the root dir of your repo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls \"../../../..\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.document_loaders import TextLoader\n",
    "\n",
    "root_dir = \"../../../..\"\n",
    "\n",
    "docs = []\n",
    "for dirpath, dirnames, filenames in os.walk(root_dir):\n",
    "    for file in filenames:\n",
    "        if file.endswith(\".py\") and \"/.venv/\" not in dirpath:\n",
    "            try:\n",
    "                loader = TextLoader(os.path.join(dirpath, file), encoding=\"utf-8\")\n",
    "                docs.extend(loader.load_and_split())\n",
    "            except Exception as e:\n",
    "                pass\n",
    "print(f\"{len(docs)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, chunk the files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "\n",
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "texts = text_splitter.split_documents(docs)\n",
    "print(f\"{len(texts)}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then embed chunks and upload them to the DeepLake.\n",
    "\n",
    "This can take several minutes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.embeddings.openai import OpenAIEmbeddings\n",
    "\n",
    "embeddings = OpenAIEmbeddings()\n",
    "embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.vectorstores import DeepLake\n",
    "\n",
    "db = DeepLake.from_documents(\n",
    "    texts, embeddings, dataset_path=f\"hub://{<org_id>}/langchain-code\"\n",
    ")\n",
    "db"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Optional`: You can also use Deep Lake's Managed Tensor Database as a hosting service and run queries there. In order to do so, it is necessary to specify the runtime parameter as {'tensor_db': True} during the creation of the vector store. This configuration enables the execution of queries on the Managed Tensor Database, rather than on the client side. It should be noted that this functionality is not applicable to datasets stored locally or in-memory. In the event that a vector store has already been created outside of the Managed Tensor Database, it is possible to transfer it to the Managed Tensor Database by following the prescribed steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from langchain.vectorstores import DeepLake\n",
    "\n",
    "# db = DeepLake.from_documents(\n",
    "#     texts, embeddings, dataset_path=f\"hub://{<org_id>}/langchain-code\", runtime={\"tensor_db\": True}\n",
    "# )\n",
    "# db"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question Answering\n",
    "First load the dataset, construct the retriever, then construct the Conversational Chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "db = DeepLake(\n",
    "    dataset_path=f\"hub://{<org_id>}/langchain-code\",\n",
    "    read_only=True,\n",
    "    embedding_function=embeddings,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retriever = db.as_retriever()\n",
    "retriever.search_kwargs[\"distance_metric\"] = \"cos\"\n",
    "retriever.search_kwargs[\"fetch_k\"] = 20\n",
    "retriever.search_kwargs[\"maximal_marginal_relevance\"] = True\n",
    "retriever.search_kwargs[\"k\"] = 20"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also specify user defined functions using [Deep Lake filters](https://docs.deeplake.ai/en/latest/deeplake.core.dataset.html#deeplake.core.dataset.Dataset.filter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def filter(x):\n",
    "    # filter based on source code\n",
    "    if \"something\" in x[\"text\"].data()[\"value\"]:\n",
    "        return False\n",
    "\n",
    "    # filter based on path e.g. extension\n",
    "    metadata = x[\"metadata\"].data()[\"value\"]\n",
    "    return \"only_this\" in metadata[\"source\"] or \"also_that\" in metadata[\"source\"]\n",
    "\n",
    "\n",
    "### turn on below for custom filtering\n",
    "# retriever.search_kwargs['filter'] = filter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "\n",
    "model = ChatOpenAI(model_name=\"gpt-3.5-turbo\")  # 'ada' 'gpt-3.5-turbo' 'gpt-4',\n",
    "qa = ConversationalRetrievalChain.from_llm(model, retriever=retriever)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "questions = [\n",
    "    \"What is the class hierarchy?\",\n",
    "    # \"What classes are derived from the Chain class?\",\n",
    "    # \"What classes and functions in the ./langchain/utilities/ forlder are not covered by unit tests?\",\n",
    "    # \"What one improvement do you propose in code in relation to the class herarchy for the Chain class?\",\n",
    "]\n",
    "chat_history = []\n",
    "\n",
    "for question in questions:\n",
    "    result = qa({\"question\": question, \"chat_history\": chat_history})\n",
    "    chat_history.append((question, result[\"answer\"]))\n",
    "    print(f\"-> **Question**: {question} \\n\")\n",
    "    print(f\"**Answer**: {result['answer']} \\n\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "-> **Question**: What is the class hierarchy? \n",
    "\n",
    "**Answer**: There are several class hierarchies in the provided code, so I'll list a few:\n",
    "\n",
    "1. `BaseModel` -> `ConstitutionalPrinciple`: `ConstitutionalPrinciple` is a subclass of `BaseModel`.\n",
    "2. `BasePromptTemplate` -> `StringPromptTemplate`, `AIMessagePromptTemplate`, `BaseChatPromptTemplate`, `ChatMessagePromptTemplate`, `ChatPromptTemplate`, `HumanMessagePromptTemplate`, `MessagesPlaceholder`, `SystemMessagePromptTemplate`, `FewShotPromptTemplate`, `FewShotPromptWithTemplates`, `Prompt`, `PromptTemplate`: All of these classes are subclasses of `BasePromptTemplate`.\n",
    "3. `APIChain`, `Chain`, `MapReduceDocumentsChain`, `MapRerankDocumentsChain`, `RefineDocumentsChain`, `StuffDocumentsChain`, `HypotheticalDocumentEmbedder`, `LLMChain`, `LLMBashChain`, `LLMCheckerChain`, `LLMMathChain`, `LLMRequestsChain`, `PALChain`, `QAWithSourcesChain`, `VectorDBQAWithSourcesChain`, `VectorDBQA`, `SQLDatabaseChain`: All of these classes are subclasses of `Chain`.\n",
    "4. `BaseLoader`: `BaseLoader` is a subclass of `ABC`.\n",
    "5. `BaseTracer` -> `ChainRun`, `LLMRun`, `SharedTracer`, `ToolRun`, `Tracer`, `TracerException`, `TracerSession`: All of these classes are subclasses of `BaseTracer`.\n",
    "6. `OpenAIEmbeddings`, `HuggingFaceEmbeddings`, `CohereEmbeddings`, `JinaEmbeddings`, `LlamaCppEmbeddings`, `HuggingFaceHubEmbeddings`, `TensorflowHubEmbeddings`, `SagemakerEndpointEmbeddings`, `HuggingFaceInstructEmbeddings`, `SelfHostedEmbeddings`, `SelfHostedHuggingFaceEmbeddings`, `SelfHostedHuggingFaceInstructEmbeddings`, `FakeEmbeddings`, `AlephAlphaAsymmetricSemanticEmbedding`, `AlephAlphaSymmetricSemanticEmbedding`: All of these classes are subclasses of `BaseLLM`. \n",
    "\n",
    "\n",
    "-> **Question**: What classes are derived from the Chain class? \n",
    "\n",
    "**Answer**: There are multiple classes that are derived from the Chain class. Some of them are:\n",
    "- APIChain\n",
    "- AnalyzeDocumentChain\n",
    "- ChatVectorDBChain\n",
    "- CombineDocumentsChain\n",
    "- ConstitutionalChain\n",
    "- ConversationChain\n",
    "- GraphQAChain\n",
    "- HypotheticalDocumentEmbedder\n",
    "- LLMChain\n",
    "- LLMCheckerChain\n",
    "- LLMRequestsChain\n",
    "- LLMSummarizationCheckerChain\n",
    "- MapReduceChain\n",
    "- OpenAPIEndpointChain\n",
    "- PALChain\n",
    "- QAWithSourcesChain\n",
    "- RetrievalQA\n",
    "- RetrievalQAWithSourcesChain\n",
    "- SequentialChain\n",
    "- SQLDatabaseChain\n",
    "- TransformChain\n",
    "- VectorDBQA\n",
    "- VectorDBQAWithSourcesChain\n",
    "\n",
    "There might be more classes that are derived from the Chain class as it is possible to create custom classes that extend the Chain class.\n",
    "\n",
    "\n",
    "-> **Question**: What classes and functions in the ./langchain/utilities/ forlder are not covered by unit tests? \n",
    "\n",
    "**Answer**: All classes and functions in the `./langchain/utilities/` folder seem to have unit tests written for them. \n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}