forked from Archives/langchain
Harrison/redo docs (#130)
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>harrison/chain_pipeline
Harrison Chase
2 years ago
committed by
GitHub
parent
f23b3ceb49
commit
b1b6b27c5f
@ -0,0 +1,25 @@
|
||||
# Core Concepts
|
||||
|
||||
This section goes over the core concepts of LangChain.
|
||||
Understanding these will go a long way in helping you understand the codebase and how to construct chains.
|
||||
|
||||
## Prompts
|
||||
Prompts generically have a `format` method that takes in variables and returns a formatted string.
|
||||
The most simple implementation of this is to have a template string with some variables in it, and then format it with the incoming variables.
|
||||
More complex iterations dynamically construct the template string from few shot examples, etc.
|
||||
|
||||
## LLMs
|
||||
Wrappers around Large Language Models (in particular, the `generate` ability of large language models) are some of the core functionality of LangChain.
|
||||
These wrappers are classes that are callable: they take in an input string, and return the generated output string.
|
||||
|
||||
## Embeddings
|
||||
These classes are very similar to the LLM classes in that they are wrappers around models,
|
||||
but rather than return a string they return an embedding (list of floats). This are particularly useful when
|
||||
implementing semantic search functionality. They expose separate methods for embedding queries versus embedding documents.
|
||||
|
||||
## Vectorstores
|
||||
These are datastores that store documents. They expose a method for passing in a string and finding similar documents.
|
||||
|
||||
## Chains
|
||||
These are pipelines that combine multiple of the above ideas.
|
||||
They vary greatly in complexity and are combination of generic, highly configurable pipelines and more narrow (but usually more complex) pipelines.
|
||||
@ -0,0 +1,10 @@
|
||||
Demos
|
||||
=====
|
||||
|
||||
The examples here are all end-to-end chains of specific applications.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
demos/*
|
||||
@ -0,0 +1,10 @@
|
||||
Integrations
|
||||
============
|
||||
|
||||
The examples here all highlight a specific type of integration.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
integrations/*
|
||||
@ -0,0 +1,10 @@
|
||||
Prompts
|
||||
=======
|
||||
|
||||
The examples here all highlight how to work with prompts.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
prompts/*
|
||||
@ -0,0 +1,38 @@
|
||||
# Using Chains
|
||||
|
||||
Calling an LLM is a great first step, but it's just the beginning.
|
||||
Normally when you use an LLM in an application, you are not sending user input directly to the LLM.
|
||||
Instead, you are probably taking user input and constructing a prompt, and then sending that to the LLM.
|
||||
|
||||
For example, in the previous example, the text we passed in was hardcoded to ask for a name for a company that made colorful socks.
|
||||
In this imaginary service, what we would want to do is take only the user input describing what the company does, and then format the prompt with that information.
|
||||
|
||||
This is easy to do with LangChain!
|
||||
|
||||
First lets define the prompt:
|
||||
|
||||
```python
|
||||
from langchain.prompts import Prompt
|
||||
prompt = Prompt(
|
||||
input_variables=["product"],
|
||||
template="What is a good name for a company that makes {product}?",
|
||||
)
|
||||
```
|
||||
|
||||
We can now create a very simple chain that will take user input, format the prompt with it, and then send it to the LLM:
|
||||
|
||||
```python
|
||||
from langchain.chains import LLMChain
|
||||
chain = LLMChain(llm=llm, prompt=prompt)
|
||||
```
|
||||
|
||||
Now we can run that can only specifying the product!
|
||||
|
||||
```python
|
||||
chain.run("colorful socks")
|
||||
```
|
||||
|
||||
There we go! There's the first chain.
|
||||
|
||||
That is it for the Getting Started example.
|
||||
As a next step, we would suggest checking out the more complex chains in the [Demos section](/examples/demos.rst)
|
||||
@ -0,0 +1,37 @@
|
||||
# Setting up your environment
|
||||
|
||||
Using LangChain will usually require integrations with one or more model providers, data stores, apis, etc.
|
||||
There are two components to setting this up, installing the correct python packages and setting the right environment variables.
|
||||
|
||||
## Python packages
|
||||
The python package needed varies based on the integration. See the list of integrations for details.
|
||||
There should also be helpful error messages raised if you try to run an integration and are missing any required python packages.
|
||||
|
||||
## Environment Variables
|
||||
The environment variable needed varies based on the integration. See the list of integrations for details.
|
||||
There should also be helpful error messages raised if you try to run an integration and are missing any required environment variables.
|
||||
|
||||
You can set the environment variable in a few ways.
|
||||
If you are trying to set the environment variable `FOO` to value `bar`, here are the ways you could do so:
|
||||
- From the command line:
|
||||
```
|
||||
export FOO=bar
|
||||
```
|
||||
- From the python notebook/script:
|
||||
```python
|
||||
import os
|
||||
os.environ["FOO"] = "bar"
|
||||
```
|
||||
|
||||
For the Getting Started example, we will be using OpenAI's APIs, so we will first need to install their SDK:
|
||||
|
||||
```
|
||||
pip install openai
|
||||
```
|
||||
|
||||
We will then need to set the environment variable. Let's do this from inside the Jupyter notebook (or Python script).
|
||||
|
||||
```python
|
||||
import os
|
||||
os.environ["OPENAI_API_KEY"] = "..."
|
||||
```
|
||||
@ -0,0 +1,11 @@
|
||||
# Installation
|
||||
|
||||
LangChain is available on PyPi, so to it is easily installable with:
|
||||
|
||||
```
|
||||
pip install langchain
|
||||
```
|
||||
|
||||
For more involved installation options, see the [Installation Reference](/installation.md) section.
|
||||
|
||||
That's it! LangChain is now installed. You can now use LangChain from a python script or Jupyter notebook.
|
||||
@ -0,0 +1,25 @@
|
||||
# Calling a LLM
|
||||
|
||||
The most basic building block of LangChain is calling an LLM on some input.
|
||||
Let's walk through a simple example of how to do this.
|
||||
For this purpose, let's pretend we are building a service that generates a company name based on what the company makes.
|
||||
|
||||
In order to do this, we first need to import the LLM wrapper.
|
||||
|
||||
```python
|
||||
from langchain.llms import OpenAI
|
||||
```
|
||||
|
||||
We can then initialize the wrapper with any arguments.
|
||||
In this example, we probably want the outputs to be MORE random, so we'll initialize it with a HIGH temperature.
|
||||
|
||||
```python
|
||||
llm = OpenAI(temperature=0.9)
|
||||
```
|
||||
|
||||
We can now call it on some input!
|
||||
|
||||
```python
|
||||
text = "What would be a good company name a company that makes colorful socks?"
|
||||
llm(text)
|
||||
```
|
||||
@ -1,18 +1,82 @@
|
||||
Welcome to LangChain
|
||||
==========================
|
||||
|
||||
Large language models (LLMs) are emerging as a transformative technology, enabling
|
||||
developers to build applications that they previously could not.
|
||||
But using these LLMs in isolation is often not enough to
|
||||
create a truly powerful app - the real power comes when you are able to
|
||||
combine them with other sources of computation or knowledge.
|
||||
|
||||
This library is aimed at assisting in the development of those types of applications.
|
||||
It aims to create:
|
||||
|
||||
1. a comprehensive collection of pieces you would ever want to combine
|
||||
2. a flexible interface for combining pieces into a single comprehensive "chain"
|
||||
3. a schema for easily saving and sharing those chains
|
||||
|
||||
The documentation is structured into the following sections:
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Getting Started
|
||||
:name: getting_started
|
||||
|
||||
getting_started/installation.md
|
||||
getting_started/environment.md
|
||||
getting_started/llm.md
|
||||
getting_started/chains.md
|
||||
|
||||
Goes over a simple walk through and tutorial for getting started setting up a simple chain that generates a company name based on what the company makes.
|
||||
Covers installation, environment set up, calling LLMs, and using prompts.
|
||||
Start here if you haven't used LangChain before.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: User API
|
||||
:maxdepth: 1
|
||||
:caption: How-To Examples
|
||||
:name: examples
|
||||
|
||||
examples/demos.rst
|
||||
examples/integrations.rst
|
||||
examples/prompts.rst
|
||||
examples/model_laboratory.ipynb
|
||||
|
||||
More elaborate examples and walk-throughs of particular
|
||||
integrations and use cases. This is the place to look if you have questions
|
||||
about how to integrate certain pieces, or if you want to find examples of
|
||||
common tasks or cool demos.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Reference
|
||||
:name: reference
|
||||
|
||||
installation.md
|
||||
integrations.md
|
||||
modules/prompt
|
||||
modules/llms
|
||||
modules/embeddings
|
||||
modules/text_splitter
|
||||
modules/vectorstore
|
||||
modules/chains
|
||||
|
||||
|
||||
Full API documentation. This is the place to look if you want to
|
||||
see detailed information about the various classes, methods, and APIs.
|
||||
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Resources
|
||||
:name: resources
|
||||
|
||||
core_concepts.md
|
||||
glossary.md
|
||||
Discord <https://discord.gg/6adMQxSpJS>
|
||||
|
||||
Higher level, conceptual explanations of the LangChain components.
|
||||
This is the place to go if you want to increase your high level understanding
|
||||
of the problems LangChain is solving, and how we decided to go about do so.
|
||||
|
||||
|
||||
@ -0,0 +1,24 @@
|
||||
# Installation Options
|
||||
|
||||
LangChain is available on PyPi, so to it is easily installable with:
|
||||
|
||||
```
|
||||
pip install langchain
|
||||
```
|
||||
|
||||
That will install the bare minimum requirements of LangChain.
|
||||
A lot of the value of LangChain comes when integrating it with various model providers, datastores, etc.
|
||||
By default, the dependencies needed to do that are NOT installed.
|
||||
However, there are two other ways to install LangChain that do bring in those dependencies.
|
||||
|
||||
To install modules needed for the common LLM providers, run:
|
||||
|
||||
```
|
||||
pip install langchain[llms]
|
||||
```
|
||||
|
||||
To install all modules needed for all integrations, run:
|
||||
|
||||
```
|
||||
pip install langchain[all]
|
||||
```
|
||||
@ -0,0 +1,33 @@
|
||||
# Integration Reference
|
||||
|
||||
Besides the installation of this python package, you will also need to install packages and set environment variables depending on which chains you want to use.
|
||||
|
||||
Note: the reason these packages are not included in the dependencies by default is that as we imagine scaling this package, we do not want to force dependencies that are not needed.
|
||||
|
||||
The following use cases require specific installs and api keys:
|
||||
|
||||
- _OpenAI_:
|
||||
- Install requirements with `pip install openai`
|
||||
- Get an OpenAI api key and either set it as an environment variable (`OPENAI_API_KEY`) or pass it to the LLM constructor as `openai_api_key`.
|
||||
- _Cohere_:
|
||||
- Install requirements with `pip install cohere`
|
||||
- Get a Cohere api key and either set it as an environment variable (`COHERE_API_KEY`) or pass it to the LLM constructor as `cohere_api_key`.
|
||||
- _HuggingFace Hub_
|
||||
- Install requirements with `pip install huggingface_hub`
|
||||
- Get a HuggingFace Hub api token and either set it as an environment variable (`HUGGINGFACEHUB_API_TOKEN`) or pass it to the LLM constructor as `huggingfacehub_api_token`.
|
||||
- _SerpAPI_:
|
||||
- Install requirements with `pip install google-search-results`
|
||||
- Get a SerpAPI api key and either set it as an environment variable (`SERPAPI_API_KEY`) or pass it to the LLM constructor as `serpapi_api_key`.
|
||||
- _NatBot_:
|
||||
- Install requirements with `pip install playwright`
|
||||
- _Wikipedia_:
|
||||
- Install requirements with `pip install wikipedia`
|
||||
- _Elasticsearch_:
|
||||
- Install requirements with `pip install elasticsearch`
|
||||
- Set up Elasticsearch backend. If you want to do locally, [this](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/getting-started.html) is a good guide.
|
||||
- _FAISS_:
|
||||
- Install requirements with `pip install faiss` for Python 3.7 and `pip install faiss-cpu` for Python 3.10+.
|
||||
- _Manifest_:
|
||||
- Install requirements with `pip install manifest-ml` (Note: this is only available in Python 3.8+ currently).
|
||||
|
||||
If you are using the `NLTKTextSplitter` or the `SpacyTextSplitter`, you will also need to install the appropriate models. For example, if you want to use the `SpacyTextSplitter`, you will need to install the `en_core_web_sm` model with `python -m spacy download en_core_web_sm`. Similarly, if you want to use the `NLTKTextSplitter`, you will need to install the `punkt` model with `python -m nltk.downloader punkt`.
|
||||
@ -0,0 +1,6 @@
|
||||
:mod:`langchain.text_splitter`
|
||||
==============================
|
||||
|
||||
.. automodule:: langchain.text_splitter
|
||||
:members:
|
||||
:undoc-members:
|
||||
@ -0,0 +1,6 @@
|
||||
:mod:`langchain.vectorstores`
|
||||
=============================
|
||||
|
||||
.. automodule:: langchain.vectorstores
|
||||
:members:
|
||||
:undoc-members:
|
||||
@ -1,6 +1,8 @@
|
||||
autodoc_pydantic==1.8.0
|
||||
myst_parser
|
||||
nbsphinx==0.8.9
|
||||
sphinx==4.5.0
|
||||
sphinx-autobuild==2021.3.14
|
||||
sphinx_rtd_theme==1.0.0
|
||||
sphinx-typlog-theme==0.8.0
|
||||
autodoc_pydantic==1.8.0
|
||||
myst_parser
|
||||
sphinx-panels
|
||||
|
||||
Loading…
Reference in New Issue