langchain/templates/rag-chroma-private

Private RAG

This template performs privae RAG (no reliance on external APIs) using:

• Ollama for the LLM
• GPT4All for embeddings

LLM

Follow instructions here to download Ollama.

The instructions also show how to download your LLM of interest with Ollama:

• This template uses llama2:13b-chat
• But you can pick from many here

Set up local embeddings

This will use GPT4All embeddings.

Chroma

Chroma is an open-source vector database.

This template will create and add documents to the vector database in chain.py.

By default, this will load a popular blog post on agents.

However, you can choose from a large number of document loaders here.

Adding the template

Create your LangServe app:

langchain serve new my-app
cd my-app

Add template:

langchain serve add rag-chroma-private

Start server:

langchain start

See Jupyter notebook rag_chroma_private for various way to connect to the template.