feat: add Momento as a standard cache and chat message history provider (#5221 )

# Add Momento as a standard cache and chat message history provider

This PR adds Momento as a standard caching provider. Implements the
interface, adds integration tests, and documentation. We also add
Momento as a chat history message provider along with integration tests,
and documentation.

[Momento](https://www.gomomento.com/) is a fully serverless cache.
Similar to S3 or DynamoDB, it requires zero configuration,
infrastructure management, and is instantly available. Users sign up for
free and get 50GB of data in/out for free every month.

## Before submitting

✅ We have added documentation, notebooks, and integration tests
demonstrating usage.

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>

2023-05-25 19:13:21 -07:00

1.5 KiB

Raw Blame History

Momento

This page covers how to use the Momento ecosystem within LangChain. It is broken into two parts: installation and setup, and then references to specific Momento wrappers.

Installation and Setup

Sign up for a free account here and get an auth token
Install the Momento Python SDK with pip install momento

Wrappers

Cache

The Cache wrapper allows for Momento to be used as a serverless, distributed, low-latency cache for LLM prompts and responses.

Standard Cache

The standard cache is the go-to use case for Momento users in any environment.

Import the cache as follows:

from langchain.cache import MomentoCache

And set up like so:

from datetime import timedelta
from momento import CacheClient, Configurations, CredentialProvider
import langchain

# Instantiate the Momento client
cache_client = CacheClient(
    Configurations.Laptop.v1(),
    CredentialProvider.from_environment_variable("MOMENTO_AUTH_TOKEN"),
    default_ttl=timedelta(days=1))

# Choose a Momento cache name of your choice
cache_name = "langchain"

# Instantiate the LLM cache
langchain.llm_cache = MomentoCache(cache_client, cache_name)

Memory

Momento can be used as a distributed memory store for LLMs.

Chat Message History Memory

See this notebook for a walkthrough of how to use Momento as a memory store for chat message history.

1.5 KiB Raw Blame History