langchain/templates/cassandra-synonym-caching

LLM-cache LangServe chain template

A simple chain template showcasing usage of LLM Caching backed by Astra DB / Apache Cassandra®.

Setup:

You need:

• an Astra Vector Database (free tier is fine!). You need a Database Administrator token, in particular the string starting with AstraCS:...;
• likewise, get your Database ID ready, you will have to enter it below;
• an OpenAI API Key. (More info here, note that out-of-the-box this demo supports OpenAI unless you tinker with the code.)

Note: you can alternatively use a regular Cassandra cluster: to do so, make sure you provide the USE_CASSANDRA_CLUSTER entry as shown in .env.template and the subsequent environment variables to specify how to connect to it.

You need to provide the connection parameters and secrets through environment variables. Please refer to .env.template for what variables are required.

Reference

Stand-alone LangServe template repo: here.