This adds a section on usage of `CassandraCache` and
`CassandraSemanticCache` to the doc notebook about caching LLMs, as
suggested in [this
comment](https://github.com/langchain-ai/langchain/pull/9772/#issuecomment-1710544100)
on a previous merged PR.
I also spotted what looks like a mismatch between different executions
and propose a fix (line 98).
Being the result of several runs, the cell execution numbers are
scrambled somewhat, so I volunteer to refine this PR by (manually)
re-numbering the cells to restore the appearance of a single, smooth
running (for the sake of orderly execution :)
"You can use Cassandra / Astra DB for caching LLM responses, choosing from the exact-match `CassandraCache` or the (vector-similarity-based) `CassandraSemanticCache`.\n",
"\n",
"Let's see both in action in the following cells."
]
},
{
"cell_type": "markdown",
"id": "a4a6725d",
"metadata": {},
"source": [
"#### Connect to the DB\n",
"\n",
"First you need to establish a `Session` to the DB and to specify a _keyspace_ for the cache table(s). The following gets you started with an Astra DB instance (see e.g. [here](https://cassio.org/start_here/#vector-database) for more backends and connection options)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "cc53ce1b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Keyspace name? my_keyspace\n",
"\n",
"Astra DB Token (\"AstraCS:...\") ········\n",
"Full path to your Secure Connect Bundle? /path/to/secure-connect-databasename.zip\n"
]
}
],
"source": [
"import getpass\n",
"\n",
"keyspace = input(\"\\nKeyspace name? \")\n",
"ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\\nAstra DB Token (\"AstraCS:...\") ')\n",
"ASTRA_DB_SECURE_BUNDLE_PATH = input(\"Full path to your Secure Connect Bundle? \")"
"print(llm(\"Why is the Moon always showing the same side?\"))"
]
},
{
"cell_type": "markdown",
"id": "8fc4d017",
"metadata": {},
"source": [
"### Semantic cache\n",
"\n",
"This cache will do a semantic similarity search and return a hit if it finds a cached entry that is similar enough, For this, you need to provide an `Embeddings` instance of your choice."