Go to file
Stefano Lottini 328d0c99f2
community[minor]: Add support for metadata indexing policy in Cassandra vector store (#22548)
This PR adds a constructor `metadata_indexing` parameter to the
Cassandra vector store to allow optional fine-tuning of which fields of
the metadata are to be indexed.

This is a feature supported by the underlying CassIO library. Indexing
mode of "all", "none" or deny- and allow-list based choices are
available.

The rationale is, in some cases it's advisable to programmatically
exclude some portions of the metadata from the index if one knows in
advance they won't ever be used at search-time. this keeps the index
more lightweight and performant and avoids limitations on the length of
_indexed_ strings.

I added a integration test of the feature. I also added the possibility
of running the integration test with Cassandra on an arbitrary IP
address (e.g. Dockerized), via
`CASSANDRA_CONTACT_POINTS=10.1.1.5,10.1.1.6 poetry run pytest [...]` or
similar.

While I was at it, I added a line to the `.gitignore` since the mypy
_test_ cache was not ignored yet.

My X (Twitter) handle: @rsprrs.
2024-06-05 11:23:26 -04:00
.devcontainer infra: Sync devcontainer.json and compose file mount location (#20461) 2024-05-01 01:32:12 -04:00
.github infra: run scheduled tests on aws, google, cohere, nvidia (#22328) 2024-05-30 13:57:12 -07:00
cookbook community[patch]: update embeddings/oracleai.py (#22240) 2024-06-03 12:38:51 -07:00
docker community[minor]: Add VDMS vectorstore (#19551) 2024-03-28 03:12:11 +00:00
docs community[minor]: add user agent for web scraping loaders (#22480) 2024-06-05 15:20:34 +00:00
libs community[minor]: Add support for metadata indexing policy in Cassandra vector store (#22548) 2024-06-05 11:23:26 -04:00
templates community[minor]: Vectara Integration Update - Streaming, FCS, Chat, updates to documentation and example notebooks (#21334) 2024-06-04 12:57:28 -07:00
.gitattributes
.gitignore community[minor]: Add support for metadata indexing policy in Cassandra vector store (#22548) 2024-06-05 11:23:26 -04:00
.readthedocs.yaml
CITATION.cff
LICENSE
Makefile docs: revamp ChatOpenAI (#22253) 2024-05-29 10:20:14 -07:00
MIGRATE.md
poetry.lock community[minor]: Add CloudBlobLoader that supports loading data from cloud buckets (#21957) 2024-05-23 10:59:55 -04:00
poetry.toml
pyproject.toml partner[patch]: Upgrade to Ruff v0.4.2 (#21108) 2024-04-30 15:06:42 -04:00
README.md docs: add v0.2 links to README (#22300) 2024-05-29 16:22:01 -07:00
SECURITY.md

🦜🔗 LangChain

Build context-aware reasoning applications

Release Notes CI PyPI - License PyPI - Downloads GitHub star chart Dependency Status Open Issues Open in Dev Containers Open in GitHub Codespaces Twitter

Looking for the JS/TS library? Check out LangChain.js.

To help you ship LangChain apps to production faster, check out LangSmith. LangSmith is a unified developer platform for building, testing, and monitoring LLM applications. Fill out this form to speak with our sales team.

Quick Install

With pip:

pip install langchain

With conda:

conda install langchain -c conda-forge

🤔 What is LangChain?

LangChain is a framework for developing applications powered by large language models (LLMs).

For these applications, LangChain simplifies the entire application lifecycle:

Open-source libraries

  • langchain-core: Base abstractions and LangChain Expression Language.
  • langchain-community: Third party integrations.
    • Some integrations have been further split into partner packages that only rely on langchain-core. Examples include langchain_openai and langchain_anthropic.
  • langchain: Chains, agents, and retrieval strategies that make up an application's cognitive architecture.
  • LangGraph: A library for building robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph.

Productionization:

  • LangSmith: A developer platform that lets you debug, test, evaluate, and monitor chains built on any LLM framework and seamlessly integrates with LangChain.

Deployment:

  • LangServe: A library for deploying LangChain chains as REST APIs.

Diagram outlining the hierarchical organization of the LangChain framework, displaying the interconnected parts across multiple layers.

🧱 What can you build with LangChain?

Question answering with RAG

🧱 Extracting structured output

🤖 Chatbots

And much more! Head to the Tutorials section of the docs for more.

🚀 How does LangChain help?

The main value props of the LangChain libraries are:

  1. Components: composable building blocks, tools and integrations for working with language models. Components are modular and easy-to-use, whether you are using the rest of the LangChain framework or not
  2. Off-the-shelf chains: built-in assemblages of components for accomplishing higher-level tasks

Off-the-shelf chains make it easy to get started. Components make it easy to customize existing chains and build new ones.

LangChain Expression Language (LCEL)

LCEL is the foundation of many of LangChain's components, and is a declarative way to compose chains. LCEL was designed from day 1 to support putting prototypes in production, with no code changes, from the simplest “prompt + LLM” chain to the most complex chains.

  • Overview: LCEL and its benefits
  • Interface: The standard Runnable interface for LCEL objects
  • Primitives: More on the primitives LCEL includes
  • Cheatsheet: Quick overview of the most common usage patterns

Components

Components fall into the following modules:

📃 Model I/O

This includes prompt management, prompt optimization, a generic interface for chat models and LLMs, and common utilities for working with model outputs.

📚 Retrieval

Retrieval Augmented Generation involves loading data from a variety of sources, preparing it, then searching over (a.k.a. retrieving from) it for use in the generation step.

🤖 Agents

Agents allow an LLM autonomy over how a task is accomplished. Agents make decisions about which Actions to take, then take that Action, observe the result, and repeat until the task is complete done. LangChain provides a standard interface for agents along with the LangGraph extension for building custom agents.

📖 Documentation

Please see here for full documentation, which includes:

  • Introduction: Overview of the framework and the structure of the docs.
  • Tutorials: If you're looking to build something specific or are more of a hands-on learner, check out our tutorials. This is the best place to get started.
  • How-to guides: Answers to “How do I….?” type questions. These guides are goal-oriented and concrete; they're meant to help you complete a specific task.
  • Conceptual guide: Conceptual explanations of the key parts of the framework.
  • API Reference: Thorough documentation of every class and method.

🌐 Ecosystem

  • 🦜🛠️ LangSmith: Tracing and evaluating your language model applications and intelligent agents to help you move from prototype to production.
  • 🦜🕸️ LangGraph: Creating stateful, multi-actor applications with LLMs, built on top of (and intended to be used with) LangChain primitives.
  • 🦜🏓 LangServe: Deploying LangChain runnables and chains as REST APIs.

💁 Contributing

As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.

For detailed information on how to contribute, see here.

🌟 Contributors

langchain contributors