langchain/cookbook
pbharti0831 049bc37111
Cookbook for applying RAG locally using open source models and tools on CPU (#24284)
This cookbook guides user to implement RAG locally on CPU using
langchain tools and open source models. It enables Llama2 model to
answer queries about Intel Q1 2024 earning release using RAG pipeline.

Main libraries are langchain, llama-cpp-python and gpt4all.

If no one reviews your PR within a few days, please @-mention one of
baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

---------

Co-authored-by: Sriragavi <sriragavi.r@intel.com>
Co-authored-by: Chester Curme <chester.curme@gmail.com>
2024-07-16 15:17:10 -04:00
..
autogpt docs[patch]: typo in AutoGPT example notebook (#22631) 2024-06-06 16:05:11 -04:00
data add optimization notebook (#18155) 2024-02-26 16:09:31 -08:00
video_captioning experimental[minor]: Create Closed Captioning Chain for .mp4 videos (#14059) 2024-03-30 01:57:53 +00:00
advanced_rag_eval.ipynb docs: format (#21132) 2024-04-30 23:32:41 +00:00
agent_vectorstore.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
airbyte_github.ipynb docs: airbyte github cookbook (#18275) 2024-02-28 18:04:15 +00:00
amazon_personalize_how_to.ipynb experimental[minor]: Amazon Personalize support (#17436) 2024-02-19 10:36:37 -08:00
analyze_document.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
anthropic_structured_outputs.ipynb Update example cookbook for Anthropic tool use (#20029) 2024-04-04 14:53:18 -07:00
apache_kafka_message_handling.ipynb docs: Fix spelling typos in apache_kafka notebook (#17998) 2024-03-01 13:58:04 -08:00
azure_container_apps_dynamic_sessions_data_analyst.ipynb [docs]: split up tool docs (#22919) 2024-06-25 13:15:08 -07:00
baby_agi_with_agent.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
baby_agi.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
camel_role_playing.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
causal_program_aided_language_model.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
code-analysis-deeplake.ipynb Community: Updating Azure Retriever and Docs to be Azure AI Search instead of Azure Cognitive Search (#19925) 2024-04-08 11:12:41 -04:00
cql_agent.ipynb community[minor]: add Cassandra Database Toolkit (#20246) 2024-04-29 15:51:43 +00:00
custom_agent_with_plugin_retrieval_using_plugnplai.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
custom_agent_with_plugin_retrieval.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
custom_agent_with_tool_retrieval.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
custom_multi_action_agent.ipynb docs, templates: update schema imports to core (#17885) 2024-02-22 15:58:44 -08:00
databricks_sql_db.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
deeplake_semantic_search_over_chat.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
docugami_xml_kg_rag.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
elasticsearch_db_qa.ipynb docs: use standard openai params (#20160) 2024-04-08 10:56:53 -05:00
extraction_openai_tools.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
fake_llm.ipynb docs: Update function "run" to "invoke" in fake_llm.ipynb (#19570) 2024-03-26 09:54:31 -04:00
fireworks_rag.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
forward_looking_retrieval_augmented_generation.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
Gemma_LangChain.ipynb docs: cookbook on gemma integrations (#18213) 2024-03-01 11:50:55 -08:00
generative_agents_interactive_simulacra_of_human_behavior.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
gymnasium_agent_simulation.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
hugginggpt.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
human_approval.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
human_input_chat_model.ipynb docs, community[patch], experimental[patch], langchain[patch], cli[pa… (#15412) 2024-01-02 15:32:16 -05:00
human_input_llm.ipynb docs, community[patch], experimental[patch], langchain[patch], cli[pa… (#15412) 2024-01-02 15:32:16 -05:00
hypothetical_document_embeddings.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
img-to_img-search_CLIP_ChromaDB.ipynb partners: add similarity search by image functionality to langchain_chroma partner package (#22982) 2024-07-15 18:48:22 +00:00
langgraph_agentic_rag.ipynb docs: Agentic and Self-RAG w/ LangGraph (#16910) 2024-03-29 11:11:35 -07:00
langgraph_crag.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
langgraph_self_rag.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
learned_prompt_optimization.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
LLaMA2_sql_chat.ipynb docs: reorg and visual refresh (#19765) 2024-04-04 00:58:36 -07:00
llm_bash.ipynb docs:Update function "run" to "invoke" in llm_bash.ipynb (#18663) 2024-03-08 09:35:36 -08:00
llm_checker.ipynb docs: Update function "run" to "invoke" (#18499) 2024-03-04 10:42:53 -08:00
llm_math.ipynb docs: Update function "run" to "invoke" in llm_math.ipynb (#18505) 2024-03-04 10:42:36 -08:00
llm_summarization_checker.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
llm_symbolic_math.ipynb docs: Update function "run" to "invoke" in llm_symbolic_math.ipynb (#19713) 2024-03-28 13:08:22 -07:00
meta_prompt.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
mongodb-langchain-cache-memory.ipynb cookbook: MongoDB Cookbook for Chat history and semantic cache (#19998) 2024-04-08 20:21:24 +00:00
multi_modal_output_agent.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
multi_modal_QA.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
multi_modal_RAG_chroma.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
Multi_modal_RAG_google.ipynb docs: standardize vertexai params (#20167) 2024-05-01 11:42:18 -04:00
multi_modal_RAG_vdms.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
Multi_modal_RAG.ipynb docs: format (#21132) 2024-04-30 23:32:41 +00:00
multi_player_dnd.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
multiagent_authoritarian.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
multiagent_bidding.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
myscale_vector_sql.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
nomic_embedding_rag.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
nomic_multimodal_rag.ipynb embeddings: nomic embed vision (#22482) 2024-06-05 09:47:17 -07:00
openai_functions_retrieval_qa.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
openai_v1_cookbook.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
optimization.ipynb add optimization notebook (#18155) 2024-02-26 16:09:31 -08:00
oracleai_demo.ipynb updated oracleai_demo.ipynb (#22635) 2024-06-06 20:29:24 +00:00
petting_zoo.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
plan_and_execute_agent.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
press_releases.ipynb docs: use standard openai params (#20160) 2024-04-08 10:56:53 -05:00
program_aided_language_model.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
qa_citations.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
qianfan_baidu_elasticesearch_RAG.ipynb text-splitters[minor], langchain[minor], community[patch], templates, docs: langchain-text-splitters 0.0.1 (#18346) 2024-02-29 18:33:21 -08:00
rag_fusion.ipynb pinecone[patch], docs: PineconeVectorStore, release 0.0.3 (#17896) 2024-02-22 08:24:08 -08:00
rag_semantic_chunking_azureaidocintelligence.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
rag_upstage_layout_analysis_groundedness_check.ipynb partner-upstage[patch]: embeddings empty list bug (#22057) 2024-05-23 11:44:30 -04:00
rag_with_quantized_embeddings.ipynb infra: rm unused # noqa violations (#22049) 2024-05-22 15:21:08 -07:00
rag-locally-on-intel-cpu.ipynb Cookbook for applying RAG locally using open source models and tools on CPU (#24284) 2024-07-16 15:17:10 -04:00
RAPTOR.ipynb Update RAPTOR.ipynb (#20586) 2024-04-18 09:47:17 -04:00
README.md Cookbook for applying RAG locally using open source models and tools on CPU (#24284) 2024-07-16 15:17:10 -04:00
retrieval_in_sql.ipynb docs: use standard openai params (#20160) 2024-04-08 10:56:53 -05:00
rewrite.ipynb cookbook[patch]: add strip of quotes (#19452) 2024-03-22 19:10:39 +00:00
sales_agent_with_context.ipynb docs: update cookbook example for SalesGPT - include Stripe Payment Link Generation (#19622) 2024-03-27 20:16:21 +00:00
selecting_llms_based_on_context_length.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
self_query_hotel_search.ipynb docs: format (#21132) 2024-04-30 23:32:41 +00:00
self-discover.ipynb add self discover notebook (#17387) 2024-02-12 09:38:43 -08:00
Semi_structured_and_multi_modal_RAG.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
Semi_structured_multi_modal_RAG_LLaMA2.ipynb patch: deprecate (a)get_relevant_documents (#20477) 2024-04-22 11:14:53 -04:00
Semi_Structured_RAG.ipynb docs: Fix link for partition_pdf in Semi_Structured_RAG.ipynb cookbook (#20763) 2024-04-22 23:22:55 +00:00
sharedmemory_for_tools.ipynb docs: Get rid of ZeroShotAgent and use create_react_agent instead (#20157) 2024-04-09 12:00:29 -05:00
smart_llm.ipynb docs:Update function "run" to "invoke" in smart_llm.ipynb (#18568) 2024-03-05 12:52:48 -08:00
sql_db_qa.mdx docs: huggingface package (#21645) 2024-05-14 03:17:40 +00:00
stepback-qa.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
together_ai.ipynb docs: update stale links in Together AI documentation (#19011) 2024-03-15 16:38:04 -07:00
tool_call_messages.ipynb docs: update tool calling cookbook (#20290) 2024-04-10 15:06:33 -04:00
tree_of_thought.ipynb docs: langchain-openai (#15513) 2024-01-06 15:54:48 -08:00
twitter-the-algorithm-analysis-deeplake.ipynb docs: use standard openai params (#20160) 2024-04-08 10:56:53 -05:00
two_agent_debate_tools.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
two_player_dnd.ipynb patch: remove usage of llm, chat model __call__ (#20788) 2024-04-24 19:39:23 -04:00
wikibase_agent.ipynb docs: use standard openai params (#20160) 2024-04-08 10:56:53 -05:00

LangChain cookbook

Example code for building applications with LangChain, with an emphasis on more applied and end-to-end examples than contained in the main documentation.

Notebook Description
LLaMA2_sql_chat.ipynb Build a chat application that interacts with a SQL database using an open source llm (llama2), specifically demonstrated on an SQLite database containing rosters.
Semi_Structured_RAG.ipynb Perform retrieval-augmented generation (rag) on documents with semi-structured data, including text and tables, using unstructured for parsing, multi-vector retriever for storing, and lcel for implementing chains.
Semi_structured_and_multi_moda... Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using unstructured for parsing, multi-vector retriever for storage and retrieval, and lcel for implementing chains.
Semi_structured_multi_modal_RA... Perform retrieval-augmented generation (rag) on documents with semi-structured data and images, using various tools and methods such as unstructured for parsing, multi-vector retriever for storing, lcel for implementing chains, and open source language models like llama2, llava, and gpt4all.
amazon_personalize_how_to.ipynb Retrieving personalized recommendations from Amazon Personalize and use custom agents to build generative AI apps
analyze_document.ipynb Analyze a single long document.
autogpt/autogpt.ipynb Implement autogpt, a language model, with langchain primitives such as llms, prompttemplates, vectorstores, embeddings, and tools.
autogpt/marathon_times.ipynb Implement autogpt for finding winning marathon times.
baby_agi.ipynb Implement babyagi, an ai agent that can generate and execute tasks based on a given objective, with the flexibility to swap out specific vectorstores/model providers.
baby_agi_with_agent.ipynb Swap out the execution chain in the babyagi notebook with an agent that has access to tools, aiming to obtain more reliable information.
camel_role_playing.ipynb Implement the camel framework for creating autonomous cooperative agents in large-scale language models, using role-playing and inception prompting to guide chat agents towards task completion.
causal_program_aided_language_... Implement the causal program-aided language (cpal) chain, which improves upon the program-aided language (pal) by incorporating causal structure to prevent hallucination in language models, particularly when dealing with complex narratives and math problems with nested dependencies.
code-analysis-deeplake.ipynb Analyze its own code base with the help of gpt and activeloop's deep lake.
custom_agent_with_plugin_retri... Build a custom agent that can interact with ai plugins by retrieving tools and creating natural language wrappers around openapi endpoints.
custom_agent_with_plugin_retri... Build a custom agent with plugin retrieval functionality, utilizing ai plugins from the plugnplai directory.
databricks_sql_db.ipynb Connect to databricks runtimes and databricks sql.
deeplake_semantic_search_over_... Perform semantic search and question-answering over a group chat using activeloop's deep lake with gpt4.
elasticsearch_db_qa.ipynb Interact with elasticsearch analytics databases in natural language and build search queries via the elasticsearch dsl API.
extraction_openai_tools.ipynb Structured Data Extraction with OpenAI Tools
forward_looking_retrieval_augm... Implement the forward-looking active retrieval augmented generation (flare) method, which generates answers to questions, identifies uncertain tokens, generates hypothetical questions based on these tokens, and retrieves relevant documents to continue generating the answer.
generative_agents_interactive_... Implement a generative agent that simulates human behavior, based on a research paper, using a time-weighted memory object backed by a langchain retriever.
gymnasium_agent_simulation.ipynb Create a simple agent-environment interaction loop in simulated environments like text-based games with gymnasium.
hugginggpt.ipynb Implement hugginggpt, a system that connects language models like chatgpt with the machine learning community via hugging face.
hypothetical_document_embeddin... Improve document indexing with hypothetical document embeddings (hyde), an embedding technique that generates and embeds hypothetical answers to queries.
learned_prompt_optimization.ipynb Automatically enhance language model prompts by injecting specific terms using reinforcement learning, which can be used to personalize responses based on user preferences.
llm_bash.ipynb Perform simple filesystem commands using language learning models (llms) and a bash process.
llm_checker.ipynb Create a self-checking chain using the llmcheckerchain function.
llm_math.ipynb Solve complex word math problems using language models and python repls.
llm_summarization_checker.ipynb Check the accuracy of text summaries, with the option to run the checker multiple times for improved results.
llm_symbolic_math.ipynb Solve algebraic equations with the help of llms (language learning models) and sympy, a python library for symbolic mathematics.
meta_prompt.ipynb Implement the meta-prompt concept, which is a method for building self-improving agents that reflect on their own performance and modify their instructions accordingly.
multi_modal_output_agent.ipynb Generate multi-modal outputs, specifically images and text.
multi_player_dnd.ipynb Simulate multi-player dungeons & dragons games, with a custom function determining the speaking schedule of the agents.
multiagent_authoritarian.ipynb Implement a multi-agent simulation where a privileged agent controls the conversation, including deciding who speaks and when the conversation ends, in the context of a simulated news network.
multiagent_bidding.ipynb Implement a multi-agent simulation where agents bid to speak, with the highest bidder speaking next, demonstrated through a fictitious presidential debate example.
myscale_vector_sql.ipynb Access and interact with the myscale integrated vector database, which can enhance the performance of language model (llm) applications.
openai_functions_retrieval_qa.... Structure response output in a question-answering system by incorporating openai functions into a retrieval pipeline.
openai_v1_cookbook.ipynb Explore new functionality released alongside the V1 release of the OpenAI Python library.
petting_zoo.ipynb Create multi-agent simulations with simulated environments using the petting zoo library.
plan_and_execute_agent.ipynb Create plan-and-execute agents that accomplish objectives by planning tasks with a language model (llm) and executing them with a separate agent.
press_releases.ipynb Retrieve and query company press release data powered by Kay.ai.
program_aided_language_model.i... Implement program-aided language models as described in the provided research paper.
qa_citations.ipynb Different ways to get a model to cite its sources.
rag_upstage_layout_analysis_groundedness_check.ipynb End-to-end RAG example using Upstage Layout Analysis and Groundedness Check.
retrieval_in_sql.ipynb Perform retrieval-augmented-generation (rag) on a PostgreSQL database using pgvector.
sales_agent_with_context.ipynb Implement a context-aware ai sales agent, salesgpt, that can have natural sales conversations, interact with other systems, and use a product knowledge base to discuss a company's offerings.
self_query_hotel_search.ipynb Build a hotel room search feature with self-querying retrieval, using a specific hotel recommendation dataset.
smart_llm.ipynb Implement a smartllmchain, a self-critique chain that generates multiple output proposals, critiques them to find the best one, and then improves upon it to produce a final output.
tree_of_thought.ipynb Query a large language model using the tree of thought technique.
twitter-the-algorithm-analysis... Analyze the source code of the Twitter algorithm with the help of gpt4 and activeloop's deep lake.
two_agent_debate_tools.ipynb Simulate multi-agent dialogues where the agents can utilize various tools.
two_player_dnd.ipynb Simulate a two-player dungeons & dragons game, where a dialogue simulator class is used to coordinate the dialogue between the protagonist and the dungeon master.
wikibase_agent.ipynb Create a simple wikibase agent that utilizes sparql generation, with testing done on http://wikidata.org.
oracleai_demo.ipynb This guide outlines how to utilize Oracle AI Vector Search alongside Langchain for an end-to-end RAG pipeline, providing step-by-step examples. The process includes loading documents from various sources using OracleDocLoader, summarizing them either within or outside the database with OracleSummary, and generating embeddings similarly through OracleEmbeddings. It also covers chunking documents according to specific requirements using Advanced Oracle Capabilities from OracleTextSplitter, and finally, storing and indexing these documents in a Vector Store for querying with OracleVS.
rag-locally-on-intel-cpu.ipynb Perform Retrieval-Augmented-Generation (RAG) on locally downloaded open-source models using langchain and open source tools and execute it on Intel Xeon CPU. We showed an example of how to apply RAG on Llama 2 model and enable it to answer the queries related to Intel Q1 2024 earnings release.