209 Commits (a8f530bc4d2692d0e1c3713643522751f85df51c)

Author	SHA1	Message	Date
Bagatur	b5d3416563	experimental[patch]: Release 0.0.51 (#17484)	5 months ago
Bagatur	c0ce93236a	experimental[patch]: fix zero-shot pandas agent (#17442)	5 months ago
Theo / Taeyoon Kang	1987f905ed	core[patch]: Support .yml extension for YAML (#16783) ... - **Description:** [AS-IS] When dealing with a yaml file, the extension must be .yaml. [TO-BE] In the absence of extension length constraints in the OS, the extension of the YAML file is yaml, but control over the yml extension must still be made. It's as if it's an error because it's a .jpg extension in jpeg support. - **Issue:** - - **Dependencies:** no dependencies required for this change,	5 months ago
Erick Friis	3a2eb6e12b	infra: add print rule to ruff (#16221) ... Added noqa for existing prints. Can slowly remove / will prevent more being intro'd	5 months ago
Charlie Marsh	24c0bab57b	infra, multiple: Upgrade configuration for Ruff v0.2.0 (#16905) ... ## Summary This PR upgrades LangChain's Ruff configuration in preparation for Ruff's v0.2.0 release. (The changes are compatible with Ruff v0.1.5, which LangChain uses today.) Specifically, we're now warning when linter-only options are specified under `[tool.ruff]` instead of `[tool.ruff.lint]`. --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	5 months ago
Eugene Yurtsev	780e84ae79	community[minor]: SQLDatabase Add fetch mode `cursor`, query parameters, query by selectable, expose execution options, and documentation (#17191) ... - **Description:** Improve `SQLDatabase` adapter component to promote code re-use, see [suggestion](https://github.com/langchain-ai/langchain/pull/16246#pullrequestreview-1846590962). - **Needed by:** GH-16246 - **Addressed to:** @baskaryan, @cbornet ## Details - Add `cursor` fetch mode - Accept SQL query parameters - Accept both `str` and SQLAlchemy selectables as query expression - Expose `execution_options` - Documentation page (notebook) about `SQLDatabase` [^1] See [About SQLDatabase](https://github.com/langchain-ai/langchain/blob/c1c7b763/docs/docs/integrations/tools/sql_database.ipynb). [^1]: Apparently there hasn't been any yet? --------- Co-authored-by: Andreas Motl <andreas.motl@crate.io>	5 months ago
Erick Friis	22b6a03a28	infra: read min versions (#17135)	5 months ago
Leonid Ganeline	563f325034	experimental[patch]: fixed import in `experimental` (#17078)	5 months ago
Giulio Zani	9f0b63dba0	experimental[patch]: Fixes issue #17060 (#17062) ... As described in issue #17060, in the case in which text has only one sentence the following function fails. Checking for that and adding a return case fixed the issue. ```python def split_text(self, text: str) -> List[str]: """Split text into multiple components.""" # Splitting the essay on '.', '?', and '!' single_sentences_list = re.split(r"(?<=[.?!])\s+", text) sentences = [ {"sentence": x, "index": i} for i, x in enumerate(single_sentences_list) ] sentences = combine_sentences(sentences) embeddings = self.embeddings.embed_documents( [x["combined_sentence"] for x in sentences] ) for i, sentence in enumerate(sentences): sentence["combined_sentence_embedding"] = embeddings[i] distances, sentences = calculate_cosine_distances(sentences) start_index = 0 # Create a list to hold the grouped sentences chunks = [] breakpoint_percentile_threshold = 95 breakpoint_distance_threshold = np.percentile( distances, breakpoint_percentile_threshold ) # If you want more chunks, lower the percentile cutoff indices_above_thresh = [ i for i, x in enumerate(distances) if x > breakpoint_distance_threshold ] # The indices of those breakpoints on your list # Iterate through the breakpoints to slice the sentences for index in indices_above_thresh: # The end index is the current breakpoint end_index = index # Slice the sentence_dicts from the current start index to the end index group = sentences[start_index : end_index + 1] combined_text = " ".join([d["sentence"] for d in group]) chunks.append(combined_text) # Update the start index for the next group start_index = index + 1 # The last group, if any sentences remain if start_index < len(sentences): combined_text = " ".join([d["sentence"] for d in sentences[start_index:]]) chunks.append(combined_text) return chunks ``` Co-authored-by: Giulio Zani <salamanderxing@Giulios-MBP.homenet.telecomitalia.it>	5 months ago
Bagatur	7d03d8f586	docs: fix docstring examples (#16889)	5 months ago
Bagatur	c2d09fb151	infra: bump exp min test reqs (#16884)	5 months ago
Bagatur	65ba5c220b	experimental[patch]: Release 0.0.50 (#16883)	5 months ago
Bagatur	b0347f3e2b	docs: add csv use case (#16756)	5 months ago
Massimiliano Pronesti	1bc8d9a943	experimental[patch]: missing resolution strategy in anonymization (#16653) ... - **Description:** Presidio-based anonymizers are not working because `_remove_conflicts_and_get_text_manipulation_data` was being called without a conflict resolution strategy. This PR fixes this issue. In addition, it removes some mutable default arguments (antipattern). To reproduce the issue, just run the very first cell of this [notebook](https://python.langchain.com/docs/guides/privacy/2/) from langchain's documentation. <!-- Thank you for contributing to LangChain! Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified. Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes if applicable, - **Dependencies:** any dependencies required for this change, - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/ If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/docs/integrations` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	5 months ago
Bagatur	bccb07f93e	core[patch]: simple prompt pretty printing (#15968)	6 months ago
Bagatur	79124fd71d	experimental[patch]: Release 0.0.49 (#15823)	6 months ago
Harrison Chase	20abe24819	experimental[minor]: Add semantic chunker (#15799)	6 months ago
Bagatur	da395f3182	experimental[patch]: loosen core max version (#15763)	6 months ago
Erick Friis	95a2c92e26	experimental[patch]: minimum version bump (#15724) ... - experimental: minimum version bump - actually 0.1.5 - actually 0.1.7	6 months ago
Erick Friis	6c9b7c2cec	experimental: minimum version bump (#15722) ... experimental relies on `from langchain_core.runnables.config import run_in_executor` which was introduced in core 0.1.5. Updated pyproject dependency as well as minimum version test.	6 months ago
Bagatur	14c5c15958	experimental[patch]: Release 0.0.48 (#15483)	6 months ago
Bagatur	54b58c03db	infra: add minimum deps pre release check (#15485)	6 months ago
Bagatur	baeac236b6	langchain[patch], experimental[patch]: update utilities imports (#15438)	6 months ago
Bagatur	1678d6ca17	langchain[patch], experimental[patch], docs: update tools imports (#15433)	6 months ago
Bagatur	fa5d49f2c1	docs, experimental[patch], langchain[patch], community[patch]: update storage imports (#15429) ... ran ```bash g grep -l "langchain.vectorstores" | xargs -L 1 sed -i '' "s/langchain\.vectorstores/langchain_community.vectorstores/g" g grep -l "langchain.document_loaders" | xargs -L 1 sed -i '' "s/langchain\.document_loaders/langchain_community.document_loaders/g" g grep -l "langchain.chat_loaders" | xargs -L 1 sed -i '' "s/langchain\.chat_loaders/langchain_community.chat_loaders/g" g grep -l "langchain.document_transformers" | xargs -L 1 sed -i '' "s/langchain\.document_transformers/langchain_community.document_transformers/g" g grep -l "langchain\.graphs" | xargs -L 1 sed -i '' "s/langchain\.graphs/langchain_community.graphs/g" g grep -l "langchain\.memory\.chat_message_histories" | xargs -L 1 sed -i '' "s/langchain\.memory\.chat_message_histories/langchain_community.chat_message_histories/g" gco master libs/langchain/tests/unit_tests/*/test_imports.py gco master libs/langchain/tests/unit_tests/**/test_public_api.py ```	6 months ago
Bagatur	480626dc99	docs, community[patch], experimental[patch], langchain[patch], cli[pa… (#15412) ... …tch]: import models from community ran ```bash git grep -l 'from langchain\.chat_models' | xargs -L 1 sed -i '' "s/from\ langchain\.chat_models/from\ langchain_community.chat_models/g" git grep -l 'from langchain\.llms' | xargs -L 1 sed -i '' "s/from\ langchain\.llms/from\ langchain_community.llms/g" git grep -l 'from langchain\.embeddings' | xargs -L 1 sed -i '' "s/from\ langchain\.embeddings/from\ langchain_community.embeddings/g" git checkout master libs/langchain/tests/unit_tests/llms git checkout master libs/langchain/tests/unit_tests/chat_models git checkout master libs/langchain/tests/unit_tests/embeddings/test_imports.py make format cd libs/langchain; make format cd ../experimental; make format cd ../core; make format ```	6 months ago
Bagatur	8e0d5813c2	langchain[patch], experimental[patch]: replace langchain.schema imports (#15410) ... Import from core instead. Ran: ```bash git grep -l 'from langchain.schema\.output_parser' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.output_parser/from\ langchain_core.output_parsers/g" git grep -l 'from langchain.schema\.messages' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.messages/from\ langchain_core.messages/g" git grep -l 'from langchain.schema\.document' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.document/from\ langchain_core.documents/g" git grep -l 'from langchain.schema\.runnable' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.runnable/from\ langchain_core.runnables/g" git grep -l 'from langchain.schema\.vectorstore' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.vectorstore/from\ langchain_core.vectorstores/g" git grep -l 'from langchain.schema\.language_model' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.language_model/from\ langchain_core.language_models/g" git grep -l 'from langchain.schema\.embeddings' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.embeddings/from\ langchain_core.embeddings/g" git grep -l 'from langchain.schema\.storage' | xargs -L 1 sed -i '' "s/from\ langchain\.schema\.storage/from\ langchain_core.stores/g" git checkout master libs/langchain/tests/unit_tests/schema/ make format cd libs/experimental make format cd ../langchain make format ```	6 months ago
Nuno Campos	eb5e250188	Propagate context vars in all classes/methods ... - Any direct usage of ThreadPoolExecutor or asyncio.run_in_executor needs manual handling of context vars	6 months ago
Ran	129a929d69	infra: Fix test filesystem paths incompatible with windows (#14388) ... - **Description:** This PR fixes test failures on Windows caused by path handling differences and unescaped special characters in regex. The failing tests are: ``` FAILED tests/unit_tests/storage/test_filesystem.py::test_yield_keys - AssertionError: assert ['key1', 'subdir\\key2'] == ['key1', 'subdir/key2'] FAILED tests/unit_tests/test_imports.py::test_importable_all - ModuleNotFoundError: No module named 'langchain_community.langchain_community\\adapters' FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_on_absolute - re.error: incomplete escape \U at position 53 FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_on_parent_dir - re.error: incomplete escape \U at position 69 FAILED tests/unit_tests/tools/file_management/test_utils.py::test_get_validated_relative_path_errs_for_symlink_outside_root - re.error: incomplete escape \U at position 64 ``` - **Issue:** fixes https://github.com/langchain-ai/langchain/issues/11775 (partially) - **Dependencies:** none	6 months ago
Leonid Ganeline	b2fd41331e	docs: docstrings `langchain_community` update (#14889) ... Addded missed docstrings. Fixed inconsistency in docstrings. **Note** CC @efriis There were PR errors on `langchain_experimental/prompt_injection_identifier/hugging_face_identifier.py` But, I didn't touch this file in this PR! Can it be some cache problems? I fixed this error.	7 months ago
Oleksandr Yaremchuk	d82a3828f2	Improve prompt injection detection (#14842) ... - **Description:** This is addition to [my previous PR](https://github.com/langchain-ai/langchain/pull/13930) with improvements to flexibility allowing different models and notebook to use ONNX runtime for faster speed. Since the last PR, [our model](https://huggingface.co/laiyer/deberta-v3-base-prompt-injection) got more than 660k downloads, and with the [public benchmark](https://huggingface.co/spaces/laiyer/prompt-injection-benchmark) showed much fewer false-positives than the previous one from deepset. Additionally, on the ONNX runtime, it can be running 3x faster on the CPU, which might be handy for builders using Langchain. **Issue:** N/A - **Dependencies:** N/A - **Tag maintainer:** N/A - **Twitter handle:** `@laiyer_ai`	7 months ago
Bob Lin	a019183a01	create mypy cache dir if it doesn't exist (#14579) ... ### Description When running `make lint` multiple times, i can see the error `mkdir: .mypy_cache: File exists`. Use `mkdir -p` to solve this problem. <img width="1512" alt="Screenshot 2023-12-12 at 11 22 01 AM" src="https://github.com/langchain-ai/langchain/assets/10000925/1429383d-3283-4e22-8882-5693bc50b502">	7 months ago
Bagatur	b72b19b593	experimental[patch]: Release 0.0.47 (#14617)	7 months ago
Erick Friis	0a9d933bb2	infra: import checking bugfix (#14569)	7 months ago
Bagatur	8bdaf55e92	experimental[patch]: Release 0.0.46 (#14572)	7 months ago
Erick Friis	482e2b94fa	infra: import CI speed (#14566) ... Was taking 10 mins. Now a few seconds.	7 months ago
Erick Friis	5418d8bfd6	infra: import CI fix (#14562) ... TIL `**` globstar doesn't work in make Makefile changes fix that. `__getattr__` changes allow import of all files, but raise error when accessing anything from the module. file deletions were corresponding libs change from #14559	7 months ago
Bagatur	ed58eeb9c5	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) ... Moved the following modules to new package langchain-community in a backwards compatible fashion: ``` mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community mv langchain/langchain/adapters community/langchain_community mv langchain/langchain/callbacks community/langchain_community/callbacks mv langchain/langchain/chat_loaders community/langchain_community mv langchain/langchain/chat_models community/langchain_community mv langchain/langchain/document_loaders community/langchain_community mv langchain/langchain/docstore community/langchain_community mv langchain/langchain/document_transformers community/langchain_community mv langchain/langchain/embeddings community/langchain_community mv langchain/langchain/graphs community/langchain_community mv langchain/langchain/llms community/langchain_community mv langchain/langchain/memory/chat_message_histories community/langchain_community mv langchain/langchain/retrievers community/langchain_community mv langchain/langchain/storage community/langchain_community mv langchain/langchain/tools community/langchain_community mv langchain/langchain/utilities community/langchain_community mv langchain/langchain/vectorstores community/langchain_community mv langchain/langchain/agents/agent_toolkits community/langchain_community mv langchain/langchain/cache.py community/langchain_community ``` Moved the following to core ``` mv langchain/langchain/utils/json_schema.py core/langchain_core/utils mv langchain/langchain/utils/html.py core/langchain_core/utils mv langchain/langchain/utils/strings.py core/langchain_core/utils cat langchain/langchain/utils/env.py >> core/langchain_core/utils/env.py rm langchain/langchain/utils/env.py ``` See .scripts/community_split/script_integrations.sh for all changes	7 months ago
Anish Nag	6da0cfea0e	experimental[patch]: SmartLLMChain Output Key Customization (#14466) ... **Description** The `SmartLLMChain` was was fixed to output key "resolution". Unfortunately, this prevents the ability to use multiple `SmartLLMChain` in a `SequentialChain` because of colliding output keys. This change simply gives the option the customize the output key to allow for sequential chaining. The default behavior is the same as the current behavior. Now, it's possible to do the following: ``` from langchain.chat_models import ChatOpenAI from langchain.prompts import PromptTemplate from langchain_experimental.smart_llm import SmartLLMChain from langchain.chains import SequentialChain joke_prompt = PromptTemplate( input_variables=["content"], template="Tell me a joke about {content}.", ) review_prompt = PromptTemplate( input_variables=["scale", "joke"], template="Rate the following joke from 1 to {scale}: {joke}" ) llm = ChatOpenAI(temperature=0.9, model_name="gpt-4-32k") joke_chain = SmartLLMChain(llm=llm, prompt=joke_prompt, output_key="joke") review_chain = SmartLLMChain(llm=llm, prompt=review_prompt, output_key="review") chain = SequentialChain( chains=[joke_chain, review_chain], input_variables=["content", "scale"], output_variables=["review"], verbose=True ) response = chain.run({"content": "chickens", "scale": "10"}) print(response) ``` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago
Erick Friis	b3f226e8f8	core[patch], langchain[patch], experimental[patch]: import CI (#14414)	7 months ago
Bagatur	52052cc7b9	experimental[patch]: Release 0.0.45 (#14418)	7 months ago
Bagatur	b2280fd874	core[patch], langchain[patch]: fix required deps (#14373)	7 months ago
kavinraj A S	ab6b41937a	Fixed a typo in smart_llm prompt (#13052) ... <!-- Thank you for contributing to LangChain! Replace this entire comment with: - **Description:** a description of the change, - **Issue:** the issue # it fixes (if applicable), - **Dependencies:** any dependencies required for this change, - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	7 months ago
Lance Martin	66848871fc	Multi-modal RAG template (#14186) ... * OpenCLIP embeddings * GPT-4V --------- Co-authored-by: Erick Friis <erick@langchain.dev>	7 months ago
Bagatur	6607cc6eab	experimental[patch]: Release 0.0.44 (#14310)	7 months ago
Eun Hye Kim	f758c8adc4	Fix #11737 issue (extra_tools option of create_pandas_dataframe_agent is not working) (#13203) ... - **Description:** Fix #11737 issue (extra_tools option of create_pandas_dataframe_agent is not working), - **Issue:** #11737 , - **Dependencies:** no, - **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17 I needed this method at work, so I modified it myself and used it. There is a similar issue(#11737) and PR(#13018) of @PyroGenesis, so I combined my code at the original PR. You may be busy, but it would be great help for me if you checked. Thank you. - **Twitter handle:** @lunara_x If you need an .ipynb example about this, please tag me. I will share what I am working on after removing any work-related content. --------- Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>	7 months ago
Abdul	82102c99b3	langchain[patch]: Running SQLDatabaseChain adds prefix "SQLQuery:\n" (#14058) ... - **Issue:** https://github.com/langchain-ai/langchain/issues/12077 --------- Co-authored-by: Abdul Kader Maliyakkal <maliyakk@amazon.com>	7 months ago
James Braza	24385a00de	core[minor], langchain[patch], experimental[patch]: Added missing `py.typed` to `langchain_core` (#14143) ... See PR title. From what I can see, `poetry` will auto-include this. Please let me know if I am missing something here. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	7 months ago
Lance Martin	cbe4753e1a	Update Open CLIP embd (#14155) ... Prior default model required a large amt of RAM and often crashed Jupyter ntbk kernel.	7 months ago
Jacob Lee	3328507f11	langchain[patch], experimental[minor]: Adds OllamaFunctions wrapper (#13330) ... CC @baskaryan @hwchase17 @jmorganca Having a bit of trouble importing `langchain_experimental` from a notebook, will figure it out tomorrow ~Ah and also is blocked by #13226~ --------- Co-authored-by: Lance Martin <lance@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	7 months ago