langchain

Commit Graph

Author	SHA1	Message	Date
Bagatur	78a1f4b264	bump 338, exp 42 (#13564 )	10 months ago
Martin Krasser	79ed66f870	EXPERIMENTAL Generic LLM wrapper to support chat model interface with configurable chat prompt format (#8295 ) ## Update 2023-09-08 This PR now supports further models in addition to Lllama-2 chat models. See [this comment](#issuecomment-1668988543) for further details. The title of this PR has been updated accordingly. ## Original PR description This PR adds a generic `Llama2Chat` model, a wrapper for LLMs able to serve Llama-2 chat models (like `LlamaCPP`, `HuggingFaceTextGenInference`, ...). It implements `BaseChatModel`, converts a list of chat messages into the [required Llama-2 chat prompt format](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) and forwards the formatted prompt as `str` to the wrapped `LLM`. Usage example: ```python # uses a locally hosted Llama2 chat model llm = HuggingFaceTextGenInference( inference_server_url="http://127.0.0.1:8080/", max_new_tokens=512, top_k=50, temperature=0.1, repetition_penalty=1.03, ) # Wrap llm to support Llama2 chat prompt format. # Resulting model is a chat model model = Llama2Chat(llm=llm) messages = [ SystemMessage(content="You are a helpful assistant."), MessagesPlaceholder(variable_name="chat_history"), HumanMessagePromptTemplate.from_template("{text}"), ] prompt = ChatPromptTemplate.from_messages(messages) memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) chain = LLMChain(llm=model, prompt=prompt, memory=memory) # use chat model in a conversation # ... ``` Also part of this PR are tests and a demo notebook. - Tag maintainer: @hwchase17 - Twitter handle: `@mrt1nz` --------- Co-authored-by: Erick Friis <erick@langchain.dev>	10 months ago
Bagatur	a9b2c943e6	bump 336, exp 44 (#13420 )	10 months ago
Predrag Gruevski	d63d4994c0	Bump all libraries to the latest `ruff` version. (#13350 ) This version of `ruff` is the one we'll be using to lint the docs and cookbooks (#12677), so I'm making it used everywhere else too.	10 months ago
Bagatur	1c67db4c18	Move OAI assistants to langchain and add callbacks (#13236 )	10 months ago
Bagatur	24386e0860	bump 334, exp 40 (#13211 )	10 months ago
Lance Martin	d2e50b3108	Add Chroma multimodal cookbook (#12952 ) Pending: * https://github.com/chroma-core/chroma/pull/1294 * https://github.com/chroma-core/chroma/pull/1293 --------- Co-authored-by: Erick Friis <erick@langchain.dev> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Bagatur	8b2a82b5ce	Bagatur/docs smith context (#13139 )	11 months ago
Bagatur	1f85ec34d5	bump 331rc3 exp 39 (#13086 )	11 months ago
Bagatur	55aeff6777	oai assistant multiple actions (#13068 )	11 months ago
Erick Friis	506f81563f	Update Deps in Experimental (#13029 )	11 months ago
Bagatur	cf481c9418	bump exp 38 (#13016 )	11 months ago
Bagatur	57e19989f6	Bagatur/oai assistant (#13010 )	11 months ago
Bagatur	eee5181b7a	bump 328, exp 37 (#12722 )	11 months ago
Noam Gat	14e8c74736	LM Format Enforcer Integration + Sample Notebook (#12625 ) ## Description This PR adds support for [lm-format-enforcer](https://github.com/noamgat/lm-format-enforcer) to LangChain. ![image](https://raw.githubusercontent.com/noamgat/lm-format-enforcer/main/docs/Intro.webp) The library is similar to jsonformer / RELLM which are supported in Langchain, but has several advantages such as - Batching and Beam search support - More complete JSON Schema support - LLM has control over whitespace, improving quality - Better runtime performance due to only calling the LLM's generate() function once per generate() call. The integration is loosely based on the jsonformer integration in terms of project structure. ## Dependencies No compile-time dependency was added, but if `lm-format-enforcer` is not installed, a runtime error will occur if it is trying to be used. ## Tests Due to the integration modifying the internal parameters of the underlying huggingface transformer LLM, it is not possible to test without building a real LM, which requires internet access. So, similar to the jsonformer and RELLM integrations, the testing is via the notebook. ## Twitter Handle [@noamgat](https://twitter.com/noamgat) Looking forward to hearing feedback! --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Predrag Gruevski	f94e24dfd7	Install and use `ruff format` instead of black for code formatting. (#12585 ) Best to review one commit at a time, since two of the commits are 100% autogenerated changes from running `ruff format`: - Install and use `ruff format` instead of black for code formatting. - Output of `ruff format .` in the `langchain` package. - Use `ruff format` in experimental package. - Format changes in experimental package by `ruff format`. - Manual formatting fixes to make `ruff .` pass.	11 months ago
Harrison Chase	eb903e211c	bump to 36 (#12487 )	11 months ago
Harrison Chase	0ca539eb85	Clean up deprecated agents and update __init__ in experimental (#12231 ) Update init paths in experimental	11 months ago
Shorthills AI	25c98dbba9	Fixed some grammatical and Exception types issues (#12015 ) Fixed some grammatical issues and Exception types. @baskaryan , @eyurtsev --------- Co-authored-by: Sanskar Tanwar <142409040+SanskarTanwarShorthillsAI@users.noreply.github.com> Co-authored-by: UpneetShorthillsAI <144228282+UpneetShorthillsAI@users.noreply.github.com> Co-authored-by: HarshGuptaShorthillsAI <144897987+HarshGuptaShorthillsAI@users.noreply.github.com> Co-authored-by: AdityaKalraShorthillsAI <143726711+AdityaKalraShorthillsAI@users.noreply.github.com> Co-authored-by: SakshiShorthillsAI <144228183+SakshiShorthillsAI@users.noreply.github.com>	11 months ago
Bagatur	c6a733802b	bump 324 and 35 (#12352 )	11 months ago
Nikhil Jha	dff24285ea	Comprehend Moderation 0.2 (#11730 ) This PR replaces the previous `Intent` check with the new `Prompt Safety` check. The logic and steps to enable chain moderation via the Amazon Comprehend service, allowing you to detect and redact PII, Toxic, and Prompt Safety information in the LLM prompt or answer remains unchanged. This implementation updates the code and configuration types with respect to `Prompt Safety`. ### Usage sample ```python from langchain_experimental.comprehend_moderation import (BaseModerationConfig, ModerationPromptSafetyConfig, ModerationPiiConfig, ModerationToxicityConfig ) pii_config = ModerationPiiConfig( labels=["SSN"], redact=True, mask_character="X" ) toxicity_config = ModerationToxicityConfig( threshold=0.5 ) prompt_safety_config = ModerationPromptSafetyConfig( threshold=0.5 ) moderation_config = BaseModerationConfig( filters=[pii_config, toxicity_config, prompt_safety_config] ) comp_moderation_with_config = AmazonComprehendModerationChain( moderation_config=moderation_config, #specify the configuration client=comprehend_client, #optionally pass the Boto3 Client verbose=True ) template = """Question: {question} Answer:""" prompt = PromptTemplate(template=template, input_variables=["question"]) responses = [ "Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like 323-22-9980. John Doe's phone number is (999)253-9876.", "Final Answer: This is a really shitty way of constructing a birdhouse. This is fucking insane to think that any birds would actually create their motherfucking nests here." ] llm = FakeListLLM(responses=responses) llm_chain = LLMChain(prompt=prompt, llm=llm) chain = ( prompt \| comp_moderation_with_config \| {llm_chain.input_keys[0]: lambda x: x['output'] } \| llm_chain \| { "input": lambda x: x['text'] } \| comp_moderation_with_config ) try: response = chain.invoke({"question": "A sample SSN number looks like this 123-456-7890. Can you give me some more samples?"}) except Exception as e: print(str(e)) else: print(response['output']) ``` ### Output ```python > Entering new AmazonComprehendModerationChain chain... Running AmazonComprehendModerationChain... Running pii Validation... Running toxicity Validation... Running prompt safety Validation... > Finished chain. > Entering new AmazonComprehendModerationChain chain... Running AmazonComprehendModerationChain... Running pii Validation... Running toxicity Validation... Running prompt safety Validation... > Finished chain. Final Answer: A credit card number looks like 1289-2321-1123-2387. A fake SSN number looks like XXXXXXXXXXXX John Doe's phone number is (999)253-9876. ``` --------- Co-authored-by: Jha <nikjha@amazon.com> Co-authored-by: Anjan Biswas <anjanavb@amazon.com> Co-authored-by: Anjan Biswas <84933469+anjanvb@users.noreply.github.com>	11 months ago
Bagatur	286a29a49e	bump 322 and 34 (#12228 )	11 months ago
Erick Friis	95ae40ff90	Fix Anthropic Functions ainvoke (#12215 ) Removes custom `NotImplementedError` in experimental anthropic functions, allowing it to fallback on default `ainvoke` implementation.	11 months ago
Bagatur	963ff93476	bump 321 (#12161 )	11 months ago
Harrison Chase	ee69116761	move csv agent to langchain experimental (#12113 )	11 months ago
Harrison Chase	03bf6ef473	add missing init files (#12114 )	11 months ago
Bagatur	85302a9ec1	Add CI check that integration tests compile (#12090 )	11 months ago
Bagatur	35c7c1f050	bump 317 (#11986 )	11 months ago
Predrag Gruevski	392df7b2e3	Type hints on varargs and kwargs that take anything should be `Any`. (#11950 ) Type hinting `args` as `List[Any]` means that each positional argument should be a list. Type hinting `*kwargs` as `Dict[str, Any]` means that each keyword argument should be a dict of strings. This is almost never what we actually wanted, and doesn't seem to be what we want in any of the cases I'm replacing here.	11 months ago
Predrag Gruevski	dcd0392423	Upgrade to newer black (23.10) and ruff (first 0.1.x!) versions. (#11944 ) Minor lint dependency version upgrade to pick up latest functionality. Ruff's new v0.1 version comes with lots of nice features, like fix-safety guarantees and a preview mode for not-yet-stable features: https://astral.sh/blog/ruff-v0.1.0	11 months ago
maks-operlejn-ds	42dcc502c7	Anonymizer small fixes (#11915 )	11 months ago
Bagatur	ba0d729961	bump 316 (#11928 )	11 months ago
Predrag Gruevski	7c0f1bf23f	Upgrade experimental package dependencies and use Poetry 1.6.1. (#11339 ) Part of upgrading our CI to use Poetry 1.6.1.	11 months ago
Bagatur	25b1d65305	bump 315 (#11850 )	11 months ago
Eugene Yurtsev	0d37b4c27d	Add python,pandas,xorbits,spark agents to experimental (#11774 ) See for contex https://github.com/langchain-ai/langchain/discussions/11680	11 months ago
Erick Friis	1861cc7100	General anthropic functions, steps towards experimental integration tests (#11727 ) To match change in js here https://github.com/langchain-ai/langchainjs/pull/2892 Some integration tests need a bit more work in experimental: ![Screenshot 2023-10-12 at 12 02 49 PM](https://github.com/langchain-ai/langchain/assets/9557659/262d7d22-c405-40e9-afef-669e8d585307) Pretty sure the sqldatabase ones are an actual regression or change in interface because it's returning a placeholder. --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Bagatur	9c0584be74	bump 313 (#11718 )	11 months ago
Suresh Kumar Ponnusamy	70f7558db2	langchain-experimental: Add allow_list support in experimental/data_anonymizer (#11597 ) - Description: Add allow_list support in langchain experimental data-anonymizer package - Issue: no - Dependencies: no - Tag maintainer: @hwchase17 - Twitter handle:	11 months ago
Kwanghoon Choi	fbb82608cd	Fixed a bug in reporting Python code validation (#11522 ) - Description: fixed a bug in pal-chain when it reports Python code validation errors. When node.func does not have any ids, the original code tried to print node.func.id in raising ValueError. - Issue: n/a, - Dependencies: no dependencies, - Tag maintainer: @hazzel-cn, @eyurtsev - Twitter handle: @lazyswamp --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	11 months ago
Bagatur	7232e082de	bump 312 (#11621 )	11 months ago
Eugene Yurtsev	c9bce5bbfb	Add version to langchain_experimental (#11613 ) Add version to langchain experimental	12 months ago
maks-operlejn-ds	f64522fbaf	Reset deanonymizer mapping (#11559 ) @hwchase17 @baskaryan	12 months ago
maks-operlejn-ds	b14b65d62a	Support all presidio entities (#11558 ) https://microsoft.github.io/presidio/supported_entities/ @baskaryan @hwchase17	12 months ago
maks-operlejn-ds	4d62def9ff	Better deanonymizer matching strategy (#11557 ) @baskaryan, @hwchase17	12 months ago
Bagatur	53887242a1	bump 310 (#11486 )	12 months ago
Qihui Xie	57ade13b2b	fix llm_inputs duplication problem in intermediate_steps in SQLDatabaseChain (#10279 ) Use `.copy()` to fix the bug that the first `llm_inputs` element is overwritten by the second `llm_inputs` element in `intermediate_steps`. *Problem description:* In [line 127]( `c732d8fffd/libs/experimental/langchain_experimental/sql/base.py (L127C17-L127C17)`), the `llm_inputs` of the sql generation step is appended as the first element of `intermediate_steps`: ``` intermediate_steps.append(llm_inputs) # input: sql generation ``` However, `llm_inputs` is a mutable dict, it is updated in [line 179](https://github.com/langchain-ai/langchain/blob/master/libs/experimental/langchain_experimental/sql/base.py#L179) for the final answer step: ``` llm_inputs["input"] = input_text ``` Then, the updated `llm_inputs` is appended as another element of `intermediate_steps` in [line 180](`c732d8fffd/libs/experimental/langchain_experimental/sql/base.py (L180)`): ``` intermediate_steps.append(llm_inputs) # input: final answer ``` As a result, the final `intermediate_steps` returned in [line 189](`c732d8fffd/libs/experimental/langchain_experimental/sql/base.py (L189C43-L189C43)`) actually contains two same `llm_inputs` elements, i.e., the `llm_inputs` for the sql generation step overwritten by the one for final answer step by mistake. Users are not able to get the actual `llm_inputs` for the sql generation step from `intermediate_steps` Simply calling `.copy()` when appending `llm_inputs` to `intermediate_steps` can solve this problem.	12 months ago
Bagatur	a3a2ce623e	Revise vowpal_wabbit notebook	12 months ago
Bagatur	8fafa1af91	merge	12 months ago
olgavrou	3b07c0cf3d	RL Chain with VowpalWabbit (#10242 ) - Description: This PR adds a new chain `rl_chain.PickBest` for learned prompt variable injection, detailed description and usage can be found in the example notebook added. It essentially adds a [VowpalWabbit](https://github.com/VowpalWabbit/vowpal_wabbit) layer before the llm call in order to learn or personalize prompt variable selections. Most of the code is to make the API simple and provide lots of defaults and data wrangling that is needed to use Vowpal Wabbit, so that the user of the chain doesn't have to worry about it. - Dependencies: [vowpal-wabbit-next](https://pypi.org/project/vowpal-wabbit-next/), - sentence-transformers (already a dep) - numpy (already a dep) - tagging @ataymano who contributed to this chain - Tag maintainer: @baskaryan - Twitter handle: @olgavrou Added example notebook and unit tests	12 months ago
maks-operlejn-ds	2aae1102b0	Instance anonymization (#10501 ) ### Description Add instance anonymization - if `John Doe` will appear twice in the text, it will be treated as the same entity. The difference between `PresidioAnonymizer` and `PresidioReversibleAnonymizer` is that only the second one has a built-in memory, so it will remember anonymization mapping for multiple texts: ``` >>> anonymizer = PresidioAnonymizer() >>> anonymizer.anonymize("My name is John Doe. Hi John Doe!") 'My name is Noah Rhodes. Hi Noah Rhodes!' >>> anonymizer.anonymize("My name is John Doe. Hi John Doe!") 'My name is Brett Russell. Hi Brett Russell!' ``` ``` >>> anonymizer = PresidioReversibleAnonymizer() >>> anonymizer.anonymize("My name is John Doe. Hi John Doe!") 'My name is Noah Rhodes. Hi Noah Rhodes!' >>> anonymizer.anonymize("My name is John Doe. Hi John Doe!") 'My name is Noah Rhodes. Hi Noah Rhodes!' ``` ### Twitter handle @deepsense_ai / @MaksOpp ### Tag maintainer @baskaryan @hwchase17 @hinthornw --------- Co-authored-by: Bagatur <baskaryan@gmail.com>	12 months ago

1 2 3

148 Commits (420a17542db56281505145ff1e3036274b09da61)