langchain/libs/standard-tests
Bagatur 236e957abb
core,groq,openai,mistralai,robocorp,fireworks,anthropic[patch]: Update BaseModel subclass and instance checks to handle both v1 and proper namespaces (#24417)
After this PR chat models will correctly handle pydantic 2 with
bind_tools and with_structured_output.


```python
import pydantic
print(pydantic.__version__)
```
2.8.2

```python
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

class Add(BaseModel):
    x: int
    y: int

model = ChatOpenAI().bind_tools([Add])
print(model.invoke('2 + 5').tool_calls)

model = ChatOpenAI().with_structured_output(Add)
print(type(model.invoke('2 + 5')))
```

```
[{'name': 'Add', 'args': {'x': 2, 'y': 5}, 'id': 'call_PNUFa4pdfNOYXxIMHc6ps2Do', 'type': 'tool_call'}]
<class '__main__.Add'>
```


```python
from langchain_openai import ChatOpenAI
from pydantic.v1 import BaseModel, Field

class Add(BaseModel):
    x: int
    y: int

model = ChatOpenAI().bind_tools([Add])
print(model.invoke('2 + 5').tool_calls)

model = ChatOpenAI().with_structured_output(Add)
print(type(model.invoke('2 + 5')))
```

```python
[{'name': 'Add', 'args': {'x': 2, 'y': 5}, 'id': 'call_hhiHYP441cp14TtrHKx3Upg0', 'type': 'tool_call'}]
<class '__main__.Add'>
```

Addresses issues: https://github.com/langchain-ai/langchain/issues/22782

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2024-07-22 20:07:39 +00:00
..
langchain_standard_tests core,groq,openai,mistralai,robocorp,fireworks,anthropic[patch]: Update BaseModel subclass and instance checks to handle both v1 and proper namespaces (#24417) 2024-07-22 20:07:39 +00:00
scripts standard-tests: a standard unit and integration test set (#20182) 2024-04-09 12:43:00 -07:00
tests standard-tests[patch]: Add pytest assert rewrites (#24408) 2024-07-18 21:41:11 +00:00
Makefile standard-tests: a standard unit and integration test set (#20182) 2024-04-09 12:43:00 -07:00
poetry.lock standard-tests[minor]: Add standard tests for cache (#23357) 2024-06-24 15:15:03 +00:00
pyproject.toml standard-tests[minor]: Add standard tests for cache (#23357) 2024-06-24 15:15:03 +00:00
README.md standard-tests: a standard unit and integration test set (#20182) 2024-04-09 12:43:00 -07:00

langchain-standard-tests

This is an INTERNAL library for the LangChain project. It contains the base classes for a standard set of tests.

Installation

This package will NOT be regularly published to pypi. It is intended to be installed directly from github at test time.

Pip:

```bash
pip install git+https://github.com/langchain-ai/langchain.git#subdirectory=libs/standard-tests
```

Poetry:

```bash
poetry add git+https://github.com/langchain-ai/langchain.git#subdirectory=libs/standard-tests
```

Usage

To add standard tests to an integration package's e.g. ChatModel, you need to create

  1. A unit test class that inherits from ChatModelUnitTests
  2. An integration test class that inherits from ChatModelIntegrationTests

tests/unit_tests/test_standard.py:

"""Standard LangChain interface tests"""

from typing import Type

import pytest
from langchain_core.language_models import BaseChatModel
from langchain_standard_tests.unit_tests import ChatModelUnitTests

from langchain_parrot_chain import ChatParrotChain


class TestParrotChainStandard(ChatModelUnitTests):
    @pytest.fixture
    def chat_model_class(self) -> Type[BaseChatModel]:
        return ChatParrotChain

tests/integration_tests/test_standard.py:

"""Standard LangChain interface tests"""

from typing import Type

import pytest
from langchain_core.language_models import BaseChatModel
from langchain_standard_tests.integration_tests import ChatModelIntegrationTests

from langchain_parrot_chain import ChatParrotChain


class TestParrotChainStandard(ChatModelIntegrationTests):
    @pytest.fixture
    def chat_model_class(self) -> Type[BaseChatModel]:
        return ChatParrotChain

Reference

The following fixtures are configurable in the test classes. Anything not marked as required is optional.

  • chat_model_class (required): The class of the chat model to be tested
  • chat_model_params: The keyword arguments to pass to the chat model constructor
  • chat_model_has_tool_calling: Whether the chat model can call tools. By default, this is set to hasattr(chat_model_class, 'bind_tools)
  • chat_model_has_structured_output: Whether the chat model can structured output. By default, this is set to hasattr(chat_model_class, 'with_structured_output')