mirror of
https://github.com/hwchase17/langchain
synced 2024-11-08 07:10:35 +00:00
181dfef118
1. Adds `.get_ls_params` to BaseChatModel which returns
```python
class LangSmithParams(TypedDict, total=False):
ls_provider: str
ls_model_name: str
ls_model_type: Literal["chat"]
ls_temperature: Optional[float]
ls_max_tokens: Optional[int]
ls_stop: Optional[List[str]]
```
by default it will only return
```python
{ls_model_type="chat", ls_stop=stop}
```
2. Add these params to inheritable metadata in
`CallbackManager.configure`
3. Implement `.get_ls_params` and populate all params for Anthropic +
all subclasses of BaseChatOpenAI
Sample trace:
https://smith.langchain.com/public/d2962673-4c83-47c7-b51e-61d07aaffb1b/r
**OpenAI**:
<img width="984" alt="Screenshot 2024-05-17 at 10 03 35 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/2ef41f74-a9df-4e0e-905d-da74fa82a910">
**Anthropic**:
<img width="978" alt="Screenshot 2024-05-17 at 10 06 07 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/39701c9f-7da5-4f1a-ab14-84e9169d63e7">
**Mistral** (and all others for which params are not yet populated):
<img width="977" alt="Screenshot 2024-05-17 at 10 08 43 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/37d7d894-fec2-4300-986f-49a5f0191b03">
|
||
|---|---|---|
| .. | ||
| langchain_standard_tests | ||
| scripts | ||
| tests | ||
| Makefile | ||
| poetry.lock | ||
| pyproject.toml | ||
| README.md | ||
langchain-standard-tests
This is an INTERNAL library for the LangChain project. It contains the base classes for a standard set of tests.
Installation
This package will NOT be regularly published to pypi. It is intended to be installed directly from github at test time.
Pip:
```bash
pip install git+https://github.com/langchain-ai/langchain.git#subdirectory=libs/standard-tests
```
Poetry:
```bash
poetry add git+https://github.com/langchain-ai/langchain.git#subdirectory=libs/standard-tests
```
Usage
To add standard tests to an integration package's e.g. ChatModel, you need to create
- A unit test class that inherits from ChatModelUnitTests
- An integration test class that inherits from ChatModelIntegrationTests
tests/unit_tests/test_standard.py:
"""Standard LangChain interface tests"""
from typing import Type
import pytest
from langchain_core.language_models import BaseChatModel
from langchain_standard_tests.unit_tests import ChatModelUnitTests
from langchain_parrot_chain import ChatParrotChain
class TestParrotChainStandard(ChatModelUnitTests):
@pytest.fixture
def chat_model_class(self) -> Type[BaseChatModel]:
return ChatParrotChain
tests/integration_tests/test_standard.py:
"""Standard LangChain interface tests"""
from typing import Type
import pytest
from langchain_core.language_models import BaseChatModel
from langchain_standard_tests.integration_tests import ChatModelIntegrationTests
from langchain_parrot_chain import ChatParrotChain
class TestParrotChainStandard(ChatModelIntegrationTests):
@pytest.fixture
def chat_model_class(self) -> Type[BaseChatModel]:
return ChatParrotChain
Reference
The following fixtures are configurable in the test classes. Anything not marked as required is optional.
chat_model_class(required): The class of the chat model to be testedchat_model_params: The keyword arguments to pass to the chat model constructorchat_model_has_tool_calling: Whether the chat model can call tools. By default, this is set tohasattr(chat_model_class, 'bind_tools)chat_model_has_structured_output: Whether the chat model can structured output. By default, this is set tohasattr(chat_model_class, 'with_structured_output')