Commit Graph

16 Commits

Author SHA1 Message Date
Bagatur
d96f67b06f
standard-tests[patch]: Update chat model standard tests (#22378)
- Refactor standard test classes to make them easier to configure
- Update openai to support stop_sequences init param
- Update groq to support stop_sequences init param
- Update fireworks to support max_retries init param
- Update ChatModel.bind_tools to type tool_choice
- Update groq to handle tool_choice="any". **this may be controversial**

---------

Co-authored-by: Chester Curme <chester.curme@gmail.com>
2024-06-17 13:37:41 -07:00
Bagatur
e2304ebcdb
standard-tests[patch]: Release 0.1.1 (#22984) 2024-06-17 15:31:34 +00:00
Bagatur
9ff249a38d
standard-tests[patch]: don't require str chunk contents (#22965) 2024-06-17 08:52:24 -04:00
ccurme
3999761201
multiple: add stop attribute (#22573) 2024-06-06 12:11:52 -04:00
ccurme
fbfed65fb1
core, partners: add token usage attribute to AIMessage (#21944)
```python
class UsageMetadata(TypedDict):
    """Usage metadata for a message, such as token counts.

    Attributes:
        input_tokens: (int) count of input (or prompt) tokens
        output_tokens: (int) count of output (or completion) tokens
        total_tokens: (int) total token count
    """

    input_tokens: int
    output_tokens: int
    total_tokens: int
```
```python
class AIMessage(BaseMessage):
    ...
    usage_metadata: Optional[UsageMetadata] = None
    """If provided, token usage information associated with the message."""
    ...
```
2024-05-23 14:21:58 -04:00
ccurme
181dfef118
core, standard tests, partner packages: add test for model params (#21677)
1. Adds `.get_ls_params` to BaseChatModel which returns
```python
class LangSmithParams(TypedDict, total=False):
    ls_provider: str
    ls_model_name: str
    ls_model_type: Literal["chat"]
    ls_temperature: Optional[float]
    ls_max_tokens: Optional[int]
    ls_stop: Optional[List[str]]
```
by default it will only return
```python
{ls_model_type="chat", ls_stop=stop}
```

2. Add these params to inheritable metadata in
`CallbackManager.configure`

3. Implement `.get_ls_params` and populate all params for Anthropic +
all subclasses of BaseChatOpenAI

Sample trace:
https://smith.langchain.com/public/d2962673-4c83-47c7-b51e-61d07aaffb1b/r

**OpenAI**:
<img width="984" alt="Screenshot 2024-05-17 at 10 03 35 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/2ef41f74-a9df-4e0e-905d-da74fa82a910">

**Anthropic**:
<img width="978" alt="Screenshot 2024-05-17 at 10 06 07 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/39701c9f-7da5-4f1a-ab14-84e9169d63e7">

**Mistral** (and all others for which params are not yet populated):
<img width="977" alt="Screenshot 2024-05-17 at 10 08 43 AM"
src="https://github.com/langchain-ai/langchain/assets/26529506/37d7d894-fec2-4300-986f-49a5f0191b03">
2024-05-17 13:51:26 -04:00
Erick Friis
c77d2f2b06
multiple: core 0.2 nonbreaking dep, check_diff community->langchain dep (#21646)
0.2 is not a breaking release for core (but it is for langchain and
community)

To keep the core+langchain+community packages in sync at 0.2, we will
relax deps throughout the ecosystem to tolerate `langchain-core` 0.2
2024-05-13 19:50:36 -07:00
ccurme
01a3228d8e
standard tests: add test for few-shot examples (#21019) 2024-05-13 10:06:12 -04:00
ccurme
2d77e5e3a1
(standard tests): add test for basic conversation sequence (#21213) 2024-05-02 13:47:10 -04:00
Erick Friis
ddc2274aea
standard-tests: split tool calling test (#20803)
just making it a bit easier to grok
2024-04-23 20:59:45 +00:00
ccurme
2238490069
mistral, openai: allow anthropic-style messages in message histories (#20565) 2024-04-17 15:55:45 -04:00
ccurme
a892f985d3
standardized-tests[patch]: test tool call messages (#20519)
Co-authored-by: Bagatur <baskaryan@gmail.com>
2024-04-16 23:25:50 +00:00
Erick Friis
77eba10f47
standard-tests: fix default fixtures (#20520) 2024-04-16 16:12:36 +00:00
Erick Friis
29282371db
core: bind_tools interface on basechatmodel (#20360) 2024-04-12 01:32:19 +00:00
Erick Friis
e6806a08d4
multiple: standard chat model tests (#20359) 2024-04-11 18:23:13 -07:00
Erick Friis
855ba46f80
standard-tests: a standard unit and integration test set (#20182)
just chat models for now
2024-04-09 12:43:00 -07:00