Archives/langchain
2
0
Fork 1
mirror of https://github.com/hwchase17/langchain synced 2024-11-06 03:20:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
38425c99d2
langchain/libs/core/langchain_core/utils
History
William FH 38425c99d2
core[minor]: Image prompt template (#14263) 
Builds on Bagatur's (#13227). See unit test for example usage (below)

```python
def test_chat_tmpl_from_messages_multipart_image() -> None:
    base64_image = "abcd123"
    other_base64_image = "abcd123"
    template = ChatPromptTemplate.from_messages(
        [
            ("system", "You are an AI assistant named {name}."),
            (
                "human",
                [
                    {"type": "text", "text": "What's in this image?"},
                    # OAI supports all these structures today
                    {
                        "type": "image_url",
                        "image_url": "data:image/jpeg;base64,{my_image}",
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,{my_image}"},
                    },
                    {"type": "image_url", "image_url": "{my_other_image}"},
                    {
                        "type": "image_url",
                        "image_url": {"url": "{my_other_image}", "detail": "medium"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "https://www.langchain.com/image.png"},
                    },
                    {
                        "type": "image_url",
                        "image_url": {"url": "data:image/jpeg;base64,foobar"},
                    },
                ],
            ),
        ]
    )
    messages = template.format_messages(
        name="R2D2", my_image=base64_image, my_other_image=other_base64_image
    )
    expected = [
        SystemMessage(content="You are an AI assistant named R2D2."),
        HumanMessage(
            content=[
                {"type": "text", "text": "What's in this image?"},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/jpeg;base64,{other_base64_image}"
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"{other_base64_image}"},
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"{other_base64_image}",
                        "detail": "medium",
                    },
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "https://www.langchain.com/image.png"},
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "data:image/jpeg;base64,foobar"},
                },
            ]
        ),
    ]
    assert messages == expected
```

---------

Co-authored-by: Bagatur <baskaryan@gmail.com>
Co-authored-by: Brace Sproul <braceasproul@gmail.com>
2024-01-27 17:04:29 -08:00
..
__init__.py core[minor]: Image prompt template (#14263) 2024-01-27 17:04:29 -08:00
_merge.py core[patch], openai[patch]: Chat openai stream logprobs (#16218) 2024-01-19 09:16:09 -08:00
aiter.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00
env.py Improve: remove extra spaces in get_from_env error (#15064) 2023-12-22 11:50:03 -08:00
formatting.py [core] prompt changes (#15186) 2023-12-26 15:52:17 -08:00
function_calling.py core[patch], community[patch], openai[patch]: consolidate openai tool… (#16485) 2024-01-25 13:18:46 -08:00
html.py core[patch]: Further restrict recursive URL loader (#15559) 2024-01-04 16:33:57 -05:00
image.py core[minor]: Image prompt template (#14263) 2024-01-27 17:04:29 -08:00
input.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00
interactive_env.py core[patch]: simple prompt pretty printing (#15968) 2024-01-12 21:08:51 -05:00
iter.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00
json_schema.py Update _retrieve_ref inside json_schema.py to include an isdigit() check (#14745) 2024-01-01 16:25:42 -08:00
loading.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00
pydantic.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00
strings.py community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463) 2023-12-11 13:53:30 -08:00
utils.py Separate out langchain_core package (#13577) 2023-11-20 13:09:30 -08:00