mirror of
https://github.com/hwchase17/langchain
synced 2024-10-29 17:07:25 +00:00
96023f94d9
We are introducing the py integration to Javelin AI Gateway www.getjavelin.io. Javelin is an enterprise-scale fast llm router & gateway. Could you please review and let us know if there is anything missing. Javelin AI Gateway wraps Embedding, Chat and Completion LLMs. Uses javelin_sdk under the covers (pip install javelin_sdk). Author: Sharath Rajasekar, Twitter: @sharathr, @javelinai Thanks!!
93 lines
2.4 KiB
Plaintext
93 lines
2.4 KiB
Plaintext
# Javelin AI Gateway
|
|
|
|
[The Javelin AI Gateway](https://www.getjavelin.io) service is a high-performance, enterprise grade API Gateway for AI applications.
|
|
It is designed to streamline the usage and access of various large language model (LLM) providers,
|
|
such as OpenAI, Cohere, Anthropic and custom large language models within an organization by incorporating
|
|
robust access security for all interactions with LLMs.
|
|
|
|
Javelin offers a high-level interface that simplifies the interaction with LLMs by providing a unified endpoint
|
|
to handle specific LLM related requests.
|
|
|
|
See the Javelin AI Gateway [documentation](https://docs.getjavelin.io) for more details.
|
|
[Javelin Python SDK](https://www.github.com/getjavelin/javelin-python) is an easy to use client library meant to be embedded into AI Applications
|
|
|
|
## Installation and Setup
|
|
|
|
Install `javelin_sdk` to interact with Javelin AI Gateway:
|
|
|
|
```sh
|
|
pip install 'javelin_sdk'
|
|
```
|
|
|
|
Set the Javelin's API key as an environment variable:
|
|
|
|
```sh
|
|
export JAVELIN_API_KEY=...
|
|
```
|
|
|
|
## Completions Example
|
|
|
|
```python
|
|
|
|
from langchain.chains import LLMChain
|
|
from langchain.llms import JavelinAIGateway
|
|
from langchain.prompts import PromptTemplate
|
|
|
|
route_completions = "eng_dept03"
|
|
|
|
gateway = JavelinAIGateway(
|
|
gateway_uri="http://localhost:8000",
|
|
route=route_completions,
|
|
model_name="text-davinci-003",
|
|
)
|
|
|
|
llmchain = LLMChain(llm=gateway, prompt=prompt)
|
|
result = llmchain.run("podcast player")
|
|
|
|
print(result)
|
|
|
|
```
|
|
|
|
## Embeddings Example
|
|
|
|
```python
|
|
from langchain.embeddings import JavelinAIGatewayEmbeddings
|
|
from langchain.embeddings.openai import OpenAIEmbeddings
|
|
|
|
embeddings = JavelinAIGatewayEmbeddings(
|
|
gateway_uri="http://localhost:8000",
|
|
route="embeddings",
|
|
)
|
|
|
|
print(embeddings.embed_query("hello"))
|
|
print(embeddings.embed_documents(["hello"]))
|
|
```
|
|
|
|
## Chat Example
|
|
```python
|
|
from langchain.chat_models import ChatJavelinAIGateway
|
|
from langchain.schema import HumanMessage, SystemMessage
|
|
|
|
messages = [
|
|
SystemMessage(
|
|
content="You are a helpful assistant that translates English to French."
|
|
),
|
|
HumanMessage(
|
|
content="Artificial Intelligence has the power to transform humanity and make the world a better place"
|
|
),
|
|
]
|
|
|
|
chat = ChatJavelinAIGateway(
|
|
gateway_uri="http://localhost:8000",
|
|
route="mychatbot_route",
|
|
model_name="gpt-3.5-turbo"
|
|
params={
|
|
"temperature": 0.1
|
|
}
|
|
)
|
|
|
|
print(chat(messages))
|
|
|
|
```
|
|
|