diff --git a/docs/extras/ecosystem/integrations/jina.mdx b/docs/extras/ecosystem/integrations/jina.mdx index be278bf7d4..7cbe5a41d4 100644 --- a/docs/extras/ecosystem/integrations/jina.mdx +++ b/docs/extras/ecosystem/integrations/jina.mdx @@ -16,3 +16,59 @@ There exists a Jina Embeddings wrapper, which you can access with from langchain.embeddings import JinaEmbeddings ``` For a more detailed walkthrough of this, see [this notebook](/docs/modules/data_connection/text_embedding/integrations/jina.html) + +## Deployment + +[Langchain-serve](https://github.com/jina-ai/langchain-serve), powered by Jina, helps take LangChain apps to production with easy to use REST/WebSocket APIs and Slack bots. + +### Usage + +Install the package from PyPI. + +```bash +pip install langchain-serve +``` + +Wrap your LangChain app with the `@serving` decorator. + +```python +# app.py +from lcserve import serving + +@serving +def ask(input: str) -> str: + from langchain import LLMChain, OpenAI + from langchain.agents import AgentExecutor, ZeroShotAgent + + tools = [...] # list of tools + prompt = ZeroShotAgent.create_prompt( + tools, input_variables=["input", "agent_scratchpad"], + ) + llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt) + agent = ZeroShotAgent( + llm_chain=llm_chain, allowed_tools=[tool.name for tool in tools] + ) + agent_executor = AgentExecutor.from_agent_and_tools( + agent=agent, + tools=tools, + verbose=True, + ) + return agent_executor.run(input) +``` + +Deploy on Jina AI Cloud with `lc-serve deploy jcloud app`. Once deployed, we can send a POST request to the API endpoint to get a response. + +```bash +curl -X 'POST' 'https://.wolf.jina.ai/ask' \ + -d '{ + "input": "Your Quesion here?", + "envs": { + "OPENAI_API_KEY": "sk-***" + } +}' +``` + +You can also self-host the app on your infrastructure with Docker-compose or Kubernetes. See [here](https://github.com/jina-ai/langchain-serve#-self-host-llm-apps-with-docker-compose-or-kubernetes) for more details. + + +Langchain-serve also allows to deploy the apps with WebSocket APIs and Slack Bots both on [Jina AI Cloud](https://cloud.jina.ai/) or self-hosted infrastructure. diff --git a/docs/extras/guides/deployments/index.mdx b/docs/extras/guides/deployments/index.mdx index d56f3365ba..09841cff14 100644 --- a/docs/extras/guides/deployments/index.mdx +++ b/docs/extras/guides/deployments/index.mdx @@ -24,6 +24,7 @@ Understanding these components is crucial when assessing serving systems. LangCh - [BentoML](https://github.com/bentoml/BentoML) - [OpenLLM](/docs/ecosystem/integrations/openllm.html) - [Modal](/docs/ecosystem/integrations/modal.html) +- [Jina](/docs/ecosystem/integrations/jina.html#deployment) These links will provide further information on each ecosystem, assisting you in finding the best fit for your LLM deployment needs. diff --git a/docs/extras/guides/deployments/template_repos.mdx b/docs/extras/guides/deployments/template_repos.mdx index aee56a203b..f1f025fff9 100644 --- a/docs/extras/guides/deployments/template_repos.mdx +++ b/docs/extras/guides/deployments/template_repos.mdx @@ -61,7 +61,7 @@ This repository contains LangChain adapters for Steamship, enabling LangChain de ## [Langchain-serve](https://github.com/jina-ai/langchain-serve) -This repository allows users to serve local chains and agents as RESTful, gRPC, or WebSocket APIs, thanks to [Jina](https://docs.jina.ai/). Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud. +This repository allows users to deploy any LangChain app as REST/WebSocket APIs or, as Slack Bots with ease. Benefit from the scalability and serverless architecture of Jina AI Cloud, or deploy on-premise with Kubernetes. ## [BentoML](https://github.com/ssheng/BentoChain)