mirror of
https://github.com/hwchase17/langchain
synced 2024-11-10 01:10:59 +00:00
community[patch]: optimize xinference llm import (#25809)
Thank you for contributing to LangChain! - [ ] **PR title**: "community: optimize xinference llm import" - [ ] **PR message**: - **Description:** from xinferece_client import RESTfulClient when there is no importing xinference. - **Dependencies:** xinferece_client - **Why do so:** the total xinference(pip install xinference[all]) is too heavy for installing, let alone it is useless for langchain user except RESTfulClient. The modification has maintained consistency with the xinference embeddings [embeddings/xinference](../blob/master/libs/community/langchain_community/embeddings/xinference.py#L89).
This commit is contained in:
parent
9decd0b243
commit
3555882a0d
@ -17,6 +17,12 @@ class Xinference(LLM):
|
|||||||
|
|
||||||
pip install "xinference[all]"
|
pip install "xinference[all]"
|
||||||
|
|
||||||
|
If you're simply using the services provided by Xinference, you can utilize the xinference_client package:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
pip install xinference_client
|
||||||
|
|
||||||
Check out: https://github.com/xorbitsai/inference
|
Check out: https://github.com/xorbitsai/inference
|
||||||
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers
|
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers
|
||||||
|
|
||||||
@ -91,10 +97,13 @@ class Xinference(LLM):
|
|||||||
):
|
):
|
||||||
try:
|
try:
|
||||||
from xinference.client import RESTfulClient
|
from xinference.client import RESTfulClient
|
||||||
|
except ImportError:
|
||||||
|
try:
|
||||||
|
from xinference_client import RESTfulClient
|
||||||
except ImportError as e:
|
except ImportError as e:
|
||||||
raise ImportError(
|
raise ImportError(
|
||||||
"Could not import RESTfulClient from xinference. Please install it"
|
"Could not import RESTfulClient from xinference. Please install it"
|
||||||
" with `pip install xinference`."
|
" with `pip install xinference` or `pip install xinference_client`."
|
||||||
) from e
|
) from e
|
||||||
|
|
||||||
model_kwargs = model_kwargs or {}
|
model_kwargs = model_kwargs or {}
|
||||||
|
Loading…
Reference in New Issue
Block a user