community[patch]: optimize xinference llm import (#25809)

Thank you for contributing to LangChain!

- [ ] **PR title**: "community: optimize xinference llm import"

- [ ] **PR message**: 
- **Description:** from xinferece_client import RESTfulClient when there
is no importing xinference.
    - **Dependencies:** xinferece_client
- **Why do so:** the total xinference(pip install xinference[all]) is
too heavy for installing, let alone it is useless for langchain user
except RESTfulClient. The modification has maintained consistency with
the xinference embeddings
[embeddings/xinference](../blob/master/libs/community/langchain_community/embeddings/xinference.py#L89).
This commit is contained in:
崔浩 2024-08-29 09:41:43 +08:00 committed by GitHub
parent 9decd0b243
commit 3555882a0d
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194

View File

@ -17,6 +17,12 @@ class Xinference(LLM):
pip install "xinference[all]" pip install "xinference[all]"
If you're simply using the services provided by Xinference, you can utilize the xinference_client package:
.. code-block:: bash
pip install xinference_client
Check out: https://github.com/xorbitsai/inference Check out: https://github.com/xorbitsai/inference
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers
@ -91,10 +97,13 @@ class Xinference(LLM):
): ):
try: try:
from xinference.client import RESTfulClient from xinference.client import RESTfulClient
except ImportError:
try:
from xinference_client import RESTfulClient
except ImportError as e: except ImportError as e:
raise ImportError( raise ImportError(
"Could not import RESTfulClient from xinference. Please install it" "Could not import RESTfulClient from xinference. Please install it"
" with `pip install xinference`." " with `pip install xinference` or `pip install xinference_client`."
) from e ) from e
model_kwargs = model_kwargs or {} model_kwargs = model_kwargs or {}