4 Commits (96cd50938a404fdfec36c078aba614295b309999)

Author	SHA1	Message	Date
William De Vena	a63cee04ac	nvidia-trt[patch]: Invoke callback prior to yielding token (#18446) ... ## PR title nvidia-trt[patch]: Invoke callback prior to yielding ## PR message - Description: Invoke on_llm_new_token callback prior to yielding token in _stream method. - Issue: https://github.com/langchain-ai/langchain/issues/16913 - Dependencies: None	7 months ago
Mikhail Khludnev	14ff1438e6	nvidia-trt[patch]: propagate InferenceClientException to the caller. (#16936) ... - **Description:** before the change I've got 1. propagate InferenceClientException to the caller. 2. stop grpc receiver thread on exception ``` for token in result_queue: > result_str += token E TypeError: can only concatenate str (not "InferenceServerException") to str ../../langchain_nvidia_trt/llms.py:207: TypeError ``` And stream thread keeps running. after the change request thread stops correctly and caller got a root cause exception: ``` E tritonclient.utils.InferenceServerException: [request id: 4529729] expected number of inputs between 2 and 3 but got 10 inputs for model 'vllm_model' ../../langchain_nvidia_trt/llms.py:205: InferenceServerException ``` - **Issue:** the issue # it fixes if applicable, - **Dependencies:** any dependencies required for this change, - **Twitter handle:** [t.me/mkhl_spb](https://t.me/mkhl_spb) I'm not sure about test coverage. Should I setup deep mocks or there's a kind of triton stub via testcontainers or so.	8 months ago
Mikhail Khludnev	2145636f1d	Nvidia trt model name for stop_stream() (#16997) ... just removing some legacy leftover.	8 months ago
William FH	583696732c	[Partner] NVIDIA TRT Package (#14733) ... Simplify #13976 and add as a separate package. - [] Add README - [X] Add doc notebook - [X] Add simple LLM integration --------- Co-authored-by: Jeremy Dyer <jdye64@gmail.com>	9 months ago