You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Ryan Kraus f027696b5f community: Added new Utility runnables for NVIDIA Riva. (#15966) ... **Please tag this issue with `nvidia_genai`** - **Description:** Added new Runnables for integration NVIDIA Riva into LCEL chains for Automatic Speech Recognition (ASR) and Text To Speech (TTS). - **Issue:** N/A - **Dependencies:** To use these runnables, the NVIDIA Riva client libraries are required. It they are not installed, an error will be raised instructing how to install them. The Runnables can be safely imported without the riva client libraries. - **Twitter handle:** N/A All of the Riva Runnables are inside a single folder in the Utilities module. In this folder are four files: - common.py - Contains all code that is common to both TTS and ASR - stream.py - Contains a class representing an audio stream that allows the end user to put data into the stream like a queue. - asr.py - Contains the RivaASR runnable - tts.py - Contains the RivaTTS runnable The following Python function is an example of creating a chain that makes use of both of these Runnables: ```python def create( config: Configuration, audio_encoding: RivaAudioEncoding, sample_rate: int, audio_channels: int = 1, ) -> Runnable[ASRInputType, TTSOutputType]: """Create a new instance of the chain.""" _LOGGER.info("Instantiating the chain.") # create the riva asr client riva_asr = RivaASR( url=str(config.riva_asr.service.url), ssl_cert=config.riva_asr.service.ssl_cert, encoding=audio_encoding, audio_channel_count=audio_channels, sample_rate_hertz=sample_rate, profanity_filter=config.riva_asr.profanity_filter, enable_automatic_punctuation=config.riva_asr.enable_automatic_punctuation, language_code=config.riva_asr.language_code, ) # create the prompt template prompt = PromptTemplate.from_template("{user_input}") # model = ChatOpenAI() model = ChatNVIDIA(model="mixtral_8x7b") # type: ignore # create the riva tts client riva_tts = RivaTTS( url=str(config.riva_asr.service.url), ssl_cert=config.riva_asr.service.ssl_cert, output_directory=config.riva_tts.output_directory, language_code=config.riva_tts.language_code, voice_name=config.riva_tts.voice_name, ) # construct and return the chain return {"user_input": riva_asr} | prompt | model | riva_tts # type: ignore ``` The following code is an example of creating a new audio stream for Riva: ```python input_stream = AudioStream(maxsize=1000) # Send bytes into the stream for chunk in audio_chunks: await input_stream.aput(chunk) input_stream.close() ``` The following code is an example of how to execute the chain with RivaASR and RivaTTS ```python output_stream = asyncio.Queue() while not input_stream.complete: async for chunk in chain.astream(input_stream): output_stream.put(chunk) ``` Everything should be async safe and thread safe. Audio data can be put into the input stream while the chain is running without interruptions. --------- Co-authored-by: Hayden Wolff <hwolff@nvidia.com> Co-authored-by: Hayden Wolff <hwolff@Haydens-Laptop.local> Co-authored-by: Hayden Wolff <haydenwolff99@gmail.com> Co-authored-by: Erick Friis <erick@langchain.dev>		8 months ago
..
adapters	community[patch]: Add safe lookup to OpenAI response adapter (#14765)	10 months ago
agent_toolkits	community[patch]: fix agent_toolkits mypy (#17050)	8 months ago
callbacks	community[patch]: MLflow callback update (#16687)	8 months ago
chat_loaders	Do not issue beta or deprecation warnings on internal calls (#15641)	9 months ago
chat_message_histories	community[patch]: chat message history mypy fixes (#17059)	8 months ago
chat_models	community[patch]: chat model mypy fixes (#17061)	8 months ago
docstore	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)	10 months ago
document_loaders	infra: add -p to mkdir in lint steps (#17013)	8 months ago
document_transformers	community[minor]: Adding asynchronous function implementation for Doctran (#15941)	9 months ago
embeddings	community[patch]: Add Progress bar to HuggingFaceEmbeddings (#16758)	8 months ago
example_selectors	refactor `langchain.prompts.example_selector` (#15369)	8 months ago
graphs	infra: add -p to mkdir in lint steps (#17013)	8 months ago
indexes	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)	10 months ago
llms	community[patch]: Fix error in `LlamaCpp` community LLM with Configurable Fields, 'grammar' custom type not available (#16995)	8 months ago
output_parsers	langchain[patch], community[minor]: move `output_parsers.ernie_functions` (#16057)	9 months ago
retrievers	community[minor]: Breebs docs retriever (#16578)	8 months ago
storage	community: revert SQL Stores (#16912)	8 months ago
tools	infra: add -p to mkdir in lint steps (#17013)	8 months ago
utilities	community: Added new Utility runnables for NVIDIA Riva. (#15966)	8 months ago
utils	openai[minor]: implement langchain-openai package (#15503)	9 months ago
vectorstores	pinecone: init pkg (#16556)	8 months ago
__init__.py	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)	10 months ago
cache.py	community: Factorize AstraDB components constructors (#16779)	8 months ago
py.typed	community[major], core[patch], langchain[patch], experimental[patch]: Create langchain-community (#14463)	10 months ago