mirror of
https://github.com/hwchase17/langchain
synced 2024-10-31 15:20:26 +00:00
bf5385592e
While using `chain.batch`, the default implementation uses a `ThreadPoolExecutor` and run the chains in separate threads. An issue with this approach is that that [the token counting callback](https://python.langchain.com/docs/modules/callbacks/token_counting) fails to work as a consequence of the context not being propagated between threads. This PR adds context propagation to the new threads and adds some thread synchronization in the OpenAI callback. With this change, the token counting callback works as intended. Having the context propagation change would be highly beneficial for those implementing custom callbacks for similar functionalities as well. --------- Co-authored-by: Nuno Campos <nuno@langchain.dev> |
||
|---|---|---|
| .. | ||
| cli | ||
| community | ||
| core | ||
| experimental | ||
| langchain | ||
| partners | ||