langchain/libs
Andreas Motl 3613d8a2ad
community[patch]: Use SQLAlchemy's bulk_save_objects method to improve insert performance (#16244)
- **Description:** Improve [pgvector vector store
adapter](https://github.com/langchain-ai/langchain/blob/v0.1.1/libs/community/langchain_community/vectorstores/pgvector.py)
to save embeddings in batches, to improve its performance.
  - **Issue:** NA
  - **Dependencies:** NA
  - **References:** https://github.com/crate-workbench/langchain/pull/1


Hi again from the CrateDB team,

following up on GH-16243, this is another minor patch to the pgvector
vector store adapter. Inserting embeddings in batches, using
[SQLAlchemy's
`bulk_save_objects`](https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session.bulk_save_objects)
method, can deliver substantial performance gains.

With kind regards,
Andreas.

NB: As I am seeing just now that this method is a legacy feature of SA
2.0, it will need to be reworked on a future iteration. However, it is
not deprecated yet, and I haven't been able to come up with a different
implementation, yet.
2024-01-18 18:35:39 -08:00
..
cli Make packages optional (#15727) 2024-01-08 17:09:21 -08:00
community community[patch]: Use SQLAlchemy's bulk_save_objects method to improve insert performance (#16244) 2024-01-18 18:35:39 -08:00
core core[minor]: RFC Add astream_events to Runnables (#16172) 2024-01-18 21:27:01 -05:00
experimental core[patch]: simple prompt pretty printing (#15968) 2024-01-12 21:08:51 -05:00
langchain Use env for revision id if not passed in as param; use git describe as backup (#16227) 2024-01-18 16:15:26 -08:00
partners google-vertexai[patch]: more integration test fixes (#16234) 2024-01-18 13:59:23 -08:00