You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

felpigeon 2791a753bf Add start index to metadata in TextSplitter (#5912) ... <!-- Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution. Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change. After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost. Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle! --> #### Add start index to metadata in TextSplitter - Modified method `create_documents` to track start position of each chunk - The `start_index` is included in the metadata if the `add_start_index` parameter in the class constructor is set to `True` This enables referencing back to the original document, particularly useful when a specific chunk is retrieved. <!-- If you're adding a new integration, please include: 1. a test for the integration - favor unit tests that does not rely on network access. 2. an example notebook showing its use See contribution guidelines for more information on how to write tests, lint etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> #### Who can review? Tag maintainers/contributors who might be interested: @eyurtsev @agola11 <!-- For a quicker response, figure out the right person to tag with @ @hwchase17 - project lead Tracing / Callbacks - @agola11 Async - @agola11 DataLoaders - @eyurtsev Models - @hwchase17 - @agola11 Agents / Tools / Toolkits - @vowelparrot VectorStores / Retrievers / Memory - @dev2049 -->		1 year ago
..
_static	Fix the shortcut conflict for document page search (#5874)	1 year ago
additional_resources	[Docs][Hotfix] Fix broken links (#5800)	1 year ago
ecosystem	Baseten integration (#5862)	1 year ago
getting_started	Update tutorials.md (#5761)	1 year ago
integrations	Refactor and update databricks integration page (#5575)	1 year ago
modules	Add start index to metadata in TextSplitter (#5912)	1 year ago
reference	Documentation fixes (linting and broken links) (#5563)	1 year ago
templates	docs `ecosystem/integrations` update 3 (#5470)	1 year ago
tracing	py tracer fixes (#5377)	1 year ago
use_cases	minor refactor GenerativeAgentMemory (#5315)	1 year ago
Makefile	Feature: linkcheck-action (#534) (#542)	2 years ago
conf.py	docs: Mendable Search integration (#2803)	1 year ago
dependents.md	docs: updated `ecosystem/dependents` (#5753)	1 year ago
index.rst	docs: Added Deploying LLMs into production + a new ecosystem (#4047)	1 year ago
integrations.rst	docs: `ecosystem/integrations` update 1 (#5219)	1 year ago
make.bat	initial commit	2 years ago
reference.rst	docs: `Deployments` page moved into `Ecosystem/` (#4949)	1 year ago
requirements.txt	Harrison/docs reqs (#2199)	1 year ago