You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
langchain/tests/unit_tests
Eugene Yurtsev e46202829f
feat #4479: TextLoader auto detect encoding and improved exceptions (#4927)
# TextLoader auto detect encoding and enhanced exception handling

- Add an option to enable encoding detection on `TextLoader`. 
- The detection is done using `chardet`
- The loading is done by trying all detected encodings by order of
confidence or raise an exception otherwise.

### New Dependencies:
- `chardet`

Fixes #4479 

## Before submitting

<!-- If you're adding a new integration, include an integration test and
an example notebook showing its use! -->

## Who can review?

Community members can review the PR once tests pass. Tag
maintainers/contributors who might be interested:

- @eyurtsev

---------

Co-authored-by: blob42 <spike@w530>
1 year ago
..
agents agent serialization (#4642) 1 year ago
callbacks [Breaking] Refactor Base Tracer(#4549) 1 year ago
chains Callbacks Refactor [base] (#3256) 1 year ago
chat_models Add ChatModel, LLM, and Embeddings for Google's PaLM APIs (#3575) 1 year ago
client Add Support for Flexible Input Format for LLM and Chat Model Runs (#4805) 1 year ago
data Prompt from file proof of concept using plain text (#127) 2 years ago
docstore Add `DocstoreFn` - lookup doc via arbitrary function (#3760) 1 year ago
document_loaders feat #4479: TextLoader auto detect encoding and improved exceptions (#4927) 1 year ago
evaluation Adding an in-context QA evaluation chain + chain of thought reasoning chain for improved accuracy (#2444) 1 year ago
examples feat #4479: TextLoader auto detect encoding and improved exceptions (#4927) 1 year ago
llms Add Invocation Params (#4509) 1 year ago
memory Zep memory (#4898) 1 year ago
output_parsers Harrison/json new line (#4646) 1 year ago
prompts Add from_file method to message prompt template (#4713) 1 year ago
retrievers Fix self query pinecone translation (#3892) 1 year ago
tools Cadlabs/python tool sanitization (#4754) 1 year ago
utilities Add Tenant ID to V2 Tracer (#4135) 1 year ago
vectorstores fix #3884 (#3475) 1 year ago
__init__.py initial commit 2 years ago
conftest.py Add pytest --only-extended and --only-core options (#4494) 1 year ago
test_bash.py Skip flaky unit test (#4591) 1 year ago
test_depedencies.py Catch changes to test group (#4802) 1 year ago
test_document_transformers.py Contextual compression retriever (#2915) 1 year ago
test_formatting.py initial commit 2 years ago
test_math_utils.py Contextual compression retriever (#2915) 1 year ago
test_pytest_config.py Block sockets for unit-tests (#4803) 1 year ago
test_python.py option for csv agent to not include df in prompt (#4610) 1 year ago
test_schema.py [simple][test] Added test case for schema.py (#3692) 1 year ago
test_sql_database.py sql: do not hard code the LIMIT clause in the table_info section (#1563) 1 year ago
test_sql_database_schema.py Suppress duckdb warning in unit tests explicitly (#3653) 1 year ago
test_text_splitter.py Add unit test for _merge_splits function (#3513) 1 year ago