Commit Graph

37 Commits

Author SHA1 Message Date
rsaryev
53de4f1fae
chore: update import statements in utils.py and llm.py to use langchain.vectorstores.FAISS 2023-11-05 12:10:28 +02:00
Saryev Rustam
ee32d8087e
Update pyproject.toml 2023-11-05 11:43:15 +02:00
Saryev Rustam
f03f52827b
Merge branch 'main' into dependabot/pip/langchain-0.0.325 2023-11-05 11:39:17 +02:00
dependabot[bot]
13e589888c
Bump langchain from 0.0.236 to 0.0.325
Bumps [langchain](https://github.com/langchain-ai/langchain) from 0.0.236 to 0.0.325.
- [Release notes](https://github.com/langchain-ai/langchain/releases)
- [Commits](https://github.com/langchain-ai/langchain/compare/v0.0.236...v0.0.325)

---
updated-dependencies:
- dependency-name: langchain
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-30 20:23:39 +00:00
dependabot[bot]
49ba7fc046
Bump urllib3 from 1.26.6 to 1.26.18
Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.6 to 1.26.18.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.6...1.26.18)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-10-18 02:21:51 +00:00
rsaryev
b978a76402
Refactor get_repo() and load_files() functions to use Repo() without root_dir.Refactored load_files added a delay when creating vector store. 2023-08-22 00:09:51 +03:00
rsaryev
3d3e2dabd5
Add check for Python version.
Add remove_model_name_local
2023-08-16 01:25:02 +03:00
dependabot[bot]
cb36a73539
Bump langchain from 0.0.181 to 0.0.236
Bumps [langchain](https://github.com/hwchase17/langchain) from 0.0.181 to 0.0.236.
- [Release notes](https://github.com/hwchase17/langchain/releases)
- [Commits](https://github.com/hwchase17/langchain/compare/v0.0.181...v0.0.236)

---
updated-dependencies:
- dependency-name: langchain
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-08-13 07:04:06 +00:00
Saryev Rustam
e03ca88549 update project version, add local model images to README, and remove default local model from consts.py 2023-07-12 03:55:35 +03:00
Saryev Rustam
e8d7b542ea update version, refactor README.md content, config.py and cli.py for improved code readability 2023-07-12 03:41:09 +03:00
Saryev Rustam
01aef1c449 Bump version to 0.1.42 refactor BaseLLM 2023-07-05 13:59:22 +03:00
Saryev Rustam
136c09c971 Update 2023-07-05 05:22:11 +03:00
Saryev Rustam
2260c6dd35 Refactor llm.py template and update version in pyproject.toml. 2023-06-30 18:04:58 +03:00
Saryev Rustam
6d72db88bb fix send_query 2023-06-30 17:51:00 +03:00
Saryev Rustam
761210d861 Add support for loading files with extensions not present in ALLOW_FILES in LOADER_MAPPING. 2023-06-30 17:34:02 +03:00
Saryev Rustam
e22c850cae fix loading files bug 2023-06-29 10:24:47 +03:00
Saryev Rustam
70fee6d501 implement embedding_search method, and modify send_query and add temperature config 2023-06-29 04:09:42 +03:00
Saryev Rustam
2a6c5ad707
Update pyproject.toml 2023-06-17 03:23:13 +03:00
Saryev Rustam
ca7f681840 Update poetry version and consts.py defaults. 2023-06-14 14:23:29 +03:00
Saryev Rustam
703b98ea82 use default gpt-3.5-turbo-0613 2023-06-14 13:58:25 +03:00
Saryev Rustam
c092d2d0d8 feat: Add Halo spinner to load_files function 2023-05-31 01:09:38 +03:00
Saryev Rustam
25b00a4aa0 fix(cli): update config_path variable 2023-05-30 13:51:46 +03:00
Saryev Rustam
675e7778ff feat: add file loading with multiprocessing
- Add multiprocessing to load files in parallel
- Update loader mapping to handle various file types
2023-05-30 11:12:32 +03:00
Saryev Rustam
572ee1bf4e Merge pull request #1 from rsaryev/feat/local
GPT4All
2023-05-30 09:24:38 +03:00
Saryev Rustam
f9a31937bb Refactored CLI and LLM classes
- Refactored the CLI and LLM classes to improve code organization and readability.
- Added a  function to create an LLM instance based on the config.
- Moved the  function to the  and  classes.
- Added a  function to handle loading an existing vector store.
- Added a  function to estimate the cost of creating a vector store for OpenAI models.
- Updated the  function to prompt for the model type and path or API key depending on the type.
- Updated the  function to use the  function and  method of the LLM instance.
- Updated the default config to include default values for  and .
- Added a  constant to store the default config values.
- Added a  constant to store the default model path.
2023-05-30 01:12:31 +03:00
Saryev Rustam
9b9a834941 Update text splitter to use recursive character text splitter
- Update  to use  instead of  for splitting documents.
- This change was made to improve document splitting and increase the number of documents available for processing.
2023-05-29 15:26:14 +03:00
Saryev Rustam
df0fc45330 Refactor file loading logic for better performance
- Refactored the file loading logic in  function to use  instead of  which should improve performance
- Removed unnecessary checks for empty directories and ignored files
- Simplified the logic for checking allowed and excluded files
2023-05-29 15:07:01 +03:00
Saryev Rustam
b63f432239 Refactor CharacterTextSplitter instantiation
- Refactor the instantiation of CharacterTextSplitter in llm.py to remove unnecessary parameters.
2023-05-29 03:29:33 +03:00
Saryev Rustam
a6aabe7d1f feat: add option to use existing vector store
- Added functionality to check for an existing vector store and prompt the user to use it
- If the user approves, the function returns the existing vector store instead of creating a new one
- Also added saving of the vector store to local storage
2023-05-29 02:28:17 +03:00
Saryev Rustam
ff4dc0d005 fix: calculation cost 2023-05-28 23:41:23 +03:00
Saryev Rustam
963e2fecfb feat: add Halo spinner when creating vector store
- Add Halo spinner when creating vector store to provide visual feedback to the user.
- Also update success message to reflect the number of documents in the vector store.
2023-05-27 17:29:46 +03:00
Saryev Rustam
d1b0c0a796 feat: add cost calculation and approval prompt for vector store creation
- Add  function to calculate the cost of creating a vector store for given documents
- Prompt user to approve the cost before creating the vector store
- Exit if user does not approve the cost
2023-05-27 17:13:39 +03:00
Saryev Rustam
11185c079f refactor: simplify chat loop
- Simplify the chat loop by moving it to a separate function called 'loop'.
- This improves code readability and makes it easier to test the chat function without needing to mock user input.
2023-05-27 05:37:31 +03:00
Saryev Rustam
5f08927c76 Refactor text loader and add repo check
- Refactored the text loader in utils.py to improve readability and maintainability.
- Added a new function called  in utils.py to check if the current directory is a git repository.
- Modified the  function in utils.py to ignore files and directories ignored by git.
- Added error handling to  function in utils.py to return  if the current directory is not a git repository.
2023-05-27 04:20:40 +03:00
Saryev Rustam
569b7d9536 Add spinner to indicate loading files
- Added a spinner using the Halo package to indicate when files are being loaded.
- The spinner appears while the load_files function is running and disappears once the function has completed.
- This should improve the user experience by providing feedback that the program is actively doing something.
2023-05-26 21:38:24 +03:00
Saryev Rustam
589a419391 refactor: update function names and remove unused imports
- Refactor cli.py and llm.py to use new function names for create_vector_store and send_question respectively
- Remove unused imports in utils.py
2023-05-26 14:59:40 +03:00
Saryev Rustam
226203e4d9 Add CLI for chatting with OpenAI model
- Add CLI functionality for chatting with OpenAI model
- Implement  function to allow users to input OpenAI API key and model name
- Implement  function to allow users to chat with OpenAI model using retrieved documents
- Add  module to handle sending questions to OpenAI model
- Add  module to load and split text documents, create retriever, and define StreamStdOut callback class
2023-05-26 04:47:22 +03:00