Commit Graph

14 Commits (main)

Author SHA1 Message Date
rsaryev 53de4f1fae
chore: update import statements in utils.py and llm.py to use langchain.vectorstores.FAISS 8 months ago
rsaryev b978a76402
Refactor get_repo() and load_files() functions to use Repo() without root_dir.Refactored `load_files` added a delay when creating vector store. 10 months ago
Saryev Rustam 761210d861 Add support for loading files with extensions not present in ALLOW_FILES in LOADER_MAPPING. 12 months ago
Saryev Rustam e22c850cae fix loading files bug 12 months ago
Saryev Rustam c092d2d0d8 feat: Add Halo spinner to load_files function 1 year ago
Saryev Rustam 675e7778ff feat: add file loading with multiprocessing
- Add multiprocessing to load files in parallel
- Update loader mapping to handle various file types
1 year ago
Saryev Rustam f9a31937bb Refactored CLI and LLM classes
- Refactored the CLI and LLM classes to improve code organization and readability.
- Added a  function to create an LLM instance based on the config.
- Moved the  function to the  and  classes.
- Added a  function to handle loading an existing vector store.
- Added a  function to estimate the cost of creating a vector store for OpenAI models.
- Updated the  function to prompt for the model type and path or API key depending on the type.
- Updated the  function to use the  function and  method of the LLM instance.
- Updated the default config to include default values for  and .
- Added a  constant to store the default config values.
- Added a  constant to store the default model path.
1 year ago
Saryev Rustam df0fc45330 Refactor file loading logic for better performance
- Refactored the file loading logic in  function to use  instead of  which should improve performance
- Removed unnecessary checks for empty directories and ignored files
- Simplified the logic for checking allowed and excluded files
1 year ago
Saryev Rustam a6aabe7d1f feat: add option to use existing vector store
- Added functionality to check for an existing vector store and prompt the user to use it
- If the user approves, the function returns the existing vector store instead of creating a new one
- Also added saving of the vector store to local storage
1 year ago
Saryev Rustam 11185c079f refactor: simplify chat loop
- Simplify the chat loop by moving it to a separate function called 'loop'.
- This improves code readability and makes it easier to test the chat function without needing to mock user input.
1 year ago
Saryev Rustam 5f08927c76 Refactor text loader and add repo check
- Refactored the text loader in utils.py to improve readability and maintainability.
- Added a new function called  in utils.py to check if the current directory is a git repository.
- Modified the  function in utils.py to ignore files and directories ignored by git.
- Added error handling to  function in utils.py to return  if the current directory is not a git repository.
1 year ago
Saryev Rustam 569b7d9536 Add spinner to indicate loading files
- Added a spinner using the Halo package to indicate when files are being loaded.
- The spinner appears while the load_files function is running and disappears once the function has completed.
- This should improve the user experience by providing feedback that the program is actively doing something.
1 year ago
Saryev Rustam 589a419391 refactor: update function names and remove unused imports
- Refactor cli.py and llm.py to use new function names for create_vector_store and send_question respectively
- Remove unused imports in utils.py
1 year ago
Saryev Rustam 226203e4d9 Add CLI for chatting with OpenAI model
- Add CLI functionality for chatting with OpenAI model
- Implement  function to allow users to input OpenAI API key and model name
- Implement  function to allow users to chat with OpenAI model using retrieved documents
- Add  module to handle sending questions to OpenAI model
- Add  module to load and split text documents, create retriever, and define StreamStdOut callback class
1 year ago