Commit Graph

68 Commits (04f824ea365898d23e00e77650e785e0c6d49b82)

Author SHA1 Message Date
冯不游 458f2a3ff3 fix: restore index back when continue process 2 years ago
Alex 046fbebf56 Enable other llm's 2 years ago
冯不游 3ab02ca959 feat: compatible with markdown 2 years ago
Alex e88ff885fe
Merge pull request #75 from arc53/rst-interpreters 2 years ago
Pavel b1a6ebffba Directives + Interpreted
Some additional filters for rst parsing
2 years ago
Alex 205be538a3 fix dbqa, with new chain type, also fix for doc export 2 years ago
Alex 9228005a7e chunked embedding 2 years ago
vintro 2a203aa547
Create __init__.py
otherwise running `python ingest.py` complains about `parser` not being a package
2 years ago
Alex 37ad3b35c8 Update ingest.py 2 years ago
Alex d642782a5a move folder 2 years ago
Pavel 8c4fcff617 requirement 2 years ago
Pavel 79b5ef9c14 Bulk ingest
Added a method based on indexGPT folder ingester. Additional rst reader included.
2 years ago
Alex 605c599b5d Create code_docs_gen.py 2 years ago
Patrick Shriwise 64fb36b3de Adding location argument to ingest scripts 2 years ago
Alex 08215248d7 Inputs folder change 2 years ago
monkish54 c94866e9e9 Add cost estimate feature
Calculates number of tokens/user cost and requires user permission to proceed.

User permission bypass is built-in to allow for non-human users.
2 years ago
Pavel 1c734727a1 Ingest rst with sphinx
Transforms all rst files in provided folder to txt format first (utilising sphinx library). In my tests size of raw sample decreased 2-3 times.
2 years ago
Alex b71a9bf5ee init2 2 years ago