28 Commits

Author	SHA1	Message	Date
Manan	e8baa46eb6	Merge branch 'main' of https://github.com/arc53/DocsGPT into main	2023-02-21 22:11:57 +05:30
Manan	d0b472ad38	Implemented html_parser: cleaning & chunk creation	2023-02-19 01:53:16 +05:30
Alex	8a5e1e8d98	cleanups	2023-02-15 20:29:09 +00:00
Alex	4d1ff8238d	switching between llms	2023-02-15 18:40:23 +00:00
Alex	f9fe3f2f48	Merge branch 'main' into custom-llm	2023-02-15 14:42:57 +00:00
EricGao888	aeac186484	Add retry strategy to increase stability	2023-02-15 17:29:39 +08:00
Pavel	d57c7b0296	-y-description	2023-02-15 13:10:30 +04:00
冯不游	b83589a308	feat: add support for directory list ... example: `python ingest.py --dir inputs1 --dir another --dir ../inputs`, the outputs will be in `outputs/input_folder_name/`	2023-02-15 02:30:39 +08:00
Alex	5883ce2685	Merge pull request #87 from arc53/ingest-cli ... Ingest cli	2023-02-14 16:20:05 +00:00
Pavel	af20c7298a	new-ingest ... Ingest with a CLI	2023-02-14 19:37:07 +04:00
冯不游	636783ca8a	fix: avoid second error issue	2023-02-14 22:29:17 +08:00
冯不游	458f2a3ff3	fix: restore index back when continue process	2023-02-14 22:05:16 +08:00
Alex	046fbebf56	Enable other llm's	2023-02-14 13:06:28 +00:00
冯不游	3ab02ca959	feat: compatible with markdown	2023-02-13 11:00:34 +08:00
Alex	e88ff885fe	Merge pull request #75 from arc53/rst-interpreters	2023-02-12 18:32:20 +00:00
Pavel	b1a6ebffba	Directives + Interpreted ... Some additional filters for rst parsing	2023-02-12 22:29:40 +04:00
Alex	205be538a3	fix dbqa, with new chain type, also fix for doc export	2023-02-12 17:58:54 +00:00
Alex	9228005a7e	chunked embedding	2023-02-12 16:25:01 +00:00
vintro	2a203aa547	Create __init__.py ... otherwise running `python ingest.py` complains about `parser` not being a package	2023-02-10 19:49:00 -05:00
Alex	37ad3b35c8	Update ingest.py	2023-02-10 17:30:37 +00:00
Alex	d642782a5a	move folder	2023-02-10 16:10:53 +00:00
Pavel	8c4fcff617	requirement	2023-02-10 19:47:51 +04:00
Pavel	79b5ef9c14	Bulk ingest ... Added a method based on indexGPT folder ingester. Additional rst reader included.	2023-02-10 19:44:42 +04:00
Patrick Shriwise	64fb36b3de	Adding location argument to ingest scripts	2023-02-07 10:20:07 -06:00
Alex	08215248d7	Inputs folder change	2023-02-07 10:52:03 +00:00
monkish54	c94866e9e9	Add cost estimate feature ... Calculates number of tokens/user cost and requires user permission to proceed. User permission bypass is built-in to allow for non-human users.	2023-02-06 18:32:06 -08:00
Pavel	1c734727a1	Ingest rst with sphinx ... Transforms all rst files in provided folder to txt format first (utilising sphinx library). In my tests size of raw sample decreased 2-3 times.	2023-02-06 23:43:23 +04:00
Alex	b71a9bf5ee	init2	2023-02-03 12:45:29 +00:00