Archives/DocsGPT
2
0
Fork 0
mirror of https://github.com/arc53/DocsGPT synced 2024-11-02 03:40:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,884 Commits 45 Branches 12 Tags 94 MiB
70bb9477c5
Commit Graph

105 Commits

Author SHA1 Message Date
Anton Larin
962becb9a5
Linting 
* validate python formatting on every build with Ruff
* fix lint warnings
 2023-05-13 10:36:17 +02:00
Anton Larin
168648e789 Proper PEP8 formatting 2023-05-12 12:02:25 +02:00
dependabot[bot]
80dfdd1cb9
Bump redis from 4.5.3 to 4.5.4 in /scripts 
Bumps [redis](https://github.com/redis/redis-py) from 4.5.3 to 4.5.4.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.3...v4.5.4)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-30 11:49:21 +00:00
dependabot[bot]
b7f1a94ba4
Bump redis from 4.5.1 to 4.5.3 in /scripts 
Bumps [redis](https://github.com/redis/redis-py) from 4.5.1 to 4.5.3.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.1...v4.5.3)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-03-27 21:34:50 +00:00
Pavel
ce8f0ef9e1
Merge pull request #168 from arc53/feature/backend-uploads 
Feature/backend uploads
 2023-03-14 19:09:37 +04:00
Pavel
c9e1c326f5 - index.plk 2023-03-14 17:56:42 +04:00
Pavel
4532b6cd8c print minus 2023-03-14 17:49:57 +04:00
Pavel
53424a5c19 Added cli commands 2023-03-14 17:33:19 +04:00
Pavel
b6c02c850a token ingeest 2023-03-14 13:32:29 +04:00
Alex
20a0800aa7 Create test_ingestion.py 2023-03-13 17:37:01 +00:00
Pavel
bac25112b7 v1 2023-03-13 19:14:33 +04:00
Alex
1d2162705d uploads backend first 2023-03-13 14:20:03 +00:00
Alex
ac0224b687 mdx format 2023-03-08 23:16:20 +00:00
Alex
0799728000 Create requirements.txt 2023-03-08 11:44:02 +00:00
Alex
1f02f3b376 Update rst_parser.py 2023-03-08 11:32:44 +00:00
Alex
f7d7244588 chunks rst 2023-03-08 00:07:53 +00:00
Alex
63e35fbdbc env vars 2023-03-01 14:16:11 +00:00
Alex
1af4ca2340
Merge pull request #129 from arc53/code-ingestion 
Code_to_dict
 2023-02-25 13:52:14 +00:00
Pavel
2c364d3c00 Code_to_dict 
3 languages added, works well with python. Java and Js require additional revieving
 2023-02-25 17:37:33 +04:00
Alex
5c2a537393
Merge pull request #116 from arc53/code-ingestion 
Code ingestion
 2023-02-22 18:46:50 +00:00
Pavel
0fb28e5213 Calc + structure 2023-02-22 21:19:13 +04:00
Manan
524e0f6f01 fix | Chunk creation error when 
title not the first element in HTML
 2023-02-22 20:20:54 +05:30
Manan
16eb503e36 Added HTML Support. read, clean-up, filter return 2023-02-21 23:06:00 +05:30
Manan
e8baa46eb6 Merge branch 'main' of https://github.com/arc53/DocsGPT into main 2023-02-21 22:11:57 +05:30
Alex
c92f5dba32 New docs gen 2023-02-20 17:15:18 +00:00
Alex
962be4d8ec
Merge pull request #109 from arc53/main 
updates
 2023-02-20 16:53:15 +00:00
Manan
d0b472ad38 Implemented html_parser: cleaning & chunk creation 2023-02-19 01:53:16 +05:30
Alex
8a5e1e8d98 cleanups 2023-02-15 20:29:09 +00:00
Alex
4d1ff8238d switching between llms 2023-02-15 18:40:23 +00:00
Alex
f9fe3f2f48 Merge branch 'main' into custom-llm 2023-02-15 14:42:57 +00:00
EricGao888
aeac186484 Add retry strategy to increase stability 2023-02-15 17:29:39 +08:00
Pavel
d57c7b0296 -y-description 2023-02-15 13:10:30 +04:00
冯不游
b83589a308 feat: add support for directory list 
example: `python ingest.py --dir inputs1 --dir another --dir ../inputs`,
the outputs will be in `outputs/input_folder_name/`
 2023-02-15 02:30:39 +08:00
Alex
d4ede13747 Merge branch 'main' into code-ingestion 2023-02-14 16:47:56 +00:00
Alex
5883ce2685
Merge pull request #87 from arc53/ingest-cli 
Ingest cli
 2023-02-14 16:20:05 +00:00
Pavel
af20c7298a new-ingest 
Ingest with a CLI
 2023-02-14 19:37:07 +04:00
冯不游
636783ca8a fix: avoid second error issue 2023-02-14 22:29:17 +08:00
冯不游
458f2a3ff3 fix: restore index back when continue process 2023-02-14 22:05:16 +08:00
Alex
046fbebf56 Enable other llm's 2023-02-14 13:06:28 +00:00
冯不游
3ab02ca959 feat: compatible with markdown 2023-02-13 11:00:34 +08:00
Alex
e88ff885fe
Merge pull request #75 from arc53/rst-interpreters 2023-02-12 18:32:20 +00:00
Pavel
b1a6ebffba Directives + Interpreted 
Some additional filters for rst parsing
 2023-02-12 22:29:40 +04:00
Alex
205be538a3 fix dbqa, with new chain type, also fix for doc export 2023-02-12 17:58:54 +00:00
Alex
9228005a7e chunked embedding 2023-02-12 16:25:01 +00:00
vintro
2a203aa547
Create __init__.py 
otherwise running `python ingest.py` complains about `parser` not being a package
 2023-02-10 19:49:00 -05:00
Alex
37ad3b35c8 Update ingest.py 2023-02-10 17:30:37 +00:00
Alex
d642782a5a move folder 2023-02-10 16:10:53 +00:00
Pavel
8c4fcff617 requirement 2023-02-10 19:47:51 +04:00
Pavel
79b5ef9c14 Bulk ingest 
Added a method based on indexGPT folder ingester. Additional rst reader included.
 2023-02-10 19:44:42 +04:00
Alex
605c599b5d Create code_docs_gen.py 2023-02-08 19:06:23 +00:00
Patrick Shriwise
64fb36b3de Adding location argument to ingest scripts 2023-02-07 10:20:07 -06:00
Alex
08215248d7 Inputs folder change 2023-02-07 10:52:03 +00:00
monkish54
c94866e9e9 Add cost estimate feature 
Calculates number of tokens/user cost and requires user permission to proceed.

User permission bypass is built-in to allow for non-human users.
 2023-02-06 18:32:06 -08:00
Pavel
1c734727a1 Ingest rst with sphinx 
Transforms all rst files in provided folder to txt format first (utilising sphinx library). In my tests size of raw sample decreased 2-3 times.
 2023-02-06 23:43:23 +04:00
Alex
b71a9bf5ee init2 2023-02-03 12:45:29 +00:00