Commit Graph

83 Commits

Author SHA1 Message Date
Anton Larin
ecfbc7b9fd count coverage 2023-08-16 16:35:48 +02:00
Anton Larin
98a97f34f5 fix packaging and imports and introduce tests with pytest.
still issues with celery worker.
2023-08-14 18:20:25 +02:00
Alex
33dce10bc3
Merge pull request #296 from larinam/revert_breaking_renaming_azure_change
Revert "Changed environment variable names OPENAI_API_BASE and OPENAI…
2023-08-08 18:15:45 +01:00
Alex
feed0b288f
Merge pull request #297 from larinam/upgrade_scripts_requirements
upgrade versions and remove transitive and unnecessary dependencies for scripts subproject
2023-08-05 22:10:49 +01:00
Anton Larin
bb1a42df91 upgrade versions and remove transitive and unnecessary dependencies 2023-08-05 15:01:53 +02:00
Anton Larin
ac5ac3e9f1 Revert "Changed environment variable names OPENAI_API_BASE and OPENAI_API_VERSION to AZURE_OPENAI_API_BASE and AZURE_OPENAI_API_VERSION"
This reverts commit ce8b29e9d0.
2023-08-05 14:08:51 +02:00
Anton Larin
bed25b317c Fix min_tokens logic for grouping documents: documents with (lengh >= min_tokens) should not be grouped into one document for indexing 2023-08-05 13:18:52 +02:00
Alex
a64a30c088 fix 2023-07-24 16:23:49 +01:00
Alex
dac76a867f fix tokens for header 2023-07-24 16:14:08 +01:00
dependabot[bot]
b1b610f4b5
Bump cryptography from 41.0.0 to 41.0.2 in /scripts
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.0 to 41.0.2.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/41.0.0...41.0.2)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-07-15 15:21:20 +00:00
Rik Schoonbeek
ce8b29e9d0 Changed environment variable names OPENAI_API_BASE and OPENAI_API_VERSION to AZURE_OPENAI_API_BASE and AZURE_OPENAI_API_VERSION 2023-07-12 17:37:56 +02:00
Alex
2f1c3075a2
Merge pull request #253 from arc53/dependabot/pip/scripts/cryptography-41.0.0
Bump cryptography from 39.0.2 to 41.0.0 in /scripts
2023-06-28 00:40:37 +07:00
Alex
5916f92f1a
Merge pull request #245 from arc53/dependabot/pip/scripts/requests-2.31.0
Bump requests from 2.28.2 to 2.31.0 in /scripts
2023-06-28 00:27:04 +07:00
Idan
897b4ef2cd Fixed a bug with reading md files 2023-06-23 14:57:29 +03:00
Anton Larin
84168e22d0 add missing variable after testin and minor fixes. 2023-06-17 16:09:22 +02:00
Alex
f6b6c2e9a3 Update requirements.txt 2023-06-16 01:36:45 +01:00
dependabot[bot]
8a98789be1
Bump transformers from 4.26.0 to 4.30.0 in /scripts
Bumps [transformers](https://github.com/huggingface/transformers) from 4.26.0 to 4.30.0.
- [Release notes](https://github.com/huggingface/transformers/releases)
- [Commits](https://github.com/huggingface/transformers/compare/v4.26.0...v4.30.0)

---
updated-dependencies:
- dependency-name: transformers
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-15 16:53:39 +00:00
Anton Larin
6d5b698c39 fix arc53/DocsGPT#199 2023-06-03 11:04:04 +02:00
Anton Larin
dd9f1abcea fix arc53/DocsGPT#199 2023-06-03 11:03:44 +02:00
Anton Larin
b4bd34fb96 fix arc53/DocsGPT#199 2023-06-03 10:58:31 +02:00
dependabot[bot]
36ed69b07e
Bump cryptography from 39.0.2 to 41.0.0 in /scripts
Bumps [cryptography](https://github.com/pyca/cryptography) from 39.0.2 to 41.0.0.
- [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/cryptography/compare/39.0.2...41.0.0)

---
updated-dependencies:
- dependency-name: cryptography
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:20:12 +00:00
dependabot[bot]
ec4fc17e3a
Bump requests from 2.28.2 to 2.31.0 in /scripts
Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-23 06:20:49 +00:00
Nazih Kalo
da5d62cc1c updating the bulk ingest file metadata to account for parsers that output lists 2023-05-19 10:29:18 -07:00
Alex
e49dd0cc6a metadata on ingestion 2023-05-17 21:41:24 +01:00
Alex
059ffe09ea
Merge pull request #232 from larinam/lint
Lint
2023-05-15 13:53:09 +01:00
GH Action - Upstream Sync
ce6f0dab56 Merge branch 'main' of https://github.com/arc53/DocsGPT 2023-05-15 12:05:18 +00:00
Alex
3001688e0e Update requirements.txt 2023-05-15 12:46:39 +01:00
dependabot[bot]
b28676d52c
Bump flask from 2.2.2 to 2.2.5 in /scripts
Bumps [flask](https://github.com/pallets/flask) from 2.2.2 to 2.2.5.
- [Release notes](https://github.com/pallets/flask/releases)
- [Changelog](https://github.com/pallets/flask/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/flask/compare/2.2.2...2.2.5)

---
updated-dependencies:
- dependency-name: flask
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-15 11:00:19 +00:00
Anton Larin
962becb9a5
Linting
* validate python formatting on every build with Ruff
* fix lint warnings
2023-05-13 10:36:17 +02:00
Anton Larin
168648e789 Proper PEP8 formatting 2023-05-12 12:02:25 +02:00
dependabot[bot]
80dfdd1cb9
Bump redis from 4.5.3 to 4.5.4 in /scripts
Bumps [redis](https://github.com/redis/redis-py) from 4.5.3 to 4.5.4.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.3...v4.5.4)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 11:49:21 +00:00
dependabot[bot]
b7f1a94ba4
Bump redis from 4.5.1 to 4.5.3 in /scripts
Bumps [redis](https://github.com/redis/redis-py) from 4.5.1 to 4.5.3.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](https://github.com/redis/redis-py/compare/v4.5.1...v4.5.3)

---
updated-dependencies:
- dependency-name: redis
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-27 21:34:50 +00:00
Pavel
ce8f0ef9e1
Merge pull request #168 from arc53/feature/backend-uploads
Feature/backend uploads
2023-03-14 19:09:37 +04:00
Pavel
c9e1c326f5 - index.plk 2023-03-14 17:56:42 +04:00
Pavel
4532b6cd8c print minus 2023-03-14 17:49:57 +04:00
Pavel
53424a5c19 Added cli commands 2023-03-14 17:33:19 +04:00
Pavel
b6c02c850a token ingeest 2023-03-14 13:32:29 +04:00
Alex
20a0800aa7 Create test_ingestion.py 2023-03-13 17:37:01 +00:00
Pavel
bac25112b7 v1 2023-03-13 19:14:33 +04:00
Alex
1d2162705d uploads backend first 2023-03-13 14:20:03 +00:00
Alex
ac0224b687 mdx format 2023-03-08 23:16:20 +00:00
Alex
0799728000 Create requirements.txt 2023-03-08 11:44:02 +00:00
Alex
1f02f3b376 Update rst_parser.py 2023-03-08 11:32:44 +00:00
Alex
f7d7244588 chunks rst 2023-03-08 00:07:53 +00:00
Alex
63e35fbdbc env vars 2023-03-01 14:16:11 +00:00
Alex
1af4ca2340
Merge pull request #129 from arc53/code-ingestion
Code_to_dict
2023-02-25 13:52:14 +00:00
Pavel
2c364d3c00 Code_to_dict
3 languages added, works well with python. Java and Js require additional revieving
2023-02-25 17:37:33 +04:00
Alex
5c2a537393
Merge pull request #116 from arc53/code-ingestion
Code ingestion
2023-02-22 18:46:50 +00:00
Pavel
0fb28e5213 Calc + structure 2023-02-22 21:19:13 +04:00
Manan
524e0f6f01 fix | Chunk creation error when
title not the first element in HTML
2023-02-22 20:20:54 +05:30