chore: use poetry as dependency manager (#242)

* Adopts [Poetry](https://python-poetry.org/) as a dependency manager
* Introduces dependency version requirements
* Deprecates Python 3.7 support

**TODO**
- [x] Update developer guide
- [x] Add back `playwright`, `manifest-ml`, and `jupyter` to dependency
group

**Not Doing => Fast Follow**
- Investigate single source for version, perhaps relying on GitHub tags
and [tackling this
issue](https://github.com/hwchase17/langchain/issues/26)
harrison/fix_logging_api
Steven Hoelscher 1 year ago committed by GitHub
parent 988cb51a7c
commit 98fb19b535
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,23 +1,35 @@
name: lint name: lint
on: [push, pull_request] on:
push:
branches: [main]
pull_request:
env:
POETRY_VERSION: "1.2.0"
jobs: jobs:
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: ["3.7"] python-version:
- "3.8"
- "3.9"
- "3.10"
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Install poetry
run: |
pipx install poetry==$POETRY_VERSION
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3 uses: actions/setup-python@v4
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
cache: poetry
- name: Install dependencies - name: Install dependencies
run: | run: |
python -m pip install --upgrade pip poetry install
pip install -r test_requirements.txt
- name: Analysing the code with our lint - name: Analysing the code with our lint
run: | run: |
make lint make lint

@ -1,23 +1,33 @@
name: test name: test
on: [push, pull_request] on:
push:
branches: [main]
pull_request:
env:
POETRY_VERSION: "1.2.0"
jobs: jobs:
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy: strategy:
matrix: matrix:
python-version: ["3.7"] python-version:
- "3.8"
- "3.9"
- "3.10"
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3
- name: Install poetry
run: pipx install poetry==$POETRY_VERSION
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3 uses: actions/setup-python@v4
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
cache: 'poetry'
- name: Install dependencies - name: Install dependencies
run: | run: poetry install
python -m pip install --upgrade pip
pip install -r test_requirements.txt
- name: Run unit tests - name: Run unit tests
run: | run: |
make tests make tests

@ -1,3 +0,0 @@
include langchain/py.typed
include langchain/VERSION
include LICENSE

@ -1,17 +1,17 @@
.PHONY: format lint tests integration_tests .PHONY: format lint tests integration_tests
format: format:
black . poetry run black .
isort . poetry run isort .
lint: lint:
mypy . poetry run mypy .
black . --check poetry run black . --check
isort . --check poetry run isort . --check
flake8 . poetry run flake8 .
tests: tests:
pytest tests/unit_tests poetry run pytest tests/unit_tests
integration_tests: integration_tests:
pytest tests/integration_tests poetry run pytest tests/integration_tests

@ -99,27 +99,90 @@ both at a short term but also at a long term level. The concept of "Memory" exis
## 🤖 Developer Guide ## 🤖 Developer Guide
To begin developing on this project, first clone to the repo locally. To begin developing on this project, first clone the repo locally.
To install requirements, run `pip install -r requirements.txt`.
This will install all requirements for running the package, examples, linting, formatting, and tests. ### Quick Start
This project uses [Poetry](https://python-poetry.org/) as a dependency manager. Check out Poetry's own [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.
To install requirements:
```bash
poetry install -E all
```
This will install all requirements for running the package, examples, linting, formatting, and tests. Note the `-E all` flag will install all optional dependencies necessary for integration testing.
Now, you should be able to run the common tasks in the following section.
### Common Tasks
#### Code Formatting
Formatting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/) and [isort](https://pycqa.github.io/isort/). Formatting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/) and [isort](https://pycqa.github.io/isort/).
To run formatting for this project, run `make format`.
To run formatting for this project:
```bash
make format
```
#### Linting
Linting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/), [isort](https://pycqa.github.io/isort/), [flake8](https://flake8.pycqa.org/en/latest/), and [mypy](http://mypy-lang.org/). Linting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/), [isort](https://pycqa.github.io/isort/), [flake8](https://flake8.pycqa.org/en/latest/), and [mypy](http://mypy-lang.org/).
To run linting for this project, run `make lint`.
To run linting for this project:
```bash
make lint
```
We recognize linting can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed. We recognize linting can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed.
#### Testing
Unit tests cover modular logic that does not require calls to outside apis. Unit tests cover modular logic that does not require calls to outside apis.
To run unit tests, run `make tests`.
To run unit tests:
```bash
make tests
```
If you add new logic, please add a unit test. If you add new logic, please add a unit test.
Integration tests cover logic that requires making calls to outside APIs (often integration with other services). Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
To run integration tests, run `make integration_tests`.
To run integration tests:
```bash
make integration_tests
```
If you add support for a new external API, please add a new integration test. If you add support for a new external API, please add a new integration test.
If you are adding a Jupyter notebook example, you can run `pip install -e .` to build the langchain package from your local changes, so your new logic can be imported into the notebook. #### Adding a Jupyter Notebook
If you are adding a Jupyter notebook example, you'll want to install the optional `dev` dependencies.
To install dev dependencies:
```bash
poetry install --with dev
```
Launch a notebook:
```bash
poetry run jupyter notebook
```
When you run `poetry install`, the `langchain` package is installed as editable in the virtualenv, so your new logic can be imported into the notebook.
#### Contribute Documentation
Docs are largely autogenerated by [sphinx](https://www.sphinx-doc.org/en/master/) from the code. Docs are largely autogenerated by [sphinx](https://www.sphinx-doc.org/en/master/) from the code.
For that reason, we ask that you add good documentation to all classes and methods. For that reason, we ask that you add good documentation to all classes and methods.
Similar to linting, we recognize documentation can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed. Similar to linting, we recognize documentation can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed.

3487
poetry.lock generated

File diff suppressed because it is too large Load Diff

@ -0,0 +1,2 @@
[virtualenvs]
in-project = true

@ -1,3 +1,53 @@
[tool.poetry]
name = "langchain"
version = "0.0.26"
description = "Building applications with LLMs through composability"
authors = []
license = "MIT"
readme = "README.md"
repository = "https://www.github.com/hwchase17/langchain"
[tool.poetry.dependencies]
python = ">=3.8.1,<4.0"
pydantic = "^1"
SQLAlchemy = "^1"
requests = "^2"
PyYAML = "^6"
numpy = "^1"
faiss-cpu = {version = "^1", optional = true}
wikipedia = {version = "^1", optional = true}
elasticsearch = {version = "^8", optional = true}
manifest-ml = {version = "^0.0.1", optional = true}
spacy = {version = "^3", optional = true}
nltk = {version = "^3", optional = true}
transformers = {version = "^4", optional = true}
[tool.poetry.group.test.dependencies]
pytest = "^7.2.0"
pytest-dotenv = "^0.5.2"
[tool.poetry.group.lint.dependencies]
flake8-docstrings = "^1.6.0"
black = "^22.10.0"
isort = "^5.10.1"
flake8 = "^6.0.0"
[tool.poetry.group.typing.dependencies]
mypy = "^0.991"
types-pyyaml = "^6.0.12.2"
types-requests = "^2.28.11.5"
[tool.poetry.group.dev]
optional = true
[tool.poetry.group.dev.dependencies]
jupyter = "^1.0.0"
playwright = "^1.28.0"
[tool.poetry.extras]
llms = ["cohere", "openai", "nlpcloud", "huggingface_hub", "manifest-ml"]
all = ["cohere", "openai", "nlpcloud", "huggingface_hub", "manifest-ml", "elasticsearch", "google-search-results", "faiss-cpu", "sentence_transformers", "transformers", "spacy", "nltk", "wikipedia"]
[tool.isort] [tool.isort]
profile = "black" profile = "black"
@ -5,3 +55,7 @@ profile = "black"
ignore_missing_imports = "True" ignore_missing_imports = "True"
disallow_untyped_defs = "True" disallow_untyped_defs = "True"
exclude = ["notebooks"] exclude = ["notebooks"]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

@ -1,7 +0,0 @@
-r test_requirements.txt
-e '.[all]'
# For trickier integrations
playwright
manifest-ml
# For development
jupyter

@ -1,40 +0,0 @@
"""Set up the package."""
from pathlib import Path
from setuptools import find_packages, setup
with open(Path(__file__).absolute().parents[0] / "langchain" / "VERSION") as _f:
__version__ = _f.read().strip()
with open("README.md", "r", encoding="utf-8") as f:
long_description = f.read()
LLM_DEPENDENCIES = ["cohere", "openai", "nlpcloud", "huggingface_hub"]
OTHER_DEPENDENCIES = [
"elasticsearch",
"google-search-results",
"wikipedia",
"faiss-cpu",
"sentence_transformers",
"transformers",
"spacy",
"nltk",
]
setup(
name="langchain",
version=__version__,
packages=find_packages(),
description="Building applications with LLMs through composability",
install_requires=["pydantic", "sqlalchemy", "numpy", "requests", "pyyaml"],
long_description=long_description,
license="MIT",
url="https://github.com/hwchase17/langchain",
include_package_data=True,
long_description_content_type="text/markdown",
extras_require={
"llms": LLM_DEPENDENCIES,
"all": LLM_DEPENDENCIES + OTHER_DEPENDENCIES,
},
)

@ -1,12 +0,0 @@
-e .
# For testing
pytest
pytest-dotenv
# For linting
black
isort
mypy
flake8
flake8-docstrings
types-requests
types-PyYAML
Loading…
Cancel
Save