chore: use poetry as dependency manager (#242)

* Adopts [Poetry](https://python-poetry.org/) as a dependency manager
* Introduces dependency version requirements
* Deprecates Python 3.7 support

**TODO**
- [x] Update developer guide
- [x] Add back `playwright`, `manifest-ml`, and `jupyter` to dependency
group

**Not Doing => Fast Follow**
- Investigate single source for version, perhaps relying on GitHub tags
and [tackling this
issue](https://github.com/hwchase17/langchain/issues/26)
harrison/fix_logging_api
Steven Hoelscher 1 year ago committed by GitHub
parent 988cb51a7c
commit 98fb19b535
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,23 +1,35 @@
name: lint
on: [push, pull_request]
on:
push:
branches: [main]
pull_request:
env:
POETRY_VERSION: "1.2.0"
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7"]
python-version:
- "3.8"
- "3.9"
- "3.10"
steps:
- uses: actions/checkout@v3
- name: Install poetry
run: |
pipx install poetry==$POETRY_VERSION
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: poetry
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r test_requirements.txt
poetry install
- name: Analysing the code with our lint
run: |
make lint

@ -1,23 +1,33 @@
name: test
on: [push, pull_request]
on:
push:
branches: [main]
pull_request:
env:
POETRY_VERSION: "1.2.0"
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7"]
python-version:
- "3.8"
- "3.9"
- "3.10"
steps:
- uses: actions/checkout@v3
- name: Install poetry
run: pipx install poetry==$POETRY_VERSION
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r test_requirements.txt
run: poetry install
- name: Run unit tests
run: |
make tests

@ -1,3 +0,0 @@
include langchain/py.typed
include langchain/VERSION
include LICENSE

@ -1,17 +1,17 @@
.PHONY: format lint tests integration_tests
format:
black .
isort .
poetry run black .
poetry run isort .
lint:
mypy .
black . --check
isort . --check
flake8 .
poetry run mypy .
poetry run black . --check
poetry run isort . --check
poetry run flake8 .
tests:
pytest tests/unit_tests
poetry run pytest tests/unit_tests
integration_tests:
pytest tests/integration_tests
poetry run pytest tests/integration_tests

@ -99,27 +99,90 @@ both at a short term but also at a long term level. The concept of "Memory" exis
## 🤖 Developer Guide
To begin developing on this project, first clone to the repo locally.
To install requirements, run `pip install -r requirements.txt`.
This will install all requirements for running the package, examples, linting, formatting, and tests.
To begin developing on this project, first clone the repo locally.
### Quick Start
This project uses [Poetry](https://python-poetry.org/) as a dependency manager. Check out Poetry's own [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.
To install requirements:
```bash
poetry install -E all
```
This will install all requirements for running the package, examples, linting, formatting, and tests. Note the `-E all` flag will install all optional dependencies necessary for integration testing.
Now, you should be able to run the common tasks in the following section.
### Common Tasks
#### Code Formatting
Formatting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/) and [isort](https://pycqa.github.io/isort/).
To run formatting for this project, run `make format`.
To run formatting for this project:
```bash
make format
```
#### Linting
Linting for this project is a combination of [Black](https://black.readthedocs.io/en/stable/), [isort](https://pycqa.github.io/isort/), [flake8](https://flake8.pycqa.org/en/latest/), and [mypy](http://mypy-lang.org/).
To run linting for this project, run `make lint`.
To run linting for this project:
```bash
make lint
```
We recognize linting can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed.
#### Testing
Unit tests cover modular logic that does not require calls to outside apis.
To run unit tests, run `make tests`.
To run unit tests:
```bash
make tests
```
If you add new logic, please add a unit test.
Integration tests cover logic that requires making calls to outside APIs (often integration with other services).
To run integration tests, run `make integration_tests`.
To run integration tests:
```bash
make integration_tests
```
If you add support for a new external API, please add a new integration test.
If you are adding a Jupyter notebook example, you can run `pip install -e .` to build the langchain package from your local changes, so your new logic can be imported into the notebook.
#### Adding a Jupyter Notebook
If you are adding a Jupyter notebook example, you'll want to install the optional `dev` dependencies.
To install dev dependencies:
```bash
poetry install --with dev
```
Launch a notebook:
```bash
poetry run jupyter notebook
```
When you run `poetry install`, the `langchain` package is installed as editable in the virtualenv, so your new logic can be imported into the notebook.
#### Contribute Documentation
Docs are largely autogenerated by [sphinx](https://www.sphinx-doc.org/en/master/) from the code.
For that reason, we ask that you add good documentation to all classes and methods.
Similar to linting, we recognize documentation can be annoying - if you do not want to do it, please contact a project maintainer and they can help you with it. We do not want this to be a blocker for good code getting contributed.

3487
poetry.lock generated

File diff suppressed because it is too large Load Diff

@ -0,0 +1,2 @@
[virtualenvs]
in-project = true

@ -1,3 +1,53 @@
[tool.poetry]
name = "langchain"
version = "0.0.26"
description = "Building applications with LLMs through composability"
authors = []
license = "MIT"
readme = "README.md"
repository = "https://www.github.com/hwchase17/langchain"
[tool.poetry.dependencies]
python = ">=3.8.1,<4.0"
pydantic = "^1"
SQLAlchemy = "^1"
requests = "^2"
PyYAML = "^6"
numpy = "^1"
faiss-cpu = {version = "^1", optional = true}
wikipedia = {version = "^1", optional = true}
elasticsearch = {version = "^8", optional = true}
manifest-ml = {version = "^0.0.1", optional = true}
spacy = {version = "^3", optional = true}
nltk = {version = "^3", optional = true}
transformers = {version = "^4", optional = true}
[tool.poetry.group.test.dependencies]
pytest = "^7.2.0"
pytest-dotenv = "^0.5.2"
[tool.poetry.group.lint.dependencies]
flake8-docstrings = "^1.6.0"
black = "^22.10.0"
isort = "^5.10.1"
flake8 = "^6.0.0"
[tool.poetry.group.typing.dependencies]
mypy = "^0.991"
types-pyyaml = "^6.0.12.2"
types-requests = "^2.28.11.5"
[tool.poetry.group.dev]
optional = true
[tool.poetry.group.dev.dependencies]
jupyter = "^1.0.0"
playwright = "^1.28.0"
[tool.poetry.extras]
llms = ["cohere", "openai", "nlpcloud", "huggingface_hub", "manifest-ml"]
all = ["cohere", "openai", "nlpcloud", "huggingface_hub", "manifest-ml", "elasticsearch", "google-search-results", "faiss-cpu", "sentence_transformers", "transformers", "spacy", "nltk", "wikipedia"]
[tool.isort]
profile = "black"
@ -5,3 +55,7 @@ profile = "black"
ignore_missing_imports = "True"
disallow_untyped_defs = "True"
exclude = ["notebooks"]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

@ -1,7 +0,0 @@
-r test_requirements.txt
-e '.[all]'
# For trickier integrations
playwright
manifest-ml
# For development
jupyter

@ -1,40 +0,0 @@
"""Set up the package."""
from pathlib import Path
from setuptools import find_packages, setup
with open(Path(__file__).absolute().parents[0] / "langchain" / "VERSION") as _f:
__version__ = _f.read().strip()
with open("README.md", "r", encoding="utf-8") as f:
long_description = f.read()
LLM_DEPENDENCIES = ["cohere", "openai", "nlpcloud", "huggingface_hub"]
OTHER_DEPENDENCIES = [
"elasticsearch",
"google-search-results",
"wikipedia",
"faiss-cpu",
"sentence_transformers",
"transformers",
"spacy",
"nltk",
]
setup(
name="langchain",
version=__version__,
packages=find_packages(),
description="Building applications with LLMs through composability",
install_requires=["pydantic", "sqlalchemy", "numpy", "requests", "pyyaml"],
long_description=long_description,
license="MIT",
url="https://github.com/hwchase17/langchain",
include_package_data=True,
long_description_content_type="text/markdown",
extras_require={
"llms": LLM_DEPENDENCIES,
"all": LLM_DEPENDENCIES + OTHER_DEPENDENCIES,
},
)

@ -1,12 +0,0 @@
-e .
# For testing
pytest
pytest-dotenv
# For linting
black
isort
mypy
flake8
flake8-docstrings
types-requests
types-PyYAML
Loading…
Cancel
Save