langchain

Commit Graph

Author	SHA1	Message	Date
Leonid Ganeline	ea0982eede	update CONTRIBUTING.md (#11872 ) Adding description of the `View deployment` button on the PR page. This nice feature was not documented. --------- Co-authored-by: Erick Friis <erickfriis@gmail.com>	9 months ago
Bagatur	bd74eba152	add azure openai sched tests (#11723 )	9 months ago
Bagatur	eedfddac2d	Restructure docs (#11620 )	9 months ago
Nuno Campos	2c11302598	Update langchain_release.yml (#11444 )	9 months ago
Predrag Gruevski	88c5349196	Revert "Rm additional file check for scheduled tests (#11192 )" (#11297 ) This reverts commit `ff90bb59bf`. Requires #11296 to merge first.	9 months ago
Predrag Gruevski	37f2f71156	Trigger Docker release workflow after new langchain release is made. (#11290 ) We want to publish a new Docker image after a new langchain Python package version is published.	9 months ago
Predrag Gruevski	d21dd72d64	Upgrade CI workflows to poetry 1.6.1. (#11344 )	9 months ago
Eugene Yurtsev	2343302fc6	Remove langserve from langchain repo (#11288 ) LangServe has been moved to a separate repo	9 months ago
CG80499	943e4f30d8	Add scoring chain (#11123 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. It lives in `docs/extras` directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->	9 months ago
Bagatur	38d5b63a10	Bedrock scheduled tests (#11194 )	9 months ago
Nuno Campos	32a8b311eb	Add base docker image and ci script for building and pushing (#10927 )	9 months ago
Kazuki Maeda	a363ab5292	rename repo namespace to langchain-ai (#11259 ) ### Description renamed several repository links from `hwchase17` to `langchain-ai`. ### Why I discovered that the README file in the devcontainer contains an old repository name, so I took the opportunity to rename the old repository name in all files within the repository, excluding those that do not require changes. ### Dependencies none ### Tag maintainer @baskaryan ### Twitter handle [kzk_maeda](https://twitter.com/kzk_maeda)	9 months ago
Eugene Yurtsev	aebdb1ad01	Ignore aadd (#11235 )	9 months ago
Bagatur	ff90bb59bf	Rm additional file check for scheduled tests (#11192 ) cc @obi1kenobi Causing issues with GHA creds https://github.com/langchain-ai/langchain/actions/runs/6342674950/job/17228926776	9 months ago
Bagatur	3508e582f1	add anthropic scheduled tests and unit tests (#11188 )	9 months ago
Eugene Yurtsev	176d71dd85	LangServe: Add release workflow (#11178 ) Add release workflow to langserve	9 months ago
Eugene Yurtsev	b05bb9e136	LangServe (#11046 ) Adds LangServe package * Integrate Runnables with Fast API creating Server and a RemoteRunnable client * Support multiple runnables for a given server * Support sync/async/batch/abatch/stream/astream/astream_log on the client side (using async implementations on server) * Adds validation using annotations (relying on pydantic under the hood) -- this still has some rough edges -- e.g., open api docs do NOT generate correctly at the moment * Uses pydantic v1 namespace Known issues: type translation code doesn't handle a lot of types (e.g., TypedDicts) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com>	9 months ago
Bagatur	040d436b3f	Add vertex scheduled test (#10958 )	10 months ago
C.J. Jameson	b4d2663beb	CONTRIBUTING.md Quick Start: focus on langchain core; clarify docs and experimental are separate (#10906 ) follow up to https://github.com/langchain-ai/langchain/pull/7959 , explaining better to focus just on langchain core no dependencies twitter @cjcjameson	10 months ago
Bagatur	dccc20b402	add model feat table (#10921 )	10 months ago
Nino Risteski	d0070040da	Update CONTRIBUTING.md (#10700 ) fiixed few typos	10 months ago
Harrison Chase	2c957de2fc	add checks on basic base modules (#10693 )	10 months ago
Harrison Chase	5442d2b1fa	Harrison/stop importing from init (#10690 )	10 months ago
Leonid Ganeline	db3369272a	fixed PR template (#10515 ) @hwchase17	10 months ago
Predrag Gruevski	ccb9e3ee2d	Install dev, lint, test, typing extra deps for linting steps. (#10249 ) `mypy` cannot type-check code that relies on dependencies that aren't installed. Eventually we'll probably want to install as many optional dependencies as possible. However, the full "extended deps" setup for langchain creates a 3GB cache file and takes a while to unpack and install. We'll probably want something a bit more targeted. This is a first step toward something better.	10 months ago
Predrag Gruevski	82d5d4d0ae	Deny creating files as a result of test runs. (#10253 ) A test file was accidentally dropping a `results.json` file in the current working directory as a result of running `make test`. This is undesirable, since we don't want to risk accidentally adding stray files into the repo if we run tests locally and then do `git add .` without inspecting the file list very closely.	10 months ago
Predrag Gruevski	7fe8bf03a0	Final poetry action fix: manually recreate softlinks broken by caching. (#10250 ) It seems the caching action was not always correctly recreating softlinks. At first glance, the softlinks it created seemed fine, but they didn't always work. Possibly hitting some kind of underlying bug, but not particularly worth debugging in depth -- we can manually create the soft links we need.	10 months ago
Predrag Gruevski	619516260d	Re-enable poetry binary caching with fix and more logging. (#10244 ) - Revert "Temporarily disable step that seems to be transiently failing. (#10234)" - Refresh shell hashtable and show poetry/python location and version.	10 months ago
Predrag Gruevski	803be5b986	Run CI when CI infra itself has changed. (#10239 ) Make sure that changes to CI infrastructure get tested on CI before being merged. Without this PR, changes to the poetry setup action don't trigger a CI run and in principle could break `master` when merged.	10 months ago
Predrag Gruevski	e34ad6fefd	Temporarily disable step that seems to be transiently failing. (#10234 )	10 months ago
Harrison Chase	c0518be1f1	fix syntax (#10155 )	10 months ago
maks-operlejn-ds	a8f804a618	Add data anonymizer (#9863 ) ### Description The feature for anonymizing data has been implemented. In order to protect private data, such as when querying external APIs (OpenAI), it is worth pseudonymizing sensitive data to maintain full privacy. Anonynization consists of two steps: 1. Identification: Identify all data fields that contain personally identifiable information (PII). 2. Replacement: Replace all PIIs with pseudo values or codes that do not reveal any personal information about the individual but can be used for reference. We're not using regular encryption, because the language model won't be able to understand the meaning or context of the encrypted data. We use Microsoft Presidio together with Faker framework for anonymization purposes because of the wide range of functionalities they provide. The full implementation is available in `PresidioAnonymizer`. ### Future works - deanonymization - add the ability to reverse anonymization. For example, the workflow could look like this: `anonymize -> LLMChain -> deanonymize`. By doing this, we will retain anonymity in requests to, for example, OpenAI, and then be able restore the original data. - instance anonymization - at this point, each occurrence of PII is treated as a separate entity and separately anonymized. Therefore, two occurrences of the name John Doe in the text will be changed to two different names. It is therefore worth introducing support for full instance detection, so that repeated occurrences are treated as a single object. ### Twitter handle @deepsense_ai / @MaksOpp --------- Co-authored-by: MaksOpp <maks.operlejn@gmail.com> Co-authored-by: Bagatur <baskaryan@gmail.com>	10 months ago
Predrag Gruevski	9aaa0fdce0	Use unified Python setup steps for release workflow.	10 months ago
XUEYANZ	f97d3a76e7	Update CONTRIBUTING.md (#9817 ) <!-- Thank you for contributing to LangChain! Replace this entire comment with: - Description: a description of the change, - Issue: the issue # it fixes (if applicable), - Dependencies: any dependencies required for this change, - Tag maintainer: for a quicker response, tag the relevant maintainer (see below), - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out! Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally. See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md If you're adding a new integration, please include: 1. a test for the integration, preferably unit tests that do not rely on network access, 2. an example notebook showing its use. These live is docs/extras directory. If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin. --> Hi LangChain :) Thank you for such a great project! I was going through the CONTRIBUTING.md and found a few minor issues.	10 months ago
Predrag Gruevski	c06f34fa35	Use new Python setup approach for scheduled tests. (#9626 ) Using the same new unified Python setup as the regular tests and the lint job, as set up in #9625.	11 months ago
Predrag Gruevski	83986ea98a	Cache poetry install + unify Python/Poetry setup for lint and test jobs. (#9625 ) With this PR: - All lint and test jobs use the exact same Python + Poetry installation approach, instead of lints doing it one way and tests doing it another way. - The Poetry installation itself is cached, which saves ~15s per run. - We no longer pass shell commands as workflow arguments to a workflow that just runs them in a shell. This makes our actions more resilient to shell code injection. If y'all like this approach, I can modify the scheduled tests workflow and the release workflow to use this too.	11 months ago
seamusp	f3ba9ce7f4	Remove -E all from installation instructions (#9573 ) Update installation instructions to only install test dependencies rather than all dependencies. --------- Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>	11 months ago
Predrag Gruevski	35812d0096	Set up concurrency groups and workflow cancelation in CI. (#9564 ) If another push to the same PR or branch happens while its CI is still running, cancel the earlier run in favor of the next run. There's no point in testing an outdated version of the code. GitHub only allows a limited number of job runners to be active at the same time, so it's better to cancel pointless jobs early so that more useful jobs can run sooner.	11 months ago
Predrag Gruevski	3c7cc4d440	Test experimental package with `langchain` on `master` branch. (#9621 ) It's possible that langchain-experimental works fine with the latest published langchain, but is broken with the langchain on `master`. Unfortunately, you can see this is currently the case — this is why this PR also includes a minor fix for the `langchain` package itself. We want to catch situations like that before releasing a new langchain, hence this test.	11 months ago
Predrag Gruevski	acb54d8b9d	Reduce cache timeouts to ensure faster builds on timeout. (#9619 ) The current timeouts are too long, and mean that if the GitHub cache decides to act up, jobs get bogged down for 15min at a time. This has happened 2-3 times already this week -- a tiny fraction of our total workflows but really annoying when it happens to you. We can do better. Installing deps on cache miss takes about ~4min, so it's not worth waiting more than 4min for the deps cache. The black and mypy caches save 1 and 2min, respectively, so wait only up to that long to download them.	11 months ago
Predrag Gruevski	a1e89aa8d5	Explicitly add the `contents: write` permission for publishing releases. (#9617 )	11 months ago
Predrag Gruevski	c75e1aa5ed	Eliminate special-casing from test CI workflows. (#9562 ) The previous approach was relying on `_test.yml` taking an input parameter, and then doing almost completely orthogonal things for each parameter value. I've separated out each of those test situations as its own job or workflow file, which eliminated all the special-casing and, in my opinion, improved maintainability by making it much more obvious what code runs when.	11 months ago
Predrag Gruevski	6c308aabae	Use the GitHub-suggested safer pattern for shell interpolation. (#9567 ) Using `${{ }}` to construct shell commands is risky, since the `${{ }}` interpolation runs first and ignores shell quoting rules. This means that shell commands that look safely quoted, like `echo "${{ github.event.issue.title }}"`, are actually vulnerable to shell injection. More details here: https://github.blog/2023-08-09-four-tips-to-keep-your-github-actions-workflows-secure/	11 months ago
Predrag Gruevski	2a3758a98e	Reminder to not report security issues as "bug" type issues. (#9554 ) Updated the issue template that pops up when users open a new issue.	11 months ago
Predrag Gruevski	9f08d29bc8	Use PyPI Trusted Publishing to publish langchain packages. (#9467 ) Trusted Publishing is the current best practice for publishing Python packages. Rather than long-lived secret keys, it uses OpenID Connect (OIDC) to allow our GitHub runner to directly authenticate itself to PyPI and get a short-lived publishing token. This locks down publishing quite a bit: - There's no long-lived publish key to steal anymore. - Publishing is only allowed via the specifically designated GitHub workflow in the designated repo. It also is operationally easier: no keys means there's nothing that needs to be periodically rotated, nothing to worry about leaking, and nobody can accidentally publish a release from their laptop because they happened to have PyPI keys set up. After this gets merged, we'll need to configure PyPI to start expecting trusted publishing. It's only a few clicks and should only take a minute; instructions are here: https://docs.pypi.org/trusted-publishers/adding-a-publisher/ More info: - https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/ - https://github.com/pypa/gh-action-pypi-publish	11 months ago
Predrag Gruevski	249752e8ee	Require manually triggering release workflows. (#9552 )	11 months ago
Predrag Gruevski	875ea4b4c6	Fix conditional that erroneously always runs. (#9543 ) The input it means to test for is `"libs/langchain"` and not `"langchain"`.	11 months ago
Predrag Gruevski	a7eba8b006	Release on push to `master` instead of on closed PRs targeting it. (#9544 ) This is safer than the prior approach, since it's safe by default: the release workflows never get triggered for non-merged PRs, so there's no possibility of a buggy conditional accidentally letting a workflow proceed when it shouldn't have. The only loss is that publishing no longer requires a `release` label on the merged PR that bumps the version. We can add a separate CI step that enforces that part as a condition for merging into `master`, if desirable.	11 months ago
Predrag Gruevski	a03003f5fd	Upgrade CI poetry version to 1.5.1. (#9479 ) Poetry v1.5.1 was released on May 29, almost 3 months ago. Probably a safe upgrade.	11 months ago
Yuki Miyake	85a1c6d0b7	🐛 fix unexpected run of release workflow (#9494 ) I have discovered a bug located within `.github/workflows/_release.yml` which is the primary cause of continuous integration (CI) errors. The problem can be solved; therefore, I have constructed a PR to address the issue. ## The Issue Access the following link to view the exact errors: [Langhain Release Workflow](https://github.com/langchain-ai/langchain/actions/workflows/langchain_release.yml) The instances of these errors take place for each PR that updates `pyproject.toml`, excluding those specifically associated with bumping PRs. See below for the specific error message: ``` Error: Error 422: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"} ``` An image of the error can be viewed here: ![Image](https://github.com/langchain-ai/langchain/assets/13769670/13125f73-9b53-49b7-a83e-653bb01a1da1) The `_release.yml` document contains the following if-condition: ```yaml if: \| ${{ github.event.pull_request.merged == true }} && ${{ contains(github.event.pull_request.labels.*.name, 'release') }} ``` ## The Root Cause The above job constantly runs as the `if-condition` is always identified as `true`. ## The Logic The `if-condition` can be defined as `if: ${{ b1 }} && ${{ b2 }}`, where `b1` and `b2` are boolean values. However, in terms of condition evaluation with GitHub Actions, `${{ false }}` is identified as a string value, thereby rendering it as truthy as per the [official documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idif). I have run some tests regarding this behavior within my forked repository. You can consult my [debug PR](https://github.com/zawakin/langchain/pull/1) for reference. Here is the result of the tests: \|If-Condition\|Outcome\| \|:--:\|:--:\| \|`if: true && ${{ false }}`\|Execution\| \|`if: ${{ false }}` \|Skipped\| \|`if: true && false` \|Skipped\| \|`if: false`\|Skipped\| \|`if: ${{ true && false }}` \|Skipped\| In view of the first and second results, we can infer that `${{ false }}` can only be interpreted as `true` for conditions composed of some expressions. It is consistent that the condition of `if: ${{ inputs.working-directory == 'libs/langchain' }}` works. It is surprised to be skipped for the second case but it seems the spec of GitHub Actions 😓 Anyway, the PR would fix these errors, I believe 👍 Could you review this? @hwchase17 or @shoelsch , who is the author of [PR](https://github.com/langchain-ai/langchain/pull/360).	11 months ago

1 2 3

116 Commits (e669f9d731af3d673885dcddc46be4cbef5959b9)