From 0dd2c21089d82445ef9ee8dd026be1f3194ed4fe Mon Sep 17 00:00:00 2001 From: Predrag Gruevski <2348618+obi1kenobi@users.noreply.github.com> Date: Thu, 17 Aug 2023 18:23:00 -0400 Subject: [PATCH] Do not bust `poetry install` cache when manually installing pydantic v2. (#9407) Using `poetry add` to install `pydantic@2.1` was also causing poetry to change its lockfile. This prevented dependency caching from working: - When attempting to restore a cache, it would hash the lockfile in git and use it as part of the cache key. Say this is a cache miss. - Then, it would attempt to save the cache -- but the lockfile will have changed, so the cache key would be *different* than the key in the lookup. So the cache save would succeed, but to a key that cannot be looked up in the next run -- meaning we never get a cache hit. In addition to busting the cache, the lockfile update itself is also non-trivially long, over 30s: ![image](https://github.com/langchain-ai/langchain/assets/2348618/d84d3b56-484d-45eb-818d-54126a094a40) This PR fixes the problems by using `pip` to perform the installation, avoiding the lockfile change. --- .github/workflows/_test.yml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/.github/workflows/_test.yml b/.github/workflows/_test.yml index 2465aa1ace..ebb0791737 100644 --- a/.github/workflows/_test.yml +++ b/.github/workflows/_test.yml @@ -46,7 +46,11 @@ jobs: elif [ "${{ matrix.test_type }}" == "core-pydantic-2" ]; then echo "Running core-pydantic-v2 tests, installing dependencies with poetry..." poetry install - poetry add pydantic@2.1 + + # Install via `pip` instead of `poetry add` to avoid changing lockfile, + # which would prevent caching from working: the cache would get saved + # to a different key than where it gets loaded from. + poetry run pip install 'pydantic>=2.1,<3' else echo "Running extended tests, installing dependencies with poetry..." poetry install -E extended_testing @@ -59,10 +63,10 @@ jobs: EXPECTED_VERSION=1 fi echo "Checking pydantic version... Expecting ${EXPECTED_VERSION}" - + # Determine the major part of pydantic version VERSION=$(poetry run python -c "import pydantic; print(pydantic.__version__)" | cut -d. -f1) - + # Check that the major part of pydantic version is as expected, if not # raise an error if [[ "$VERSION" -ne $EXPECTED_VERSION ]]; then