Poetry caching issue

[Syffe]

• Poetry version: 1.4.1

• Python version: 3.11.0

• OS version and name: Ubuntu 20.04.6 LTS

• pyproject.toml: https://gist.github.com/Syffe/80bc7fe55dc2dcb7d59dd295b8e9c7c0

• I am on the latest stable Poetry version, installed using a recommended method.

• I have searched the issues of this repo and believe that this is not a duplicate.

• I have consulted the FAQ and blog for any relevant entries or release notes.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Poetry seems to be flaky when executing our test suite.
We're running a GitHub Actions workflow where we use poetry to install our dependencies based on pyproject.toml

• Installing python-lsp-jsonrpc (1.0.0)
  • Installing pyyaml (6.0)
  • Installing restructuredtext-lint (1.4.0)
  • Installing rich (13.3.2)
  • Installing ruamel-yaml (0.17.21)
  • Installing setproctitle (1.2.3)
  • Installing sphinx (6.1.3)
  • Installing sqlalchemy (2.0.7)

  StopIteration

  0

  at /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/json/decoder.py:353 in raw_decode
      349│         have extraneous data at the end.
      350│ 
      351│         """
      352│         try:
    → 353│             obj, end = self.scan_once(s, idx)
      354│         except StopIteration as err:
      355│             raise JSONDecodeError("Expecting value", s, err.value) from None
      356│         return obj, end
      357│ 

The following error occurred when trying to handle this error:


  Stack trace:

  27  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:272 in _execute_operation
        result = self._do_execute_operation(operation)

  26  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:374 in _do_execute_operation
        result: int = getattr(self, f"_execute_{method}")(operation)

  25  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:494 in _execute_install
        status_code = self._install(operation)

  24  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:532 in _install
        archive = self._download(operation)

  23  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:707 in _download
        return self._download_link(operation, link)

  22  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/executor.py:743 in _download_link
        archive = self._chef.prepare(archive, output_dir=output_dir)

  21  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/chef.py:106 in prepare
        return self._prepare_sdist(archive, destination=output_dir)

  20  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/chef.py:188 in _prepare_sdist
        return self._prepare(

  19  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/chef.py:121 in _prepare
        env.install(builder.build_system_requires)

  18  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/chef.py:85 in install
        installer.run()

  17  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/installer.py:116 in run
        return self._do_install()

  16  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/installation/installer.py:[263](https://github.com/actions/runs/4509960669/jobs/7940340850?pr=5157#step:6:264) in _do_install
        ops = solver.solve(use_latest=self._whitelist).calculate_operations()

  15  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/puzzle/solver.py:74 in solve
        packages, depths = self._solve()

  14  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/puzzle/solver.py:157 in _solve
        result = resolve_version(self._package, self._provider)

  13  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/mixology/__init__.py:18 in resolve_version
        return solver.solve()

  12  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/mixology/version_solver.py:112 in solve
        next = self._choose_package_version()

  11  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/mixology/version_solver.py:428 in _choose_package_version
        package = self._provider.complete_package(package)

  10  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/puzzle/provider.py:578 in complete_package
        self._pool.package(

   9  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/repositories/repository_pool.py:124 in package
        return repo.package(name, version, extras=extras)

   8  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/repositories/cached_repository.py:76 in package
        return self.get_release_info(canonicalize_name(name), version).to_package(

   7  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/repositories/cached_repository.py:53 in get_release_info
        cached = self._release_cache.remember(

   6  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/utils/cache.py:162 in remember
        value = self.get(key)

   5  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/utils/cache.py:110 in get
        return self._get_payload(key)

   4  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/utils/cache.py:175 in _get_payload
        payload = self._deserialize(f.read())

   3  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/site-packages/poetry/utils/cache.py:196 in _deserialize
        data = json.loads(data_str[10:])

   2  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/json/__init__.py:346 in loads
        return _default_decoder.decode(s)

   1  /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/json/decoder.py:337 in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())

  JSONDecodeError

  Expecting value: line 1 column 1 (char 0)

  at /opt/hostedtoolcache/Python/3.11.2/x64/lib/python3.11/json/decoder.py:355 in raw_decode
      351│         """
      352│         try:
      353│             obj, end = self.scan_once(s, idx)
      354│         except StopIteration as err:
    → 355│             raise JSONDecodeError("Expecting value", s, err.value) from None
      356│         return obj, end
      357│ 

  • Installing tenacity (8.2.2)
  • Installing tokenize-rt (5.0.0)

[dimbleby]

looks duplicate #4210

anyway no-one will care to debug poetry 1.3, sorry. please close

[Syffe]

looks duplicate #4210

anyway no-one will care to debug poetry 1.3, sorry. please close

Corrected the version, It is indeed on 1.4.1, not 1.3

[dimbleby]

clear your cache directory

[sileht]

clear your cache directory

We don't use cache.

[dimbleby]

your stack trace and your issue title both disagree.

If you mean that you don't want to use the cache, try --no-cache.

[sileht]

When I say no cache, because this is run in a GitHub action without any poetry cache/pip cache enabled. We don't explicitly set --no-cache (but I will try) as the container is freshly created and poetry install is the first command we run.

[sileht]

Whatever the root cause is, we can make some improvements to the error message itself.

The non-verbose version is just:

 StopIteration

  0

JSONDecodeError

  Expecting value: line 1 column 1 (char 0)

This error doesn't report anything valuable here, not even the package name that failed to be installed.

[martin-kokos]

Has anyone looked into how it gets corrupted. I can't really image much more than it being an empty file - that can happen for example if there are two write-mode file objects writing to the same file.

[Kuba314]

@Syffe Could you reduce the pyproject.toml file and still keep this issue reproducible? I might be encountering the same issue since I switched from 1.3.1 to 1.4.1 and I'd really like to narrow this down.

[uckelman]

We're seeing this too, in a fresh container that should have no preexisting cache.

[runfalk]

I recently spotted this in one of our pipeline where we do poetry install for 10 ish projects in parallel. It works most of the time, but 1/25 runs fail. We use Poetry 1.4.2.

I think this is fixed on master already in #7453. I've worked around it for now using --no-cache.

[dimbleby]

as above = #7453, now released - please close