Skip to content

Concurrent poetry install fails due to cache issue #5295

Description

@abowden1989
  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
    • because this is a concurrency issue it doesn't always happen, so there isn't a guaranteed reproducer to execute with -vvv
  • Ubuntu 18.04:v1.0.2 (docker container)
  • Seen on multiple versions (issue seems to be in dependency)
  • pyproject.toml not particularly relevant

Issue

We have a build script that runs the poetry install command concurrently to install into virtual environments for multiple different python versions. This fails sporadically in CI with an exception from the cachy library:

STDOUT:     Updating dependencies
STDOUT:     Resolving dependencies...
STDOUT:     
STDOUT:       ValueError
STDOUT:     
STDOUT:       invalid literal for int() with base 10: b''
STDOUT:     
STDOUT:       at /usr/local/lib/python3.6/dist-packages/cachy/stores/file_store.py:65 in _get_payload
STDOUT:            61│ 
STDOUT:            62│         with open(path, 'rb') as fh:
STDOUT:            63│             contents = fh.read()
STDOUT:            64│ 
STDOUT:         →  65│         expire = int(contents[:10])
STDOUT:            66│ 

The multiple concurrent poetry install commands are using the same cache directory, and it looks like the code in the cachy library is not thread safe - https://github.com/sdispater/cachy/blob/master/cachy/stores/file_store.py.

It appears that if one process is performing a cache write at https://github.com/sdispater/cachy/blob/master/cachy/stores/file_store.py#L102-L103 while another is doing a read at https://github.com/sdispater/cachy/blob/master/cachy/stores/file_store.py#L62-L63, the read could result in reading an empty file before the contents are written, and then L65 attempts to cast an empty set of bytes to an integer and throws.

Ideally the cache would be thread safe, but documentation saying that concurrent installs are not safe would be an alternative.

A possible workaround that we will attempt is to set the cache-dir in the config to different directories for different concurrent installs.

UPDATE:
The attempted workaround of setting a specific cache-dir in the config does not work, likely due to the issue described in #2445 meaning the repository cache is still shared. As a workaround we are now going to try setting both the poetry cache-dir and XDG_CACHE_HOME to different directories for different installs

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working as expectedstatus/triageThis issue needs to be triaged

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions