Dependency walk failed

[SunnyR]

• Poetry version: 1.3.2
• Python version: 3.8
• OS version and name: wsl ubuntu 20.04
• pyproject.toml: https://gist.github.com/SunnyR/eedd0f0d974b4c046a660f354eb65dee

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When using poetry export with dev dependencies (following command), i get stack trace below with error:
poetry export -f requirements.txt -o ./dev_dependencies --without-hashes --with dev

Stack trace:

  12  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  11  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  10  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

   9  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

   8  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   7  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   6  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/command.py:62 in execute
        60│ 
        61│         try:
     →  62│             return self.handle()
        63│         except KeyboardInterrupt:
        64│             return 1

   5  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/command.py:107 in handle
       105│         exporter.with_credentials(self.option("with-credentials"))
       106│         exporter.with_urls(not self.option("without-urls"))
     → 107│         exporter.export(fmt, Path.cwd(), output or self.io)
       108│ 

   4  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/exporter.py:86 in export
        84│             raise ValueError(f"Invalid export format: {fmt}")
        85│ 
     →  86│         getattr(self, self.EXPORT_METHODS[fmt])(cwd, output)
        87│ 
        88│     def _export_generic_txt(

   3  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/exporter.py:101 in _export_generic_txt
        99│         )
       100│ 
     → 101│         for dependency_package in get_project_dependency_packages(
       102│             self._poetry.locker,
       103│             project_requires=root.all_requires,

   2  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:103 in get_project_dependency_packages
       101│         selected.append(dependency)
       102│ 
     → 103│     for package, dependency in get_project_dependencies(
       104│         project_requires=selected,
       105│         locked_packages=repository.packages,

   1  ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:129 in get_project_dependencies
       127│         )
       128│ 
     → 129│     nested_dependencies = walk_dependencies(
       130│         dependencies=project_requires,
       131│         packages_by_name=packages_by_name,

  RuntimeError

  Dependency walk failed at click (>=6.5)

  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry_plugin_export/walker.py:155 in walk_dependencies
      151│             requirement, packages_by_name, nested_dependencies
      152│         )
      153│ 
      154│         if not locked_package:
    → 155│             raise RuntimeError(f"Dependency walk failed at {requirement}")
      156│ 
      157│         if requirement.extras:
      158│             locked_package = locked_package.with_features(requirement.extras)
      159│ 

[neersighted]

You have not provided a pyproject.toml as requested in the issue template -- can you please provide that information so that people can help you debug?

[finswimmer]

I guess this is a duplicate of #173

[SunnyR]

@neersighted updated the issue description with pyproject.toml

@finswimmer i'm already using a python constraint so i don't believe this is a duplicate. Let me know.

[dimbleby]

It's hard to know what to make of this. So far as I can see poetry simply doesn't support source distributions with file suffix .tgz, which is all that is available at https://pypi.org/project/amqplib/1.0.2/#files, therefore the given pyproject.toml can't even lock.

So I don't know how this becomes an export problem...

[SunnyR]

I've updated the pyproject.toml file with packages that are the culprit - click and black.

Also, this issue might be related - psf/black#2964, but i'm still not sure why export is not working?

[SunnyR]

Tweaked my pyproject.toml a bit more and was able to resolve the issue by specifying black versions for each python version.

Marking closed.

[neersighted]

I don't know that this should be closed -- we shouldn't fail to export a valid lock file generated from a valid pyproject.

[mhamiltonj]

I'm having the same issue, not sure if it's the same cause. I think in our case it is related to #28. Our poetry.lock conatins duplicate entries for scipy (versions 1.8.1 and 1.10.0). This doesn't seem to be a problem, the poetry instally 1.10 and the project works fine. But export fails with Dependency walk failed at scipy (>=1.3,<1.9). I have no idea where the <1.9 constraint is coming from.

In our case a workaround is adding a direct scipy = ">=1.10" dependency in our pyproject.toml, but this seems artificial, since scipy is a subdependency, not a direct dependency for us.

[marcorichetta]

Same as @mhamiltonj
In our case, the issue is statsmodels/statsmodels#8543
Workaround: poetry add "statsmodels@^0.13.2,!=0.13.3,!=0.13.4,!=0.13.5"

[neersighted]

Adding a direct dependency is not exactly a workaround; you should add constraints for anything you care to perturb/alter, not just anything you import directly. That being said, it is a workaround in this specific case as you are trying to perturb the lock file for us with this plugin, and not the dependency tree as a whole.