Environment: transformers==4.57.6, tokenizers==0.23.1
Problem: transformers 4.57.6 requires tokenizers>=0.22.0,<=0.23.0, but tokenizers 0.23.0 does not exist on PyPI (only 0.23.1). This causes a hard ImportError on any fresh install.
ImportError: tokenizers>=0.22.0,<=0.23.0 is required for a normal functioning of this module, but found tokenizers==0.23.1.
Suggested fix: Loosen the transformers version constraint in requirements.txt (e.g. transformers>=4.38.0,<5.0.0).
Workaround: We patched transformers' dependency_versions_check.py to skip runtime version checks, but this workaround shouldn't be necessary.
Environment: transformers==4.57.6, tokenizers==0.23.1
Problem: transformers 4.57.6 requires
tokenizers>=0.22.0,<=0.23.0, but tokenizers 0.23.0 does not exist on PyPI (only 0.23.1). This causes a hard ImportError on any fresh install.Suggested fix: Loosen the transformers version constraint in requirements.txt (e.g.
transformers>=4.38.0,<5.0.0).Workaround: We patched transformers' dependency_versions_check.py to skip runtime version checks, but this workaround shouldn't be necessary.