Handle `ERROR: While executing gem ... (Zlib::BufError)`?

[dentarg]

I've seen this happening a couple of times:

Run gem install gem-compare --version 1.1.0
Successfully installed rainbow-3.1.1
Successfully installed gemnasium-parser-0.1.9
Successfully installed diffy-3.4.2
Successfully installed gem-compare-1.1.0
4 gems installed
Run gem compare $COMPARE_GEM $COMPARE_FROM_VERSION $COMPARE_TO_VERSION > compare_output.txt
NOTE: Gem::Specification#has_rdoc is deprecated with no replacement. It will be removed in Rubygems 4
Gem::Specification#has_rdoc called from /opt/hostedtoolcache/Ruby/3.1.2/x64/lib/ruby/gems/3.1.0/gems/gem-compare-1.1.0/lib/rubygems/comparator/utils.rb:112.
NOTE: Gem::Specification#has_rdoc is deprecated with no replacement. It will be removed in Rubygems 4
Gem::Specification#has_rdoc called from /opt/hostedtoolcache/Ruby/3.1.2/x64/lib/ruby/gems/3.1.0/gems/gem-compare-1.1.0/lib/rubygems/comparator/utils.rb:112.
Run gem compare --diff $COMPARE_GEM $COMPARE_FROM_VERSION $COMPARE_TO_VERSION > compare_diff.txt
ERROR:  While executing gem ... (Zlib::BufError)
    buffer error

This is probably not the fault of this plugin, rather a temporary transport/network issue. Maybe rubygems.org does some rate limiting. Nevertheless, it would be cool to handle it, if possible.

We can see that the first gem compare command worked, is the second one fetching the gems again from RubyGems.org?

GitHub Actions log at https://github.com/Starkast/wikimum/runs/7173057743?check_suite_focus=true#step:2:454

(I've made an action that runs this plugin on Dependabot PRs and makes comment with the comparison, https://github.com/dentarg/gem-compare)

[strzibny]

Do you have a suggestion on how to address this?

[dentarg]

Not yet :)

[dentarg]

For now I've worked around this issue with retries outside gem compare.

[dentarg]

Maybe ruby/zlib#61 will make a difference for this (just a guess)

[martinemde]

Could you try with the latest rubygems? We recently added something that will give a better report when a file reaches an eof unexpectedly. It might at least reveal if this is a problem with the downloaded file.

For what it's worth, gem-compare is simply calling Gem::Package#extract_files. It's not doing anything "fancy" so this is either internal to rubygems, or zlib (as you mentioned) or caused by network issues when downloading.

[strzibny]

Yes, it would be better to open an issue in RubyGems and/or Zlib, and just link the issue.

[dentarg]

Could you try with the latest rubygems?

I'm not running gem compare by hand, I'm using my action https://github.com/dentarg/gem-compare to run it on Dependabot PRs. It uses whatever RubyGems setup-ruby picks, https://github.com/ruby/setup-ruby#rubygems, which seems to be the RubyGems that comes with Ruby.

Here's a recent run using Ruby 3.1.4 where the problem still showed: https://github.com/Starkast/wikimum/actions/runs/6154045239/job/16698800533#step:2:85

Here's a recent run using Ruby 3.2.2 where the problem still showed:
https://github.com/dentarg/burd/actions/runs/5905770058/job/16020494333#step:2:95

Looks like Ruby 3.2.2 ships with RubyGems 3.4.10? Is that recent enough?

I guess I could always install the latest RubyGems at the cost of somewhat slower CI runs.

[dentarg]

Latest Ruby 3.4.5 and latest RubyGems (3.7.1)

The error is more clear now, Gem::Package::FormatError, and the culprit seems to be that the binary version is fetched? Fetching pg-1.6.0-aarch64-linux.gem

$ docker run -it --rm ruby:3.4.5 bash
...

root@d3ac2348b667:/# gem install gem-compare
Fetching gemnasium-parser-0.1.9.gem
Fetching rainbow-3.1.1.gem
Fetching diffy-3.4.4.gem
Fetching gem-compare-1.2.1.gem
Successfully installed rainbow-3.1.1
Successfully installed gemnasium-parser-0.1.9
Successfully installed diffy-3.4.4
Successfully installed gem-compare-1.2.1
4 gems installed

A new release of RubyGems is available: 3.6.9 → 3.7.1!
Run `gem update --system 3.7.1` to update your installation.

root@d3ac2348b667:/# gem compare pg 1.5.9 1.6.0
Fetching pg-1.5.9.gem
Fetching pg-1.6.0-aarch64-linux.gem
ERROR:  While executing gem ... (Gem::Package::FormatError)
    No such file or directory @ rb_sysopen - /tmp/d20250802-22-g61prp/pg-1.6.0.gem
	/usr/local/lib/ruby/3.4.0/rubygems/package.rb:633:in 'Gem::Package#verify'
	/usr/local/lib/ruby/3.4.0/rubygems/package.rb:596:in 'Gem::Package#spec'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:289:in 'Gem::Comparator#use_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:233:in 'Gem::Comparator#download_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:221:in 'Gem::Comparator#get_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:83:in 'block in Gem::Comparator#compare_versions'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:81:in 'Array#each'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:81:in 'Gem::Comparator#compare_versions'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/commands/compare_command.rb:100:in 'Gem::Commands::CompareCommand#execute'
	/usr/local/lib/ruby/3.4.0/rubygems/command.rb:326:in 'Gem::Command#invoke_with_build_args'
	/usr/local/lib/ruby/3.4.0/rubygems/command_manager.rb:253:in 'Gem::CommandManager#invoke_command'
	/usr/local/lib/ruby/3.4.0/rubygems/command_manager.rb:194:in 'Gem::CommandManager#process_args'
	/usr/local/lib/ruby/3.4.0/rubygems/command_manager.rb:152:in 'Gem::CommandManager#run'
	/usr/local/lib/ruby/3.4.0/rubygems/gem_runner.rb:57:in 'Gem::GemRunner#run'
	/usr/local/bin/gem:12:in '<main>'

root@d3ac2348b667:/# gem update --system 3.7.1
Fetching rubygems-update-3.7.1.gem
Successfully installed rubygems-update-3.7.1
Installing RubyGems 3.7.1
  Successfully built RubyGem
  Name: bundler
  Version: 2.7.1
  File: bundler-2.7.1.gem
Bundler 2.7.1 installed
RubyGems 3.7.1 installed
Regenerating binstubs
Regenerating plugins
...

root@d3ac2348b667:/# gem compare pg 1.5.9 1.6.0
Fetching pg-1.5.9.gem
Fetching pg-1.6.0-aarch64-linux.gem
ERROR:  While executing gem ... (Gem::Package::FormatError)
    No such file or directory @ rb_sysopen - /tmp/d20250802-44-c83a8b/pg-1.6.0.gem
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/package.rb:633:in 'Gem::Package#verify'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/package.rb:596:in 'Gem::Package#spec'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:289:in 'Gem::Comparator#use_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:233:in 'Gem::Comparator#download_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:221:in 'Gem::Comparator#get_package'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:83:in 'block in Gem::Comparator#compare_versions'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:81:in 'Array#each'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/comparator.rb:81:in 'Gem::Comparator#compare_versions'
	/usr/local/bundle/gems/gem-compare-1.2.1/lib/rubygems/commands/compare_command.rb:100:in 'Gem::Commands::CompareCommand#execute'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/command.rb:326:in 'Gem::Command#invoke_with_build_args'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/command_manager.rb:253:in 'Gem::CommandManager#invoke_command'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/command_manager.rb:194:in 'Gem::CommandManager#process_args'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/command_manager.rb:152:in 'Gem::CommandManager#run'
	/usr/local/lib/ruby/site_ruby/3.4.0/rubygems/gem_runner.rb:57:in 'Gem::GemRunner#run'
	/usr/local/bin/gem:12:in '<main>'

[dentarg]

The problem comes and goes, in the same container, just 35 minutes later, no problem, but we see Fetching pg-1.6.0.gem

root@d3ac2348b667:/# gem compare pg 1.5.9 1.6.0
Fetching pg-1.5.9.gem
Fetching pg-1.6.0.gem
Compared versions: ["1.5.9", "1.6.0"]

...

[dentarg]

I think it is the order in the array returned from Gem::SpecFetcher.fetcher.spec_for_dependency dep that varies. One Gem::Specification below has s.platform = #<Gem::Platform:0x0000ffff46af13d8 @cpu="aarch64", @os="linux", @version=nil>.

root@4e30b3d9eba7:/app# irb
irb(main):001> gem_name = "pg"
=> "pg"
irb(main):002> version = "1.6.0"
=> "1.6.0"
irb(main):003> dep = Gem::Dependency.new gem_name, version
=> Gem::Dependency.new("pg", Gem::Requirement.new(["= 1.6.0"]), :runtime)
irb(main):004> specs_and_sources, _errors = Gem::SpecFetcher.fetcher.spec_for_dependency dep
=>
[[[Gem::Specification.new do |s|
...
irb(main):005> specs_and_sources
=>
[[Gem::Specification.new do |s|
    s.name = "pg"
    s.version = Gem::Version.new("1.6.0")
    s.installed_by_version = Gem::Version.new("0")
    s.authors = ["Michael Granger", "Lars Kanis"]
    s.date = Time.utc(1980, 1, 2)
    s.description = "Pg is the Ruby interface to the PostgreSQL RDBMS. It works with PostgreSQL 10 and later."
    s.email = ["ged@FaerieMUD.org", "lars@greiz-reinsdorf.de"]
    s.homepage = "https://github.com/ged/ruby-pg"
    s.licenses = ["BSD-2-Clause"]
    s.metadata = {"homepage_uri" => "https://github.com/ged/ruby-pg",
     "source_code_uri" => "https://github.com/ged/ruby-pg",
     "changelog_uri" => "https://github.com/ged/ruby-pg/blob/master/CHANGELOG.md",
     "documentation_uri" => "http://deveiate.org/code/pg",
     "msys2_mingw_dependencies" => "postgresql"}
    s.platform = #<Gem::Platform:0x0000ffff46af13d8 @cpu="aarch64", @os="linux", @version=nil>
    s.require_paths = ["lib"]
    s.required_ruby_version = Gem::Requirement.new([">= 2.7", "< 3.5.dev"])
    s.rubygems_version = "3.6.9"
    s.specification_version = 4
    s.summary = "Pg is the Ruby interface to the PostgreSQL RDBMS"
    end,
  #<Gem::Source[Remote: https://rubygems.org/ API URI: https://rubygems.org/]>],
 [Gem::Specification.new do |s|
    s.name = "pg"
    s.version = Gem::Version.new("1.6.0")
    s.installed_by_version = Gem::Version.new("0")
    s.authors = ["Michael Granger", "Lars Kanis"]
    s.date = Time.utc(2025, 7, 27)
    s.description = "Pg is the Ruby interface to the PostgreSQL RDBMS. It works with PostgreSQL 10 and later."
    s.email = ["ged@FaerieMUD.org", "lars@greiz-reinsdorf.de"]
    s.homepage = "https://github.com/ged/ruby-pg"
    s.licenses = ["BSD-2-Clause"]
    s.metadata = {"homepage_uri" => "https://github.com/ged/ruby-pg",
     "source_code_uri" => "https://github.com/ged/ruby-pg",
     "changelog_uri" => "https://github.com/ged/ruby-pg/blob/master/CHANGELOG.md",
     "documentation_uri" => "http://deveiate.org/code/pg",
     "msys2_mingw_dependencies" => "postgresql"}
    s.require_paths = ["lib"]
    s.required_ruby_version = Gem::Requirement.new([">= 2.7"])
    s.rubygems_version = "3.6.2"
    s.specification_version = 4
    s.summary = "Pg is the Ruby interface to the PostgreSQL RDBMS"
    end,
  #<Gem::Source[Remote: https://rubygems.org/ API URI: https://rubygems.org/]>]]