RetroSearch Browse

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/rubygems/rubygems/issues/7432 below:

on musl systems, bundler does not include the relevant `musl` native gem platform in the lock file · Issue #7432 · rubygems/rubygems · GitHub

Describe the problem as clearly as you can

The desired behavior is:

Given I am on a linux musl system,
And I depend on a gem that with native (precompiled) gems,
And that gem ships a -linux-musl variant,
When I bundle lock,
The PLATFORMS section of my Gemfile.lock should include my -linux-musl platform,
And I should bundle the -linux-musl variant of the gem's native package.

However, in some scenarios, the correct *-linux-musl platform is omitted by bundler, leading to either choosing a less-appropriate platform gem (scenario 1 below), or a non-resolvable lock file (scenario 3 below).

Please note that this issue is probably a blocker for general consumption of musl-specific native gems.

/cc @segiddins @deivid-rodriguez

Did you try upgrading rubygems & bundler?

Yes, this behavior is present with Rubygems 3.5.5 and Bundler 2.5.5.

Post steps to reproduce the problem

Here are three scenarios to explore this problem space. I'm running these commands using the ruby:3.3-alpine docker image, and deleting Gemfile.lock at the beginning of each scenario.

To summarize what you're about to see:

when bundling a gem with linux and linux-musl variants, bundler omits the correct platform from the lock file
when bundling a gem with linux-gnu and linux-musl variants, everything works as expected
but when combining scenario 2 with a second gem that publishes a linux variant, the problem re-emerges

Here we go.

1. a gem that publishes *-linux and *-linux-musl native gem variants

Let's use rcee_precompiled v0.5.0 (see https://rubygems.org/gems/rcee_precompiled/versions), which publishes

rcee_precompiled-0.5.0-x86_64-linux.gem
rcee_precompiled-0.5.0-x86_64-linux-musl.gem

Gemfile:

# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.0"

run bundle lock (or bundle install), and the lock file contains:

GEM
  remote: https://rubygems.org/
  specs:
    rcee_precompiled (0.5.0-aarch64-linux)
    rcee_precompiled (0.5.0-aarch64-linux-musl)
    rcee_precompiled (0.5.0-arm-linux)
    rcee_precompiled (0.5.0-arm-linux-musl)
    rcee_precompiled (0.5.0-arm64-darwin)
    rcee_precompiled (0.5.0-x86-linux)
    rcee_precompiled (0.5.0-x86-linux-musl)
    rcee_precompiled (0.5.0-x86_64-darwin)
    rcee_precompiled (0.5.0-x86_64-linux)

PLATFORMS
  aarch64-linux
  aarch64-linux-musl
  arm-linux
  arm-linux-musl
  arm64-darwin
  x86-linux
  x86-linux-musl
  x86_64-darwin
  x86_64-linux

DEPENDENCIES
  rcee_precompiled (= 0.5.0)

BUNDLED WITH
   2.5.5

Note the absence of x86_64-linux-musl in the PLATFORMS section.

If I run bundle show --paths I can see that it's incorrectly chosen the *-linux flavor:

# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/rcee_precompiled-0.5.0-x86_64-linux

I can fix this by specifically adding the correct platform to the lock file:

bundle lock --add-platform x86_64-linux-musl
bundle install

then the Gemfile.lock contains x86_64-linux-musl:

PLATFORMS
  aarch64-linux
  aarch64-linux-musl
  arm-linux
  arm-linux-musl
  arm64-darwin
  x86-linux
  x86-linux-musl
  x86_64-darwin
  x86_64-linux
  x86_64-linux-musl

and I can see it's now choosing the correct musl native gem variant:

# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/rcee_precompiled-0.5.0-x86_64-linux-musl

2. a gem that publishes *-linux-gnu and *-linux-musl native gem variants

Now let's use rcee_precompiled v0.5.1 which switched from *-linux to *-linux-gnu, and publishes

rcee_precompiled-0.5.0-x86_64-linux-gnu.gem
rcee_precompiled-0.5.0-x86_64-linux-musl.gem

Gemfile:

# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"

run bundle lock (or bundle install), and the lock file contains (in part):

PLATFORMS
  aarch64-linux-gnu
  aarch64-linux-musl
  arm-linux-gnu
  arm-linux-musl
  arm64-darwin
  x86-linux-gnu
  x86-linux-musl
  x86_64-darwin
  x86_64-linux-gnu
  x86_64-linux-musl

Yay! This works. I see x86_64-linux-musl in the PLATFORMS section and bundle show --paths tells me it's choosing the right variant.

Cool cool cool.

3. but now add another gem that publishes a *-linux platform native gem

Let's add nokogiri:

# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"
gem "nokogiri"

run bundle lock (or bundle install), and the lock file contains (in part):

PLATFORMS
  aarch64-linux
  aarch64-linux-gnu
  aarch64-linux-musl
  arm-linux
  arm-linux-gnu
  arm-linux-musl
  arm64-darwin
  x86-linux
  x86-linux-gnu
  x86-linux-musl
  x86_64-darwin
  x86_64-linux
  x86_64-linux-gnu

Note that x86_64-linux-musl is now missing again! And if I attempt to bundle install I get an error:

Could not find gems matching 'rcee_precompiled (= 0.5.1)' valid for all resolution platforms (aarch64-linux,
aarch64-linux-gnu, aarch64-linux-musl, arm-linux, arm-linux-gnu, arm-linux-musl, arm64-darwin, x86-linux,
x86-linux-gnu, x86-linux-musl, x86_64-darwin, x86_64-linux, x86_64-linux-gnu) in rubygems repository
https://rubygems.org/ or installed locally.

However if I go through the process of:

bundle lock --add-platform x86_64-linux-musl
bundle install

then bundler will include x86_64-linux-musl in the PLATFORMS section of the lock file:

PLATFORMS
  aarch64-linux
  aarch64-linux-gnu
  aarch64-linux-musl
  arm-linux
  arm-linux-gnu
  arm-linux-musl
  arm64-darwin
  x86-linux
  x86-linux-gnu
  x86-linux-musl
  x86_64-darwin
  x86_64-linux
  x86_64-linux-gnu
  x86_64-linux-musl

and will choose the correct variant again:

# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/nokogiri-1.16.0-x86_64-linux
/usr/local/lib/ruby/gems/3.3.0/gems/racc-1.7.3
/usr/local/bundle/gems/rcee_precompiled-0.5.1-x86_64-linux-musl

What were you expecting to happen?

In scenarios 1 and 3, I expect

bundle lock to add x86_64-linux-musl to the PLATFORMS section of the lock file
bundle install to resolve to the x86_64-linux-musl native gem variant for rcee_precompiled

What actually happened?

Scenarios 1 and 3 seem to confuse bundler, which omits the correct local platform, x86_64-linux-musl, from the lock file.

If not included with the output of your command, run bundle env and paste the output below

At the end of scenario 3, this is what bundle env emits:

Environment

Bundler       2.5.5
  Platforms   ruby, x86_64-linux-musl
Ruby          3.3.0p0 (2023-12-25 revision 5124f9ac7513eb590c37717337c430cb93caa151) [x86_64-linux-musl]
  Full Path   /usr/local/bin/ruby
  Config Dir  /usr/local/etc
RubyGems      3.5.5
  Gem Home    /usr/local/bundle
  Gem Path    /root/.local/share/gem/ruby/3.3.0:/usr/local/lib/ruby/gems/3.3.0:/usr/local/bundle
  User Home   /root
  User Path   /root/.local/share/gem/ruby/3.3.0
  Bin Dir     /usr/local/bundle/bin
Tools         
  Git         not installed
  RVM         not installed
  rbenv       not installed
  chruby      not installed

Bundler Build Metadata

Built At          2024-01-18
Git SHA           2efa8cec93
Released Version  true

Bundler settings

app_config
  Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
silence_root_warning
  Set via BUNDLE_SILENCE_ROOT_WARNING: true

Gemfile Gemfile

source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"
gem "nokogiri"

Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    nokogiri (1.16.0-aarch64-linux)
      racc (~> 1.4)
    nokogiri (1.16.0-arm-linux)
      racc (~> 1.4)
    nokogiri (1.16.0-arm64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.0-x86-linux)
      racc (~> 1.4)
    nokogiri (1.16.0-x86_64-darwin)
      racc (~> 1.4)
    nokogiri (1.16.0-x86_64-linux)
      racc (~> 1.4)
    racc (1.7.3)
    rcee_precompiled (0.5.1-aarch64-linux-gnu)
    rcee_precompiled (0.5.1-aarch64-linux-musl)
    rcee_precompiled (0.5.1-arm-linux-gnu)
    rcee_precompiled (0.5.1-arm-linux-musl)
    rcee_precompiled (0.5.1-arm64-darwin)
    rcee_precompiled (0.5.1-x86-linux-gnu)
    rcee_precompiled (0.5.1-x86-linux-musl)
    rcee_precompiled (0.5.1-x86_64-darwin)
    rcee_precompiled (0.5.1-x86_64-linux-gnu)
    rcee_precompiled (0.5.1-x86_64-linux-musl)

PLATFORMS
  aarch64-linux
  aarch64-linux-gnu
  aarch64-linux-musl
  arm-linux
  arm-linux-gnu
  arm-linux-musl
  arm64-darwin
  x86-linux
  x86-linux-gnu
  x86-linux-musl
  x86_64-darwin
  x86_64-linux
  x86_64-linux-gnu
  x86_64-linux-musl

DEPENDENCIES
  nokogiri
  rcee_precompiled (= 0.5.1)

BUNDLED WITH
   2.5.5

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4