Search code examples
rubyrubygemsbundler

Why private repo requires *.rb file for a custom gem?

I have code from a project I am trying to make into a gem stored in a private git repo so we can re-use it in other projects.

Even though the gem stores the code, the *.rb files containing the code seem to be required. This would seem redundant to me as the *.rb files are now both in the repo of the gem and the gem itself. That, or my Gemfile / bundler setup is wrong.

Let's say I make this file:

~/code/holaPrj$ cat > hola.rb 
class Hola

self.say(lang: :en)
end
end

... and I build the gem:

~/code/holaPrj$ cat hola.gemspec 
Gem::Specification.new do |s|
  s.name        = 'hola'
  s.version     = '0.0.1'
  s.date        = '2019-08-14'
  s.summary     = "Custom gem in private repo testing"
  s.authors     = ["Julien Lamarche"]
  s.email       = ["jlam@credil.org"]
  s.files       = ["lib/hola.rb"]
  s.license     = 'Nonstandard'
end

~/code/holaPrj$ rvm default do gem build hola.gemspec 
WARNING:  no homepage specified
WARNING:  See http://guides.rubygems.org/specification-reference/ for help
  Successfully built RubyGem
  Name: hola
  Version: 0.0.1
  File: hola-0.0.1.gem

I copy this gem to my ~/tmp/, then unpack it. I can see the files in the file system:

~/code/holaPrj$ cp hola-0.0.1.gem ~/tmp/
'hola-0.0.1.gem' -> '/home/jlam/tmp/hola-0.0.1.gem'

~/code/holaPrj$ cd ~/tmp/
lusk 16:33:18 ~/tmp$ rvm default do gem unpack hola-0.0.1.gem 
Unpacked gem: '/home/jlam/tmp/hola-0.0.1'

lusk 16:33:28 ~/tmp$ ls hola-0.0.1/lib/hola.rb 
hola-0.0.1/lib/hola.rb

Thus, it is esblish that a *.gem file will contain the ruby files in quesation.

Copying this gem and the gemspec file into a repo for internal publication:

~/code/holaPrj$ cp hola-0.0.1.gem ../holaGem/
'hola-0.0.1.gem' -> '../holaGem/hola-0.0.1.gem'
~/code/holaPrj$ cp hola.gemspec ../holaGem/
'hola.gemspec' -> '../holaGem/hola.gemspec'
~/code/holaPrj$ 

~/code/holaGem$ git add *
~/code/holaGem$ git commit -am "adding the gem and gemspec file"
[master (commit racine) 9c7807c] adding the gem and gemspec file
 2 files changed, 11 insertions(+)
 create mode 100644 hola-0.0.1.gem
 create mode 100644 hola.gemspec

~/code/holaGem$ git remote add origin git+ssh://$ourServer/gems/hola
~/code/holaGem$ git push origin master
Décompte des objets: 4, fait.
Delta compression using up to 8 threads.
Compression des objets: 100% (4/4), fait.
Écriture des objets: 100% (4/4), 1.54 KiB | 0 bytes/s, fait.
Total 4 (delta 0), reused 0 (delta 0)
To git+ssh://$ourServer/gems/hola
 * [new branch]      master -> master

Added in Gemfile:

~/tmp/importMyGem$ cat Gemfile

source 'https://rubygems.org'
git_source(:our_gems){ |repo_name| "git+ssh://$ourServer/gems/#{repo_name}" }

gem 'hola', our_gems: 'hola'

Then we install it:

 ~/tmp/importMyGem$ rvm default do bundle  install 

Warning, new version of rvm available '1.29.9-next', you are using older version '1.29.3'.
You can disable this warning with:    echo rvm_autoupdate_flag=0 >> ~/.rvmrc
You can enable  auto-update  with:    echo rvm_autoupdate_flag=2 >> ~/.rvmrc
Fetching git+ssh://$ourServer/gems/hola
Fetching gem metadata from https://rubygems.org/
Resolving dependencies...
Using bundler 2.0.2
Using hola 0.0.1 from git+ssh://$ourServer/gems/hola (at master@9c7807c)
Bundle complete! 1 Gemfile dependency, 2 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

... and bundle knows where it is:

~/tmp/importMyGem$ rvm default do bundle info hola
  * hola (0.0.1 9c7807c)
    Summary: Custom gem in private repo testing
    Path: /usr/local/rvm/gems/ruby-2.3.1/bundler/gems/hola-9c7807c88e90

~/tmp/importMyGem$ ls /usr/local/rvm/gems/ruby-2.3.1/bundler/gems/hola-9c7807c88e90/
hola-0.0.1.gem  hola.gemspec

Loading up irb and the gem:

~/tmp/importMyGem$ rvm default do bundle exec irb
2.3.1 :001 > require 'hola'
LoadError: cannot load such file -- hola
    from (irb):1:in `require'
    from (irb):1
    from /usr/local/rvm/rubies/ruby-2.3.1/bin/irb:11:in `<top (required)>'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `load'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:28:in `run'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli.rb:465:in `exec'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli.rb:27:in `dispatch'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/cli.rb:18:in `start'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/exe/bundle:30:in `block in <top (required)>'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
    from /usr/local/rvm/gems/ruby-2.3.1/gems/bundler-2.0.2/exe/bundle:22:in `<top (required)>'
    from /usr/local/rvm/gems/ruby-2.3.1/bin/bundle:23:in `load'
    from /usr/local/rvm/gems/ruby-2.3.1/bin/bundle:23:in `<main>'
    from /usr/local/rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
    from /usr/local/rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'

This I would have expected to work.

Lets add the .rb files:

 ~/code/holaGem$ cp -r ../holaPrj/lib ./
'../holaPrj/lib' -> './lib'
'../holaPrj/lib/hola.rb' -> './lib/hola.rb'

~/code/holaGem$ ls
hola-0.0.1.gem  hola.gemspec  lib

~/code/holaGem$ git add lib/
~/code/holaGem$ git commit -am "adding the .rb files of the gem"
[master 1c6ee59] adding the .rb files of the gem
 1 file changed, 7 insertions(+)
 create mode 100644 lib/hola.rb

~/code/holaGem$ git push origin master 

Décompte des objets: 4, fait.
Delta compression using up to 8 threads.
Compression des objets: 100% (3/3), fait.
Écriture des objets: 100% (4/4), 381 bytes | 0 bytes/s, fait.
Total 4 (delta 1), reused 0 (delta 0)
To git+ssh://$ourServer/gems/hola
   9c7807c..1c6ee59  master -> master

... and lets upgrade the gem version to 0.0.2 just to make sure we know we have a new gem version (though I suppose the hash in bunlder or the Gemfile.lock would suffice):

~/code/holaGem$ cat hola.gemspec 
Gem::Specification.new do |s|
  s.name        = 'hola'
  s.version     = '0.0.2'
  s.date        = '2019-08-14'
  s.summary     = "Custom gem in private repo testing"
  s.authors     = ["Julien Lamarche"]
  s.email       = ["jlam@credil.org"]
  s.files       = ["lib/hola.rb"]
  s.license     = 'Nonstandard'
end
~/code/holaGem$ vi hola.gemspec 

~/code/holaGem$ git commit -am "wo#15499 - project .rb files added"
[master 87742bc] wo#15499 - project .rb files added
 1 file changed, 1 insertion(+), 2 deletions(-)
lusk 17:51:38 (master) ~/code/holaGem$ git push origin master 
Décompte des objets: 3, fait.
Delta compression using up to 8 threads.
Compression des objets: 100% (3/3), fait.
Écriture des objets: 100% (3/3), 366 bytes | 0 bytes/s, fait.
Total 3 (delta 1), reused 0 (delta 0)
To git+ssh://$ourServer/gems/hola
   1c6ee59..87742bc  master -> master

Going to the importing project, importMyGem, I update to 0.0.2!

~/tmp/importMyGem$ rvm default do bundle update
Fetching git+ssh://$ourServer/gems/hola
Fetching gem metadata from https://rubygems.org/
Resolving dependencies...
Using bundler 2.0.2
Using hola 0.0.2 (was 0.0.1) from git+ssh://$ourServer/gems/hola (at master@87742bc)
Bundle updated!

The *.rb files are now there:

$ ls /usr/local/rvm/gems/ruby-2.3.1/bundler/gems/hola-5b37daed4c3a
hola-0.0.1.gem  hola.gemspec  lib

And irb can find the file:

~/tmp/importMyGem$ rvm default do bundle exec irb
2.3.1 :001 > require 'hola'
 => true

So what's the point of having *.rb files inside the *.gem file if my private git repo requires it's own copy of the *.rb files for the importing project to find the *.rb files? Or rather, is there something wrong with my bundler or Gemfile setup?

Solution

  • A *.gem file is really just a zip file that contains the contents (source code and other metadata) of your gem.

    It is also a build artifact that developers don't usually check into a repo as it can be reproduced from the source code. And indeed, the contents of the *.gem archive are the source code of the gem, so checking it in would be duplicating the contents of the repo.

    For that reason, when Bundler implemented installing gems from a git source (the gem install command itself doesn't support this) it required the git repo to include the source code and a .gemspec file. From this it will actually build a *.gem file and then install it with gem install. Bundler is not set up and doesn't expect there to be a *.gem file in a repo.

    So, what you should do is check in the source code and not the *.gem file.