Search code examples
activerecordpuppetjruby

puppetserver gem install adds gem but it's not available for custom functions in the puppet code

I have simple setup: CentOS7 and Puppet7 running on it. Server is up and running. I installed activerecord-jdbcsqlite3-adapter adapter for my custom function. This is what I can see on the server with puppetserver gem list

*** LOCAL GEMS ***

activemodel (6.1.7.3)
activerecord (6.1.7.3)
activerecord-jdbc-adapter (61.2 java)
activerecord-jdbcsqlite3-adapter (61.2 java)
activesupport (6.1.7.3)
bundler (default: 2.2.33)
cmath (default: 1.0.0)
concurrent-ruby (1.1.5)
csv (default: 3.2.5)
deep_merge (1.2.2, 1.0.1)
e2mmap (default: 0.1.0)
fast_gettext (1.1.2)
ffi (default: 1.15.4 java)
fileutils (default: 1.4.1)
forwardable (default: 1.2.0)
gettext (3.2.2)
hiera-eyaml (3.3.0)
highline (2.0.3)
hocon (1.3.1)
i18n (1.14.1)
io-console (default: 0.5.9 java)
ipaddr (default: 1.2.2)
irb (default: 1.0.0)
jar-dependencies (default: 0.4.1)
jdbc-sqlite3 (3.28.0)
jruby-openssl (default: 0.14.0 java)
jruby-readline (default: 1.3.7 java)
json (default: 2.5.1 java)
locale (2.1.3, 2.1.2)
logger (default: 1.5.1)
matrix (default: 0.3.0)
mini_portile2 (2.8.2)
minitest (5.18.0)
multi_json (1.15.0)
mutex_m (default: 0.1.0)
optimist (3.0.1)
ostruct (default: 0.5.5)
prime (default: 0.1.0)
psych (default: 3.3.4 java)
puppet-resource_api (1.8.16)
puppetserver-ca (2.5.0)
racc (default: 1.5.2 java)
rake-ant (default: 1.0.4)
rdoc (default: 6.3.3)
rexml (default: 3.2.5)
rss (default: 0.2.7)
rubygems-update (default: 3.2.33)
scanf (1.0.0)
semantic_puppet (1.0.4, 1.0.2)
shell (default: 0.7)
sync (default: 0.5.0)
text (1.3.1)
thwait (default: 0.1.0)
tracer (default: 0.1.0)
tzinfo (2.0.6)
webrick (default: 1.7.0)
zeitwerk (2.6.8)

[root@puppet-server certs]# puppetserver gem environment

RubyGems Environment:
  - RUBYGEMS VERSION: 3.2.33
  - RUBY VERSION: 2.6.8 (2022-10-24 patchlevel 0) [java]
  - INSTALLATION DIRECTORY: /opt/puppetlabs/server/data/puppetserver/jruby-gems
  - USER INSTALLATION DIRECTORY: /root/.gem/jruby/2.6.0
  - RUBY EXECUTABLE: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre/bin/java -cp :/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar:/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/facter.jar:/opt/puppetlabs/server/data/puppetserver/jars/* org.jruby.Main
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /opt/puppetlabs/server/data/puppetserver/jruby-gems/bin
  - SPEC CACHE DIRECTORY: /root/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: uri:classloader:/META-INF/jruby.home/etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - universal-java-1.8
  - GEM PATHS:
     - /opt/puppetlabs/server/data/puppetserver/jruby-gems
     - /opt/puppetlabs/server/data/puppetserver/vendored-jruby-gems
     - /opt/puppetlabs/puppet/lib/ruby/vendor_gems
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "install" => "--env-shebang"
     - "update" => "--env-shebang"
     - "setup" => "--env-shebang"
     - "pristine" => "--env-shebang"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /root/.rbenv/shims
     - /root/.rbenv/bin
     - /usr/local/sbin
     - /usr/local/bin
     - /usr/sbin
     - /usr/bin
     - /opt/puppetlabs/bin
     - /root/bin

Here is code of my custom_function.rb (it prints list of all gems which can be "seeing" from the function invoked by puppet. (Output is below)):

require "timeout"
require "erb"
require "rubygems"
#require "activerecord-jdbcsqlite3-adapter"
#require "sqlite3"

#ActiveRecord::Base.establish_connection(:adapter => "sqlite3",
#                                        :database => "/my-db/sqlite.db",
#                                        :pool => 5,
#                                        :timeout => 10000)

# Documentation for the function
Puppet::Functions.create_function(:custom_function) do
  dispatch :custom_function do
    required_param 'String', :device_name
    required_param 'String', :template_name
  end

  def custom_function(device_name, template_name)
    my_local_gems = Gem::Specification.sort_by{ |g| [g.name.downcase, g.version] }.group_by{ |g| g.name }
    puts my_local_gems.map{ |name, specs|
      [
        name,
        specs.map{ |spec| spec.version.to_s }.join(',')
      ].join(' ')
    }
    puts RUBY_VERSION
  end
end

output:

>puppet lookup classes --compile
benchmark 0.1.0
bigdecimal 2.0.0
bundler 2.1.4
cgi 0.1.0.2
concurrent-ruby 1.1.9
csv 3.1.2
date 3.0.3
deep_merge 1.2.2
delegate 0.1.0
did_you_mean 1.4.0
etc 1.1.0
facter 4.3.1
fast_gettext 1.1.2
fcntl 1.0.0
ffi 1.15.5
fiddle 1.0.0
fileutils 1.4.1
forwardable 1.3.1
getoptlong 0.1.0
gettext 3.2.2
hiera 3.12.0
hiera-eyaml 3.3.0
highline 2.0.3
hocon 1.3.1
io-console 0.5.6
ipaddr 1.2.2
irb 1.2.6
json 2.3.0
locale 2.1.3
logger 1.4.2
matrix 0.2.0
minitest 5.13.0
multi_json 1.15.0
mutex_m 0.1.0
net-pop 0.1.0
net-smtp 0.1.0
net-ssh 4.2.0
net-telnet 0.2.0
observer 0.1.0
open3 0.1.0
openssl 2.1.4
optimist 3.0.1
ostruct 0.2.0
power_assert 1.1.7
prime 0.1.1
pstore 0.1.0
psych 3.1.0
puppet 7.24.0
puppet-resource_api 1.8.16
puppetserver-ca 2.5.0
racc 1.4.16
rake 13.0.1
rdoc 6.2.1.1
readline 0.0.2
readline-ext 0.1.0
reline 0.1.5
rexml 3.2.3.1
rss 0.2.8
scanf 1.0.0
sdbm 1.0.0
semantic_puppet 1.0.4
singleton 0.1.0
stringio 0.1.0
strscan 1.0.3
sys-filesystem 1.4.1
test-unit 3.3.4
text 1.3.1
thor 1.2.1
timeout 0.1.0
tracer 0.1.0
uri 0.10.0
webrick 1.6.1
xmlrpc 0.3.0
yaml 0.1.0
zlib 1.1.0
2.7.7

And if I add require "activerecord-jdbcsqlite3-adapter" I am getting error like:

Error: Could not run: cannot load such file -- activerecord-jdbcsqlite3-adapter

And finally. It works well if I run it in puppetserver irb:

[root@puppet-server ~]# puppetserver irb
irb(main):001:0> require "activerecord-jdbcsqlite3-adapter"
=> true
irb(main):002:0> ActiveRecord::Base.establish_connection(:adapter => "sqlite3", :database => "/db/sqlite.db")
=> #<ActiveRecord::ConnectionAdapters::ConnectionPool:0x2cd31214 @connection_klass=ActiveRecord::Base, @connections=[], @thread_cached_conns=#<Concurrent::Map:0x5d32f5db entries=0 default_proc=nil>, @checkout_timeout=5.0, @mon_data_owner_object_id=4020, @now_connecting=0, @pool_config=#<ActiveRecord::ConnectionAdapters::PoolConfig:0x16f2d883
...
...
...
Solution

  • Puppet's puppetserver and puppet subsystems have separate Ruby installations. The former is JRuby, whereas the latter is MRI / CRuby. The puppetserver gem command operates on the Ruby supporting puppetserver, but for your custom function you probably need the one supporting puppet, as that's what the catalog compiler uses. To install gems for that one, use the gem command bundled alongside the puppet command, typically at /opt/puppetlabs/puppet/bin/gem.