Search code examples
chef-infrachef-recipe

Chef: searching node inside recipe returns error 500.


I'm using Chef 11 (opensource server) and chef-client 11 also. I'm trying to search the nodes with a specific role. Using knife I am able to get the results I wish using: knife search node "run_list:role\[cassandra\]". Also I'm able to search using search(:node, "run_list:role\\[cassandra\\]") inside the recipe. But if I try to access the role using an attribute I cannot get it working. I have tried the most obvious way search(:node, "run_list:role\\[#{node[:role][:cassandra]}\\]") and also trying to compose the query outside the search line (query = 'run_list:role\[' + node[:cassandra][:seed_role]) but I get the same error. I'm getting error 500 during Chef-client compile phase (I think). Here it is the part of the code I'm using and the output log.

 # Searching for Cassandra nodes inside the same Chef environment
  cassandra_seeds = []
  cassandra_nodes_macs = []
  query = 'run_list:role\[' + node[:cassandra][:seed_role] + '\] AND chef_environment:#{node.chef_environment}'
  log query
  search(:node, query).each do |n|
#  search(:node, "run_list:role\\[cassandra\\]  AND \
#    chef_environment:#{node.chef_environment}").each do |n|
#  search(:node, "roles:*#{node[:cassandra][:seed_role]}  AND \
#    chef_environment:#{node.chef_environment}").each do |n|
    cassandra_seeds << n[:fqdn] unless n[:fqdn] == node[:fqdn]
  end

$ kitchen converge ub
-----> Starting Kitchen (v1.2.1)
-----> Converging <default-ubuntu-1404>...
       Preparing files for transfer
       Resolving cookbook dependencies with Berkshelf 3.2.1...
       Removing non-cookbook files before transfer
       Preparing data bags
       Preparing nodes
       Preparing encrypted data bag secret
       bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
       bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
       Transfering files to <default-ubuntu-1404>
       bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
       [2015-01-19T15:33:52+00:00] INFO: Starting chef-zero on port 8889 with repository at repository at /tmp/kitchen
         One version per cookbook

       [2015-01-19T15:33:52+00:00] INFO: Forking chef instance to converge...
       Starting Chef Client, version 11.10.4
       [2015-01-19T15:33:52+00:00] INFO: *** Chef 11.10.4 ***
       [2015-01-19T15:33:52+00:00] INFO: Chef-client pid: 2711
       [2015-01-19T15:33:52+00:00] INFO: Setting the run_list to ["recipe[apt]", "recipe[cassandra::default]"] from JSON
       [2015-01-19T15:33:52+00:00] INFO: Run List is [recipe[apt], recipe[cassandra::default]]
       [2015-01-19T15:33:52+00:00] INFO: Run List expands to [apt, cassandra::default]
       [2015-01-19T15:33:52+00:00] INFO: Starting Chef Run for default-ubuntu-1404
       [2015-01-19T15:33:52+00:00] INFO: Running start handlers
       [2015-01-19T15:33:52+00:00] INFO: Start handlers complete.
       [2015-01-19T15:33:52+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: /reports/nodes/default-ubuntu-1404/runs
       resolving cookbooks for run list: ["apt", "cassandra::default"]
       [2015-01-19T15:33:53+00:00] INFO: Loading cookbooks [apt, ark, cassandra, java, line, sysctl, ulimit, user, yum]
       Synchronizing Cookbooks:
         - apt
       [2015-01-19T15:33:53+00:00] INFO: Storing updated cookbooks/cassandra/recipes/configure.rb in the cache.
         - cassandra
         - user
         - java
         - ulimit
         - yum
         - ark
         - sysctl
         - line
       Compiling Cookbooks...
       [2015-01-19T15:33:54+00:00] WARN: Cloning resource attributes for execute[apt-get-update-periodic] from prior resource (CHEF-3694)
       [2015-01-19T15:33:54+00:00] WARN: Previous execute[apt-get-update-periodic]: /tmp/kitchen/cache/cookbooks/apt/recipes/default.rb:80:in `from_file'
       (...)
       [2015-01-19T15:33:54+00:00] ERROR: #<RuntimeError: Error on token 'node.chef_environment' at 72 of 'run_list:role\[cassandra\] AND chef_environment:#{node.chef_environment}': Expected TO in range query>
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/solr/solr_parser.rb:117:in `parse_error'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/solr/solr_parser.rb:147:in `read_single_expression'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/solr/solr_parser.rb:100:in `read_expression'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/solr/solr_parser.rb:17:in `parse'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/endpoints/search_endpoint.rb:95:in `search'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/endpoints/search_endpoint.rb:13:in `get'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/rest_base.rb:29:in `call'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/rest_router.rb:23:in `call'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-zero-1.7.3/lib/chef_zero/server.rb:335:in `block in make_app'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/server.rb:412:in `call'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/server.rb:412:in `handle_request'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/server.rb:306:in `process_client'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/server.rb:215:in `block in run'

       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/thread_pool.rb:94:in `call'
       /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/puma-1.6.3/lib/puma/thread_pool.rb:94:in `block in spawn_thread'
       [2015-01-19T15:33:54+00:00] INFO: HTTP Request Returned 500 Internal Server Error: error

       ================================================================================
       Recipe Compile Error in /tmp/kitchen/cache/cookbooks/cassandra/recipes/default.rb
       ================================================================================


       Net::HTTPFatalError
       -------------------
       500 "Internal Server Error"


       Cookbook Trace:
       ---------------
         /tmp/kitchen/cache/cookbooks/cassandra/recipes/configure.rb:41:in `from_file'
         /tmp/kitchen/cache/cookbooks/cassandra/recipes/default.rb:35:in `from_file'


       Relevant File Content:
       ----------------------
       /tmp/kitchen/cache/cookbooks/cassandra/recipes/configure.rb:

        34:  else
        35:    old_token = node[:cassandra][:token]
        36:    # Searching for Cassandra nodes inside the same Chef environment

        37:    cassandra_seeds = []
        38:    cassandra_nodes_macs = []
        39:    query = 'run_list:role\[' + node[:cassandra][:seed_role] + '\] AND chef_environment:#{node.chef_environment}'
        40:    log query
        41>>   search(:node, query).each do |n|
        42:  #  search(:node, "run_list:role\\[cassandra\\]  AND \
        43:  #    chef_environment:#{node.chef_environment}").each do |n|
        44:  #  search(:node, "roles:*#{node[:cassandra][:seed_role]}  AND \
        45:  #    chef_environment:#{node.chef_environment}").each do |n|
        46:      cassandra_seeds << n[:fqdn] unless n[:fqdn] == node[:fqdn]
        47:    end
        48:  
        49:    cassandra_nodes_macs << node[:macaddress] unless cassandra_nodes_macs.include?(node[:macaddress])
        50:  




       Running handlers:
       [2015-01-19T15:33:54+00:00] ERROR: Running exception handlers
       Running handlers complete

       [2015-01-19T15:33:54+00:00] ERROR: Exception handlers complete
       [2015-01-19T15:33:54+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       Chef Client failed. 0 resources updated in 1.955433697 seconds
       [2015-01-19T15:33:54+00:00] ERROR: 500 "Internal Server Error"
       [2015-01-19T15:33:54+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <default-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sudo -E chef-client -z --config /tmp/kitchen/client.rb --log_level info --json-attributes /tmp/kitchen/dna.json]
>>>>>> ----------------------

Thanks


Solution

  • After comment I played with chef-shell (to try search in the same context as a recipe)

    Valid syntax I found to work (no quote around role name):

    • search(:node, 'run_list:role\[mysql-server-wso2\] AND chef_environment:I1')
    • search(:node, "run_list:role\\[mysql-server-wso2\\] AND chef_environment:I1")

    The double quote way sounds easiest for variable interpolation. Exemple with your case:

    query = "run_list:role\\[#{node[:cassandra][:seed_role]}\\] AND chef_environment:#{node.chef_environment}"
    

    You may ensure it is properly interpolated by adding a log line just after like this:

    Chef::Log.info "Query is: #{query}"
    

    Old Answer: I don't know why you set this in single quotes, but its likely your problem:

     query = 'run_list:role\[' + node[:cassandra][:seed_role] + '\] AND chef_environment:#{node.chef_environment}'
    

    Being in single quote, #{node.chef_environment} is not substitued and is sent to chef server as is, which the server can't resolv properly and fail with a 500 error.

    Written like this the query should be ok:

    query = "run_list:role['#{node[:cassandra][:seed_role]}'] AND chef_environment:#{node.chef_environment}"