Pre-requisites
Issue
Attempting to run 'chef-client' after bootstrapping a specific node on windows is resulting in: -
knife winrm <machine> chef-client -m -x <user> -P <password> --verbose
[2017-03-15T07:48:28+00:00] INFO: *** Chef 12.19.36 ***
[2017-03-15T07:48:28+00:00] INFO: Platform: x64-mingw32
[2017-03-15T07:48:28+00:00] INFO: Chef-client pid: 16184
[2017-03-15T07:48:50+00:00] INFO: Client key C:\chef\client.pem is not present - registering
[2017-03-15T07:48:50+00:00] ERROR: Running exception handlers
[2017-03-15T07:48:50+00:00] ERROR: Exception handlers complete
[2017-03-15T07:48:50+00:00] FATAL: Stacktrace dumped to c:/chef/cache/chef-stacktrace.out
[2017-03-15T07:48:50+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-03-15T07:48:50+00:00] FATAL: NoMethodError: undefined method `closed?' for nil:NilClass
Did you mean? clone
Correctly, it has not found the client.pem and is using the validator for the first time but upon reading the stack trace, I can see the root cause is that the machine is unable to acquire the private key from the chef server. See below: -
>>>> Caused by Net::HTTPServerException: 407 "Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )"
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http/response.rb:120:in `error!'
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http/response.rb:129:in `value'
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http.rb:920:in `connect'
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http.rb:852:in `start'
C:/opscode/chef/embedded/lib/ruby/2.3.0/net/http.rb:1398:in `request'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http/basic_client.rb:70:in `request'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:340:in `block in send_http_request'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:372:in `block in retrying_http_errors'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:370:in `loop'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:370:in `retrying_http_errors'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:333:in `send_http_request'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:148:in `request'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/http.rb:131:in `post'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/api_client/registration.rb:96:in `create'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/api_client/registration.rb:87:in `create_or_update'
C:/opscode/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.19.36-universal-mingw32/lib/chef/api_client/registration.rb:58:in `run'
What have I tried?
Manually creating and downloading the private key from the chef server - still get the 407 at a later stage. This leads me to believe that there actually is some proxy issue however I'm able to communicate with the chef server just fine from the node
I've run the WinRM command using a domain controller's credentials so I'm certain there is no lack of permissions and have enabled the following WinRM settings on the node.
cmd.exe /c winrm quickconfig -q
cmd.exe /c winrm quickconfig -transport:http
cmd.exe /c winrm set winrm/config @{MaxTimeoutms="1800000"}
cmd.exe /c winrm set winrm/config/winrs @{MaxMemoryPerShellMB="3000"}
cmd.exe /c winrm set winrm/config/service @{AllowUnencrypted="true"}
cmd.exe /c winrm set winrm/config/service/auth @{Basic="true"}
cmd.exe /c winrm set winrm/config/client/auth @{Basic="true"}
cmd.exe /c winrm set winrm/config/listener?Address=*+Transport=HTTP @{Port="5985"}
cmd.exe /c netsh advfirewall firewall set rule name="Windows Remote Management (HTTP-In)" profile=public protocol=tcp localport=5985 remoteip=localsubnet new remoteip=any
cmd.exe /c net stop winrm
cmd.exe /c sc config winrm start= auto
cmd.exe /c net start winrm
Update running chef-client local to the machine, without even using WinRM results in the same error. Leaving WinRM settings and commands in post for historic reference
Knife.rb
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "admin"
client_key "#{current_dir}/admin_user.pem"
validation_client_name 'org-validator'
validation_key "#{current_dir}/org-validator.pem"
chef_server_url "https://chef.fqdn/organizations/org"
cookbook_path ["#{current_dir}/../cookbooks"]
Client.rb created by bootstrap process
log_level :info
log_location STDOUT
chef_server_url "https://chef.fqdn/organizations/org"
validation_client_name "org-validator"
file_cache_path "c:/chef/cache"
file_backup_path "c:/chef/backup"
cache_options ({:path => "c:/chef/cache/checksums", :skip_expires => true})
# Using default node name (fqdn)
trusted_certs_dir "c:/chef/trusted_certs"
It's worth noting that I'm not having this problem on any other machines on the network. I'm struggling to find any more debugging information from the chef-client or the chef-server.
ANSWER
As mentioned in the accepted answer, the machine had HTTP_PROXY set in the windows environment variables which chef recognises and attempts to use. The solution was to add no_proxy to the client.rb or to remove the HTTP_PROXY environment variable.
no_proxy 'chef.fqdn'
If the proxy isn't required for Chef either unset the proxy environment variables or set no_proxy appropriately.