I've set up an Artifactory instance and set up a remote Chef Supermarket on it which replicates from a local repository on another Artifactory. I'm happy with the pipeline that is in place as it means the delivery of the artifacts is automated to the customer through push/pull replication.
At the customer end we also have a Chef Server which is intended to orchestrate the deployments through running the cookbooks on various servers according to their roles/environment setup.
ORG: Artifactory (Local Chef Repo)
-----------------------------------
|
<web>
|
-----------------------------------
CUST: Artifactory (Remote Chef Repo)
Chef Server
I have an empty directory in which I created .chef/knife.rb, obtained the trust between chef server and configured my supermarket.
knife[:supermarket_site] = 'http://<user>:<api_key>@customer-artifactory:8081/artifactory/api/chef/org-chef'
When I run knife supermarket list I get a list of all the cookbooks from the Chef Supermarket hosted in the artifactory. However, the issue I face is that when I engage Chef Server into the mix, it fails to resolve the cookbooks.
user@workstation:~/chef-ops$ knife ssh 'name:vm01' sudo chef-client -x <user> -P <password>
vm01 knife sudo password:
vm01
vm01 Starting Chef Infra Client, version 15.3.14
vm01 resolving cookbooks for run list: ["repository"]
vm01
vm01 ================================================================================
vm01 Error Resolving Cookbooks for Run List:
vm01 ================================================================================
vm01
vm01 Missing Cookbooks:
vm01 ------------------
vm01 The following cookbooks are required by the client but don't exist on the server:
vm01 * repository
vm01
vm01
vm01 Expanded Run List:
vm01 ------------------
vm01 * repository
vm01
vm01 System Info:
vm01 ------------
vm01 chef_version=15.3.14
vm01 platform=ubuntu
vm01 platform_version=18.04
vm01 ruby=ruby 2.6.4p104 (2019-08-28 revision 67798) [x86_64-linux]
vm01 program_name=/usr/bin/chef-client
vm01 executable=/opt/chef/bin/chef-client
vm01
vm01
vm01 Running handlers:
vm01 [2019-10-11T10:28:05+00:00] ERROR: Running exception handlers
vm01 Running handlers complete
vm01 [2019-10-11T10:28:05+00:00] ERROR: Exception handlers complete
vm01 Chef Infra Client failed. 0 resources updated in 04 seconds
vm01 [2019-10-11T10:28:05+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
vm01 [2019-10-11T10:28:05+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
vm01 [2019-10-11T10:28:05+00:00] FATAL: Net::HTTPServerException: 412 "Precondition Failed"
I realise this is likely because the setting in knife.rb is not propagated to Chef Server, so I would need to configure Chef Server to look at Artifactory.
I've looked at the following documentation regarding using private Chef Supermarkets, but they do not seem to fit my use case.
Having looked through these various resources (and more) I am unsure that Artifactory and Chef Server can be set up to achieve an automated cookbook resolution. There doesn't seem to be any documentation on this. For example, what would the Chef Server callback URL be under Artifactory?
oc_id['applications'] = {
'supermarket' => {
'redirect_uri' => 'http://customer-artifactory:8081/artifactory/api/oauth2/???'
}
}
Am I right in thinking that Chef Server is unable to resolve the cookbooks from Artifactory directly and that we would need to create some kind of pipeline to synchronize Chef Server with the Artifactory chef repository?
The only option I can see is to create a procedure that regularly does a knife supermarket list and then iterates through the results, downloading them from the Artifactory Supermarket and then uploads all new versions to Chef Server via a shell script on a regular basis (cron job/jenkins). I thought it would be better for Chef Server to request cookbooks from the supermarket on an ad-hoc basis as it would only then request what it needs at any given moment.
Am I wrong in thinking that this feels like something Artifactory/Chef Server could/should be doing for me?
AFAIK: Chef Supermarket and Chef Infra Server (Bookshelf) are not connected so there is no automatic cookbook resolution. You would need a process that uploads the cookbooks from the supermarket/Artifactory to the Chef server or use the Policyfiles https://docs.chef.io/policyfile.html approach.