I am trying to create a cloudformation template which installs puppet and also aws puppet module. I am able to create my instance with puppet, define the security group etc. and it seems to work fine, but I also want to install the aws puppet module as part of my template. This is the code for my puppet instance
"PuppetMasterInstance" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"puppet3" : [],
"puppet3-server" : [],
"ruby-devel" : [],
"gcc" : [],
"make" : [],
"rubygems" : []
},
"rubygems" : {
"json" : []
}
},
"files": {
"/etc/yum.repos.d/epel.repo": {
"source": "https://s3.amazonaws.com/cloudformation-examples/enable-epel-on-amazon-linux-ami",
"mode": "000644",
"owner": "root",
"group": "root"
},
"/etc/puppet/autosign.conf": {
"content": "*.internal\n",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/fileserver.conf": {
"content": "[modules]\n allow *.internal\n",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/puppet.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\n",
" logdir=/var/log/puppet\n",
" rundir=/var/run/puppet\n",
" ssldir=$vardir/ssl\n",
" pluginsync=true\n",
"[agent]\n",
" classfile=$vardir/classes.txt\n",
" localconfig=$vardir/localconfig\n"
]
]
},
"mode": "000644",
"owner": "root",
"group": "root"
},
"/etc/puppet/modules/cfn/manifests/init.pp": {
"content": "class cfn {}",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/modules/cfn/lib/facter/cfn.rb": {
"source": "https://s3.amazonaws.com/cloudformation-examples/cfn-facter-plugin.rb",
"mode": "100644",
"owner": "root",
"group": "wheel"
}
},
"services": {
"sysvinit": {
"puppetmaster": {
"enabled": "true",
"ensureRunning": "true"
}
}
}
}
}
},
"Properties": {
"InstanceType": {
"Ref": "InstanceType"
},
"SecurityGroups": [
{
"Ref": "PuppetGroup"
}
],
"ImageId": {
"Ref": "AmiID"
},
"KeyName": {
"Ref": "KeyName"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash\n",
"yum update -y \n",
"/opt/aws/bin/cfn-init --region ",
{
"Ref": "AWS::Region"
},
" -s ",
{
"Ref": "AWS::StackName"
},
" -r PuppetMasterInstance ",
" --access-key ",
{
"Ref": "CFNKeys"
},
" --secret-key ",
{
"Fn::GetAtt": [
"CFNKeys",
"SecretAccessKey"
]
},
"\n",
"/opt/aws/bin/cfn-signal -e $? '",
{
"Ref": "PuppetMasterWaitHandle"
},
"'\n"
]
]
}
}
}
}
This works fine, however I want to execute the following commands after puppet is installed:
"gem install aws-sdk-core",
"gem install retries",
"export AWS_ACCESS_KEY_ID=my_key",
"export AWS_SECRET_ACCESS_KEY=my_secret",
"puppet module install puppetlabs-aws"
I tried using the "commands:" tag before "files:" and the template failed. I tried to put the code inside the "UserData": but it failed again. I couldn't find information about the order of execution of the different sections in the template and I assume the failures are due to wrong order of execution (puppet & ruby are not installed when the commands run).
Any help will be much appreciated.
I found a very informative post on the aws forum regarding the order of execution. AWS::CloudFormation::Init executes in the following order:
packages -> groups -> users-> sources -> files -> commands -> services
source: https://forums.aws.amazon.com/message.jspa?messageID=414670
The way I fixed the problem is probably far from ideal but it works:
"PuppetMasterInstance" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init" : {
"configSets" : {
"ascending" : [ "config" , "config2" ],
"descending" : [ "config2" , "config" ]
},
"config" : {
"packages" : {
"yum" : {
"puppet3" : [],
"puppet3-server" : [],
"ruby-devel" : [],
"gcc" : [],
"make" : [],
"rubygems" : []
},
"rubygems" : {
"json" : []
}
},
"files": {
"/etc/yum.repos.d/epel.repo": {
"source": "https://s3.amazonaws.com/cloudformation-examples/enable-epel-on-amazon-linux-ami",
"mode": "000644",
"owner": "root",
"group": "root"
},
"/etc/puppet/autosign.conf": {
"content": "*.internal\n",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/fileserver.conf": {
"content": "[modules]\n allow *.internal\n",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/puppet.conf": {
"content": {
"Fn::Join": [
"",
[
"[main]\n",
" logdir=/var/log/puppet\n",
" rundir=/var/run/puppet\n",
" ssldir=$vardir/ssl\n",
" pluginsync=true\n",
"[agent]\n",
" classfile=$vardir/classes.txt\n",
" localconfig=$vardir/localconfig\n"
]
]
},
"mode": "000644",
"owner": "root",
"group": "root"
},
"/etc/puppet/modules/cfn/manifests/init.pp": {
"content": "class cfn {}",
"mode": "100644",
"owner": "root",
"group": "wheel"
},
"/etc/puppet/modules/cfn/lib/facter/cfn.rb": {
"source": "https://s3.amazonaws.com/cloudformation-examples/cfn-facter-plugin.rb",
"mode": "100644",
"owner": "root",
"group": "wheel"
}
},
"services": {
"sysvinit": {
"puppetmaster": {
"enabled": "true",
"ensureRunning": "true"
}
}
}
},
"config2" : {
"commands" : {
"1" : {
"command" : "gem install aws-sdk-core"
},
"2" : {
"command" : "gem install retries"
},
"3" : {
"command" : "export _MYAWSKEY_"
},
"4" : {
"command" : "export MY_AWS_SECRET_"
},
"5" : {
"command" : "puppet module install puppetlabs-aws"
}
}
}
}
},
"Properties": {
"InstanceType": {
"Ref": "InstanceType"
},
"SecurityGroups": [
{
"Ref": "PuppetGroup"
}
],
"ImageId": {
"Ref": "AmiID"
},
"KeyName": {
"Ref": "KeyName"
},
"UserData": {
"Fn::Base64": {
"Fn::Join": [
"",
[
"#!/bin/bash\n",
"yum update -y \n",
"/opt/aws/bin/cfn-init -c ascending --region ",
{
"Ref": "AWS::Region"
},
" -s ",
{
"Ref": "AWS::StackName"
},
" -r PuppetMasterInstance ",
" --access-key ",
{
"Ref": "CFNKeys"
},
" --secret-key ",
{
"Fn::GetAtt": [
"CFNKeys",
"SecretAccessKey"
]
},
"\n",
"/opt/aws/bin/cfn-signal -e $? '",
{
"Ref": "PuppetMasterWaitHandle"
},
"'\n"
]
]
}
}
}
}
By specifying order of execution of the configSets I am able to run everything I need to install and configure puppet and after that run the commands to install the plugin.