How can i install puppet cluster on Amazon EC2 instances?

Question

I'm using ubuntu 12.04 AMI in EC2 for creating puppet cluster and i'm facing problems while configuring it. The problem is that the master is not able to recognize the slaves. Do i need more packages other than mysql

/etc/mysql/my.cnf

what changes do i need in the above file?

Answer 1

Puppet is a configuration management tool that allows automating the process of defining and maintaining consistent state of several developer workstations. It is a descriptive, centralized and client-server based system. The central server is configured and the clients synchronize themselves to it to ensure that all systems end in the described state. For instance, the task of ensuring the same development environment on all developer systems in a project can be easily accomplished using Puppet. Here is a quick procedure to set up a Puppet server and one Puppet client on Amazon EC2 instance having Ubuntu OS, and also installing Puppet Dashboard on server to view the status of the clients.

Prerequisites

• Two ec2 instances set up with Ubuntu ami.
• One instance named as puppetserver and other as puppetclient.

Procedure

Puppet server and client set up

• Configuring hosts files View the /etc/hostname file on puppetserver and puppetclient. These are the Puppet server and client hostnames respectively Edit /etc/hosts file on both the systems. Add server and client IPs and corresponding hostnames.

• Setting up the Puppet Server Enabling the Puppet Labs Package repository

• Download the "puppetlabs-release" package for the OS (here, Ubuntu 12.04) on Puppet server
• Install the package by running dpkg -i
• Run apt-get update to get new list of available packages.

For example, to enable the repository for Ubuntu 12.04, Precise Pangolin:

wget https://apt.puppetlabs.com/puppetlabs-release-precise.deb 
sudo dpkg -i puppetlabs-release-precise.deb 
sudo apt-get updateInstall Puppet

Install Puppet

Install puppetmaster

sudo apt-get update sudo apt-get install puppetmaster

Setting up the Puppet Client

Install Puppet on the puppet client(s)

sudo apt-get update sudo apt-get install puppet

Specify the Puppet server domain name on the client. To do this, modify the
/etc/puppet/puppet.conf file and add the line server=. The client can now connect to the Puppet master.

Start the Puppet agent service for establishing first communication between server and client.

sudo puppet agent --verbose --no-daemonize --onetime

This starts a connection to the Puppet master process that is listening on port 8140 on the Puppet server. The output will be verbose, and the agent will not continue running in the background as a daemon. Also, it will run only one time, that is, after the connection is closed, the agent process will exit. The output looks like:

The client has made itself known to the server by sending an SSL certificate request. The server needs to certify the client. To view the list of yet-to-be signed certificates on the server

sudo puppet cert --list

This lists the following

Sign the client node's SSL certificate

sudo puppet cert --sign <puppet client name>

Client can now establish full connection to the server and poll the Puppet master for any configuration updations.

Defining Configurations

We have set up puppet on both Puppet server and client and have also established communication between the two machines. Next step is to define the configuration for the target systems using puppet manifest. These manifests are specified in site.pp file.

As an example, we define a manifest that will create a helloworld.txt file on the client.

Defining manifest

Put the following manifest definition in /etc/puppet/manifests/site.pp file,

node "<puppet client hostname>" { file { "/home/ubuntu/helloworld.txt": content => "This is test content", ensure => file, owner => "ubuntu", group => "ubuntu", mode => 0644 } }

This manifest defines that the puppet client must have a helloworld.txt file in /home/ubuntu/ folder with content, This is test content.

Getting changes on client

On puppet client, run the following command.

sudo puppet agent -t

The puppet client pulls the manifests defined in the site.pp file on the puppet server. It learned that a file named helloworld.txt with defined specifications, is expected to exist at location /home/ubuntu. Since, no such file exists on the client, the agent takes action and creates the file.

View the 'helloworld.txt' file

To verify that the client exists in a state defined by the Puppet server, run the following command

sudo vi /home/ubuntu/helloworld.txt

The file contents are same as defined in the manifest definition on the server.

Installing Puppet Dashboard

Overview Puppet Dashboard is a GUI that interfaces with Puppet. It can be used to view and report the status of all the client nodes. Puppet dashboard runs on port 3000 on the puppet server.

Following are the steps for set up

1. Installing external dependencies

Dashboard is a Ruby on Rails web app and thus requires certain software to be installed RubyGems Rake version 0.8.3 or newer MySQL database server version 5.x Ruby-MySQL bindings version 2.7.x or 2.8.x

Install the packages

sudo apt-get install -y build-essential irb libmysql-ruby libmysqlclient-dev libopenssl-ruby libreadline-ruby mysql-server rake rdoc ri ruby ruby-dev

Install RubyGems package system

( URL="http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz" PACKAGE=$(echo $URL | sed "s/\.[^\.]*$//; s/^.*\///") cd $(mktemp -d /tmp/install_rubygems.XXXXXXXXXX) && \ wget -c -t10 -T20 -q $URL && \ tar xfz $PACKAGE.tgz && \ cd $PACKAGE && \ sudo ruby setup.rb )

Create gem as an alternative name for gem1.8

sudo update-alternatives --install /usr/bin/gem gem /usr/bin/gem1.8 1

Installing Puppet Dashboard

Install puppet-dashboard from puppetlabs package repository

sudo apt-get update sudo apt-get install puppet-dashboard

Configuring Dashboard

Modify the database.yml file. It can be found at /usr/share/puppet-dashboard/config/database.yml.

Under the key-value pairs for production environment, the database value 'dashboard_production' specifies the dashboard database name, and username value 'dashboard' specifies the user for this database. In the next step, we will create both the database and the user. password value is the password for MySQL.

Creating and Configuring MySQL database

Create the user and database for puppet-dashboard. Navigate to MySQL command line

CREATE DATABASE dashboard_production CHARACTER SET utf8; CREATE USER 'dashboard'@'localhost' IDENTIFIED BY 'my_password'; GRANT ALL PRIVILEGES ON dashboard_production.* TO 'dashboard'@'localhost';

Configure MySQL's maximum packet size to permit larger rows in database

set global max_allowed_packet = 33554432;

Also modify the mysql configuration file /etc/mysql/my.cnf

Allowing 32MB allows an occasional 17MB row with plenty of spare room

max_allowed_packet = 32M

To create dashboard tables, run the following command in the puppet-dashboard folder

cd /usr/share/puppet-dashboard rake RAILS_ENV=production db:migrate

Testing that Dashboard is working

Start the dashboard using Ruby’s built-in WEBrick server

cd /usr/share/puppet-dashboard
sudo ./script/server -e production

Dashboard instance starts on port 3000 using the “production” environment. Dashboard’s UI can be viewed at :3000

Configure puppet

Both the puppet server and client need to be configured for the dashboard to receive reports. Configure agent nodes to submit reports to master by turning their reporting ON.

puppet.conf (on each agent)

[agent]
report = true

Configure the server. Add the http report handler to puppet server's reports setting and set reporturl to Dashboard instance’s reports/upload URL

puppet.conf (on puppet master)

[master]
reports = store, http
reporturl = http://<server hostname>:3000/reports/upload

For enabling dashboard's external node classifier(ENC),

puppet.conf (on puppet master)

[master]
node_terminus = exec
external_nodes = /usr/bin/env PUPPET_DASHBOARD_URL=http://<server hostname>:3000 /usr/share/puppet-dashboard/bin/external_node

Testing Puppet's connection to Dashboard

Restart the puppet master Run one of the puppet agents to test the configurations

sudo puppet agent -t

The output will be:

This means that the report has arrived. To process it, we will activate the delayed_job workers.

Starting delayed_job workers

Run the following command

cd /usr/share/puppet-dashboard
sudo env RAILS_ENV=production script/delayed_job -p dashboard -n 1 -m start

This starts the delayed_job workers, and completes the pending task.

Thus, puppet is now installed on two EC2 instances, out of which one is server and the other is client. Also, puppet-dashboard is installed to view the status of the client nodes.