Tuesday, July 06, 2010

Chef installation and minimal configuration

I started to play with Chef the other day. The instructions on the wiki are a bit confusing, but help on twitter (thanks @jtimberman) and on the #chef IRC channel (thanks @kallistec) has been great. I am at the very minimal stage of having a chef client talking to a chef server. I hasten to write down what I've done so far, both for my own sake and for others who might want to do the same. My OS is Ubuntu 10.04 32-bit on both machines.

First of all: as the chef wiki says, make sure you have FQDNs correctly set up on both client and server, and that they can ping each other at a minimum using the FQDN. I added the FQDN to the local IP address line in /etc/hosts, so that 'hostname -f' returned the FQDN correctly. In what follows, my Chef server machine is called chef.example.com and my Chef client machine is called client.example.com.

Installing the Chef server


Here I went the Ruby Gems route, because the very latest Chef (0.9.4) had not been captured in the Ubuntu packages yet when I tried to install it.

a) install pre-requisites

# apt-get install ruby ruby1.8-dev libopenssl-ruby1.8 rdoc ri irb build-essential wget ssl-cert

b) install Ruby Gems

# wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
# tar xvfz rubygems-1.3.7.tgz
# cd rubygems-1.3.7
# ruby setup.rb
# ln -sfv /usr/bin/gem1.8 /usr/bin/gem

c) install the Chef gem

# gem install chef

d) install the Chef server by bootstrapping with the chef-solo utility

d1) create /etc/chef/solo.rb with contents:


file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
recipe_url "http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz"

d2) create /etc/chef/chef.json with contents:

{
"bootstrap": {
"chef": {
"url_type": "http",
"init_style": "runit",
"path": "/srv/chef",
"serve_path": "/srv/chef",
"server_fqdn": "chef.example.com",
"webui_enabled": true
}
},
"run_list": [ "recipe[bootstrap::server]" ]
}

d3) run chef-solo to bootstrap the Chef server install:

# chef-solo -c /etc/chef/solo.rb -j /etc/chef/chef.json

e) create an initial admin client with the Knife utility, to interact with the API

#knife configure -i
Where should I put the config file? [~/.chef/knife.rb]
Please enter the chef server URL: [http://localhost:4000] http://chef.example.com
Please enter a clientname for the new client: [root]
Please enter the existing admin clientname: [chef-webui]
Please enter the location of the existing admin client's private key: [/etc/chef/webui.pem]
Please enter the validation clientname: [chef-validator]
Please enter the location of the validation key: [/etc/chef/validation.pem]
Please enter the path to a chef repository (or leave blank):

f) create an intial Chef repository

I created a directory called /srv/chef/repos , cd-ed to it and ran this command:

# git clone git://github.com/opscode/chef-repo.git

At this point, you should have a functional Chef server, although it won't help you much unless you configure some clients.

Installing a Chef client

Here's the bare minimum I did to get a Chef client to just talk to the Chef server configured above, without actually performing any cookbook recipe yet (I leave that for another post).

The first steps are very similar to the ones I followed when I installed the Chef server.


a) install pre-requisites

# apt-get install ruby ruby1.8-dev libopenssl-ruby1.8 rdoc ri irb build-essential wget ssl-cert

b) install Ruby Gems

# wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz
# tar xvfz rubygems-1.3.7.tgz
# cd rubygems-1.3.7
# ruby setup.rb
# ln -sfv /usr/bin/gem1.8 /usr/bin/gem

c) install the Chef gem

# gem install chef

d) install the Chef client by bootstrapping with the chef-solo utility

d1) create /etc/chef/solo.rb with contents:

file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
recipe_url "http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz"
Caveat for this stepCaveat for this step
d2) create /etc/chef/chef.json with contents:

{
"bootstrap": {
"chef": {
"url_type": "http",
"init_style": "runit",
"path": "/srv/chef",
"serve_path": "/srv/chef",
"server_fqdn": "chef.example.com",
"webui_enabled": true
}
},
"run_list": [ "recipe[bootstrap::client]" ]
}

Note that the only difference so far between the Chef server and the Chef client bootstrap files is the one directive at the end of chef.json, which is bootstrap::server for the server and bootstrap::client for the client.

Caveat for this step: if you mess up and bootstrap the client using the wrong chef.json file containing the bootstrap::server directive, you will end up with a server and not a client. I speak from experience -- I did exactly this, then when I tried to run chef-client on this box, I got:

WARN: HTTP Request Returned 401 Unauthorized: Failed to authenticate!

/usr/lib/ruby/1.8/net/http.rb:2097:in `error!': 401 "Unauthorized" (Net::HTTPServerException)

d3) run chef-solo to bootstrap the Chef client install:

# chef-solo -c /etc/chef/solo.rb -j /etc/chef/chef.json

At this point, you should have a file called client.rb in /etc/chef on your client machine, with contents similar to:

#
# Chef Client Config File
#
# Dynamically generated by Chef - local modifications will be replaced
#

log_level :info
log_location STDOUT
ssl_verify_mode :verify_none
chef_server_url "http://chef.example.com:4000"

validation_client_name "chef-validator"
validation_key "/etc/chef/validation.pem"
client_key "/etc/chef/client.pem"

file_cache_path "/srv/chef/cache"
pid_file "/var/run/chef/chef-client.pid"

Mixlib::Log::Formatter.show_time = false
Caveat for this stepCaveat for this stepCaveat for this step
e) validate the client against the server

e1) copy /etc/chef/validation.pem from the server to /etc/chef on the client
e2) run chef-client on the client; for debug purposes you can use:

# chef-client -l debug

If everything goes well, you should see a message of the type:

# chef-client
INFO: Starting Chef Run
INFO: Client key /etc/chef/client.pem is not present - registering
WARN: Node client.example.com has an empty run list.
INFO: Chef Run complete in 1.209376 sec WARN: HTTP Request Returned 401 Unauthorized: Failed to authenticate!
31 < ggheo > 30 /usr/lib/ruby/1.8/nCaveat for this stepet/http.rb:2097:in `error!': 401 "Unauthorized" (Net::HTTPServerException)onds
INFO: Running report handlers
INFO: Report handlers complete

You should also have a file called client.pem containing a private key that the client will be using when talking to the server. At this point, you should remove validation.pem from /etc/chef on the client, as it is not needed any more.

You can also run this command on the server to see if the client got registered with it:

# knife client list -c /etc/chef/knife.rb

The output should be something like:

[
"chef-validator",
"chef-webui",
"chef.example.com",
"root",
"client.example.com"
]

That's it for now. As I warned you, nothing exciting happened here except for having a Chef client that talks to a server but doesn't actually DO anything. Stay tuned for more installments in my continuing chef saga though...


5 comments:

Anonymous said...

Ohai!

Thank you for this post, quite detailed and helpful.

A note for you and anyone who happens by here, the bootstrap cookbook is now considered deprecated, and the chef cookbook should be used instead.

- bootstrap::client -> chef::bootstrap_client
- bootstrap::server -> chef::bootstrap_server

The "Bootstrap Chef RubyGems installation" page on the wiki is updated with this, along with usage information, and the chef cookbook's readme has more detail as well.

Running the bootstrap cookbook's recipes will print a message about this deprecation.

Also, the Opscode apt repository (apt.opscode.com) is updated with the 0.9.6 packages, and we are working with a Debian developer to get this version uploaded to Debian, too.

Wes Winham said...

The opscode apt repo doesn't work at least on Ubuntu 8.04:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
chef-server: Depends: chef-server-webui (= 0.9.6+dfsg-2) but it is not going to be installed
Depends: chef-solr (= 0.9.6+dfsg-2) but it is not going to be installed
chef-server-api: Depends: libmerb-assets-ruby but it is not installable
Depends: libmerb-haml-ruby but it is not installable
Depends: libmerb-helpers-ruby but it is not installable
Depends: rabbitmq-server (>= 1.6) but it is not installable
Depends: couchdb (>= 0.9.1) but it is not installable
Depends: thin but it is not installable
Depends: chef-solr (= 0.9.6+dfsg-2) but it is not going to be installed
E: Broken packages

Going the rubygems route like in the post seems to be the only way to go for now.

bharath said...

nice post

thank you

Ruxandra T. said...

Hi, thanks a lot for the series of posts about Chef. They are very helpful if you are starting with it, like me.
I wonder if you mind showing the full dates of the comments, so that I can relate them to the current status-quo. At the time of writing I can see only the hour of the comments.

Grig Gheorghiu said...

Thanks for the feedback, Ruxandra. I fixed the timestamp format for the comments so hopefully it's clearer now ;-)

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...