The examples

Over the last two weeks (part one & part two) I have slowly going into detail about module set up and some architecture, well nows the time for real world.

To save me writing loads of puppet code I am going to abbreviate and leave some bits out. First things first a simple module.

init.pp

class javaapp ( $conf_dir = $javaapp::params::conf_dir) inherits javaapp::params {

  class {
    "javaapp::install":
    conf_dir => $conf_dir
  }

}

install.pp

class javaapp::install (conf_dir = $javaapp::params::conf_dir ) inherits javaapp::params {

 package {
    "javaapp":
    name => "$javaapp::params::package",
    ensure => installed,
    before => Class["javaapp::config"],
  }

  file {
    "/var/lib/tomcat6/shared":
    ensure => directory,
  }

}

config.pp

class javaapp::config (app_var1 = $javaapp::params::app_var1,
                       app_var2 = $javaapp::params::app_var2) inherits javaapp::params {

  file {
    "/etc/javaapp/javaapp.conf":
    content => template("javaapp/javaapp.conf"),
    owner   => 'tomcat',
    group   => 'tomcat'
  }
}

params.pp

class javaapp::params ( ) {

$conf_dir = "/etc/javaapp"
$app_var1 = "1.2.3.4/32"
$app_var2 = "host.domain.com"

}

One simple module in the module directory. As you can see I have put all parameters into one file, it use to be that you’d specify the same defaults in every file so in init and config you would duplicate the same variables. Well that is just insane and if you have complicated modules with several manifests in each one it gets difficult to maintain all the defaults. This way they are all in one file and are easy to identify and maintain, it by far isn’t perfect, it does work though and i’m not even sure if puppet supports it and if it doesn’t that is a failing of puppet but it does work with the latest 2.7.18 and i’m sure i’ve had it on all 2.7 variants at some point.

You should be aiming to set sensible defaults set every parameter regardless, but make sure it’s sensible, if you want to enforce the variable is set you can still not put an entry in params and just specify it without a default.

Now the /etc/puppet directory

Matthew-Smiths-MacBook-Pro-2:puppet soimafreak$ ls
auth.conf	extdata		hiera.yaml	modules
autosign.conf	fileserver.conf	manifests	puppet.conf

the auth, autosign and fileserver configs will depend on your infrastructure, but the two important configurations here are puppet.conf and hiera.conf

puppet.conf

[master]
certname=server.domain.com
modulepath = /etc/puppet/modules 
[main]
    # The Puppet log directory.
    # The default value is '$vardir/log'.
    logdir = /var/log/puppet

    # Where Puppet PID files are kept.
    # The default value is '$vardir/run'.
    rundir = /var/run/puppet

    # Where SSL certificates are kept.
    # The default value is '$confdir/ssl'.
    ssldir = $vardir/ssl

		autosign = true
		autosign = /etc/puppet/autosign.conf

[agent]
    # The file in which puppetd stores a list of the classes
    # associated with the retrieved configuratiion.  Can be loaded in
    # the separate ``puppet`` executable using the ``--loadclasses``
    # option.
    # The default value is '$confdir/classes.txt'.
    classfile = $vardir/classes.txt

    # Where puppetd caches the local configuration.  An
    # extension indicating the cache format is added automatically.
    # The default value is '$confdir/localconfig'.
    localconfig = $vardir/localconfig
    pluginsync = true

The only real change worth making to this is in the agent sector, plugin sync ensures that any plugins you install in puppet, like Firewall, VCSRepo, hiera etc are loaded by the agent, obviously on the agent you do not want all of the master config at the top.

Now the hiera.yaml file

hiera.yaml

---
:hierarchy:
      - %{env}
:backends:
    - yaml
:yaml:
    :datadir: '/etc/puppet/extdata'

Okay, to the trained eye this is sort of pointless, it tells puppet that it should look in a directory called /etc/puppet/extdata for a file called %{env}.yaml so in this case if env were to equal bob it would like for a file /etc/puppet/extdata/bob.yaml The advantage to this is that at some point if needed that file could be changed to for example

hiera.yaml

---
:hierarchy:
      - common
      - %{location}
      - %{env}
      - %{hostname}
:backends:
    - yaml
:yaml:
    :datadir: '/etc/puppet/extdata'

This basically provides a location for all variables that you are not able to tie down to a role which will be defined by the manifests.

Matthew-Smiths-MacBook-Pro-2:puppet soimafreak$ ls manifests/roles/
tomcat.pp	default.pp	app.pp	apache.pp

So we’ll look at the default node and tomcat to get a full picture of the variables being passed around.

default.pp

node default {
	
	#
	# Default node - base packages for all systems
	#

  # Define stages
  class {
    "sshd":     stage =>  first;
    "ntp":      stage =>  first;
  }
	
  # Needed for Facter to generate OS related information
  package {
    "redhat-lsb":
    ensure => "installed"
  }

  # mcollective
  class {
    "mcollective":
    mc_password   => "bobbypassword6",
    puppet_server => "puppet.domain.com",
    activemq_host => "mq.domain.com",
  }

  # Manage puppet
  include puppet
}

As you can see, this default node sets up some classes that must be on every box and ensures that packages that are vital are also installed. If you so feel the need to extrapolate this further you could have the default node inherit another node, for example you may have a company manifest as follows:

company.pp

node company {
$mc_password = "bobbypassword6"
$activemq_host = "mq.domain.com"

$puppet_server = $env ? {
    "bob" => 'bobpuppet.domain.com',
    default => 'puppet.domain.com',
  }
}

This company node manifest could be inherited by the default and then instead of having puppet_server => “puppet.domain.com”, you could have puppet_server => $puppet_server, which I think is nice and clear. The only recommendation is to keep your default and your role manifests as simple as possible, try and keep if statements out of them, can you push the decision into hiera? do you have a company.pp that would be sensible to have some env logic in it? are you able to take some existing logic and turn it into a fact ?

Be ruthless and push as much logic out as possible, use the tools to do the leg work and keep puppet manifests and modules simple to maintain.

Now finally the role,

tomcat.pp

node /*tomcat.*/ inherits default {

  include tomcat6

  # Installs java app using the init/install classes and default params, 
  include javaapp

  class {
    "javaapp::config"
    app_var1 => hiera('app_var1') 
    app_var2 => $fqdn
  }
}

The role should be “simple” but it also needs to make it clear what it’s setting, if you notice that several roles use the same class and in most cases the params are the same, change the params file, remove the options from the roles, try and keep it so what is in the roles is only overrides and as minimal as possible. The hiera vars and any set in the default / other node inheritance can all be referenced here.

Summary

Hopefully that helps some of you understand the options for placing variables within puppet in different locations. As I mentioned in the other posts, this method has 3 files, the params, the role the hiera file that’s it, all variables are in one fo those three so there’s no need to hunt through all of the manifests in a module to identify where the the variable may or may not be set, it is either defaulted of overridden, if it’s overridden it will be in the role manifest, from there you can work out if it’s in your default or hiera and so on.

Category:
Linux, Puppet
Tags:
,

Join the conversation! 6 Comments

  1. […] have an attributes directory, which s exactly what I try and do within puppet with my params stuff, Here but because it’s a well known structure it’s used more, this does mean that people can […]

    Reply
  2. […] part one for a generic overview Puppet with out barriers part two for manifest consolidation and Puppet with out barriers part three for params & […]

    Reply
  3. As it’s been some time now, are you still using this method? I want to find out as we are working with Puppet and Foreman and I want to try and get the setup of this correct from the outside. It seems like using Hiera and or Foreman to store params/variables is the way to go. I don’t like the idea of having defined variables in more than one place though. Do you think that having them in 3 places is still ok, or does it things?

    I am thinking about using dynamic environments within Puppet and Hiera to define variables and parameters for modules. We can also use the ENC features of Foreman to give hosts specific host groups (parent/child) association.

    Splitting your puppet environments into project named environments would be a good solution, using Hiera to then define project specific variables, i.e. Java version, etc.

    Also, do you think you can over do it with parameterising everything? Surely standardising installs would be a better concept, maybe not hard coding, but defaulting everything in Hiera?

    Reply
    • I’m not sure how foreman works but the principles are the same, modules should have sensible defaults set in one location (Params.pp) there should be one manifest per server roll where the varibles are either hard coded if they are the same across all environments or loaded from hiera / third party.

      We started with hiera and gradually moved I so the files were generated by a third party app and downloaded onto the server. The third party app we had created our environments through a nice little web gui an automatically generated the appropriate variables that were hen loaded into the environment.

      As for the paramerterising everything, if for example it is a RHEL only platform and it’s a bespoke module then maybe you don’t need to but it’s one of those things where if it is done up front you won’t get into the state of people then hacking in params later because they don;t understand how to populate the module properly or the hiera file etc etc. It will make the module more agile later.

      Reply

Don't be Shy, Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: