Check usage of classes in your Puppet-infra

Written by Ger Apeldoorn. Posted in Manageable Puppet Infrastructure, Sysadmin

It’s always good to know what the impact is of your change. With these scripts, we can easily see which hosts use a specific class.

Example usage

$ module_usage.sh Random
=================================
DTA Server
Count Modulename
=================================
4 Profile::Random
4 Profile::Random::Accounts
4 Profile::Random::Filesystems
4 Profile::Random::Initd
4 Profile::Random::Ulimit
2 Role::Random

=================================
PRODUCTION Server
Count Modulename
=================================
7 Profile::Random
7 Profile::Random::Accounts
7 Profile::Random::Filesystems
7 Profile::Random::Initd
7 Profile::Random::Ulimit
3 Role::Random

Continue reading for info on how to set this up!

Emergency-stop

Written by Ger Apeldoorn. Posted in Sysadmin

When upgrading the Puppet master (or messing things up) it can be useful to have a quick script that enables the firewall and stops incoming connections.

This is a simple script that enables that. Note that it assumes that the normal situation does not have any firewall-rules running.

/usr/local/bin/puppetstop:

#!/bin/bash
# This script blocks all connections. Exceptions can be made in /etc/sysconfig/iptables-emergencystop
iptables-restore < /etc/sysconfig/iptables-emergencystop
mv /etc/sysconfig/iptables-emergencystop /etc/sysconfig/iptables

/etc/sysconfig/iptables-emergencystop:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [611449:52637218]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -s [puppetmaster-IP] -m tcp -j ACCEPT
-A INPUT -p tcp -s [testmachine-ip] -m tcp -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT

/usr/local/bin/puppetstopundo:

#!/bin/bash
iptables -F
mv /etc/sysconfig/iptables /etc/sysconfig/iptables-emergencystop

/usr/local/bin/puppetenable

#!/bin/bash
if [ $# -eq 0 ]
  then
    echo "No arguments supplied"
fi
iptables -I INPUT -p tcp -s $1 -m tcp -j ACCEPT

MPI – Managing multiple customers within a single environment

Written by Ger Apeldoorn. Posted in Manageable Puppet Infrastructure

Intro

This is set in the Manageable Puppet Infrastructure, but should be quite useable if you’re not using it.

You want to manage multiple clients within a single environment. (You probably don’t want to create seperate environments or puppetmasters, because that creates quite a bit of overhead in the long run).

But how do we keep things flexible? It’s quite simple, use Hiera and classes in a smart way and you’re there!

One note; this simple setup only works if you do not have any naming conflicts in the modules. e.g. customer1 needs puppetlabs/apache and another needs another module named apache. In that case, you do need seperate environments.

Get that $customer variable filled

You’ll need a way to let Puppet know what customer a node belongs to. There are several ways:

Facter variables can be overridden quite easily. Use the ENC or Hiera!

  • Getting the info from Hiera, see below.
  • Set it from the ENC.

Adding a global $customer variable

In my site.pp:

$customer = hiera('customer')

node default {
  hiera_include('roles')
}

Example: hieradata/hosts/myserver.customer.com.yaml

---
customer: mycustomer1

That’s an easy way to get the $customer (global) variable filled!

Putting it to use!

Roles

With the MPI, you’ve created a baserole that applies to all hosts.

We can still use that, just be aware that those resources/includes apply to ALL customers.

Also, we’d like a baseclass for each customer. Within the roles-module, that’s a piece of cake.

modules/role/
└── manifests
    ├── base.pp   # class role::base { (applies to all nodes)
    ├── mycustomer1
    │   ├── app.pp       # class role::mycustomer1::app {
    │   ├── base.pp      # class role::mycustomer1::base { (applies to all nodes of mycustomer1)
    │   ├── db.pp        # class role::mycustomer1::db {
    │   └── smarthost.pp # class role::mycustomer1::smarthost {
    └── mycustomer2
        ├── app.pp       # class role::mycustomer2::app {
        ├── base.pp      # class role::mycustomer2::base {
        ├── db.pp        # class role::mycustomer2::db {
        └── lamp.pp      # class role::mycustomer2::lamp {

Hiera

You can add a level for customer-specific settings.

:hierarchy:
  - "hosts/%{clientcert}"        # Host-specific settings
  - "customer/%{customer}"       # Customer specific settings
  - "osfamily/%{osfamily}"       # Settings based on OS
  - "environment/%{environment}" # Environment specific settings
  - common                       # Common settings for all nodes.

Now, we can add a yaml file for each customer and assign the role::customer::base there.

Example: hieradata/customer/mycustomer1.yaml

---
roles:
  - role::mycustomer1::base

Of course, you can add other customer-specific settings as well here!

Feedback is welcome, leave a message!

Script to generate certificates & Puppet-code

Written by Ger Apeldoorn. Posted in Sysadmin

My laziness knows no bounds, I use the script below to automate writing automation puppet code for creating new users (and generating, converting and mailing their certificates too)…

This ruby script will read information from a CSV-file and:

  • generate certificate with a random passphrase
  • create a Putty version
  • email the (encrypted) certificates to the user
  • prints the Puppet-resource definition to STDOUT
  • store the passphrase in a textfile

It will check if the certificates have already been generated (check if directory exists), so you can just add to the CSV-file.

Installing

Prerequisites:

  • Ruby of course
  • some rubygems: mail, csv
  • pwgen
  • ssh-keygen
  • puttygen

Just put the files below in the same directory.

Usage

Edit the userlist.txt to add users; the format is:

userid,username,name,email,groups (groups ; seperated)

Example:

"1001","gertest","Ger Apeldoorn","g.apeldoorn@example.nl","admin;dev;prutsor"
"1002","gertest2","Ger Apeldoorn","g.apeldoorn@example.nl","admin;dev;prutsor"

After that’s done, just run the script:

./gencerts.rb

Example output:

@pe_accounts::user { 'gertest':
	locked  => false,
	comment => 'Ger Apeldoorn',
	uid     => '999',
	gid     => '999',
	groups  => ["admin", "dev", "prutsor"],
	sshkeys => ['ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnCWb6yiDpOdtzJ3w0Hf/MOIvXswpqu3XnotDtgAkJaH/ZtC/wNrwAsm+ugyGtTmWTX22LdJ1M4mr0KYk7cj1m6QlKp78R4485uERFn0/q6LAJSqKYzEFY53SGECCn/BVZWYDJQ6UcoslouSiIsDivC+rwrG4UGAwdyDSED4PrT0U2kTkPeadbxqvT3FcHcO1HuQF8nMZ g.apeldoorn@example.nl'],
}

After running this, the user receives an email with mailtext.txt as contents and three attachments:

  • id_rsa
  • id_rsa.pub
  • putty_private.ppk

Note that the private keys are protected with the passphrase… 🙂

Files

gencerts.rb:

#!/usr/bin/env ruby
CSVFILE = 'userlist.txt'
EMAILCONTENT = 'mailtext.txt'
CURDIR = Dir.pwd

require 'csv'
require 'rubygems'
gem 'mail'
require 'mail'

def which(cmd)
  exts = ENV['PATHEXT'] ? ENV['PATHEXT'].split(';') : ['']
  ENV['PATH'].split(File::PATH_SEPARATOR).each do |path|
    exts.each { |ext|
      exe = "#{path}/#{cmd}#{ext}"
      return exe if File.executable? exe
    }
  end
  return nil
end

def preflight_checks
        #see if apps are installed and in path
        puts "pwgen is not installed"      if not which('pwgen')
        puts "ssh-keygen is not installed" if not which('ssh-keygen')
        puts "puttygen is not installed"   if not which('puttygen')
end

def send_email(username, email)
        mail = Mail.new do
                from    'linuxbeheer@fundeon.nl'
                to              email
                subject 'Your SSH keys'
                body    File.read(File.join(CURDIR, EMAILCONTENT))
        end
        mail.add_file(File.join(CURDIR, username, "id_rsa"))
        mail.add_file(File.join(CURDIR, username, "id_rsa.pub"))
        mail.add_file(File.join(CURDIR, username, "putty_private.ppk"))
        mail.delivery_method :sendmail
        mail.deliver
end
def print_puppetconfig(userid,username,name,email,groups)
        puts "@pe_accounts::user { '#{username}':"
        puts "  locked  => false,"
        puts "  comment => '#{name}',"
        puts "  uid     => '#{userid}',"
        puts "  gid     => '#{userid}',"
        puts "  groups  => #{groups.split(";").inspect},"
        pubkey = File.read(File.join(CURDIR, username, "id_rsa.pub")).chomp
        puts "  sshkeys => ['#{pubkey}'],"
        puts "}"

end

preflight_checks

CSV.foreach(CSVFILE) do |row|
        userid,username,name,email,groups = row
        if File.exists?(File.join(CURDIR, username))
                puts "Directory for user #{username} already exists"
                next
        end
        Dir.mkdir(File.join(CURDIR, username))
        Dir.chdir(File.join(CURDIR, username))
        passphrase=`pwgen -c -n -1 10`
        passphrase.chomp!
        #Create SSH keypair
        `ssh-keygen -C #{email} -P #{passphrase} -f id_rsa`
        #Convert SSH keypair to Putty
        `echo #{passphrase} | puttygen id_rsa -O private -o putty_private.ppk`
        #Save the passphrase
        File.open("passphrase", 'w') {|f| f.write(passphrase) }
        send_email(username, email)
        print_puppetconfig(userid,username,name,email,groups)
end

mailtext.txt:

Hi,

You will find your OpenSSH and Putty keys attached. You can use these keys to login to any Linux server for which you are authorized.

You can get the passphrase from -----.

Kind regards.
Le Pinguin

Have fun!