Getting python-ldap installed on OS X

July 7th, 2008

If you want to use Fink, and build a bunch of crap you don’t need, follow these instructions.

If you are like me and prefer to use the libraries that are already installed, just do this:

  1. If you haven’t already, install XCode.
  2. Download python-ldap
  3. and extract it.

  4. Edit setup.cfg. The relevant lines are below:
    library_dirs = /Developer/SDKs/MacOSX10.5.sdk/usr/lib/
    include_dirs = /Developer/SDKs/MacOSX10.5.sdk/usr/include/ /Developer/SDKs/MacOSX10.5.sdk/usr/include/sasl
  5. sudo python setup.py install
  6. That’s all!

My First Conference

June 23rd, 2008

I’m at Velocity 2008 right now. So far I’ve learned several things:

  1. Twitter can’t handle a conference.
  2. Sun might succeed in its attempt to restore its relevance.
  3. SSDs are going to be abother layer in server memory, between disks and RAM.
  4. Whenever a member of the Ops team is unavailable, something breaks.
  5. Con food isn’t remarkable, in either positive or negative ways.
  6. Arrive early to talks so you can snag a spot on the power strip.
  7. I FAIL at socialization.

At least I managed to score some Puppet schwag (whoever made these shirts has a different definition of XXL than me…) and managed to track down Luke Kanies and Andrew Shafer of Reductive Labs to say “Hi.” Haven’t socialized much with them; they look like they’re working.

About to watch a talk on measuring performance; a topic I’ve always had big questions about; hopefully I’ll learn something good. I’m kinda regretting my decision to sit towards the front, though.

Crickets

June 18th, 2008

It’s quiet in here. Life has been hectic and I’m somewhat confused as to what I’m allowed to say without getting in trouble at work.

Hopefully I’ll have some more Puppet content soon.

Creating Puppet Modules

April 18th, 2008

In Puppet parlance, a module is a collection of manifests (including classes and definitions), templates, and files which, taken together, describe a recipe for configuring something via Puppet. Puppet has a number of facilities which make modules incredibly useful and labor-saving.

Modules have a standard internal organization which is described in detail on the ModuleOrganisation wiki page. A trivial module can be as simple as a manifests directory containing only a single manifest: init.pp. However, as modules grow more complex, you’ll want to break up your manifests and add templates in a templates directory, files in a files directory, and a README file which explains how to make use of your module.

I’m in the process of polishing up several modules, including LDAP, Kerberos, memcached, and ntp modules. In some ways, I’m duplicating work; several implementations of these modules are available. However, Puppet modules are still evolving, and I wanted to try my hand at module writing. Also, the existing modules did not work quite right for us. Some of them fail to properly isolate site-specific information from the recipe, for example. Others had complex interdependencies that I didn’t like.

There are several techniques that I think will evolve as “best practices” for Puppet module design. These are:

  1. Keep site-specific customization out of your modules.
  2. Your module should implement a minimal working configuration “out of the box”.
  3. Use variables to make it easier to patch your module for use on other platforms.
  4. Break a module up into sensible classes and defined types to make it easy to customize via inheritance.
  5. Use init.pp for module-wide variables and/or common functionality, but break all other classes/defines into separate files.

Let’s take a look at each of these in detail:

Keep site-specific customization out of your modules.

Modules are, at heart, meant to be shared with others. Try to write your modules so they are as site-neutral as they can be. Use variables that can be set in a higher-level scope like site.pp to control how the module works, instead of baking your site’s settings into the module. For example, in my LDAP module I do this:

class ldap::common {
  case $ldap_base_dn {
    "": { $ldap_base_dn = "dc=example,dc=com"
      warning("ldap_base_dn not set, using default $ldap_base_dn")
    }
  }
 
  case $ldap_admin_dn {
    "": { $ldap_admin_dn = "cn=admin,$ldap_base_dn"
      warning("ldap_admin_dn not set, using default $ldap_admin_dn")
    }
  }
 
  case $ldap_admin_password {
    "": { fail("ldap_admin_password not set!")
    }
  }

Then, in site.pp, I set the variables appropriately:

# Site Variables
$ldap_server           = "ash001.example.com"
$ldap_admin_password   = "testing"
$ldap_base_dn          = "dc=example,dc=com"

Your module should implement a minimal working configuration “out of the box”.

This principle is also tied in with the fact that modules were meant to be shared. When someone is looking for a module, they are going to want something that works with little-to-no configuration, because this will give them confidence that the module will work once it is fully configured. Just like full-blown pieces of software, if a module doesn’t have an easy initial setup, people will get confused as to whether or not it even works. My LDAP module provides a ldap::master class which implements a very basic configuration: the slapd package is installed, a working configuration file is set up (without SSL or any of the other goodies), and the service is started. If the user sets just one variable, $ldap_admin_password in their init.pp and includes ldap::master on a node, they will be able to verify that LDAP is up and running with an example configuration. Even better, if they set the other variables, this configuration will be customized to their site with little effort on their part. It might be better to add in all the bells and whistles (SSL at a minimum), but I’m not sure yet where I stand on that.

Use variables to make it easier to patch your module for use on other platforms.

Currently, in the init.pp of my LDAP module, I set variables like this:

  $ldappackage       = "slapd"
  $ldapservice       = "slapd"
  $ldapdir           = "/etc/ldap"
  $ldaputilpackage   = "ldap-utils"
  $ldapclientpackage = "libnss-ldap"

and use them like this:

  package {
    $ldappackage:     ensure => installed;
  }
 
  file { "$ldapdir/slapd.conf":
    content => template("ldap/slapd.conf.erb"),
    require => Package[$ldappackage],
    notify  => Service["$ldapservice"],
  }
 
  service { $ldapservice:
    require   => [ Package[$ldappackage], File["$ldapdir/slapd.conf"] ],
    ensure    => running,
    enable    => true,
  }

These variables are set up for Debian now, because that is the distribution I’ve standardized on. If later I (or someone I’ve shared the module with) wants to add support for another distribution, they can set up case statements, and not have to modify the rest of the module!

  case $operatingsystem {
    debian: {
      $ldappackage       = "slapd"
      $ldapservice       = "slapd"
      $ldapdir           = "/etc/ldap"
      $ldaputilpackage   = "ldap-utils"
      $ldapclientpackage = "libnss-ldap"
    }
    centos: {
      $ldappackage       = "openldap"
      $ldapservice       = "slapd"
      $ldapdir           = "/etc/openldap"
      $ldaputilpackage   = "openldap-utils"
      $ldapclientpackage = "libnss-ldap"
    }
  }

I have not tested this, as I don’t use CentOS, so don’t use these!

Break a module up into sensible classes and defined types to make it easy to customize via inheritance.

Since you’ve pulled all the site-specific customization out of your package, and you are providing a minimal working configuration, people are going to want to do other things with your module that are appropriate to their site. If you have everything lumped into one big “ldap” class, this is going to be difficult for them. I break thinks up into ldap::common for things common to all the other classes, ldap::client for things needed to query the LDAP servers, ldap::master for the primary LDAP server, and later I’ll provide ldap::slave for replication slaves.

Use init.pp for module-wide variables and/or common functionality, but break all other classes/defines into separate files.

This is for your sanity as well as the sanity of those who might want to modify your module. I started with everything in one big init.pp file and it rapidly went out of control. Puppet does some awesome automagical lookups that you can take advantage of. For example, my ldap::master class is defined in a file called master.pp; when someone tries to load ldap::master, Puppet automatically searches for a .pp file named “master” in the “ldap” module!

So far, this is all the wisdom I have to impart on the subject. I’ll be sure to post links here when these modules are ready for prime-time.

SSH tab completion

March 24th, 2008

This is awesome. Put the following into your ~/.bash_profile file:

complete -W "$(echo `cat ~/.ssh/known_hosts | cut -f 1 -d ' ' \
| sed -e s/,.*//g | uniq | grep -v "\["`;)" ssh

Open a new shell and type ssh tab tab. I love tab completion, don’t you?

FreeBSD Update

March 17th, 2008

I just followed these instructions to update my server from FreeBSD 6.2-RELEASE to FreeBSD 7.0-RELEASE.

Aside from taking almost a day, it went flawlessly. I’m extremely impressed!

Virtualization Article

March 12th, 2008

This article might be interesting to some of you.

Bash History

March 12th, 2008

Here are a couple of tips for more effective use of your Bash command history. I think the Ctrl-R trick will save me a lot of typing.

I have some articles planned, but have been busy starting my new job. Those few of you who actually read this, don’t worry - I’ll be back.

Useful SSH-isms

February 23rd, 2008

Two useful SSH-isms

Simple SSH speed-up

February 14th, 2008

I just discovered something that is very simple but saves a significant amount of time. Specifically, the SSH ControlMaster feature, which allows you to re-use an existing SSH connection whenever you connect to a host you are already connected to.

In your ~/.ssh/config file, add:

Host *
        ControlMaster auto
        ControlPath /tmp/%r@%h:%p

Whenever you connect to server.example.com as user joeuser, SSH will create a named pipe at /tmp/joeuser@server.example.com:22. If you open another connection to the same server (as the same user), instead of creating a new TCP/IP connection, SSH will automatically multiplex the new session with the existing connection (through the named pipe)! This reduces time spent setting up new connections. Although this usual only saves a couple of seconds per connection, I’m constantly opening several simultaneous sessions; this will definitely add up to a significant savings over the long term!