Paul Lathrop


You Want...

An Operations Engineer specializing in designing and deploying stable, fault-tolerant, scalable operations architectures that grow as fast as your company. Take advantage of ten years of professional experience combined with independence, initiative, and creativity to obtain innovative solutions to complex systems problems and infrastructure needs.

I Have...

  • Expert-level understanding of Linux/Unix administration and internals.
  • Expert-level understanding of scalable infrastructure design principles: automated installation, configuration management, lifecycle management, asynchronous systems, etc.
  • Intermediate to expert-level understanding of source-code management principles and systems, particularly git.
  • Intermediate to expert programming/scripting skills; developed non-trivial applications and utility scripts in Python, Node.js, Clojure, Common Lisp, Ruby, and bash.
  • Intermediate understanding of distributed data storage and processing technologies: Cassandra, Hadoop, ActiveMQ, RabbitMQ, etc.
  • Intermediate understanding of the JVM platform esp. garbage collection and tuning.
  • Intermediate understanding of software development methodologies: test-driven development, behavior-driven design, object-oriented programming, functional programming, etc.
  • Comprehensive high-level understanding of programming concepts and talent for applying those concepts to rapidly develop proficiency in unfamiliar programming languages.
  • Operational, high-level understanding of network protocols.
  • Basic to intermediate network troubleshooting/monitoring skills: traceroute, tcpdump, nmap, wireshark, netcat, netstat, ping, iproute, dig, Nagios, etc.
  • Operational understanding of relational database systems; intermediate understanding of database scaling techniques: replication, optimization, de-normalization, sharding, etc.
  • Built and lead an Operations team from the ground up; ensured Operations stayed highly integrated with Engineering as both teams grew.
  • Built and automated multi-AZ infrastructure on top of Amazon Web Services.
  • Participated in design and implementation of custom cluster management software, used to manage 1000 servers across bi-coastal datacenters.
  • Implemented Puppet modules to automate configuration of a broad range of services including Apache, Cassandra, Hadoop, MySQL, and RabbitMQ clusters.
  • Installed and maintained a broad range of standard high-scale infrastructure software.
  • Authored and implemented policies and procedures that balance security, ease of use, ease of administration, and executive oversight.
  • Procurement experience: selecting & documenting standard hardware configurations, creating & maintaining vendor relationships, managing purchasing budgets, etc.
  • Contributed to the maintenance and improvement of open-source tools.

I've Been...

Senior Linux Systems Administrator

Yammer, Inc. (2011-current)

I've just joined the team at Yammer and I'm excited to see what we'll build together!

Evil Puppetmaster / Engineer

SimpleGeo, Inc. (2010-2011)

  • Yes, that's actually my title.
  • Assisted in the design of an "Operations API" which we will use to control our systems programatically for ad-hoc tasks like restarting services, running consumption tests, kicking off Puppet runs, etc.
  • Built and open-sourced piro, a system for remotely controlling services in a plugabble and programmatic fashion.
  • Created a system which fully records incoming requests and spools a rolling window of requests to disk in order to allow us to replay requests in the event of a failure where we would otherwise lose data.
  • Contributed to development of user-facing API services.
  • Responded to a malicious attack on infrastructure by a privileged user; prevented attack from destroying critical systems. Led recovery from attack and implementation of new security measures. Led investigation of attack which ended in prosecution of responsible party.
  • Built Puppet manifests and bootstrap scripts to allow us to bootstrap instances to various roles without having to maintain AMIs.
  • Built Operations team and acted as team lead: represented operational interests to the development team leads, established processes (scrum, kanban), set priorites in response to business needs, delegated tasks to team members.
  • Migrated configuration management from Chef to Puppet.

Senior Systems Engineer

Digg, Inc. (2008-2010)

  • Actively worked to close gap between engineering and operations, foster transparency,
    and empower engineering to iterate rapidly while maintaining site stability.
  • Participated in development of internal cluster mangagement software.
  • Wrote command-line application to automatically allocate and provision servers,
    including options to specify minimum hardware requirements.
  • Redesigned Digg infrastucture to take advantage of heavy automation & configuration management.
  • Developed Puppet modules to automate deployment, configuration, and lifecycle management of key clusters.
  • Developed FAI scripts which bootstrap systems from 'bare metal' to functioning Puppet clients.
  • Assisted in migration from Subversion to Git as primary source code management system.
  • Responsible for ensuring reliable operation of production, staging, and development systems.
  • Perform code pushes and maintain change documentation.

Contract Infrastructure Architect

Zicasso, LLC. (2007-2009)

  • Built an infrastucture which has run smoothly for over two years with minimal intervention.
  • Automated system configuration using Puppet.
  • Firewall design, implementation, and maintenance.
  • Create, deploy, and manage Xen virtualized servers.
  • Patch managment and server maintenance.
  • Proposed and implemented automated backup system.
  • Deploy & maintain split-horizon DNS services.
  • Replace Zenoss monitoring system with Nagios.
  • Implement SNMP infrastructure.
  • Deploy Zenoss monitoring system, notifications, and escalations.
  • Created extensive documentation, including straightforward how-to procedures for common administrative tasks.
  • Configure & maintain MySQL database systems.

Senior Systems Administrator

Kapor Enterprises, Inc. (2007)

  • Created Twiki-based project management application.
  • Identified key areas of network and process improvement, proposed solutions.
  • Proposed, planned, and implemented single sign-on solution and corporate directory service.
  • Provided desktop support for a heterogenous network of Mac OS X, Windows, and Linux desktops.
  • Responsible for researching and procurement of best-of-breed equipment to implement the needs of supported organizations.

Systems Administrator

SquareTrade, Inc. (2005-2007)

  • 24x7 pager support for critical production systems.
  • Planned and implemented migration from SiteScope to Nagios network monitoring, implemented custom service plugins, distributed & redundant architecture, performance metrics, and custom reporting interface.
  • Planned and implemented OpenLDAP directory service; researched and proposed site-wide integration of applications with directory service.
  • Administered heterogeneous network of several hundred nodes spread across several sites; maintained WAN links, VPN, remote administration between locations; maintained two data center locations with over 125 servers.
  • Performed audit of poorly maintained RAID systems, implemented automated maintenance and reporting of RAID array performance and issues.
  • Proposed, planned, and test-deployed Linux desktop solution for sales representatives to replace costly Windows desktops.
  • Performed extensive documentation of server and network infrastructure.
  • Researched, planned, and assisted deployment of Novell eDirectory and ZenWorks infrastructure.
  • Assisted in migration from Checkpoint firewalls to OpenBSD firewalls.
  • Assisted in software license compliance audit.
  • Oversaw replacement of aging desktop-class hardware with server-class hardware.
  • Researched and proposed migration plan from NT-style domain to Active Directory infrastructure.
  • Researched and proposed deployment of enterprise XMPP chat service for internal messaging.
  • Researched and proposed replacement of overloaded internal mail architecture with distributed, scalable mailing architecture.

Systems Administrator

OnYourMark, LLC (2004-2005)

  • Planned and implemented migration from archaic Sendmail-based email system to a database-backed Postfix mail server, delegating specific tasks to system administration staff. Supervised programming of customized email administration software for new system.
  • Replaced several outdated firewalls with OpenBSD firewalls. Wrote sophisticated firewall rule sets to filter and monitor network traffic. Created automatic log processing scripts to analyze firewall logs and produce daily reports.
  • Created standardized server configuration procedures and documentation. Oversaw redeployment of servers to comply with standardized configuration.
  • Created and administered network access/security policies and procedures.
  • Maintained and administered heterogeneous intra-office network, including Windows 2000/XP and Mac OS X workstations, Unix file and web servers, wireless access points, and network printers.
  • Planned migration of intra-office network to Windows 2003 Active Directory infrastructure.
  • Remotely administered and maintained DNS, web, and database servers in several co-location facilities.
  • Provided Tier II technical support.
  • Oversaw system backup and network maintenance as well as server monitoring (Nagios) and IDS.
  • Performed software license audit and inventory.
  • Maintained relationships with hardware, software, and bandwidth vendors.
  • Installed and administered DHCP server for automatic network configuration.
  • Automated a variety of administrative tasks through shell scripts and custom programming.

Systems Consultant

Independent (2003-2004)

  • Molecular Genetics Laboratory, Michigan State University
    • Updated laboratory website.
    • Trained laboratory staff in HTML and web authoring software for performing site updates.
  • Nocturnal Theater
    • Administered web, DNS, and email services.
  • The Intangible
    • Administered DNS and web services.

Senior Systems Administrator

Web Media Works (2002-2003)

  • Administered web, DNS, and database servers running FreeBSD, NT 4, and Windows 2000 Server.
  • Administered MySQL and MS-SQL 2000 databases.
  • Achieved goal of 99% uptime (up from 70% at time of hire).
  • Created, implemented, and enforced security policies on network, server, and individual workstation levels.
  • Managed server migration to the FreeBSD operating system.
  • Planned and installed internal network: CAT5 cabling, network ports, high-bandwidth switches, wireless access points, routers, firewall, and VPN.
  • Secured wireless network segment against unauthorized access and packet monitoring.

Student Administrator/Programmer

J. Robert Van Pelt Library (1999-2001)

  • Proposed, planned, and implemented conversion of aging collection of 486 PCs into Citrix Metaframe thin-client terminals.
  • Assisted in migration of servers from Windows NT4 to 2000 Server.
  • Supported a variety of Windows PCs ranging from Windows 95 to Windows 2000 Server.
References Available Upon Request
Copyright © 2007-2010 Paul Lathrop. All Rights Reserved. Design by Crystal Odenkirk