Volume Number: 24 (2008)
Issue Number: 08
Column Tag: Network Management

SNMP Primer for OSX Leopard

An article on "Understanding, Configuring, and Customizing SNMP on OS X Leopard" that might change the way you think about your Mac.

by Noah Gift

If you haven't done any work with SNMP before, you might be thinking, "who cares?" SNMP has gotten the somewhat deserved reputation as being complex to administer and understand. As a result, many people don't care, and there are not that many articles that talk about it.

This article will attempt to present the information in a gentle enough manner that no experience with SNMP will be required. At the same time, we will dive into a few obscure, but absolutely fascinating things that be done with SNMP that few delve into. It is quite fun to let a cat out of the bag, so let's dive into SNMP and see how it might just change the way you think about your Mac.

Because SNMP can be incredibly boring to talk about, let's save the obligatory overview of what SNMP is, the history of SNMP, configuring SNMP etc. Instead we are going to immediately do something useful. Just follow these steps on a OS X Leopard machine that is not a production server.

Step 1:  cp /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.bak.052108
Step 2: echo "rocommunity public" > /etc/snmp/snmpd.conf
Step 3:  Restart the snmpd daemon by simply typing the executable:
mini# snmpd
Step 4:  Query the disk usage of your machine via snmp:
mini# snmpdf –v 2c –c public localhost
Description            size (kB)          Used        Available    Used%
Physical memory        1310720            821440      489280         62%
Swap space               65532                 0       65532          0%
/                     77814832          51153396    26661436         65%
/dev                         1                 1           0        100%
/dev                         1                 1           0        100%
/net                         0                 0           0          0%
/home/net                    0                 0           0          0%

Hopefully these simple four steps did the equivalent of dropping a suitcase nuke on your preconceptions about SNMP. Let's now go over what we did in detail now. In the first step we simply made a backup of the snmp configuration file, and added a date stamp on it.

In the second step we overwrote the whole snmpd.conf file by simply echoing out one line of configuration syntax; this effectively erases pages upon pages of almost completely useless configuration data you will never use. This in fact is one of the biggest stumbling blocks to understanding new technology, too much information.

In the third step we told snmpd to start the daemon so we can query it. (We will get into launchd configurations later). Finally in the fourth step, we ran a specialized snmp tool that is effectively the same as running df –k on a local machine.

Now that we have removed some of the magic behind SNMP, and brought it down to the level of complexity of ssh or tar let's fill in a few of the details. The very first detail to understand is the cryptic syntax for the snmpdf command. If we look at the command again, we can break it down into the following sections:

snmpdf –v [SNMP Version] –c [Name of community string] IP Address or hostname

Fortunately, all of the Net–SNMP command line tools follow this same syntax, so you will only need to understand one tool to understand the rest. The –v option refers to which version of SNMP you wish to use. The available options are 1, 2c, and 3. Versions 1 and 2c are not encrypted so they are only safe to use in a secure firewalled environment, and version 3 requires more explanation and setup. If you just want to setup snmp at home behind a firewall, then 2c is the version you will most likely want to use.

The next flag –c refers to the community string. With SNMP versions 1 and 2c, the authentication system revolves around setting community strings to grant access to the snmp daemon. There are ways to set community strings to allow read access, which is what we did in the section above, and also to allow read/write access. In this case, we went with a convention and used public as the ro, or read only community string.

Finally, we just typed in the name of the local machine. We could have typed in the name of any machine on the internet though, if they had snmp running and had the configuration we setup. This is really almost 80% of what someone needs to know about SNMP, we will cover a few of other things later. Given that we are now SNMP experts, let's write some code to monitor a home network for disk usage problems.

Since we are now SNMP experts, and have bragged to all of our friends how we can monitor the disk usage of our machines remotely, we got thrown into a consulting gig "by accident." Our task is to write a simple nightly monitoring script that checks all of the machines on a local network to make sure they do not have critical disk space issues. This client has several Final Cut Pro Suites where the editors constantly overfill their RAID volumes, and it is our job to write code that will prevent it before it happens and send the company owner an email when disk space on any volume exceeds 80%.

With Python and SNMP this is actually a trivial, yet very useful, problem to solve. The first step is to write some code that flags disk usage that exceeds 80%.

#!/usr/bin/env python

""" Parses the output of snmpdf and performs a regular expression match that searches for 80–99% disk usage pattern. If a match is found it prints out the volume that exceeds our "quota"."""

from subprocess import PIPE, Popen
import re
def snmpdf(machine):
    """Returns snmpdf output as file object"""
    p = Popen("snmpdf –v 2c –c public %s" % machine,
                shell=True,
                stdout=PIPE,
                stderr=PIPE)
    return p.stdout
def parse(file):
    """Parses file object and determines if critical match
    between 80–99% disk usage is met.
    Returns collected results with new stamp line."""
    collection = []
    pattern = "[8–9][0–9]%"
    outline = (line.split() for line in file)
    flag = (" ".join(row) for row in outline \
        if re.search(pattern, row[–1]))
    for line in flag:
        newline = "%s DISK USAGE CRITICAL" % line
        collection.append(newline)
    if len(collection) > 0:
        return collection
if __name__ == "__main__":
    #prints results
    out = snmpdf("localhost")
    result = parse(out)
    if result:
        for line in result:
            print line

All this script does is to take the results from the snmpdf command and look for volumes that have between 80–99 % utilization. If you are dealing with editing on a local RAID volume, often exceeding 80% capacity will cause performance problems. If we run this script on a machine with utilization problems, we get something that looks like this:

mini# python2.5 snmpdf_alert.py
/ 97349872 88678460 8671412 91% DISK USAGE CRITICAL

We can see that this machine's root volume or "/", is at 91% capacity. This is obviously a big problem that we need to address very soon.

In order to turn this into something that manages a network and emails a warning, it would be fairly easy to just run this in a cron job or via launchd, and mail an alert message if the output of the script was not None, which is what happens when there is no match for our regular expression.

If you are new to Python some of the code may look a little weird, so here are a few things to remember. First, whitespace is significant, so indentations are there to control the flow of the program. Second, if you are coming from another language, Python is fairly easy to pick up. If you would like a reasonable introduction to the language please refer to tutorial listed in references.

There are few things we glossed over in the first parts of the article because they were boring, but let's get them out of the way. This should bring you a bit closer, if not all the way, to knowing 80% SNMP.

First, in order to get the SNMP daemon to run upon startup you will need to modify the plist for:

/System/Library/LaunchDaemons/org.net–snmp.snmpd.plist

And make it look as follows:

<?xml version="1.0" encoding="UTF–8"?>
<!DOCTYPE plist PUBLIC "–//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList–1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/>
    <key>KeepAlive</key>
    <true/>
    <key>Label</key>
    <string>org.net–snmp.snmpd</string>
    <key>OnDemand</key>
    <false/>
    <key>Program</key>
    <string>/usr/sbin/snmpd</string>
    <key>ProgramArguments</key>
    <array>
        <string>snmpd</string>
        <string>–f</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>ServiceIPC</key>
    <false/>
</dict>
</plist>

What this does is to tell the snmpd daemon to start up upon reboot and then stay alive. Arguably, this could be set to OnDemand instead, but this is one of the newer features of launchd and it hasn't been fully tested with snmp much.

Next, we need to talk a little bit about the snmpwalk command, OIDs, and MIBs. The very short explanation of an OID is that it is a string of numbers, with a human readable name, that lives inside of a MIB file. This whole system is a hierarchically–assigned namespace for the SNMP protocol to keep track of what an agent can provide when you query it.

Let's take a look at how the snmpwalk command is used to shed some light on this. Here is a basic snmp query of our local machine again using the OID sysdescr:

mini# snmpwalk –v 2c –c public localhost sysdescr

SNMPv2–MIB::sysDescr.0 = STRING: Darwin mini.local 9.2.2
 Darwin Kernel Version 9.2.2: Tue Mar  4 21:17:34 PST 2008;
 root:xnu–1228.4.31~1/RELEASE_I386 i386

By examining the command we ran you may notice the syntax is identical to the syntax of the snmpdf command, with the exception of the word sysdescr. This word is an OID, and it lives in the MIB–2 file. You can refer to the references for an actual breakdown of this OID and where it lives in the larger MIB structure if you are curious, but if just want to use SNMP you only need to know important OID's to query.

As our last mention of this topic, it is important to know that the snmpwalk command retrieves all of the OIDs that are listed below it in the hierarchical structure. To get the superset of the OID that holds sysdescr, we could type in this:

mini# snmpwalk –v 2c –c public localhost system
  
SNMPv2–MIB::sysDescr.0 = STRING: Darwin mini.local 9.2.2 Darwin Kernel
 Version 9.2.2: Tue Mar  4 21:17:34 PST 2008; root:xnu–1228.4.31~1/RELEASE_I386 i386

SNMPv2–MIB::sysObjectID.0 = OID: NET–SNMP–MIB::netSnmpAgentOIDs.255
DISMAN–EVENT–MIB::sysUpTimeInstance = Timeticks: (1892500) 5:15:25.00
[OUTPUT SHORTENED FOR SPACE] 

This gives us not only the sysdescr OID, but many others as well. For writing a monitoring system you almost never query something in this manner though, you typically just find out the important lower level OIDs, and then write some scripts using them.

This is all we are going to cover about the very nitty gritty details about how SNMP is implemented. If you are curious, you do a Google search for these terms and you should get all of the reference material you need.

At this point we have the basics covered for configuring, configuring, scripting, and querying SNMP on the Mac. As a final trick, we are going to extend the Net–SNMP agent on our Mac and do something fun.

We are going to write a script that tells iTunes to start up and play something on "Party Shuffle" if we query a specific OID. In order to do that we need double our already massive configuration file, or add one more line. Let's make our /etc/snmp/snmpd.conf look like this:

rocommunity public
exec PlayItunes /usr/bin/python /tmp/playitunes.py

Next, we need to write that script so that it executes when we run. Here is what that script looks like:

#!/usr/bin/env python
from subprocess import Popen, PIPE
cmd = """osascript<<END
tell application "iTunes"
play playlist "Party Shuffle"
end tell
END"""
def play_iTunes():
     Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE)
     print "Started ITunes Party Shuffle"
play_iTunes()

Next we need to send the snmpd daemon a HUP to tell it to reread it's configuration file. Do a ps –ef | grep snmpd and then give the PID a kill –1. This is what it looks like when I do it:

mini# ps –ef | grep snmpd                                   
    0    26     1   0   0:01.82 ??         0:02.26 snmpd –f
    0   515   355   0   0:00.00 ttys000    0:00.00 grep snmpd
mini# kill –1 26

Finally, we are ready to "query" our machine and make it trigger our custom script. There is a standard OID that responds to custom scripts and it is used in the snmpwalk call below:

mini# snmpwalk –v 2c –c public localhost .1.3.6.1.4.1.2021.8
UCD–SNMP–MIB::extIndex.1 = INTEGER: 1
UCD–SNMP–MIB::extNames.1 = STRING: PlayItunes
UCD–SNMP–MIB::extCommand.1 = STRING: /usr/bin/python
UCD–SNMP–MIB::extResult.1 = INTEGER: 0
UCD–SNMP–MIB::extOutput.1 = STRING: Started ITunes Party Shuffle
UCD–SNMP–MIB::extErrFix.1 = INTEGER: noError(0)
UCD–SNMP–MIB::extErrFixCmd.1 = STRING: 

When we run the command, and see the output, we can see the print statement that we included with our script. If we actually take a look at our machine, we will notice that ITunes indeed pops up, and starts on Party Shuffle.

This is a slightly silly example for extending a Net–SNMP agent, but if you are a home power user, maybe it isn't. Who doesn't want to brag and tell your spouse, you turned on iTunes via SNMP to the downstairs computer?

We covered a lot of ground in this article, and hopefully got you to understand how SNMP might be useful in your computing ecosystem. There were quite a few things we glossed over, mostly the boring stuff, but they aren't entirely necessary for you to start hacking around with SNMP.

We covered about 80% of the most important material, and I will leave the remaining 20% for you to pick up on your own. If you do plan on learning more about SNMP theory it would make sense to read a book or two on the subject or read a few articles on Wikipedia on SNMP. For casual use of SNMP though, you have more than your fair share to explore with the ideas from this article.

One final word of caution though is to make sure that you only use SNMP version 1, and 2 if you are communicating on a secure LAN behind a firewall. There have been some high profile break–ins of machines that have occurred by using insecure versions of SNMP over the internet. If you do need to remotely query or a control a machine across a the internet you must use SNMP v3 to be secure.

Noah Gift. "Using Net–SNMP and IPython". IBM Developerworks,AIX and Unix, http://www.ibm.com/developerworks/aix/library/au-netsnmpnipython/.

Noah Gift. "Python and Applescript". O'Reilly. http://www.oreilly.com/pub/a/mac/2007/05/08/using-python-and-applescript-to-get-the-most-out-of-your-mac.html

Noah Gift and Jeremy Jones. Python For Unix and Linux Systems Administration. O'Reilly . ISBN: 0596515820

Guido van Rossum. Python Tutorial: http://docs.python.org/tut.

OID Value/Description of sysdescr: http://www.alvestrand.no/objectid/1.3.6.1.2.1.1.1.html

Noah Gift has been a Mac user since his family bought a Macintosh Performa 6300 in 1992, and started connected to BBS networks immediately and then eventually the World Wide Web in 1993 when it become open to the public. He is the co-author of Python For Unix and Linux Systems Administration by O�Reilly. Noah has a couple of decades of experience in the Television and Film industry starting off as an editor for ABC Network News as a teenager. He contributed to the first feature animated film for Disney Feature Animation and Sony Imageworks. He also had stints at Turner Studios and Caltech, where he worked for the Nobel Prize-winning President as a Mac expert. He has a Master�s degree in CIS, and is LPI and ACSA certified. He also organizes PyAtl, the local Python programmers user group in Atlanta. Currently Noah is consultant, writer and speaker, specializing in OS X/Unix, Linux, Python, and Web development for his company, Giftcs, www.giftcs.com. Many of his projects and writing are available at www.noahgift.com. He can be contacted at noah.gift@giftcs.com