Volume Number: 20 (2004)
Issue Number: 11
Column Tag: Programming

Looking for Open Directory

by Dean Shavit

In the Beginning, there was NetInfo

I remember being awed and mystified by NetInfo at Apple Presentations by former NeXT System Engineers of Mac OS X Server version 1 in 1999. The idea of hierarchies and domains, administrator accounts and authentication, entries and attributes seemed downright weird to a Mac OS devotee like myself. Growing into the UNIX-y side of things with OS X Server as it evolved, I found myself more and more intrigued with the control a Directory Service brought to a group of Mac users, the mount records, machine accounts, and URLs visible in NetInfo Manager provided a valuable head-scratching and learning experience, to say the least.

Initially, NetInfo was a brilliant solution to a common UNIX problem--how to distribute and consolidate user, group, and host information over a network without having to maintain copies of /etc/passwd /etc/users, /etc/groups and /etc/hosts over multiple machines. As a matter of fact, extracting and capturing valid host, passwd, and group flat files for use on a UNIX system is as easy as:

$ nidump passwd . > mypasswds.txt
$ nidump hosts  . > myhosts.txt
$ nidump group  . > mygroup.txt

The real magic came in having applications, not just users, access authentication information in a central repository--without having to duplicate flat files all over the network. Mail Services, HTTP Services, FTP Services and file services can all share the same source of authentication. NetInfo, as implemented in NeXTSTEP (the ancestor to OS X), also allowed for a hierarchical arrangement of NetInfo "Domains" or databases of information, where defining administrator accounts at the top (or root) of the hierarchy allowed for them to authenticate at any point on all of its branches, as in the illustration below, where an administrator who had an account in the Corporate domain could authenticate any workstation in either the Sales or Marketing domain, while a user with an account in Sales wouldn't be able to authenticate at a workstation in the Marketing domain.

Each OS X system up to OS X 10.3, either server or client, still has a local NetInfo database that's consulted first by services and applications in need of authentication, regardless of whether the OS X workstation or server is configured to consult shared directories or foreign directories, such as Microsoft's Active Directory or Novell's eDirectory. With NetInfo as a directory service, things got much more interesting with an additional shared NetInfo database created on OS X 10.1 or 10.2 (thereby becoming a directory server), then offered up to other OS X computers on a network via a process of "binding, " which took place at reboot by automatic configuration (DHCP), static configuration (specifying the IP address of the server which hosted the shared Network NetInfo database, along with the "tag" or name of the database) or by broadcast binding (which required the creation of machine records in the shared NetInfo and was widely ignored in OS X Server, a legacy from when NexTSTEP didn't support DHCP services).

With OS X Server 10.1, configuring a shared NetInfo domain could be a hit or miss proposition for all but the most learned gurus or the most patient and stalwart administrators, with the process of creating and populating a shared domain at times resulting in a spinning disc of death. Destroying a shared NetInfo database was sometimes cause for a reinstall of the whole server. OS X Server 10.1 relied heavily on a properly configured DNS server (which is still a requirement for most OS X Servers) but with many Mac professionals just getting into UNIX (myself included) DNS configuration was a black art, soon mastered with long nights of hair-pulling.

OS X Server 10.1 stored a lot of system and service configuration information in NetInfo that would be moved to XML property list (.plist) files in OS X Server 10.2 and 10.3, notably settings for the IP Firewall Service, Mail Server, and Apple File Server.

NetInfo Becomes Open Directory, with Its Apprentice, OpenLDAP

Open Directory (I), which premiered with OS X Server 10.2, was Apple's new moniker for the collection of properties and values it had previously collected in NetInfo. It had some significant enhancements other than its capability to serve up directory information through LDAP via the OpenLDAP slapd daemon. Open Directory had new features that allowed for preference management by workgroup and computer accounts with presets that made creation of users and groups more efficient. With the release of OS X Server 10.2, NetInfo was still the default method for storing shared directory data, but it had been extended into something much more interesting. Previous to the release of OS X Server 10.2, there were rumblings that NetInfo has been thrown under the bus in favor of OpenLDAP or "slapd." In reality, OS X 10.2, which ushered in the Open Directory era, was an evolutionary step towards OpenLDAP. OS X 10.2 used NetInfo for its directory storage, or "domains" which is Apple's term for the actual databases that contain the directory information, but did integrate OpenLDAP so that the shared NetInfo domains could be accessed through an LDAPv3 plug-in to the Directory Access application, via a pre-defined set of Open Directory LDAP mappings, allowing for a smoother transition to Open Directory II in OS X Server 10.3. Also a significant advance of Open Directory was the addition of the Password Server which supported multiple authentication protocols based on the Simple Authentication and Security Layer (SASL) Standard which superseded the deprecated and insecure crypt password algorithm, that OS X 10.1 (not OS 9!) clients are still forced to use to this day.

Open Directory services in OS X 10.2 also introduced the Workgroup Manager Application, which allowed for the easy insertion and editing of mcx (managed user and workgroup settings) but offered no real interface for adding LDAP attributes for populating such entries as email address, phone number, department, and title that would make the LDAP support useful with clients such as Apple's Address Book or Microsoft Entourage. Insertion of such data could easily be done with good 'ol NetInfo Manager, by creating the requisite properties and values within the user records.

One of the best tools for populating user records with contact and organizational information is the open-source X Windows application Directory Administrator (available at fink.sourceforge.net ) that provide pre-set fields that makes entering personal data a pleasure.

Another nice touch was the Open Directory Assistant, which guided admins through the setup of directory domains and the password server and came in handy as a troubleshooting tool for those who forgot that the password server was necessary for authenticating Windows clients and gave themselves a "Basic" (now labeled "Crypt") password in Workgroup Manager denying them the privilege to set password server passwords for other users. With OS X Server 10.3, it's done by deleting the .AppleSetupDone file in /var/db and rebooting, which forces the Server Setup Assistant to run, allowing the administrator to reset their password and become a password server administrator once again. Upon setting up Open Directory, the personal installing OS X Server had three basic choices for a directory. First, they could store user account information in a local directory (single local NetInfo Domain). Second, they could connect to a directory on another server either by NetInfo or LDAP or NIS. Third, they could choose to host a directory that other computers and servers could access over the network (shared NetInfo domain) and could enable LDAP support for NetInfo. Each change or session of the Open Directory Assistant required a reboot, but it was a vast improvement over the riskiness of adding or deleting directory domains in OS X Server 10.1. Jaguar Server also proved the value of Apple's Workgroup Management scheme, modeled loosely on the interface and features of Macintosh Manager, the Classic Mac Managed Workgroup solution, but delivered with the robustness and reliability that Macintosh Manager administrators could only dream of.

Open Directory II: The Apprentice is now the Master

They went ahead and did it, they really did! It's extremely unusual, in the spirit of NIH (not invented here), that a computer company with as rich of a tradition as Apple (and NeXT) would dump its home-brewed network directory structure in favor of an open-source replacement, but that's exactly what happened in OS X Server 10.3. Shared NetInfo domains were no more, replaced by Open Directory Masters that used OpenLDAP exclusively and BerkleyDB as the back-end storage for the directory domain. An Open Directory Master with sufficient memory and storage can operate smoothly with 100,000 user accounts and can handle 250 simultaneous LDAP connections! For larger networks, this meant that it might no longer be necessary to arrange servers in hierarchies, as with NetInfo, allowing more flexibility in management and authentication.

Setting up and installing OS X Server is still relatively straightforward, the Open Directory Assistant morphed into the new Server Setup Assistant and gave up some of its capabilities to the new Server Admin application. On first look, Open Directory II appears little changed from Open Directory for Jaguar Server. However, the nomenclature has changed significantly. A shared domain is now an "Open Directory Master." A local directory is labeled a "Standalone Server." With these shades of Microsoft-speak, Apple accomplished once again what the others dared not to imagine: with Panther Server, an administrator could change the structure (role) of Open Directory without rebooting. Try that on a Windows Server! How did Apple do it? Well, OpenLDAP is quite a different beast than NetInfo. While NetInfo domains bound together at boot time, Open Directory II acts as an external directory domain connected locally at /LDAPv3/127.0.0.1, which is the same as a connection to an LDAP directory running on a different server at another IP address.

Note that the Server Admin or Server Assistant adds the Open Directory Master LDAP node with the "Custom path" as if it were manually connected computer. As a matter of fact, aside from a few very important differences, which I'll get to later, there's really little difference between the way an OS X client would connect to an Open Directory II Master and the way the OS X Server that hosts it actually connects.

Connecting an OS X Server to another Open Directory Server is also accomplished through the LDAPv3 plug-in in Directory Access, using the "Open Directory Server" built-in mappings. There's a plug-in for Active Directory that doesn't require editing the Active Directory Schema to leverage accounts on a Windows 2000 or 2003 Server. Open Directory Masters can emulate Windows Domain Controllers for Windows Workstations allowing a single OS X Server to act as both home directory host and authentication server to mixed environments.

In addition to Apple's Password Server, Open Directory II features a new authentication authority: Kerberos. Given a properly configured DNS Server on the network or on the Open Directory Server itself, an Open Directory Master auto-configures itself as a KDC (Key Distribution Center) for OS X 10.3 clients (without additional configuration) and OS X 10.2 clients configured with the proper /etc/authorization edits, /etc/ krb5.keytab files, and edu.mit.Kerberos preference files. All OS X 10.3 clients receive their edu.mit.Kerberos preferences through the kerberosautoconfig command. The following from the man page describes the process:

the kerberosautoconfig command creates the 
edu.mit.Kerberos file from information stored in the open 
directory config record named KerberosClient. The existing 
edu.mit.Kerberos file is only replaced if the autogenerated 
header is present and the generation_id in the KerberosClient 
config record is greater than that within the file. The default 
location of the output file is /Library/Preferences/edu.mit.Kerberos.

This addition to Kerberos is Apple's unique method of simplifying the configuration of an authentication protocol that typically requires much work at the command-line. Even the task of adding other OS X Servers to the Kerberos Realm is a relatively simple process that allows multiple servers connected to a an Open Directory Master to grant access to services like ssh, afp, ftp, pop and imap without having users re-authenticate. When I first got single-sign on going here at our training lab, I was startled to be able to ssh into a server without authenticating again! It's easy to get used to, though! With Kerberos being the main authentication protocol for Windows Servers, and being supported by every major OS (even OS 9!), it's great to see Apple embracing an open standard and improving on it....wait did I just say that? Let me repeat myself, and repeat....

Only once an administrator starts scratching the surface of the Panther Server's Workgroup Manager application, specifically by going to its Preferences dialog and checking the "All Records" tab and inspector, does it becomes clear how much Apple has enriched Open Directory II.

Of course, such enrichment can't come without a requisite warning:

Looking at the raw directory data is an interesting way to twiddle an evening or two away.

Clicking on the "New Attribute..." button brings up a window with a popup of all supported Open Directory attributes. However, Workgroup Manager falls way short of NetInfo Manager in one significant way. It does not allow the person editing the directory data to see the hierarchy, structure, or relationships between much the attributes. NetInfo Manager still exists, but none of the Open Directory Data is really visible there on an Open Directory Master anymore. For instance, it is possible to add a KDCConfigData attribute, but it is not possible, without working with the command-line dscl tool, to get a sense of where in the LDAP directory it should go.

rclark:~ rclark$ dscl -u rclark -p /LDAPv3/127.0.0.1/ -list /Config                 
Password: 
KerberosClient
KerberosKDC
ldapreplicas
macosxodconfig
mcx_cache
passwordserver

Mcx_cache and passwordserver are familiar holdovers from Open Directory II. KerberosClient and KerberosKDC are new additions to the family. A closer look at the KerberosClient attribute reveals the magic of Apple's Kerberos implementation:

dscl -u rclark -p /LDAPv3/127.0.0.1/ -read /Config/KerberosClient

cn: KerberosClient
objectClass: apple-configuration top
AppleMetaNodeLocation: /LDAPv3/127.0.0.1
PasswordPlus: ********
RecordName: KerberosClient
XMLPlist: <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" 
     "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>edu.mit.kerberos</key>
        <dict>
                <key>domain_realm</key>
                <dict>
                        <key>.most.com</key>
                        <string>RCLARK.MOST.COM</string>
                        <key>most.com</key>
                        <string>RCLARK.MOST.COM</string>
                </dict>
                <key>libdefaults</key>
                <dict>
                        <key>default_realm</key>
                        <string>RCLARK.MOST.COM</string>
                </dict>
                <key>realms</key>
                <dict>
                        <key>RCLARK.MOST.COM</key>
                        <dict>
                                <key>KADM_List</key>
                                <array>
                                        <string>rclark.most.com</string>
                                </array>
                                <key>KDC_List</key>
                                <array>
                                        <string>rclark.most.com</string>
                                </array>
                        </dict>
                </dict>
        </dict>
        <key>generationID</key>
        <integer>1114977345</integer>
</dict>
</plist>

Here's the property list (.plist) used to automatically generate the edu.mit.Kerberos file when an OS X 10.3 client logs into an Open Directory Master with a running Kerberos KDC (Key Distribution Center).

Open Directory II also allows setting the passwords of multiple users simultaneously, as well as setting other attributes, such as home directory path. Just make a multiple selection of user accounts in Workgroup Manager and set a password for all of the users, check the option to "change password at next login" and with the correct password policies in place, there's no more need to worry about setting unique passwords and recording them. Password policies have expanded to include password length, composition of number and letters, difference from account name, and difference from last x number of passwords used when a change of password is required, though Open Directory still lacks an allow/deny schedule for users and groups.

The ldapreplicas attribute is a new addition to Open Directory as well. With two unique serial numbers (don't try it without them), an Open Directory Master will replicate the directory, password server, and Kerberos KDC to another OS X 10.3 Server in its entirety, and quickly. It takes just a few minutes to replicate an Open Directory Master on a 100-megabit Ethernet network with a few hundred users and a KDC. This is great for large educational institutions and enterprises concerned about downtime. Replication can occur on a schedule, or whenever the Open Directory Master is changed. Failover occurs naturally and the client logs into the first available replica.

Searching for the "Look"

OS X Server 10.3 is certainly one beast of a Directory server, but it does share one unfortunate characteristic with Windows 2000 and 2003 Server that its predecessors didn't. Though Workgroup Manager gives administrators access to all LDAP attributes and values, it doesn't allow for the hierarchical view that NetInfo manager provided in previous iterations of OS X Server. As with Windows Servers, administrators can see parts of the directory in certain applications, but never the "whole" thing. Certain important elements, such as the /Config container, aren't readily visible, even from using the command-line utilities. I, for one, despite have been dipped in the river of UNIX, am still a very visual person--things tend to make more sense to me if I can visualize them, eyeball them. GUIs make me smile.

So, my search for a utility (Directory Administrator, despite being a great editor, doesn't have a hierarchical mode) to view Open Directory II was on. I toyed around with phpldapadmin (http://phpldapadmin.sourceforge.net/) other web-based utilities, frankly I wanted something I could run on an iBook or Powerbook without having to configure a web server. Eventually, I stumbled upon a great utility called LBE.jar (short for LDAP Browser/Editor), a Java applet that finally gave me that "look" I'd been missing from previous version of OS X Server. LBE (http://www-unix.mcs.anl.gov/~gawor/ldap/) is old software (it hasn't been updated since 2002), but seems to work perfectly with Open Directory, which is either a testament to Open Directory's adherence to the OpenLDAP standard, or LBE's adherence to it, or both.

Downloading and unpacking LBE is simple, and requires no further tweaking to work on OS X 10.3.5. It launches with a double-click on the LBE.jar file. Configuring LBE to connect to Open Directory isn't very straightforward--it took me a few tries to get it right, as in the screenshot below:

Browsing Open Directory with LBE is certainly an eye-opening experience! Finally, I have a hierarchical tool that not only reveals all of the attributes, but shows them in proper relation to their container, sub containers, and organizational units. LBE is a great learning tool for connecting the "LDAP dots", but I'm not about to start editing Open Directory attributes with it...I'm a little concerned that it might not write the data properly, so I'd have to recommend that any editing be done on a test server, rather than a production server. I'm also a little concerned that it may be sending my account and password in the clear, so I'm careful not to run it over any network I'm unsure about, especially WAPs (Wireless Access Points).

Just about everything's connecting to LDAP these days, and not just address books. Soon, even freeware content management systems may have the ability to connect directly to and OS X Server for user and group information, ecommerce accounts created at online stores will live in LDAP on OS X Servers. Some of the latest builds of the PHP (Pre Hypertext Processor) library have LDAP support rolled in. And who knows what Apple will have up its sleeve for Open Directory III due out in mid-2005 with Tiger Server? Given the pre-announcement of several features, I can safely guestimate that new attributes will appear for access control lists (ACLs), Windows Group and User Policies (yes, there'll probably be registry information living, like alien larvae, in Open Directory), and new Kerberos Features that allow Windows clients authenticated by Windows Server KDCs to access services on an OS X Server, not to mention some new home directory options. It's great to see Apple embracing an open standard and improving on it....wait did I just say that again? Let me repeat myself, and repeat....and repeat.

Dean Shavit is an ACSA (Apple Certified System Administrator) who leads training sessions and manages consulting projects for MOST (Mac OS Training & Consulting) in Chicago. If you have questions or feedback you can contact him at dean@macworkshops.com.