Volume Number: 27
Issue Number: 01
Column Tag: MacEnterprise

MacEnterprise: Managing Software Installs with Munki-Part 4

Crafting pkginfo files for munki

By Greg Neagle, MacEnterprise.org

Over the past several months, we've been looking at munki, a set of open-source tools that can manage software installation and removal on Mac OS X machines. Munki is available for download at http://code.google.com/p/munki. Munki can install software packaged in Apple's Installer package format, software delivered for "drag-and-drop" installs on disk images, and Adobe CS3, CS4 and CS5 products and updates using Adobe's supported enterprise deployment tools.

We set up a demonstration munki server on a Mac OS X "client" machine, and used that server along with the munki client tools to install, update, and remove some software packages on a client machine. The munki server is simply a web server, containing three types of information:

Installer items: these are packages or disk images containing the software to be installed. In many cases, you can use a package or disk image provided by the software vendor without having to repackage or convert the installer package. For example, munki can install Firefox directly from the disk image that you download from http://www.mozilla.com - you do not have to "repackage" Firefox in order to install it with munki.

Catalogs: these are lists of available software, containing metadata about the installer items. The munki administrator builds these catalogs using tools provided with munki.

Manifests: A manifest is a list of what software should be installed on or removed from a given machine. You can have a different manifest for every machine, or one manifest for all of your machines. Manifests can include the contents of other manifests, allowing you to group software for easy addition to client manifests. For example, you could create a manifest listing all of the software every machine in your organization must have. The manifest for a specific client could then include the "common software" manifest, and additionally have software unique to that client.

There is a fourth class of data that is commonly stored on the munki server as well, but munki clients do not access it directly. This data is the "pkginfo" files - typically one per installer item. These contain the metadata for each installer item. Munki clients do not access these files directly; instead they use the catalogs, which are themselves built from the pkginfo files.

Pkginfo files provide metadata about each installer item or package, information that either cannot be determined from the package itself, or info that would be too slow to get from every package each time munki runs. Much of the learning curve around munki involves crafting pkginfo files. Therefore, this month we will look at this topic in detail.

Fortunately, munki provides two tools to help with creating pkginfo files. The first we used in the previous two columns: munkiimport. munkiimport helps with importing new packages into the munki repository. It in turn calls a second tool to do most of the initial pkginfo generation. This tool is called, strangely enough, makepkginfo, and can be found at /usr/local/munki/makepkginfo. You can call it directly if you'd like. In its most common use, you give makepkginfo a package or disk image, and it outputs pkginfo. Here's an example:

% cd /usr/local/munki
% ./makepkginfo ~/Downloads/GoogleSketchUpMEN.dmg 
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>autoremove</key>
   <false/>
   <key>catalogs</key>
   <array>
      <string>testing</string>
   </array>
   <key>description</key>
   <string></string>
   <key>display_name</key>
   <string>Google SketchUp 8.0 (English)</string>
   <key>installed_size</key>
   <integer>105008</integer>
   <key>installer_item_hash</key>
   <string>d079e58569142aa5ac6c60617146b398a974be1b6d0aec3c270bfa3f70dbc07d</string>
   <key>installer_item_location</key>
   <string>GoogleSketchUpMEN.dmg</string>
   <key>installer_item_size</key>
   <integer>39800</integer>
   <key>minimum_os_version</key>
   <string>10.4.0</string>
   <key>name</key>
   <string>Google SketchUp 8.0 Installer</string>
   <key>receipts</key>
   <array>
      <dict>
         <key>filename</key>
         <string>Google_SU8_EN_SketchUp_Application.pkg</string>
         <key>installed_size</key>
         <integer>87812</integer>
         <key>name</key>
         <string>Google SketchUp Application</string>
         <key>packageid</key>
         <string>com.google.sketchup8.sketchup.application</string>
         <key>version</key>
         <string>8.0.3161.0.0</string>
      </dict>
      <dict>
         <key>filename</key>
         <string>Google_SU8_EN_SketchUp_Support.pkg</string>
         <key>installed_size</key>
         <integer>17192</integer>
         <key>name</key>
         <string>Google SketchUp Support</string>
         <key>packageid</key>
         <string>com.google.sketchup8.sketchup.support</string>
         <key>version</key>
         <string>8.0.3161.0.0</string>
      </dict>
      <dict>
         <key>filename</key>
         <string>Google_SU8_EN_SketchUp_Free.pkg</string>
         <key>installed_size</key>
         <integer>4</integer>
         <key>name</key>
         <string>Google SketchUp 8.0 (English) Add-Ons</string>
         <key>packageid</key>
         <string>com.google.sketchup8.sketchup_free.addons</string>
         <key>version</key>
         <string>3.0.3161.0.0</string>
      </dict>
   </array>
   <key>uninstall_method</key>
   <string>removepackages</string>
   <key>uninstallable</key>
   <true/>
   <key>version</key>
   <string>3.0.3161.0.0</string>
</dict>
</plist>

You could copy and paste this into a text editor for editing, or use shell redirection to create a text file directly; something like:

makepkginfo GoogleSketchUpMEN.dmg > GoogleSketchUp.plist

In either case, the pkginfo is probably not ideal as-is, and will need to be edited further. Let's look at some common edits.

Name

   <key>name</key>
   <string>Google SketchUp 8.0 Installer</string>

name is the most important key in a pkginfo file. It is this name that is used in manifest files to specify a package to be installed. In order for munki to find the latest version of a package, all the versions of a package must have the same name in their pkginfo item.

While you could use the name that makepkginfo pulled out of the Installer package, you probably want to remove the version number and simplify it:

   <key>name</key>
   <string>GoogleSketchUp</string>

(I remove the spaces from the name - but there's really no requirement to do this; it's just an old habit. It does make it easier to remember the name without having to check if it's "Google SketchUp" or "Google Sketch Up"…) If you later download a newer version of Google SketchUp, you'd want to make sure the name for its pkginfo was the same as the name you chose for this one).

Version

   <key>version</key>
   <string>3.0.3161.0.0</string>

version is the next most important key. This is how munki finds the most recent version (or a specific version) of a package. You'll see that the version that makepkginfo generated seems to be wrong. This is because the version of the SketchUp installer package is 3.0.3161.0.0. (Note - this is not common. Usually you'll find the installer package version more accurately reflects the version of the software it installs). You'll probably want to edit the version string to match the version of the Google SketchUp application:

   
   <key>version</key>
   <string>8.0.3161.0.0</string>

Catalogs

   <key>catalogs</key>
   <array>
      <string>testing</string>
   </array>

By default, makepkginfo puts newly generated items into a "testing" catalog. This is a useful default. You can configure a subset of your munki clients to look first in the testing catalog for package information. By placing new versions of software in the testing catalog, only your "testing" subset of machines will install the latest version. Later, when you are confident a new version won't cause issues for you, you can move it to a "production" or "release" catalog. All of your machines would be configured to check the production catalog.

   <key>catalogs</key>
   <array>
      <string>production</string>
   </array>

Note that the value of the catalogs key is an array - you can place an item in multiple catalogs. This is not used often, but one reason to do this would be to create multiple testing groups. Perhaps you are testing a new version of Firefox. You have a group of testers (for example, a group of web developers) that shouldn't get every new application, but do want newer versions of Firefox sooner than the general population. You can make this happen by creating a "firefox-testing" catalog. You create a munki catalog simply by defining at least one item as being in that catalog. So by adding "firefox-testing" to the list of catalogs for the latest version of Firefox, you cause a "firefox-testing" catalog to be created with at least one item.

   <key>catalogs</key>
   <array>
      <string>testing</string>
      <string>firefox-testing</string>
   </array>

You would also then list the "firefox-testing" catalog in the list of catalogs for your Firefox testers' manifest(s):

   <key>catalogs</key>
   <array>
      <string>firefox-testing</string>
      <string>production</string>
   </array>

This causes munki to look for items in the firefox-testing catalog first, looking in the production catalog only if no matching package is found in firefox-testing.

name and version are the most important keys, and so you should always check these and edit if needed. If you are using a testing catalog for new packages, you can often leave the catalogs key as-is, but if you want to move a package directly into production or have a special configuration, you may need to edit this key as well.

Display Name and Description

   <key>description</key>
   <string></string>
   <key>display_name</key>
   <string>Google SketchUp 8.0 (English)</string>

The display_name and description keys are optional, but help provide a better end-user experience. If provided, the display_name is used instead of the name when information is displayed to the user in Managed Software Update.app. The description, if it exists, is displayed below when a user selects an item in Managed Software Update.app.

   <key>description</key>
   <string>3D design software from Google.</string>
   <key>display_name</key>
   <string>Google SketchUp</string>

If I edit the two keys as shown above, it appears in Managed Software Update.app as in Figure 1.

Figure 1 - Google SketchUp displayed in Managed Software Update.app

For the display name, I removed the version number, since that is displayed elsewhere. The description can be as long as you'd like.

The pkginfo for Google SketchUp we've worked on so far is perfectly useable and functional. But there are more munki features that can be controlled by additional pkginfo edits.

Receipts vs. Installs

Our Google SketchUp pkginfo has a receipts key, listing the receipts for the Apple packages that are installed. This info can be used in two different ways. The first use for the receipts information is when removing an item. Munki uses the receipts to determine what files were installed, and compares that against all the other receipts on the machine. Any files listed in the item's receipts that are not in any other receipts are removed.

The second use for receipts is as a method to determine whether or not an item is installed, and therefore if munki should attempt to install the item. If munki has no other information, it will look for the existence of the receipts that are listed under the receipts key. If any of these are missing or an older version, the item will be installed.

But there are problems with this double duty for receipts. It is not uncommon for metapackages (packages that contain other packages) to install only a subset of the available packages, depending on the OS version and other installed software. This means that the list of receipts generated by makepkginfo may contain receipts for packages that don't actually get installed, or worse, are installed on some machines, but not others. If any receipt is missing, munki attempts to install the package. But if installing the package does not leave every receipt munki has on file, munki will continue to attempt to install the package again and again.

There are two ways to deal with this problem. The first is to edit the pkginfo, deleting any receipt for subpackages that are not actually installed. If the subpackages that don't get installed are only installed on Tiger machines, for example, and you don't have any Tiger machines, this solution may work. But if the subpackages are installed on some machines you manage and not others (perhaps due to hardware differences, or the existence of other software on the machine), this solution causes another problem. If you delete a reference to a receipt that does get installed, munki will not be able to accurately remove the item, as you have removed some of the information it needs to determine what needs to be removed.

A better solution to this problem is to provide munki with alternate information it can use to decide whether or not to install the item, leaving the receipts key to be used only for removing the item. Pkginfo files can contain an installs key, which lists items that are installed by the item. This key must exist (and is generated by default) for installer items that aren't Apple packages (like drag-n-drop disk images), as these items don't leave receipts. makepkginfo cannot (currently) automatically generate an installs key for Apple packages, but you can do so manually.

As an example of an automatically generated installs key, we can review the pkginfo items created for Firefox and Google Chrome in previous installments, as these items are distributed as drag-n-drop disk images. Here's the installs key for Firefox 3.6.13:

   <key>installs</key>
   <array>
      <dict>
         <key>CFBundleIdentifier</key>
         <string>org.mozilla.firefox</string>
         <key>CFBundleName</key>
         <string>Firefox</string>
         <key>CFBundleShortVersionString</key>
         <string>3.6.13</string>
         <key>path</key>
         <string>/Applications/Firefox.app</string>
         <key>type</key>
         <string>application</string>
      </dict>
   </array>
Here's the same key for a recent version of Google Chrome:
   <key>installs</key>
   <array>
      <dict>
         <key>CFBundleIdentifier</key>
         <string>com.google.Chrome</string>
         <key>CFBundleName</key>
         <string>Chrome</string>
         <key>CFBundleShortVersionString</key>
         <string>8.0.552.215</string>
         <key>minosversion</key>
         <string>10.5.0</string>
         <key>path</key>
         <string>/Applications/Google Chrome.app</string>
         <key>type</key>
         <string>application</string>
      </dict>
   </array>

There's a fair amount of information there. Fortunately, you don't have to generate these items by hand; makepkginfo can help you. Let's create one for Google SketchUp. First we must install Google SketchUp on a machine, so the application is at its correct path. We can then use makepkginfo with the -f flag to generate an item for the installs list:

% cd /usr/local/munki/
% ./makepkginfo -f /Applications/Google\ SketchUp\ 8/SketchUp.app
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>installs</key>
   <array>
      <dict>
         <key>CFBundleIdentifier</key>
         <string>com.google.sketchupfree8</string>
         <key>CFBundleName</key>
         <string>SketchUp</string>
         <key>CFBundleShortVersionString</key>
         <string>8.0.3161</string>
         <key>path</key>
         <string>/Applications/Google SketchUp 8/SketchUp.app</string>
         <key>type</key>
         <string>application</string>
      </dict>
   </array>
</dict>
</plist>

All we need is the actual installs section, which we can copy and paste into the Google SketchUp pkginfo:

   
   <key>installs</key>
   <array>
      <dict>
         <key>CFBundleIdentifier</key>
         <string>com.google.sketchupfree8</string>
         <key>CFBundleName</key>
         <string>SketchUp</string>
         <key>CFBundleShortVersionString</key>
         <string>8.0.3161</string>
         <key>path</key>
         <string>/Applications/Google SketchUp 8/SketchUp.app</string>
         <key>type</key>
         <string>application</string>
      </dict>
   </array>

This tells munki to check for the SketchUp.app inside the /Applications/Google SketchUp 8 folder, and if it exists, it must be version 8.0.3161 or later. If it doesn't exist, or is older, munki will attempt to install Google SketchUp 8.0.3161.

You may have multiple items in the installs list, and these items are not limited to applications. You can check for other bundle types, like Internet plug-ins or System Preferences panes, or Info.plist files at specific paths. These can all be compared to a specific version. You may also check for the existence of directories or files. With individual files, a checksum is generated - if the file on disk exists but its checksum doesn't match the one in the installs key, the item will be installed. For all of these file types, you can use makepkginfo -f to generate installs info.

The installs key provides a flexible and powerful way to help munki decide if a given item should be installed. It also provides for a certain amount of "self-repair". Using our prior example where we provide an installs key for our Google SketchUp item that lists the SketchUp.app application, if a privileged user were to delete the app or its entire enclosing folder, on its next run, munki would once again schedule an install of Google SketchUp.

Updates

Munki supports two kinds of package dependencies. Both require certain keys in pkginfo files. The first type of dependency allows you to mark a given package as an update for another package. We'll use Microsoft Office 2011 as an example. Our initial package is the disk image for the Office 2011 installer. Its pkginfo name is "Office2011" and its version is "14.0.0.0.0". Later, Microsoft releases an update for Office 2011. We create a new pkginfo item for the update, which we name "Office2011_Update", version "14.0.1.0.0". But we don't want to have to modify all the manifests that include Office 2011 and manually add the update to the list of managed_installs - instead, we want munki to discover and apply the update "automagically." We can do this by adding a new key - update_for - to the Office2011_Update pkginfo item.

   <key>update_for</key>
   <array>
      <string>Office2011</string>
   </array>

This key informs munki that this item is an update for Office2011. On any machine whose manifest contains "Office2011" in its managed_installs list, this update will also be considered for installation. When Microsoft releases the Office 2011 14.0.2 update, we can repeat the procedure for this update, and the next, and the next. In this way, all your managed machines with Office 2011 automatically find and install the updates.

You can also use the update_for key to mark other packages that aren't strictly updates so that they will be installed with (and removed with) another package. There are several pieces of software for which we need to install some additional files - licensing configuration files, or other site or organization-specific customizations. Instead of modifying the vendor's installer or repackaging the software, we package our additions and mark them as an update for the third-party software.

A good example of this is Firefox. Firefox is updated frequently, so it is convenient to be able to use the unmodified disk image as downloaded from Mozilla.com as the installation source. We also have some Firefox extensions we want everyone to have. By packaging these extensions separately, but marking them as updates for Firefox, any machine that gets Firefox also gets the extensions. When Firefox is removed, so are the extensions.

Requirements

Sometimes you need to ensure packages are installed in a certain order - for example, Adobe Acrobat 9 Pro has several updates that must be installed in the order they were released. If you name all the updates "AcrobatPro9Update" and you mark all them like so:

   <key>update_for</key>
   <array>
      <string>AcrobatPro9</string>
   </array>

munki will find the latest "AcrobatPro9Update" (as of this writing, 9.4.1) and try to install that. Unless the currently installed version of Acrobat Pro is 9.4.0, this is likely to fail. So you need to tell munki that before you install the 9.4.1 update, you must ensure the 9.4.0 update is installed. To do this, you use the requires key.

   <key>requires</key>
   <array>
      <string>AcrobatPro9Update-9.4.0.0.0</string>
   </array>

As it turns out, the 9.4.0 update requires the 9.3.4 update:

   <key>requires</key>
   <array>
      <string>AcrobatPro9Update-9.3.4.0.0</string>
   </array>

This update in turn requires the 9.3.3 update, which in turn requires the 9.3.2 update, and so on, all the way back to the 9.1.0 update. Manually installing all of these updates in the right order is a giant pain. Getting them all into munki and installing in the right order is also a pain, but once you've done it, munki can then do it on every machine that has or needs Acrobat Pro 9.

The requires key can also be used for other relationships: perhaps you have a tool to install that requires that the Xcode tools also be installed. You could add a requires key specifying that your tool requires Xcode, and so when someone tried to install your tool on a machine using munki, Xcode would be installed first.

There are more optional pkginfo keys that can be useful in some situations. Here are a few.

   <key>minimum_os_version</key>
   <string>10.5.0</string>
   <key>maximum_os_version</key>
   <string>10.5.8</string>

You can use these keys to specify that a given version of a package should not be installed on an OS version lower than, say 10.5.0, or higher than, say, 10.5.8. (This is really useful with Xcode.)

   <key>supported_architectures</key>
   <array>
      <string>i386</string>
      <string>x86_64</string>
   </array>

This key can be used to limit the installation of a given version of a package to machines matching a certain processor architecture. In the above example, this would not install on a PowerPC-based machine.

Version 0.7.0 and later of munki support a few more interesting keys:

   <key>forced_install</key>
   <true/>
   <key>forced_uninstall</key>
   <true/>

These keys can be used to indicate that a package is safe to install and/or uninstall without the user's consent. A package marked with forced_install equal to true will be silently installed in the background without the user being notified. Use this carefully.

   <key>blocking_applications</key>
   <array>
        <string>Firefox</string>
      <string>Safari</string>
      <string>Opera</string>
   </array>

This key lists applications that may block the installation of a package. This key can come into effect in two circumstances. The first: a user is notified of updates to install and elects to install without logging out. In this scenario, if one or more of the applications in the blocking_applications list is open, the user is warned to quit the application(s) before being allowed to proceed.

The second scenario is in conjunction with the forced_install and forced_uninstall keys. If any application in the blocking_applications list is open, the forced operation will not be attempted for that package. The intent is to prevent munki from trying to update or remove open applications and potentially causing crashing and data loss.

Even More Keys

There are many more keys that may appear in a pkginfo item, but most of the ones not mentioned so far are automatically created by makepkginfo based on the contents of an installer item. Here are some additional keys:

installer_item_hash: a checksum of the installer item so we can verify the downloaded item matches the original item. Auto-generated.

installer_item_location: relative path to the installer item (package) inside the pkgs directory on the munki server. Auto-generated, but may need to be edited if you move or modify the path to the package.

RestartAction: does the package require a logout or restart? Auto-generated from Apple packages, but you can add or modify this and may need to for non-Apple packages. Set to "RequireLogout" if logout is needed, or "RequireRestart" if a restart is needed.

installer_type/uninstall_method: keys used by munki to determine the correct install and unsinstall methods. Auto-generated.

uninstallable: a Boolean that indicates whether or not the package is uninstallable. Auto-generated based on the installer type, but you may need to set it to false for items (especially Apple updates) that you know cannot be safely uninstalled.

Remember that when you edit a pkginfo file on the munki server, you must run makecatalogs /path/to/munki/repo to get your changes incorporated into the munki catalogs. A common mistake is to make changes and forget to run makecatalogs.

Whew - that's a lot of info to digest. Pkginfo files are easily the most complicated part of munki, but that's because they contain virtually all of the information munki needs to do its job. Pkginfo files contain both metadata extracted from the packages themselves, and additional information that only the munki admin can provide. munkiimport and makepkginfo can help create the pkginfo files, but you will sometimes need to manually edit these files to take advantage of all of munki's features.

That concludes (for now) our look at munki. We haven't exhausted all of munki's features, but hopefully we've covered enough for you to decide whether or not it's worth further investigation. If you'd like to continue your exploration of this set of tools, visit the munki website at http://code.google.com/p/munki, and read (or join) the munki-dev Google Group at http://groups.google.com/group/munki-dev.

Greg Neagle is a member of the steering committee of the Mac OS X Enterprise Project (macenterprise.org) and is a senior systems engineer at a large animation studio. Greg has been working with the Mac since 1984, and with OS X since its release. He can be reached at gregneagle@mac.com.