TweetFollow Us on Twitter

Mac in the Shell: Reading and Writing plist files with Python

Volume Number: 25
Issue Number: 09
Column Tag: Mac in the Shell

Mac in the Shell: Reading and Writing plist files with Python

Tame those pesky plists

by Edwarcd Marczak

Welcome

Property list files, also known as 'plists,' are pervasive in OS X. This article teaches you the basic inner-workings of the plist format, system level methods of working with plist files and how to interact with these files using Python under OS X.

Anatomy

Plist files are structured XML (eXtensible Markup Language) files and easily understandable. Essentially, a plist file is a way to store standard types of data. By "standard," I mean string, integer, Boolean and so on, although there are ways to store arbitrary data as well. A plist file can easily be read into and written out from an NSDictionary object. Thanks to PyObj-C, an NSDictionary can be mapped onto and manipulated with a Python-based dictionary object.

Given the following dictionary:

{
    color:'blue',
    count:15,
    style:'fruit'
}

the plist in Listing 1 would be created

Listing 1-example plist file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Let's take a closer look at this plist. The header declares this file as an XML

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

Ultimately, this header isn't up to you. In this article, you'll see that Apple's Cocoa APIs will properly generate this upon writing a plist. For more information about XML, see the specification page as http://xml.org, or the Wikipedia entry at http://en.wikipedia.org/wiki/Xml.

The plist tag wraps the entire file:

<plist version="1.0">

Again, Apple's APIs will write this out as appropriate. Next, we find a dictionary tag:

<dict>

As I mentioned earlier, the structure we wrote out was a dictionary. In fact, that's all you'll ever really do with plist files: read a plist into a dictionary or create one from scratch, and then let Apple's APIs write it out.

Wrapped in the dictionary are its values:

<key>color</key>
<string>blue</string>
<key>count</key>
<integer>15</integer>
<key>style</key>
<string>fruit</string>
Following this, the tags are closed and the file ends:
</dict>
</plist>

Each tag should lead to a new level of indentation. It's easy to see the structure here. Best of all, it's easily human-readable.

However, beginning with OS X 10.5, the bulk of plist files found on the system are stored in a binary format, not plain text. While this does have the effect of using less space on disk and faster load times, it takes the human-readable part out of the picture. Of course, there are ways to deal with that.

System Tools

There are several ways to work with plist files, both graphically and from the command line. Apple's Property List Editor is installed as part of the free developer tools suite (Xcode et al). In a standard install, it is found at /Developer/Applications/Utilities/Property List Editor.app. This is the easiest way to visualize a plist. It's also useful for creating a plist from scratch. Property List Editor can also edit entries in a plist file.


Figure 1-Property List Editor.app displaying the hierarchy of a plist file.

While Property List Editor is fine for one-off plist work, it doesn't really scale too well. That it doesn't have a dictionary to use with AppleScript is just one example. What if you need to modify a plist on thousands of machines? (Or even 15 machines-it's a pain to walk around to each machine and potentially interrupt people's work. You may even want to update them after hours, while you're home). Once again, it's scripting to the rescue.

There are several utilities for standard shell scripting or ad-hoc use. plutil, defaults and PlistBuddy all have different purposes and capabilities.

plutil is the most basic and utilitarian of the three. plutil, the plist utility, converts plist files between text (xml) and binary formats and can also verify the structure of a plist. An example is in order. If you want to view the contents of a binary plist-com.apple.nat.plist, for example-but don't care to open it in Property List Editor you can run this:

plutil -convert xml1 -o - /Library/Preferences/com.apple.nat.plist

(This makes a very nice alias: alias viewplist="plutil -convert xml1 -o - $1". Keep that in your .bash_profile). Running this command tells plutil to convert the plist to text ("xml1") and send the output ("-o") to standard out. You could certainly write the output to another file on disk if you choose.

plutil can also lint a file; that is, check it for consistency and basic errors. What it cannot do is verify that your key-names and data are correct. Running a lint check is as simple as passing in the -lint switch:

$ plutil -lint /Library/Preferences/com.apple.loginwindow.plist 
/Library/Preferences/com.apple.loginwindow.plist: OK

If the lint process encounters an error (or errors, perhaps), you're told the error and on which line:

$ plutil -lint someplist 
someplist: Encountered unknown tag stringblue</string on line 6

The defaults command gives you access to the user defaults system. The "user defaults system" is a fancy way of saying "preferences," which, you'll probably recognize as data stored in a plist file. The name is derived from the Cocoa API that performs the same task: NSUserDefaults. The defaults utility allows for reading and writing individual keys and their data to and from a plist file, reading a plist in whole and more.

Perhaps the simplest use of the defaults command is reading an entire plist file. This is equivalent to the plutil command given earlier:

$ defaults read /Library/Preferences/com.apple.nat
{
    NatPortMapDisabled = 0;
}

The defaults command reads plist files of either xml or binary. However, it will only write a plist out in the binary variety. It will even go so far as to convert an xml plist into binary if used to update a value in that plist. Do note that the target plist is specified without the .plist extension.

The defaults command, however, is not exactly a general-purpose plist utility like plutil or Propery List Editor.app. As mentioned, it works within the bounds of the user defaults system. The upshot of this is that it expects plists to reside in specific places: one of the Library/Preferences directories on the system. Do not rely on the defaults command to read and write arbitrary plists. (In 10.5 and 10.6, accessing arbitrary plist files is possible, however, that functionality is said to be going away. Plus, you're reading this article and will be learning better ways of handling this). One other small problem with defaults: it's virtually impossible to work with values in nested dictionaries. Which brings us to PlistBuddy.

PlistBuddy started off as a utility that was only found embedded into packages for Apple updates. Clearly, Apple realized they needed a utility like this and developed it for their own use. As of Leopard, though, it's a real part of the OS: it is found at /usr/libexec/PlistBuddy and even has a man page. While the defaults command can handle most tasks, PlistBuddy excels at editing keys and values in a nested dictionary.

Let's imagine our example plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>color</key>
   <string>blue</string>
   <key>count</key>
   <integer>15</integer>
   <key>cust_info</key>
   <dict>
      <key>pid</key>
      <string>98234573</string>
      <key>uid</key>
      <string>348576</string>
   </dict>
   <key>style</key>
   <string>fruit</string>
</dict>
</plist>

Notice that the key, "cust_info" is a dictionary, rather than a simple, single value. PlistBuddy can easily update the values in this nested dictionary. PlistBuddy can work interactively, which I will not cover, but can also pass in all commands using the "-c" switch. To set the value of a key, you need the path to the key and the set command. The path to the key starts with a colon (":") and uses a colon as the separator for each level in the hierarchy. Here's how to change ("set") the value of the existing "pid" key to 94758476, in the plist, "com.mactech.example.plist":

/usr/libexec/PlistBuddy -c "Set :cust_info:pid 94758476" ./com.mactech.example.plist

(This is running the command in the same directory as the target plist. Otherwise, you'd need to specify the full path to the plist to edit). See the PlistBuddy man page (note the capitalization!) for more information on the utility. PlistBuddy is capable of much, much more, including copying values and merging plist files.

Accessing plists Via Python

From time to time, as a system administrator, you'll find yourself in a position where you'd like a script to store its own preferences. Or, simply have a script analyze a plist and act on the contents in some manner. In many cases, bash scripting that uses the commands already presented (plutil, PlistBuddy and, particularly, defaults) will be perfectly acceptable. However, for anything with a little more complexity, you may already be scripting in Python (or perl, or Ruby, etc.). Since Mac in the Shell has been focusing on Python for the last several columns, we'll use it here as well.

Python, with PyObj-C, makes this trivial. More interestingly, you get the best of both worlds: Apple's APIs along with Python's ease of use and the speed of the edit and run cycle (skipping the compile step of C-based languages). To see this in action, let's start with nearly the most simple example possible. Listing 2 contains write_plist.py, which demonstrates creating a dictionary that gets written to a plist.

Listing 2-write_plist.py

#!/usr/bin/python2.5
from Foundation import NSMutableDictionary
my_dict = NSMutableDictionary.dictionary()
my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'
success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)
if not success:
  print "plist failed to write!"
  sys.exit(1)

Upon running this program, com.mactech.example.plist will be created in the same working directory as the program itself. The plist file will match the output that is shown in Listing 1. Let's examine this line-by-line to see how it works.

The very first line-#!/usr/bin/python2.5-is a good reminder that Python version 2.5 or higher is required for PyObj-C integration. This will not work on Tiger systems out of the box.

from Foundation import NSMutableDictionary

This import is responsible for all of the magic here. While we could import all of Foundation, we'll just import the portion we need: NSMutableDictionary.

my_dict = NSMutableDictionary.dictionary()
-

Typically, creating a dictionary in Python would use curly braces, like this:

new_dict = {}

or, you can even fill it on creation:

new_dict = {'color':'blue', 'count':15, 'style':'fruit'}

However, we need to create a real Cocoa NSMutableDictionary object, so that's what we've done. Nicely, we can no go on and treat that just like a Python dictionary:

my_dict['color'] = 'blue'
my_dict['count'] = 15
my_dict['style'] = 'fruit'

You can use the Cocoa API for adding entries to a dictionary as well:

my_dict.setValue_forKey_('stop', 'state')

This would set the key 'state' to store the value 'stop', and add the following to the plist once written out:

<key>state</key>
<string>stop</string>

But, really... if you're using Python, take advantage of it where you can! (I suggest using the Python method). You will need to use the Cocoa API to write the dictionary out to disk as a plist file:

success = my_dict.writeToFile_atomically_('com.mactech.example.plist', 1)

The Cocoa writeToFile:atomically: method of NSDictionary (and, by extension, NSMutableDictionary) writes a property list representation of the contents of the dictionary to the path given.

if not success:
  print "plist failed to write!"
  sys.exit(1)

This final conditional tests to see if the writeToFile:atomically: method returned a True ("success") or False ("failure") value. While not strictly necessary for this program to run, checking these values is a good habit to get into.

Python Ease

Just as a reminder, once you create the NSMutableDictionary, you can use standard Python mechanisms to manipulate and traverse it. Adding a key with a dictionary as its value is as simpe as you'd expect. Just create the dictionary and then assign it to the parent dictionary. For example, to recreate the com.mactech.example.plist shown earlier, we would add the following to our program, after creating the initial dictionary:

sub_dict = {}
sub_dict['uid'] = '348576'
sub_dict['pid'] = '98234573'
my_dict['cust_info'] = sub_dict

Also, as shown earlier, you can also use all of the Cocoa APIs available to you to manipulate the dictionary as well. The style you choose may be situation dependent. Some situations may call for using the Cocoa-way, while others may favor more Pythonic writing. When working with any Cocoa API, though, as always, you'll want to keep the documentation handy.

Use It or Lose It

This was an incredibly fun article to write. The topic is incredibly practical for everyday use. The plist format is pervasive throughout OS X. Every technical person should have a familiarity with it, and System Admins should be even more deeply involved. While many cases can simply be solved with a single command-line call to defaults or PlistBuddy, anything with deeper involvement should use a scripting language like Python. The nice thing about the scripting solution is that once you build up your library of routines, they're written and ready for re-use. Reading about it here only gets you so far. Go write a script and run it on a test system so you're ready for the real thing when the opportunity arrives.

Media of the month: Kick it old-school with vintage computer brochures and manuals at http://assemblyman-eph.blogspot.com/2009/04/vintage-computer-brochures.html. Full PDFs of how it used to be. I remember when taking one of my first computer courses, the teacher launching into the history of computing. Naturally, I just wanted to get into sitting at a computer and coding. But now, perhaps more than ever before, it's really useful to be able to frame our current experience with that which it was built on and evolved from.

Next month, we'll be covering Snow Leopard related topics! It'll be two issues before we get back to Python and scripting in general. Until then, keep practicing.

References

"About Property Lists": https://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/AboutPropertyLists/AboutPropertyLists.html#/apple_ref/doc/uid/20001010-46719

"Understanding XML Property Lists": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1

"Introduction to Property List Programming Topics for Core Foundation": http://developer.apple.com/iphone/library/documentation/CoreFoundation/Conceptual/CFPropertyLists/CFPropertyLists.html

"Introduction to User Defaults": http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/PropertyLists/UnderstandXMLPlist/UnderstandXMLPlist.html#/apple_ref/doc/uid/10000048i-CH6-SW1


Ed Marczak is the Executive Editor of MacTech Magazine. He has written for MacTech since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Macs Fan Control 1.5.0.0 - Monitor and c...
Macs Fan Control allows you to monitor and control almost any aspect of your computer's fans, with support for controlling fan speed, temperature sensors pane, menu-bar icon, and autostart with... Read more
Daylite 6.7.7 - Dynamic business organiz...
Daylite helps businesses organize themselves with tools such as shared calendars, contacts, tasks, projects, notes, and more. Enable easy collaboration with features such as task and project... Read more
OpenOffice 4.1.7 - Free and open-source...
OpenOffice.org is both an Open Source product and a project. The product is a multi-platform office productivity suite. It includes the key desktop applications, such as a word processor, spreadsheet... Read more
Backup and Sync 3.46 - File backup and s...
Backup and Sync (was Google Drive) is a place where you can create, share, collaborate, and keep all of your stuff. Whether you're working with a friend on a joint research project, planning a... Read more
iClock 5.5 - Customizable menu bar clock...
iClock replaces the old Apple's default menu bar clock with more features, customization and increases your productivity. Features: Have your Apple or Google calendar instantly available from the... Read more
Garmin Express 6.18.0.0 - Manage your Ga...
Garmin Express is your essential tool for managing your Garmin devices. Update maps, golf courses and device software. You can even register your device. Update maps Update software Register your... Read more
MarsEdit 4.3.5 - Quick and convenient bl...
MarsEdit is a blog editor for OS X that makes editing your blog like writing email, with spell-checking, drafts, multiple windows, and even AppleScript support. It works with with most blog services... Read more
Xcode 11.0 - Integrated development envi...
Xcode includes everything developers need to create great applications for Mac, iPhone, iPad, and Apple Watch. Xcode provides developers a unified workflow for user interface design, coding, testing... Read more
DaisyDisk 4.8 - $9.99
DaisyDisk allows you to visualize your disk usage and free up disk space by quickly finding and deleting big unused files. The program scans your disk and displays its content as a sector diagram... Read more
VMware Fusion 11.5.0 - Run Windows apps...
VMware Fusion and Fusion Pro - virtualization software for running Windows, Linux, and other systems on a Mac without rebooting. The latest version includes full support for Windows 10, macOS Mojave... Read more

Latest Forum Discussions

See All

Marvel Strike Force is adding Agent Coul...
Marvel Strike Force, the popular squad-based RPG, is set to receive a bunch of new content over the next few weeks. [Read more] | Read more »
Lots of premium games are going free (so...
You may have seen over the past couple weeks a that a bunch of premium games have suddenly become free. This isn’t a mistake, nor is it some last hurrah before Apple Arcade hits, and it’s important to know that these games aren’t actually becoming... | Read more »
Yoozoo Games launches Saint Seiya Awaken...
If you’re into your anime, you’ve probably seen or heard of Saint Seiya. Based on a shonen manga by Masami Kurumada, the series was massively popular in the 1980s – especially in its native Japan. Since then, it’s grown into a franchise of all... | Read more »
Five Nights at Freddy's AR: Special...
Five Nights at Freddy's AR: Special Delivery is a terrifying new nightmare from developer Illumix. Last week, FNAF fans were sent into a frenzy by a short teaser for what we now know to be Special Delivery. Those in the comments were quick to... | Read more »
Rush Rally 3's new live events are...
Last week, Rush Rally 3 got updated with live events, and it’s one of the best things to happen to racing games on mobile. Prior to this update, the game already had multiplayer, but live events are more convenient in the sense that it’s somewhat... | Read more »
Why your free-to-play racer sucks
It’s been this way for a while now, but playing Hot Wheels Infinite Loop really highlights a big issue with free-to-play mobile racing games: They suck. It doesn’t matter if you’re trying going for realism, cart racing, or arcade nonsense, they’re... | Read more »
Steam Link Spotlight - The Banner Saga 3
Steam Link Spotlight is a new feature where we take a look at PC games that play exceptionally well using the Steam Link app. Our last entry talked about Terry Cavanaugh’s incredible Dicey Dungeons. Read about how it’s a great mobile experience... | Read more »
Combo Quest (Games)
Combo Quest 1.0 Device: iOS Universal Category: Games Price: $.99, Version: 1.0 (iTunes) Description: Combo Quest is an epic, time tap role-playing adventure. In this unique masterpiece, you are a knight on a heroic quest to retrieve... | Read more »
Hero Emblems (Games)
Hero Emblems 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: ** 25% OFF for a limited time to celebrate the release ** ** Note for iPhone 6 user: If it doesn't run fullscreen on your device... | Read more »
Puzzle Blitz (Games)
Puzzle Blitz 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: Puzzle Blitz is a frantic puzzle solving race against the clock! Solve as many puzzles as you can, before time runs out! You have... | Read more »

Price Scanner via MacPrices.net

Amazon is offering new 10.5″ iPad Airs for up...
Amazon has Apple’s new 10.5″ iPad Airs on sale today for up to $50 off MSRP, starting at $469. Shipping is free: – 10.5″ 64GB WiFi iPad Air: $469 $50 off MSRP – 10.5″ 256GB WiFi + Cellular iPad Air... Read more
Apple now offering a full line of 13″ 2.4GHz...
Apple has a full line of Certified Refurbished 2019 13″ 2.4GHz 4-Core Touch Bar MacBook Pros now available starting at $1529 and up to $300 off MSRP. Apple’s one-year warranty is included, shipping... Read more
2019 iMacs available for up to $350 off MSRP,...
Apple has Certified Refurbished 2019 21″ & 27″ iMacs now available starting at $929 and up to $350 off the cost of new models. Apple’s one-year warranty is standard, shipping is free, and each... Read more
Get a clearance 2018 13″ 2.3GHz Quad-Core Tou...
Apple has Certified Refurbished 2018 13″ 2.3GHz 4-Core Touch Bar MacBook Pros available starting at $1489. Apple’s one-year warranty is included, shipping is free, and each MacBook has a new outer... Read more
11″ WiFi iPad Pros on sale today for up to $2...
Amazon has new 2018 Apple 11″ WiFi iPad Pros in stock today and on sale for up to $200 off Apple’s MSRP. These are the same iPad Pros sold by Apple in its retail and online stores. Be sure to select... Read more
Select 12″ iPad Pros on sale for $200 off App...
Amazon has select 2018 Apple 12″ iPad Pros in stock today and on sale for $200 off Apple’s MSRP. These are the same iPad Pros sold by Apple in its retail and online stores. Be sure to select Amazon... Read more
Get one of Apple’s new 2019 iPhone 11 models...
Boost Mobile is offering the new 2019 Apple iPhone 11, iPhone 11 Pro, and 11 Pro Max for $100 off MSRP. Their discount reduces the cost of an iPhone 11 to $599 for the 64GB models, $899 for the 64GB... Read more
13″ 1.4GHz Silver MacBook Pros on sale for $1...
B&H Photo has new 2019 13″ 1.4GHz 4-Core Touch Bar Silver MacBook Pros on sale for $100 off Apple’s MSRP. Overnight shipping is free to many addresses in the US. These are the same MacBook Pros... Read more
4-core and 6-core 2018 Mac minis available at...
Apple has Certified Refurbished 2018 Mac minis available on their online store for $120-$170 off the cost of new models. Each mini comes with a new outer case plus a standard Apple one-year warranty... Read more
$250 prepaid Visa card with any Apple iPhone,...
Xfinity Mobile will include a free $250 prepaid Visa card with the purchase of any new iPhone, new line activation, and transfer of phone number to Xfinity Mobile. Offer is valid through October 27,... Read more

Jobs Board

Best Buy *Apple* Computing Master - Best Bu...
**734646BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Store Associates **Location Number:** 001220-Issaquah-Store **Job Description:** The Read more
Best Buy *Apple* Computing Master - Best Bu...
**734517BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000685-El Camino Real-Store **Job Description:** **What does Read more
*Apple* Mobility Pro - Best Buy (United Stat...
**734830BR** **Job Title:** Apple Mobility Pro **Job Category:** Store Associates **Location Number:** 000423-West Broad-Store **Job Description:** At Best Buy, our Read more
Systems Analyst ( *Apple* & Android) (Jo...
Systems Analyst ( Apple & Android) (Job ID: 572513) + 11751 Meadowville Ln, Chester, VA 23836, USA + Full-time Company Description Computer Consultants International, Read more
*Apple* Mobile App Developer - eiWorkflow So...
…eiWorkflow Solutions, LLC is currently looking for a consultant for the following role. Apple Mobile App Developer Tasks the role will be performing: ? Mobile App Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.