TweetFollow Us on Twitter

Mac in the Shell-Learning Python on the Mac: Modules

Volume Number: 25
Issue Number: 03
Column Tag: Mac in the Shell

Learning Python on the Mac: Modules

Organize code into reusable entities

by Edward Marczak

Introduction

Python has an incredible amount to offer, and its concept of modules is near the top of the list of power. Modules bring a lot of functionality to the language, and offer the author the ability to reuse code in as many different applications as they wish. Since the start of our journey down the path of Python, we've covered basic data types, flow control and functions. Modules will be one of the last very high-level concepts we cover for a bit, but it's an important one. Like most Mac in the Shell columns, this one, too, is best read with a Terminal.app window open and your favorite text editor up and running. So, clear your brain, and let's dive in.

What is a Module?

Modules, really, are just files. Like many things in Python, some very powerful features come with little fanfare, and they 'just work.' By placing functions, variables and code into a standalone file, you create an organized, self-contained, reusable package known as a namespace. In the past two months, you've seen the import keyword, and just had to trust me that it did something useful.

In actuality, import reaches out into the file system, locates, reads and runs an external module. The imported file becomes an object in the importing script, and all definitions in the file become attributes of that object. For illustrative purposes, let's imagine this: top-level ("main") file a.py imports b.py. b.py also imports c.py and also imports some modules from the standard Python library of modules. This is shown in Figure 1.


Figure 1 - Python high-level program architecture.

In essence, the introductory Mac in the Shell articles introducing Python were incredibly simplified; it's rare that a Python program won't have the need to import a module. While we'll be concentrating on importing modules written in Python, do note that Python can actually import a variety of different module types.

How does python know where to find modules? The python interpreter walks a pre-defined set of locations to know where to look. After looking in the same directory as the importing program, an environment variable named PYTHONPATH is consulted. PYTHONPATH is a simple list of directories in which to search for modules. It is constructed just like the PYTHONDOCS variable, covered in the second article introducing Python (December 2008, issue 24.12). After searching PYTHONPATH, python will search among the standard library (location defined at compile time). Finally, python includes any paths listed in .pth files (a slightly advanced topic that we won't be covering this month).

Most people don't ever have to create a PYTHONPATH variable, as the default locations serve most needs. However, if you're sharing development among a few people and need to have a common location for specific modules, you'll need to point PYTHONPATH to your custom location. Like the PATH variable, PYTHONPATH is simply a colon-separated list of paths, searched in order (from left to right). For example, in the bash shell:

export PYTHONPATH="/usr/share/python/"

With this path set, python will additionally search for modules in the /usr/share/python directory.

All of this comes together for the python interpreter as sys.path. From a running program, or, via the interactive interpreter, import the built-in sys module and print sys.path. Here's an abridged listing:

>>> import sys
>>> print sys.path
['', '/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python25.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5']

Note that the path starts with an empty entry, which is how the program's directory is searched.

Let's start with a simple example to demonstrate modules.

Using Modules

Sticking with the a.py, b.py and c.py concept, let's define them for real. Follow along with these short listings.

Listing 1 - a.py

#!/usr/bin/env python
import b
if b.IsInAdmin('root'):
  print "root is in the admin group"
else:
  print "root is not in the admin group"

As stated earlier, the import statement reads a module file and executes it. So, the line import b reads the file b.py and executes it like any other Python program. The if statement tests the return value from an object imported from b. We can also say that the object is in b's namespace. Listing 2 reveals what b.py is doing.

Listing 2 - b.py

#!/usr/bin/env python
import c
def IsInAdmin(user):
  admins = c.GetAdminGroup();
  if user in admins:
    return True
  else:
    return False

b.py starts off with its own import: c.py. b.py defines one function, IsInAdmin(). Finally, look at c.py to see how it works its magic.

Listing 3 - c.py

#!/usr/bin/env python
import plistlib
def GetAdminGroup():
  admin_plist = plistlib.readPlist('/var/db/dslocal/nodes/Default/groups/admin.plist')
  return admin_plist['users']

c.py imports plistlib-something you don't have to define, as it's part of the standard modules shipped with OS X. plistlib allows the author an easy manner of reading and writing OS X's plist files (ASCII only, currently). Here, we'll directly grab the admin.plist file and return the contents from the users key.

Save all three of these files in the same directory, mark a.py executable (chmod 770 a.py), and run it via sudo (it needs root privileges as the admin.plist file is protected). The output looks simply like this:

$ sudo ./a.py 
root is in the admin group

Overall, that was fairly little work. Of course, in a real-world scenario, you wouldn't need to break it down as far as we did here. However, GetAdminGroup() might make a really nice addition to a larger OS X-specific library.

Imports and Namespaces

We've spoken a little about the concept of namespaces without formally defining it. A namespace is a mapping from names to objects. The important thing to realize is that there is no relation between names in different namespaces. Two modules can define x, and there will be no conflict. Internally to python, namespaces are implemented as dictionaries (remember those? Covered briefly in our introduction - November 2008, issue 24.11). To see this in action, run the interactive python shell (by typing python at the shell prompt) in the same directory as the modules we've been working on. A module's dictionary is hidden away in the __dict__ attribute. It responds to all standard dictionary messages. For example, to list the contents of the object, import our c.py module:

>>> import c
and then list the keys:
>>> c.__dict__.keys()
['GetAdminGroup', 'plistlib', '__builtins__', '__file__', '__name__', '__doc__']

This is the same list that you get already with the dir() function, covered in earlier columns. However, it does lead to some fairly (advanced) trickery and techniques.

The point of this is that an import statement will import objects into their own namespace. This name of a given namespace is derived from the filename. This means that module filenames must conform to good variable names. You can certainly create a module named while.py, but since "while" is also a built-in keyword, you won't be able to import it. Our c.py module has been imported into a namespace of 'c'.

This also means that you should be careful to name your module something useful, as you'll be referring to it in code... or will you? Well, that depends. It depends somewhat on how you import a module, and somewhat on style. I'll say it now: my personal preference is that you always import as we've been showing-by using the import keyword-and referencing the namespace in code. Some people consider this bulky, though. Look at our plistlib usage:

plistlib.readPlist()

Instead of simply calling readPlist, we have to qualify it. This prevents it from conflicting with any other function (object, really) of that name. Importing plistlib created a namespace for that module. However, there is a way to import into the current namespace, and that relies on the from keyword variant.

To import an entire module, or just a select part of it into the current namespace, you tell python, "from [module] import [what]". The "what" can reference a specific object in the module, or use a wildcard. To import all of c.py directly into the current namespace, b.py can issue:

from c.py import *

b.py could then make the following change:

admins = GetAdminGroup();

See? No need to qualify the function call. Also, be aware that a subset of a module can be imported by specifying it explicitly:

from plistlib import readPlist

While all of this may seem to lessen the burden on the python interpreter, it doesn't in reality. The problem with this is much more from a maintenance-for-the-author perspective. This is especially true when multiple people are working on a project, each by contributing a module. As you may guess, conflicting object names simply get crushed, and the last name defined 'wins.' You, the author, lose, of course. Additionally, since python reads the entire module file in any case, or, has a pre-compiled byte-code version, there are no real performance/time savings by importing a single object. You may change a module that is imported in this manner and pollute the namespace of the importing application without realizing it. Using a standard import, there will never be conflicting objects.

To Be Executed

I mentioned above that when a module is imported, it is run like any other Python application. This is true; it's how objects get created, just like in the top-level file. We can prove this, too. Consider the following code:

Listing 4 - msgtest.py

#!/usr/bin/env python
print "Running top-level python file"
import msgmod
print "Finishing up top-level file"

Listing 5 - msgmod.py

#!/usr/bin/env python print "Running msgmod.py module" Running msgtest.py yields the following output: Running top-level python file Running msgmod.py module Finishing up top-level file

Not surprising if you've been following along! The same goes for any assignment statements in a module. Of course, functions are read and turned into an object, but the contents of which are not executed until explicitly called. Sometimes, however, a file has value as both a top-level file and as something to be imported. As a top-level file, we want it to execute, but as an imported file, we just want it imported, ready at our beck and call. There's a Python-ism that addresses this. A built-in variable, __name__, is assigned in each module at runtime. When the top-level application is run, __name__ is "__main__". Instead of just defining statements at the top indent-level, every statement in the top-level file is put into a function (or class) definition. The only top level statement tests __name__, and call your main function as appropriate. For example, our a.py program would be altered like this:

#!/usr/bin/env python
import b
def main():
  if b.IsInAdmin('root'):
    print "root is in the admin group"
  else:
    print "root is not in the admin group"
if __name__ == '__main__':
  main()

This way, if we had other useful functions in a.py, it could be run directly, or imported as a module.

I will be using this idiom going forward.

Fin

For this month, that's enough. Modules are such a core concept in Python, it's worth it to make sure it's understood before moving on. Next month, we'll talk a little more about style, and then cover classes.

As an aside, I met many people at Macworld who were really exited by this column turning its attention to Python. I'd love to hear ideas from all readers, so, letters@mactech.com and emarczak@mactech.com are ready and waiting for mail.

Media of the month: "Synthetic Worlds: The Business and Culture of Online Games," by Edward Castronova. Interesting work that covers, mainly, the economy of virtual worlds (ok, MMOs). The book is a touch dated at this point, but overall, it should make the reader think about how much more our technology is touching and influencing our 'non-technological' world.

See you next month!


Ed Marczak is the Executive Editor of MacTech Magazine. He lives in New York with his wife, two daughters and various pets. He has been involved with technology since Atari sucked him in, and has followed Apple since the Apple I days. He spends his days on the Mac team at Google, and free time with his family and/or playing music. Ed is the author of the Apple Training Series book, "Advanced System Administration v10.5," and has written for MacTech since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

CleanMyMac X 4.4.4 - Delete files that w...
CleanMyMac makes space for the things you love. Sporting a range of ingenious new features, CleanMyMac lets you safely and intelligently scan and clean your entire system, delete large, unused files... Read more
TeamViewer 14.4.2669 - Establish remote...
TeamViewer gives you remote control of any computer or Mac over the Internet within seconds or can be used for online meetings. Find out why more than 200 million users trust TeamViewer! Free for non... Read more
Paperless 3.0.6 - $69.95
Paperless is a digital documents manager. Remember when everyone talked about how we would soon be a paperless society? Now it seems like we use paper more than ever. Let's face it - we need and we... Read more
BetterTouchTool 3.141 - Customize multi-...
BetterTouchTool adds many new, fully customizable gestures to the Magic Mouse, Multi-Touch MacBook trackpad, and Magic Trackpad. These gestures are customizable: Magic Mouse: Pinch in / out (zoom)... Read more
TextMate 2.0.rc.29 - Code/markup editor...
TextMate is a versatile plain text editor with a unique and innovative feature set which caused it to win an Apple Design Award for Best Mac OS X Developer Tool in August 2006 A rapidly growing... Read more
Little Snitch 4.4.1 - Alerts you about o...
Little Snitch gives you control over your private outgoing data. Track background activity As soon as your computer connects to the Internet, applications often have permission to send any... Read more
Little Snitch 4.4 - Alerts you about out...
Little Snitch gives you control over your private outgoing data. Track background activity As soon as your computer connects to the Internet, applications often have permission to send any... Read more
MPlayer OSX Extended 16 - Multimedia pla...
MPlayer OSX Extended is the future of MPlayer OSX. Leveraging the power of the MPlayer and FFmpeg open source projects, MPlayer OSX Extended aims to deliver a powerful, functional and no frills video... Read more
Google Chrome 75.0.3770.142 - Modern and...
Google Chrome is a Web browser by Google, created to be a modern platform for Web pages and applications. It utilizes very fast loading of Web pages and has a V8 engine, which is a custom built... Read more
Notability 4.0.4 - Note-taking and annot...
Notability is a powerful note-taker to annotate documents, sketch ideas, record lectures, take notes and more. It combines, typing, handwriting, audio recording, and photos so you can create notes... Read more

Latest Forum Discussions

See All

TEPPEN guide - Tips and tricks for new p...
TEPPEN is a wild game that nobody asked for, but I’m sure glad it exists. Who would’ve thought that a CCG featuring Capcom characters could be so cool and weird? In case you’re not completely sure what TEPPEN is, make sure to check out our review... | Read more »
Dr. Mario World guide - Other games that...
We now live in a post-Dr. Mario World world, and I gotta say, things don’t feel too different. Nintendo continues to squirt out bad games on phones, causing all but the most stalwart fans of mobile games to question why they even bother... | Read more »
Strategy RPG Brown Dust introduces its b...
Epic turn-based RPG Brown Dust is set to turn 500 days old next week, and to celebrate, Neowiz has just unveiled its biggest and most exciting update yet, offering a host of new rewards, increased gacha rates, and a brand new feature that will... | Read more »
Dr. Mario World is yet another disappoin...
As soon as I booted up Dr. Mario World, I knew I wasn’t going to have fun with it. Nintendo’s record on phones thus far has been pretty spotty, with things trending downward as of late. [Read more] | Read more »
Retro Space Shooter P.3 is now available...
Shoot-em-ups tend to be a dime a dozen on the App Store, but every so often you come across one gem that aims to shake up the genre in a unique way. Developer Devjgame’s P.3 is the latest game seeking to do so this, working as a love letter to the... | Read more »
Void Tyrant guide - Guildins guide
I’ve still been putting a lot of time into Void Tyrant since it officially released last week, and it’s surprising how much stuff there is to uncover in such a simple-looking game. Just toray, I finished spending my Guildins on all available... | Read more »
Tactical RPG Brown Dust celebrates the s...
Neowiz is set to celebrate the summer by launching a 2-month long festival in its smash-hit RPG Brown Dust. The event kicks off today, and it’s divided into 4 parts, each of which will last two weeks. Brown Dust is all about collecting, upgrading,... | Read more »
Flappy Royale is an incredibly clever ta...
I spent the better part of my weekend playing Flappy Royale. I didn’t necessarily want to. I just felt like I had to. It’s a hypnotic experience that’s way too easy to just keep playing. | Read more »
Void Tyrant guide - General tips and tri...
Void Tyrant is a card-based dungeon-crawler that doesn’t fit in the mold of other games in the genre. Between the Blackjack-style combat and strange gear system alone, you’re left to your own devices to figure out how best to use everything to your... | Read more »
Webzen’s latest RPG First Hero is offici...
You might be busy sending your hulking Dark Knight into the midst of battle in Webzen’s other recent release: the long-anticipated MU Origin 2. But for something a little different, the South Korean publisher has launched First Hero. Released today... | Read more »

Price Scanner via MacPrices.net

Amazon drops prices, now offers clearance 13″...
Amazon has new dropped prices on clearance 13″ 2.3GHz Dual-Core non-Touch Bar MacBook Pros by $200 off Apple’s original MSRP, with prices now available starting at $1099. Shipping is free. Be sure to... Read more
2018 15″ MacBook Pros now on sale for $500 of...
Amazon has dropped prices on select clearance 2018 15″ 6-Core MacBook Pros to $500 off Apple’s original MSRP. Prices now start at $1899 shipped: – 2018 15″ 2.2GHz Touch Bar MacBook Pro Silver: $1899.... Read more
Price drop! Clearance 12″ 1.2GHz Silver MacBo...
Amazon has dropped their price on the recently-discontinued 12″ 1.2GHz Silver MacBook to $849.99 shipped. That’s $450 off Apple’s original MSRP for this model, and it’s the cheapest price available... Read more
Apple’s 21″ 3.0GHz 4K iMac drops to only $936...
Abt Electronics has dropped their price on clearance, previous-generation 21″ 3.0GHz 4K iMacs to only $936 shipped. That’s $363 off Apple’s original MSRP, and it’s the cheapest price we’ve seen so... Read more
Amazon’s Prime Day savings on Apple 11″ iPad...
Amazon has new 2018 Apple 11″ iPad Pros in stock today and on sale for up to $250 off Apple’s MSRP as part of their Prime Day sale (but Prime membership is NOT required for these savings). These are... Read more
Prime Day Apple iPhone deal: $100 off all iPh...
Boost Mobile is offering Apple’s new 2018 iPhone Xr, iPhone Xs, and Xs Max for $100 off MSRP. Their discount reduces the cost of an Xs to $899 for the 64GB models and $999 for the 64GB Xs Max. Price... Read more
Clearance 13″ 2.3GHz Dual-Core MacBook Pros a...
Focus Camera has clearance 2017 13″ 2.3GHz/128GB non-Touch Bar Dual-Core MacBook Pros on sale for $169 off Apple’s original MSRP. Shipping is free. Focus charges sales tax for NY & NJ residents... Read more
Amazon Prime Day deal: 9.7″ Apple iPads for $...
Amazon is offering new 9.7″ WiFi iPads with Apple Pencil support for $80-$100 off MSRP as part of their Prime Day sale, starting at only $249. These are the same iPads found in Apple’s retail and... Read more
Amazon Prime Day deal: 10% (up to $20) off Ap...
Amazon is offering discounts on new 2019 Apple AirPods ranging up to $20 (10%) off MSRP as part of their Prime Day sales. Shipping is free: – AirPods with Charging Case: $144.99 $15 off MSRP –... Read more
Amazon Prime Day deal: $50-$80 off Apple Watc...
Amazon has Apple Watch Series 4 and Series 3 models on sale for $50-$80 off Apple’s MSRP as part of their Prime Day deals with prices starting at only $199. Choose Amazon as the seller rather than a... Read more

Jobs Board

*Apple* Graders/Inspectors (Seasonal/Hourly/...
…requirements. #COVAentryleveljobs ## Minimum Qualifications Some knowledge of agricultural and/or the apple industry is helpful as well as the ability to comprehend, Read more
Best Buy *Apple* Computing Master - Best Bu...
**710003BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Store Associates **Location Number:** 000171-Winchester Road-Store **Job Description:** Read more
Best Buy *Apple* Computing Master - Best Bu...
**709786BR** **Job Title:** Best Buy Apple Computing Master **Job Category:** Sales **Location Number:** 000430-Orange Park-Store **Job Description:** **What does a Read more
Geek Squad *Apple* Master Consultation Agen...
**709918BR** **Job Title:** Geek Squad Apple Master Consultation Agent **Job Category:** Services/Installation/Repair **Location Number:** 000106-Palmdale-Store Read more
*Apple* Systems Architect/Engineer, Vice Pre...
…its vision to be the world's most trusted financial group. **Summary:** Apple Systems Architect/Engineer with strong knowledge of products and services related to Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.