TweetFollow Us on Twitter

Mac in the Shell: Debugging Python with vim

Volume Number: 26
Issue Number: 01
Column Tag: Mac in the Shell

Mac in the Shell: Debugging Python with vim

IDE-like debugging without leaving vim

by Edward Marczak

Welcome

Mac in the Shell has been focusing on Python for the System Administrator over the last few months. Last month re-introduced the vi(m) text editor. This month, we're going to tie them together a bit. I've covered pdb - the python debugger - already. It's really handy to have a debugger at your disposal. However, pdb's output and display can get a little unwieldy. Never fret: Vim to the rescue! This article will show you how to get the latest MacVim and start using it to debug Python code.

Use the Source, Luke

First thing is first: this will all fail miserably with the version of vim that's built into the base OS. Best solution: use MacVim. MacVim comes built with Python, GUI support and more. There's a few ways to get MacVim, and I'll detail both.

First way is the easy way: download a binary release, typically of the latest snapshot. Do that directly from the MacVim project page, currently on Google Code:

http://code.google.com/p/macvim/

But wait, there's more! If you like to customize your experience, like I do, you'll want the source code. This is a multi-part step. First, you'll need to make sure that the (free) Apple developer tools are installed on your machine. (and honestly, what good is a machine without those?). You'll also need git installed to check out the MacVim source. If you're a fink or MacPorts person, you can use either of those systems to install git. If you're a pre-built person, or just need to be in this case, go pick up a pre-built git from:

http://code.google.com/p/git-osx-installer/

Once git is up and running, check out the MacVim source by changing to the destination directory and issuing a clone command:

git clone git://repo.or.cz/MacVim.git vim7

Once that completes, you'll need to configure and build MacVim. This is where you can customize the binary a bit. I want the whole experience, so, I change to the vim7 directory and run configure with my options:

./configure 
—enable-perlinterp —enable-pythoninterp 
—enable-rubyinterp 
—with-features=huge 
—enable-gui=macvim 
—with-tlib=ncurses 
—enable-multibyte —with-python-config-dir=
/System/Library/Frameworks/Python.framework/Versions/2.6//lib/python2.6/config/

No matter how you configure Vim, I'd highly recommend the -enable-gui option and, for this article, you must enable the python interpreter. Of course, you're reading an article about debugging python using Vim, so why wouldn't you enable this?

Once configure finishes its business (successfully), type make and...wait. Barring any errors, you'll find the MacVim application in the src/MacVim/build/Release directory. If you're following this method, when you want to 'upgrade' (read: compile a new version), change into the vim7 directory and ask git to pull down the latest changes:

$ git pull

followed by make. Go pick up your fresh binary in the Release directory as before.

You may also want to copy the mvim script from the src/MacVim subdirectory and place it somewhere in your $PATH. Running mvim from the command-line will try to do the right thing: if you're logged in via ssh, you'll use console-based vim. If you have a GUI, you'll run GUI-based MacVim. You can also create alias or functions that call mvim as a different name. This will cause Vim to run in different modes. I have the following in my ~/.bash_profile initialization script:

export VIM_APP_DIR=${HOME}/Applications/MacVim/
vi() { ${HOME}/Applications/MacVim/mvim ${@}; }
vim() { ${HOME}/Applications/MacVim/mvim ${@}; }
gvim () { ${HOME}/Applications/MacVim/mvim ${@}; }
vimdiff() { ${HOME}/Applications/MacVim/mvim -d ${@}; }
export -f vi
export -f vim
export -f gvim
export -f vimdiff

I use bash functions rather than aliases as there are several problems with using aliases in this case. Primarily, a bash alias will fork off a new shell and there's too much juggling around of variables. Bash aliases have been deprecated in favor of bash functions in any case.

In my example .bash_profile snippet, I explicitly define where MacVim lives as I keep it in my home directory. Then I define the four ways I may call vi(m). One of the most important to me is the definition for vimdiff. When integrating with version control programs, I want to make sure it doesn't fork off, and it should come back after an individual window is closed. Once these are defined, I separately do the same for gvim and git integration if I'm logged in to a GUI session with:

export GIT_EDITOR='gvim -f -c "au VimLeave * !open -a Terminal"'

The final group of exports causes functions to be exported to subshells.

One last note: no matter how you choose to call vim, ensure that your method gets found before the system vi(m). Keep it early in your path, or use my method above, as functions are always used before searching $PATH.

The Setup

The entire point of the earlier Vim exercise is to ensure that you are using a Python-enabled version of vim. The vim binary that ships with OS X is not compiled with the Python, or any scripting option. Once you've downloaded or compiled MacVim and have it in place, launch it and test for Python support. Enter ex mode by typing a colon (":") followed by py and a python command. Something like:

:py print 6*7

You'll see the result on the command line. Not exceptionally exiting, perhaps...not at the outset, anyway. However, having Python support built in means that Python can be used to script just about anything. That is exciting! This is also, not coincidentally, how we're going to be able to debug python code entirely within vim. If you receive an error from 'py' - specifically "E319: Sorry, the command is not available in this version" - ensure that you're running the updated version of vim that you downloaded (or built!) and not the system vi(m).

Configuring Vim for Debugging

There are plenty of hoops one can jump through to debug directly in vim. My favorite, though, is a freely downloadable plugin. Go grab vimpdb from its project page:

http://code.google.com/p/vimpdb/

From there, click on the download link. Or, you can just grab the source directly using subversion:

svn checkout http://vimpdb.googlecode.com/svn/trunk/ vimpdb

Pleasantly, subversion is built into OS X as of 10.5, so, you can just run that command with no fuss.

However you've obtained vimpdb, you need to drop VimPdb.py and VimPdb.vim into your vim plugin directory (~/.vim/plugin). Now you're ready...maybe.

By default, vimpdb uses function keys for instructing it on what action to take. If you're using a 'real' keyboard, you can likely just use the default key bindings. By 'real' keyboard, I'm essentially talking about any stand-alone keyboard. It's the keyboards that are built into the MacBooks and MacBook Pro machines that are the problem. If you're like me, you'll occasionally want to debug on the go. The trouble starts with the way Apple overloads the function keys. Despite a setting in System Preferences that allows you to "use all" function keys "as standard function keys," this doesn't do what you expect initially. You'll still need to undo those key settings in the "Keyboard Shortcuts" section of the Keyboard pref pane. It's about picking your battles, I suppose. However, those shortcuts are really nice to have. (I know, this is a fairly lame argument, but, hey - I'm the end-user here and I demand satisfaction). I've experimented a bit with creating a wrapper script for vim that swaps out the plist file that controls this (~/Library/Preferences/com.apple.symbolichotkeys.plist) with one that has no F-key definitions, but that causes other issues for me. Ultimately, I reached a compromise and only undo a few default keyboard shortcuts: gone are ^F2, ^F4 and ^F8. So, let's go set up our vim key mappings.


Figure 1 - Assigning - or unassigning - keyboard shortcuts.

Again, if you want to leave the default keys for vimpdb and disable the OS X integrated keys in the Keyboard pref pane, feel free. For this article, I'm going to make some very simple changes, but you can set up the keys as you like. Open up the vimpdb.vim file that you just placed in your ~/.vim/plugin directory (using Vim, of course!). Move to line 540 (in Vim, type "540 G"). I've altered these lines to map F3 and F4 to step into and step over, respectively. They now read:

   " Was F7 and F8
   map <buffer> <silent> <F3> :call PdbStepInto()<CR>
   map <buffer> <silent> <F4> :call PdbStepOver()<CR>

Also, I've needed to change out the places that F3 and F4 were already used. Scroll down to line 560 or so, and make these changes:

   " Was F4
   map <buffer> <silent> <F7> :call PdbEvalCurrentWord()<CR>
   map <buffer> <silent> <C-F7> :call PdbEvalCurrentWORD()<CR>
   " Was F3
   map <buffer> <silent> <F8> :call PdbEvalExpression()<CR>

Save the file, and you're ready.

Lock and Load

The default vimpdb key binding along with the changes we've made leave us with the following keys for controlling a debug session:

Start and stop debugging

- F5 - Start/continue debug session of current file.

- Ctrl-F5 - Start debugging and do not pause at first line

- Ctrl-Shift-F5 - Start debugging with a given list of parameters.

- Shift-F5 - Stop the current debug session.

- Ctrl-Alt-Shift-F5 - Restart the current debug session.

Breakpoints

- F2 - Toggle breakpoint.

- Ctrl-F2 - Toggle conditional breakpoint

- Shift-F2 - Toggle temporary breakpoint

- Ctrl-Shift-F2 - Clear all breakpoints in current file

- Ctrl-Alt-Shift-F2 - Clear all breakpoints in all files

- F11 - Print condition of conditional breakpoint under the cursor

Step/continue

- F3 - Step into

- F4 - Step over

- Ctrl-F4 - Continue running until reaching a return from function

Cursor movement

- F6 - Move cursor to currently debugged line.

- Ctrl-F6 - Change current debugged line to where the cursor is currently placed.

Stack

- F9 - Move up in stack frame.

- F10 - Move down in stack frame.

- F12 - Print stack trace

Evaluate/Execute

- F7 - Evaluate a given expression (in the current debug context)

- Ctrl-F7 - Exec a given statement (in the current debug context)

- F8 - Evaluate the current word under the cursor (in the current debug context)

- Ctrl-F8 - Evaluate the current WORD under the cursor (in the current debug context)

Again, depending on the keyboard you use and the setting you have in Keyboard Shortcuts, some of these key mappings may (will?) have conflicts.

So, what does all of this get you? Figure 2 should give you an idea.


Figure 2 - VimPDB in action.

Unlike the pure command-line environment of pdb, vimpdb lets you keep your code in view and step along with it as it runs. Visually. In figure 2, line 3 - highlighted in green - is the current line. Line 9 - highlighted in red - has a breakpoint set on it. You can use any Python code that you like. I'm going to use the code that I offered up in my article on PyObjC from August 2009 that reads the Apple Address Book.

Debugging

Once you're set up, the debugging is fairly easy. Load the python code that you'd like to debug into the current buffer and press F5. This begins the debugger, and vim should print a status message that says "Starting debugger." You should also now see your first executable line highlighted in green. Now, the "first executable" line is the first thing that the parser needs to look at, not necessarily what you'd expect as "executable."

To step through the program, you can press F4 to step 1 instruction. This will step over any intervening code between the current line and what is visible as next. The current line will execute, and the green line will advance. If the current line is a function call that you want to inspect, press F3 to step into the function. You'll need to do this at least once if you use the Python convention of creating a main() function and calling that conditionally:

if __name__ == '__main__':
  main()

Once you're inside a function, often, you'll get to the part you're interested in, and then not want to have to step through the remainder of the function. You can accomplish that by pressing control-F4.

The great thing about using a debugger is not only can you step through and watch the flow of your code, but you can inspect variables as they change, too. Vimpdb doesn't let us down here. Position the cursor over the variable name you want to inspect and press F7. The value of the variable is displayed in the output area at the bottom of the vim screen followed by a note to press any key to return to debugging.

For example, if I were to press F7 over the "abgroups" variable showing in Figure 2, I receive this output:

 (
    "ABGroup (0x10835cb40) {\n\tCreation     : 2009-07-14 15:13:36 
    -0400\n\tGroupName    : WS\n\tModification : 2009-10-27 19:24:14 
    -0400\n\tUnique ID    : 8BBAEDE9-0D31-4B48-A967-826523BF9DA7:ABGroup\n}",
    "ABGroup (0x1006f3ab0) {\n\tCreation     : 2009-01-14 15:13:36 
    -0400\n\tGroupName    : MacTech Editorial\n\tMembers    
    : 28 members\n\tModification : 2009-11-23 08:04:24 
    -0500\n\tUnique ID    : 5712EB9A-9743-4744-BA05-354726896614:ABGroup\n}"
)

Note that variables you inspect must be defined. If the current line contains a variable that hasn't been defined yet, you can't inspect that variable until the line has been executed. If you try, you'll receive a NameError:

NameError: Name 'blah' is not defined.

Just ensure that the variable you're trying to inspect is defined. If it's first defined on the current line, execute the current line first.

Let's look at a typical short debug session. First, you'd open vim and load your code into the buffer:

vim ~/dev/py/address_book.py

(or, open vim and type :e ~/dev/py/address_book.py).

Start the debug session by pressing F5 to initialize the debugger and start the session. You should note the "starting debugger" message. No other vimpdb keys will do anything until this step is taken.

You can navigate down to the call to main() and set a breakpoint - using F2 while the cursor is on the line with said call. Once set, press F5 to run the program as usual until it hits your break point. If you stopped on main(), press F4 to step into the main function. Execute a line that assigns a value to a variable, cursor up to the variable name and press F7 to inspect the value that was assigned; it'll appear at the bottom of the vim window. Press F3 to execute the current line.

Rinse, repeat.

One extra nice feature of VimPDB: the ability to save and load breakpoints. In a larger program, you may set several breakpoints in different locations that you want to recall after quitting vim and then trying to debug again. While in debug mode with all of the breakpoints set, type \s (that's backslash, and then s). You'll be prompted for a filename. Type one and press return. When you later come back to vim and load your project, start the debugger and type \l (backslash, then lowercase L, for "load").

In Closing

VimPDB provides the best of all worlds for debugging Python code. It doesn't make you change code to drop in a bunch of print or logging statements. It gives you a larger view of your code as it's running. Finally, it can easily be run remotely over an ssh session (win!).

Also, VimPDB is pretty easy in general, and a great addition to your toolkit. This article focused mainly on the preparation: install and introduction. Once you start using it, it becomes second nature. Unfortunately, unless you wildly remap the default keys, you'll need to choose between OS X functions on the F-keys and bindings in Vim (and yes, OS X "wins"). This is of particular annoyance on a MacBook/Pro, but not insurmountable.

Media of the month: No better time to recommend "Learning the vi and Vim Editors" by Arnold Robbins, Elbert Hannah, and Linda Lamb. This is a pretty comprehensive look at learning and customizing vi(m). Available in dead-tree and electronic versions.

Hope to see everyone in San Francisco for Macworld!


Ed Marczak is the Executive Editor of MacTech Magazine. He has written the Mac in the Shell column since 2004.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Latest Forum Discussions

See All

Six fantastic ways to spend National Vid...
As if anyone needed an excuse to play games today, I am about to give you one: it is National Video Games Day. A day for us to play games, like we no doubt do every day. Let’s not look a gift horse in the mouth. Instead, feast your eyes on this... | Read more »
Old School RuneScape players turn out in...
The sheer leap in technological advancements in our lifetime has been mind-blowing. We went from Commodore 64s to VR glasses in what feels like a heartbeat, but more importantly, the internet. It can be a dark mess, but it also brought hundreds of... | Read more »
Today's Best Mobile Game Discounts...
Every day, we pick out a curated list of the best mobile discounts on the App Store and post them here. This list won't be comprehensive, but it every game on it is recommended. Feel free to check out the coverage we did on them in the links below... | Read more »
Nintendo and The Pokémon Company's...
Unless you have been living under a rock, you know that Nintendo has been locked in an epic battle with Pocketpair, creator of the obvious Pokémon rip-off Palworld. Nintendo often resorts to legal retaliation at the drop of a hat, but it seems this... | Read more »
Apple exclusive mobile games don’t make...
If you are a gamer on phones, no doubt you have been as distressed as I am on one huge sticking point: exclusivity. For years, Xbox and PlayStation have done battle, and before this was the Sega Genesis and the Nintendo NES. On console, it makes... | Read more »
Regionally exclusive events make no sens...
Last week, over on our sister site AppSpy, I babbled excitedly about the Pokémon GO Safari Days event. You can get nine Eevees with an explorer hat per day. Or, can you? Specifically, you, reader. Do you have the time or funds to possibly fly for... | Read more »
As Jon Bellamy defends his choice to can...
Back in March, Jagex announced the appointment of a new CEO, Jon Bellamy. Mr Bellamy then decided to almost immediately paint a huge target on his back by cancelling the Runescapes Pride event. This led to widespread condemnation about his perceived... | Read more »
Marvel Contest of Champions adds two mor...
When I saw the latest two Marvel Contest of Champions characters, I scoffed. Mr Knight and Silver Samurai, thought I, they are running out of good choices. Then I realised no, I was being far too cynical. This is one of the things that games do best... | Read more »
Grass is green, and water is wet: Pokémo...
It must be a day that ends in Y, because Pokémon Trading Card Game Pocket has kicked off its Zoroark Drop Event. Here you can get a promo version of another card, and look forward to the next Wonder Pick Event and the next Mass Outbreak that will be... | Read more »
Enter the Gungeon review
It took me a minute to get around to reviewing this game for a couple of very good reasons. The first is that Enter the Gungeon's style of roguelike bullet-hell action is teetering on the edge of being straight-up malicious, which made getting... | Read more »

Price Scanner via MacPrices.net

Take $150 off every Apple 11-inch M3 iPad Air
Amazon is offering a $150 discount on 11-inch M3 WiFi iPad Airs right now. Shipping is free: – 11″ 128GB M3 WiFi iPad Air: $449, $150 off – 11″ 256GB M3 WiFi iPad Air: $549, $150 off – 11″ 512GB M3... Read more
Apple iPad minis back on sale for $100 off MS...
Amazon is offering $100 discounts (up to 20% off) on Apple’s newest 2024 WiFi iPad minis, each with free shipping. These are the lowest prices available for new minis among the Apple retailers we... Read more
Apple’s 16-inch M4 Max MacBook Pros are on sa...
Amazon has 16-inch M4 Max MacBook Pros (Silver and Black colors) on sale for up to $410 off Apple’s MSRP right now. Shipping is free. Be sure to select Amazon as the seller, rather than a third-party... Read more
Red Pocket Mobile is offering a $150 rebate o...
Red Pocket Mobile has new Apple iPhone 17’s on sale for $150 off MSRP when you switch and open up a new line of service. Red Pocket Mobile is a nationwide MVNO using all the major wireless carrier... Read more
Switch to Verizon, and get any iPhone 16 for...
With yesterday’s introduction of the new iPhone 17 models, Verizon responded by running “on us” promos across much of the iPhone 16 lineup: iPhone 16 and 16 Plus show as $0/mo for 36 months with bill... Read more
Here is a summary of the new features in Appl...
Apple’s September 2025 event introduced major updates across its most popular product lines, focusing on health, performance, and design breakthroughs. The AirPods Pro 3 now feature best-in-class... Read more
Apple’s Smartphone Lineup Could Use A Touch o...
COMMENTARY – Whatever happened to the old adage, “less is more”? Apple’s smartphone lineup. — which is due for its annual refresh either this month or next (possibly at an Apple Event on September 9... Read more
Take $50 off every 11th-generation A16 WiFi i...
Amazon has Apple’s 11th-generation A16 WiFi iPads in stock on sale for $50 off MSRP right now. Shipping is free: – 11″ 11th-generation 128GB WiFi iPads: $299 $50 off MSRP – 11″ 11th-generation 256GB... Read more
Sunday Sale: 14-inch M4 MacBook Pros for up t...
Don’t pay full price! Amazon has Apple’s 14-inch M4 MacBook Pros (Silver and Black colors) on sale for up to $220 off MSRP right now. Shipping is free. Be sure to select Amazon as the seller, rather... Read more
Mac mini with M4 Pro CPU back on sale for $12...
B&H Photo has Apple’s Mac mini with the M4 Pro CPU back on sale for $1259, $140 off MSRP. B&H offers free 1-2 day shipping to most US addresses: – Mac mini M4 Pro CPU (24GB/512GB): $1259, $... Read more

Jobs Board

All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.