Volume Number: 21 (2005)
Issue Number: 3
Column Tag: Programming

QuickTime Toolkit

by Tim Monroe

The Incredibles

Recommending QuickTime Programming Books

One of the first questions that new QuickTime programmers ask -- or should ask -- is this: "Where do I get started?" Usually this means: "Where can I find some good documentation on programming QuickTime?"

Introduction

One of the first questions that new QuickTime programmers ask -- or should ask -- is this: "Where do I get started?" Usually this means: "Where can I find some good documentation on programming QuickTime?" Almost without exception, the most up-to-date and most authoritative information is to be found on the QuickTime developer web pages, starting at http://developer.apple.com/quicktime. That page contains links to the latest on-line documentation, sample code, technical notes, and related materials such as QuickTime tools and information about the Apple-sponsored user- and API-level mailing lists.

The documentation links can be particularly daunting for newbies, however. There are links to over fifty separate documents (available in both HTML and PDF formats), which range from introductory overviews to highly-specific discussions of each major area of QuickTime programming, including sprites, movie importers and exporters, QuickTime VR, movie capture and compression, and so forth. There are also documents that assist in developing QuickTime applications in Cocoa or on Windows. So where should we begin if we want to write an image capture application in Cocoa? With the capture documentation? With the introductory documents? With the Cocoa documentation? Or suppose we wanted to write the application in Java instead; is there a good tutorial on capturing sound and video data using QuickTime for Java?

I dunno; or rather, I didn'tno until I got ahold of a copy of a new book in the O'Reilly Developer's Notebook series, QuickTime for Java(TM): A Developer's Notebook by Chris Adamson. In that book, Chris shows how to develop a Java application that captures video and sound data from, for example, an iSight camera. What's amazing about this is that Apple has explicitly claimed not to support sequence grabbing in QuickTime for Java versions 6.1 and later (while tantalizing us with the suggestion that "it may be provided in future releases"). Chris figured out the magic necessary to get things to work and delivers a working Java-based capture tool. Very nice. What's even nicer is that the entire book is a thoroughly readable tutorial on using Java to develop QuickTime applications.

The excellence of Chris' book set me to thinking about the general state of QuickTime programming books. If our imagined newbie wanted some recommendations on paper-and-glue books to hold in her hands and peruse from the safety of her armchair, what could we offer? In this article, I want to recommend five books on QuickTime and QuickTime applications development. Weighing in at more than 2600 pages total, I doubt that even the most persistent newbie would slog though them all before launching into a programming project. But, in my opinion, these are some of the best sources of information available to QuickTime programmers today.

Disclaimer: I wrote two of these books, and I assisted in reviewing several others. I know all the authors personally, and I have the greatest respect for each of them. However, I have no financial stake in any of these publications. You can purchase a zillion copies of each of these books without fear that you are contributing to my retirement fund.

QuickTime for the Web

QuickTime for the Web by Steven Gulie is easily the most misnamed book in this bunch (or in just about any other bunch). From the sound of the title, you might think that it was aimed at Internet-savvy webmeisters looking to optimize delivery of their web-based QuickTime movies. Indeed, the subtitle of the first edition was "A Hands-On Guide for Webmasters, Site Designers, and HTML Authors". Thankfully, the publisher dropped that subtitle in following editions. This book, though it does of course discuss issues involved with deploying QuickTime content on the web, is much more general and useful. It is, quite simply, the best available introduction to QuickTime as a multimedia technology. It explains in great detail the many, varied parts of the QuickTime architecture, from video and sound to QuickTime VR to sprites and wired actions. To SMIL. To Flash. To still images. To text movies. And beyond.

Now, this is not primarily a programming book. It does at one point show how to use JavaScript to control QuickTime movies playing back in a browser window, but it's not by any means an API-level investigation of QuickTime. So why am I recommending it to our newbie QuickTime programmer? Simply because a thorough understanding of the QuickTime architecture is essential for making intelligent decisions about the design and implementation of QuickTime applications. Time and again I have seen developers (and not just rank newbies) overlook easy solutions that would have been more obvious had they had a broader view of the scope of QuickTime and QuickTime movies. Steve's book provides that broad view and deserves to be the first stop on any prospective QuickTime programmer's reading list.

QuickTime Toolkit

The two QuickTime Toolkit books, which I wrote, grew out of articles that appeared in MacTech magazine from 2000 to 2003. In fact, this series of articles was originally planned precisely to allow the easy transition from magazine to book form. Those books present, in my opinion, the most complete and most accessible general introduction to QuickTime programming that is available today. They also contain a fair amount of previously undocumented material that has come in handy over the years.

The first book, QuickTime Toolkit, Volume One: Basic Movie Playback and Media Types, begins by showing how to use QuickTime functions to open and display a movie in a window on the screen. It also shows how to perform basic editing operations on a movie and how to save an edited movie into a file. This book then shows how to work with a variety of media types, including video, still images, text, timecode, and sprites. It introduces concepts that are fundamental to QuickTime programming: movies, tracks, media, time scales, track references, atoms, atom containers, data references, media samples, sample descriptions, and a host of others. This first book ends with an in-depth look at one of the cornerstones of QuickTime interactivity: wired actions and wired sprites.

The second book in this series, QuickTime Toolkit, Volume Two: Advanced Movie Playback and Media Types, continues this journey by looking at a handful of the more advanced media types supported by QuickTime: video effects, skins, Flash, and QuickTime VR. It shows how to capture movies from sound and video input sources, broadcast movies to the Internet or a LAN, play movies full screen, and load movies asynchronously. Together, these two books present a detailed narrative that covers a substantial amount of what's involved in QuickTime application programming on both Macintosh and Windows computers.

Both of these books present example code using the C programming language and reflect what we now call the "Carbon" programming model. If you are programming in REALbasic, or Revolution, or Java, or Cocoa, or any other non-C language, then you'll need to look elsewhere for introductory material geared to developing QuickTime applications in your particular development environment. However, chances are that the basic algorithms presented in these books can be transferred fairly easily to other languages. More than once, for instance, I've taken code from these books and reworked it to run in a Cocoa application. And, of course, over the past year and a half, we've had the opportunity to play with a number of these alternative development languages and tools in this column in MacTech.

Interactive QuickTime

Matthew Peterson is a recognized master of QuickTime's interactive capabilities -- particularly of wired sprites and wired actions. I've seen audiences in awe over the amazing things he has done building intelligent, active QuickTime movies. Movies that read data off a remote server and chart graphs in real time. Movies that look and act like a pocket calculator. Movies that track the mouse and alter the appearance and location of objects in the movie based on mouse movements. Movies that simply knock your socks off, no matter how long you've worked with QuickTime.

Interactive QuickTime is Matthew's eagerly-awaited explanation of how he does some of those amazing things. He guides us deftly through QuickTime's interactive landscape, discussing how to add dynamic, active behaviors to sprites, video, text tracks, Flash, QuickTime VR, and movie tracks. He also covers the much-neglected topic of communicating with servers using QTLists. This book is chock-full of useful algorithms and interesting approaches to programming interactive behaviors in QuickTime movies.

The examples in this book are almost exclusively presented in QScript, a scripting language supported by the LiveStage Pro application. To get the most out of this book, therefore, you need to have that tool available, and indeed the CD included with the book provides a demo version of LiveStage Pro. I can attest from long experience that building interactive movie using raw C code can be a painful process. Nonetheless, the algorithms themselves are clear enough that you could easily reimplement them in C or any other high-level language.

QuickTime for Java(TM): A Developer's Notebook

I know what you are thinking: "I don't do Java; I have no interest in doing Java; so this book is not for me." Fair enough. At least let me say this: if you do program in Java and you are interested in developing QuickTime applications, then QuickTime for Java(TM): A Developer's Notebook by Chris Adamson (O'Reilly, 2005) should be on your bookshelf. Better yet, it should be propped open on your desk as you work your way though it. We've already seen hints of the kind of careful analysis Chris brings to the table in his ability to get QuickTime for Java applications to capture video and audio sources. He shows that same insight throughout this slim volume, demonstrating how to open and display still images and QuickTime movies, support movie editing and undo operations, export movies and still images, build movies from raw data, and work with the QuickTime video effects architecture. All in Java, and all skillfully integrated with the major Java windowing toolkits, AWT and Swing.

Actually, however, I dispute the idea that this is a book exclusively for Java developers. In several instances, Chris trots out some code that, mutatis mutandis, could easily be useful to C programmers. In particular, he presents a careful dissection of the format of the metadata attached to iTunes AAC files and shows how to construct a set of Java classes for reading and displaying the author, title, and album information (among other kinds). I imagine it would be easy to reimplement his algorithm in C and hence make it more widely available.

I do have one quibble with this book. This isn't your typical O'Reilly book: there's no animal on the cover and the editorial direction is much more focused. These Developer's Notebooks are supposed to harness "the often-frantic scribbling and notes that a true-blue alpha geek mentally makes when working with a new language, API, or project." The book design reflects this "lab manual" approach: the pages have a graph-paper ruled background and the cover shows the stains of a coffee mug. That's all well and good. But the book is printed with a bluish-purplish ink that is particularly unwelcome for the screen shots and other graphics. The text is perfectly legible, to be sure, but the images just look odd. It would have been less jarring to stick with black ink, I think.

Book Information

Books in the Morgan-Kaufmann QuickTime Developer Series:

QuickTime for the Web, by Steven Gulie. Morgan-Kaufmann (Third Edition, July 2003). 825 pages. ISBN: 1-55860-904-0.

QuickTime Toolkit, Volume One: Basic Movie Playback and Media Types, by Tim Monroe. Morgan-Kaufmann (First Edition, June 2004). 640 pages. ISBN: 0-12-088401-1.

QuickTime Toolkit, Volume Two: Advanced Movie Playback and Media Types, by Tim Monroe. Morgan-Kaufmann (First Edition, June 2004). 528 pages. ISBN: 0-12-088402-X.

Interactive QuickTime, by Matthew Peterson. Morgan-Kaufmann (First Edition, August 2003). 597 pages. ISBN: 1-55860-746-3.

Books in the O'Reilly Developer's Notebooks series:

QuickTime for Java, by Chris Adamson. O'Reilly (First Edition, January 2005). 255 pages. ISBN: 0-596-00822-8.

Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him at monroe@mactech.com. The views expressed here are not necessarily shared by his employer.