Volume Number: 21 (2005)
Issue Number: 5
Column Tag: Programming

QuickTime Toolkit

Back To The Future

by Tim Monroe

Developing Applications With The QuickTime For Cocoa Kit

In several earlier articles ("The Cocoanuts" in MacTech, December 2002 and "Animal Crackers" in January 2003), we took a look at developing QuickTime applications using the Cocoa classes NSMovie and NSMovieView. We found that these classes provide reasonably good support for displaying and editing QuickTime movies. They are tightly integrated with the Cocoa application framework as a whole, and they succeed marvelously at shielding the Cocoa programmer from the Carbon data structures and APIs native to QuickTime.

Introduction

In several earlier articles ("The Cocoanuts" in MacTech, December 2002 and "Animal Crackers" in January 2003), we took a look at developing QuickTime applications using the Cocoa classes NSMovie and NSMovieView. We found that these classes provide reasonably good support for displaying and editing QuickTime movies. They are tightly integrated with the Cocoa application framework as a whole, and they succeed marvelously at shielding the Cocoa programmer from the Carbon data structures and APIs native to QuickTime.

Nonetheless, the two classes NSMovie and NSMovieView provide only a limited selection of methods for managing QuickTime movies. We saw that we fairly quickly needed to subclass NSMovieView to override its default behaviors (for instance to support a contextual menu that was sensitive to the type of movie being displayed in a view). And we needed to add categories to these classes to extend them in fairly simple ways (for instance to show and hide specific buttons in the movie controller bar). NSMovie and NSMovieView are fine as far as they go; the main problem with them is that they don't go very far at all. They are intentionally limited to movie playback and simple editing, and even there they sidestep the opportunity to incorporate movie opening and playback behaviors that even a simple application would probably like to exhibit (such as loading movies asynchronously or automatically finding external target movies for wired actions).

Apple has recently introduced an enhanced set of Cocoa classes that support QuickTime movie playback and editing, called the QuickTime For Cocoa Kit or (more briefly) the QTKit. This new framework first shipped with Mac OS X 10.4 ("Tiger") and is also available for Mac OS X 10.3.9 ("Panther") as part of QuickTime 7. In this article and the next several articles, I want to examine the QTKit framework in some detail. The QTKit is an important piece of new Cocoa technology that is already used by key applications such as QuickTime Player (on Tiger and Panther) and Spotlight (on Tiger). It promises to significantly simplify the process of adding multimedia capabilities to Cocoa applications.

In this article, we'll take an introductory look at QTKit's classes and their methods, as well as the structures and functions it uses to manipulate times and time ranges. We'll see how to use the QTKit to display a QuickTime movie in a Cocoa document window and control the playback of that movie. We'll also see how to use QTKit methods in a command-line tool. In the following articles, we'll look at more advanced uses of these new classes. We'll learn how to use the QTKit to export and flatten movies, and how to perform more refined movie editing.

QuickTime for Cocoa Kit Overview

The QTKit is a Cocoa framework that provides support for opening and displaying QuickTime movies, and for performing various sorts of manipulations on those movies. Note that I did not write: "QTKit is a framework that provides a Cocoa wrapper for all of QuickTime". There are large pieces of QuickTime functionality that are not addressed by the version of QTKit released in QuickTime 7. For instance, there are no classes for capturing video or sound data (thereby wrapping QuickTime's sequence grabbing capabilities), and there are no classes for working directly with sprites or wired actions or atom containers or QuickTime VR hot spots. The version of QTKit released in QuickTime 7 should be viewed as a very good start to the Cocoaization of QuickTime, not the completed journey.

QTKit Classes

The QTKit framework currently exposes five classes: QTMovie, QTTrack, QTMedia, QTDataReference, and QTMovieView. QTMovieView is just what you would expect: it's a subclass of NSView that can display QuickTime movies and allow the user to interact with them. Figure 1 shows a Cocoa window whose content area is completely filled by a QTMovieView.

Figure 1: A window that contains a QTMovieView

As you can see, this QTMovieView contains the standard movie controller bar. However, you can hide the bar by invoking the setControllerVisible: method, like this:

[movieView setControllerVisible:NO];

The QTMovieView class provides methods for controlling movie playback, positioning the current movie time, and performing edit operations on the current movie selection (cut, copy, paste, etc.). It also provides a contextual menu that allows access to a handful of these methods, as shown in Figure 2.

Figure 2: A contextual menu in a QTMovieView

In QTMovieView, this menu is truly contextual; for a Flash movie, the menu is the one shown in Figure 3.

Figure 3: A contextual menu in a Flash movie

You associate a QuickTime movie with a QTMovieView object by calling the setMovie: method, which takes a QTMovie object as its parameter.

QTMovie is the key class in QTKit. It is a Cocoa wrapper for a QuickTime movie and a QuickTime movie controller. QTMovie provides an extensive selection of methods for opening QuickTime movies from files, URLs, the Pasteboard, an NSData object, and data specified by a QuickTime data reference. For instance, if filename is the full pathname of a QuickTime movie file, this line of code creates and initializes a QTMovie object associated with that movie file:

[[QTMovie alloc] initWithFile:filename error:nil];

Notice the error parameter; it's a pointer to an NSError object that is used to return to the caller an indication of any error that occurs during the movie initialization. All QTMovie initialization methods use that parameter.

QTMovie also provides an extensive selection of methods for getting and setting movie properties, controlling movie playback and current time, getting and setting the movie selection, and editing the movie selection. We'll encounter a number of these methods in the following discussion.

The remaining three classes, QTTrack, QTMedia, and QTDataReference provide Cocoa representations of QuickTime tracks, media, and data references, respectively. For simple movie playback and editing, you will not need to use these classes and we will not discuss them further in this article.

Time Structures and Functions

Time positions and durations are managed in QTKit using two structures, QTTime and QTTimeRange. Note that these are not objects, since creating an object each time we needed to specify a location in movie or a duration of a movie segment would have been onerous and inefficient. Here is the declaration of the QTTime structure:

typedef struct {
   long long         timeValue;
   long              timeScale;
   long              flags;
} QTTime;

The timeValue field is a 64-bit value that indicates a time value. This value is interpreted relative to the timeScale field, which indicates the number of units per second. The flags field is usually 0; in rare circumstances the kQTTimeIsIndefinite flag may be set, indicating that the other two fields are not to be trusted. (This might happen if you try to do an operation on a QTTime structure whose timeScale field is 0.)

The QTKit provides a handful of utility methods for initializing and manipulating QTTime structures. For instance, QTMakeTime takes a time value and a time scale and then returns a QTTime structure. We can specify the four-second point in a movie like this:

QTTime startTime = QTMakeTime(4, 1);

The QTTimeRange structure specifies a range or duration of time values; it's declared like this:

typedef struct {
   QTTime            time;
   QTTime            duration;
} QTTimeRange;

QTKit Behaviors

One metric of the value of a framework is not the number and variety of the methods you can call, but (so to speak) the methods you don't need to call -- that is, capabilities that are provided "for free" by the framework. This is an especially valuable metric for a QuickTime framework, as the operations required to (say) open and display a QuickTime movie can easily become quite numerous. As we've seen in earlier articles (particularly "Loaded" in MacTech, September 2002), loading a movie from a remote server without blocking the application (that is, asynchronously) requires a fair amount of work. We need to make special API calls to start the asynchronous loading and then we need to monitor the movie's load state until it has reached a certain threshold. And we need to explicitly task the movie all the while. In QTKit, all movies are loaded asynchronously, with absolutely no extra work required by the developer. And QTKit takes care of tasking the movie at appropriate intervals. Loading a remote movie asynchronously is as simple as executing initWithURL:error:.

Here are a few more of the behaviors provided for free to QTKit client applications:

Multi-level undo for editing operations. QTKit takes advantage of existing Cocoa classes to support multi-level undo and redo for editing operations on a QTMovieView object. In earlier versions of QuickTime Player, for instance, only the most recent edit operation is undoable.

Intermovie communication. QTMovie automatically finds the target of a wired action that is targeted at another movie.

Drag and drop editing. QTKit supports dragging a movie selection out of a QTMovieView and dropping it into any other object that can accept dragged movie data; dragging between Cocoa and Carbon applications is fully supported.

Live resizing. A QTMovieView object can be resized while the associated movie is playing. Earlier versions of QuickTime Player drew a ghost outline of the movie frame and window frame during resizing and redrew the movie data only when the final size had been reached.

Constant pitch audio. In previous versions of QuickTime, selecting a faster playback rate would alter the pitch of the audio (resulting in the infamous "chipmunk" effect). QTKit automatically takes advantage of improvements in the QuickTime 7 audio architecture to maintain a constant pitch during accelerated movie playback.

Keep in mind that these are only a few of the improved behaviors that all QTKit client applications exhibit as part of using QTKit.

Movie Editing

Let's begin our hands-on exploration of the QTKit framework by seeing how to delete a segment of a QuickTime movie using QTKit APIs. First, we need to open the movie file; if the movie file is located at /tmp/sample.mov, we can use this line of code:

QTMovie *movie = [QTMovie movieWithFile:@"/tmp/sample.mov" error:nil];

Next we need to specify the movie segment that is to be deleted. We can use the QTTime and QTTimeRange structures and associated functions to do that:

QTTime startTime = QTMakeTime(4, 1);
QTTime duration = QTMakeTime(2, 1);
QTTimeRange range = QTMakeTimeRange(startTime, duration);

These lines of code specify a segment that is two seconds long beginning four seconds into the movie.

We can delete the segment by using the QTMovie method deleteSegment:, like so:

[movie deleteSegment:range];

There is however one gotcha: by default, a QTMovie object is uneditable. So we first need to set the editable state of the movie like this:

[movie setAttribute:[NSNumber numberWithBool:YES] 
                   forKey:QTMovieEditableAttribute];

The method setAttribute:forKey: is the standard way of setting an attribute of a QTMovie object. The header file QTMovie.h lists a large number of keys for this method.

Finally, to save the edited movie back into the original file, we can execute the updateMovieFile method:

[movie updateMovieFile];

Movie Exporting

Let's consider another easy example. Suppose we want to save an existing QuickTime movie file as a 3GPP file (which is streamable to cell phones and other terminal devices that conform to the 3GPP standard). In this case, we can use the writeToFile:withAttributes: method supplied by QTMovie. This method is used to export or flatten a QuickTime movie, depending on the contents of the attributes dictionary. Listing 1 shows a complete command-line tool that creates the dictionary, adds the appropriate objects and keys, and then exports the movie.

Listing 1: Exporting a movie as a 3GPP file

#import <QTKit/QTKit.h>

int main ()
{
   NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

   QTMovie *movie = [QTMovie movieWithFile:@"/tmp/sample.mov" error:nil];
   NSDictionary *dictionary = [NSDictionary dictionaryWithObjectsAndKeys:
         [NSNumber numberWithBool:YES], QTMovieExport, 
         [NSNumber numberWithLong:kQTFileType3GPP], 
         QTMovieExportType, nil];

   [movie writeToFile:@"/tmp/sample.3gp" withAttributes:dictionary];

   [pool release];
	
   exit(1);
}

If this code is saved into the file mov23gp.m, we can create the executable command-line tool by issuing this command:

% cc -o mov23gp -g mov23gp.m -framework Foundation -framework QTKit

Movie Display

Command-line tools are useful enough, but chances are you're more interested in opening and displaying QuickTime movies in a window. Let's see how to work with a QTMovieView object. For this, we'll need to use Xcode and Interface Builder.

Launch Xcode; select "New Project..." from the File menu and then select "Cocoa Document-based Application", as shown in Figure 4.

Figure 4: Creating a new multi-document project

The Assistant then prompts us for a project name and location; let's call our project "KitEez" and save it in our home directory. Xcode then displays the project window shown in Figure 5. (I've opened some of the disclosure triangles to reveal the contents of each group.)

Figure 5: The KitEez project window

The "Product" -- that is, our application -- is shown in red because it hasn't been built yet. We can build the application by selecting Build (Command-B) from the Build menu. And we can run the application by selecting Run (Command-R). If we do that, we'll see the document window shown in Figure 6.

Figure 6: The default document window

That's certainly not what we want; remember, we want to display QuickTime movies in this window, not a short text string.

Modifying the Document Window Appearance

Let's open MyDocument.nib with Interface Builder. We can do this by double-clicking the entry in the project window. Figure 7 shows the main window for this nib file.

Figure 7: The document nib file

Now let's remove the text from the main document window and replace it with a QTMovieView object. By default, the QTKit palette item is not automatically loaded into the Palettes toolbar, so we need to choose "Preferences..." and then select the Palettes pane; click "Add..." (Figure 8).

Figure 8: The Palettes pane

Navigate to /Developer/Extras/Palettes and select the QTKit.palette item, as shown in Figure 9.

Figure 9: The QTKit palette

At this point, the Palettes toolbar looks like the one in Figure 10.

Figure 10: The QTKit palette in the toolbar

Drag the QuickTime icon from the palette into the document window and resize it to fill the content area of the window, as shown in Figure 11. In the Size pane of the QTMovieView Inspector palette, set the movie view autosizing so that the movie view remains pegged to the size of the window.

Figure 11: The revised document window

Now, we need a way for our custom document class (MyDocument) to keep track of this new object that we've added to the document window. We do this by adding an outlet to the document class. An outlet is essentially just an instance variable that holds a reference to some object. We'll add the instance variable to our class declaration in a moment, but we also need to add it to our nib file. Click on the Classes tab in the MyDocument.nib window and navigate to the MyDocument class, as shown in Figure 12.

Figure 12: The Class hierarchy window

Now select the "Add Outlet to MyDocument" menu item in the Classes menu. Interface Builder will display the Class Info window shown in Figure 13.

Figure 13: The Class Info window

Change the default outlet name "myOutlet" to "movieView".

So far, we've created a new outlet for the document class. Now we need to link the outlet with the movie view that we added to the document window. Click on the Instances tab in the MyDocument.nib window; then hold down the Control key, click on the File's Owner icon, and drag until the cursor is over the movie view in the document window. When we release the mouse button, Interface Builder displays the window shown in Figure 14.

Figure 14: Connections of the File's Owner

Click the Connect button to make the desired connection. Notice that the Destination field updates to indicate that movieView is connected to an object of type QTMovieView. For the moment, we are done modifying the nib file. Let's save our work and quit Interface Builder.

Configuring the Application

Move back to Xcode. The first thing we need to do is add the QTKit framework to the project. Select "Add to Project..." in the Project menu and then navigate to /System/Library/Frameworks; select QTKit.framework and click OK and then Add.

At this point, we need to add an instance variable for the movieView outlet we created in Interface Builder. Open the file MyDocument.h and add a movieView instance variable. Let's also add an instance variable to keep track of the QTMovie object associated with the movie view. (This is not strictly necessary, as we can always get the QTMovie object associated with a QTMovieView object by calling its movie method.) Also, import the QTKit.h header file from the QTKit framework. MyDocument.h now looks like this:

#import <QTKit/QTKit.h>

@interface MyDocument : NSDocument
{
   QTMovieView             *movieView;
   QTMovie                 *movie;
}
@end

Now select the KitEez target in the project window and bring up the inspector window by selecting "Get Info" in the File menu. In the Properties pane, set the document types for the MyDocument class to include the extension "mov" and the OS type "MooV". This will allow files with names ending in ".mov" or of file type "MooV" to appear in the file-opening dialog box. (Later we'll see how to modify KitEez to allow the user to select any kind of file openable by QuickTime, regardless of filename extension or file type.)

Opening Movie Files

When the user selects a file in the file-opening dialog box, our multi-document application creates a new document window, based on the information in the nib file. When it has loaded the nib and initialized all the objects specified therein, it calls our document's windowControllerDidLoadNib method. This is where we want to assign the movie to the movie view.

By the time windowControllerDidLoadNib is called, an instance of the MyDocument class has already been created. All the instance variables associated with outlets, and in particular movieView, are already set up for us. It remains only for us to do any final configuration of the movie or view before the new document window is displayed to the user.

As you know, we need to create a QTMovie object and assign it to the movie view. Once again we'll use the initWithFile:error: method, which initializes a movie object with a QuickTime movie specified by a filename:

movie = [[QTMovie alloc] initWithFile:[self filename] error:nil];

We'll set the movie associated with the movie view like this:

[movieView setMovie:movie];

And we'll set the movie to be editable, just like we did earlier using setAttribute:forKey:.

Setting the Window Size

Before showing the new window to the user, we need to set the size of the window's content region to exactly enclose the movie at its natural size. We can determine the natural size of the movie like this:

NSSize size = [[movie attributeForKey:QTMovieNaturalSizeAttribute] sizeValue];

As you can see, we get a movie attribute by calling the attributeForKey: method.

We can determine the appropriate content region size for a given movie size using the windowContentSizeForMovieSize: method defined in Listing 2. Notice that we use the isControllerVisible method on QTMovieView to see whether the controller bar is showing; if it is, we need to increase the height of the window by the height of the movie controller bar.

Listing 2: Finding a content region size for a movie size

- (NSSize)windowContentSizeForMovieSize:(NSSize)size
{
   NSSize contentSize = size;

   if ([movieView isControllerVisible])
      contentSize.height += [movieView controllerBarHeight];

   if (contentSize.width == 0)
      contentSize.width = [[movieView window] frame].size.width;

   return contentSize;
}

Notice also that we look to see whether the size passed in has a width of 0; if it does, we set the width of the window's content region to the current width of the window.

In windowControllerDidLoadNib, we'll call windowContentSizeForMovieSize: and set the size of the movie window like this:

[[movieView window] setContentSize:[self windowContentSizeForMovieSize:size]];

There is one final task that we need to perform. Since our movie view entirely fills the content region of the document window, we need to disable the resize box provided by NSWindow and enable the resize box displayed by QTMovieView. We can do that with two calls to setShowsResizeIndicator::

[[movieView window] setShowsResizeIndicator:NO];
[movieView setShowsResizeIndicator:YES];

Listing 3 shows our complete implementation of windowControllerDidLoadNib.

Listing 3: Opening a movie window

- (void)windowControllerDidLoadNib:(NSWindowController *) aController
{
   [super windowControllerDidLoadNib:aController];
   if ([self fileName]) {
      movie = [[QTMovie alloc] initWithFile:[self fileName] error:nil];
      if (movie) {
         // set the view's movie
         [movieView setMovie:movie];

         // make the movie editable
         [movie setAttribute:[NSNumber numberWithBool:YES] forKey:QTMovieEditableAttribute];
			
         // set the size of the movie window to exactly enclose the movie at its natural size
         NSSize size = [[movie attributeForKey:QTMovieNaturalSizeAttribute] sizeValue];
         [[movieView window] setContentSize:[self windowContentSizeForMovieSize:size]];

         // use the QTKit's resize indicator, not NSWindow's
         [[movieView window] setShowsResizeIndicator:NO];
         [movieView setShowsResizeIndicator:YES];
      }
   }
}

If we build and run the application, we will be able to open and display QuickTime movies. The movies are fully editable, either using items in the Edit menu or their command-key equivalents.

Setting Openable Document Types

The current limitation on KitEez is that it can open only files whose filename extension is ".mov" or that are of HFS file type "MooV". We'd really like it to be able to open any kind of file the QuickTime can open. We could of course add more and more file extensions and types to the application's property list; a better solution, however, is to determine dynamically what files should appear in the file-opening dialog box. To do that, we need to add an application delegate to the project. That delegate will override the openDocument: and panel:shouldShowFilename: methods. In particular, we'll add the override method shown in Listing 4.

Listing 4: Specifying openable document types

- (BOOL)panel:(id)sender shouldShowFilename:(NSString *)filename
{
   BOOL isDir = NO;

   [[NSFileManager defaultManager] fileExistsAtPath:filename isDirectory:&isDir];

   return isDir ? YES : [QTMovie canInitWithFile:filename];
}

The key element here is to use the QTMovie class method canInitWithFile:, which returns a value of type BOOL that indicates whether the specified file can be used to initialize a QTMovie object. QTMovie also supplies the class methods canInitWithURL:, canInitWithPasteboard:, and canInitWithDataReference:.

Conclusion

In this article, we've taken a preliminary first look at the QTKit framework introduced in QuickTime 7 on both Tiger and Panther. We've seen how to use QTKit to build a simple command-line tool that manipulates QuickTime movies, and we've seen how to create a multi-document Cocoa application that can open and display QuickTime movies. In the next two articles, we'll continue investigating this important new framework. We'll finish KitEez by allowing it to save edited movies, and we'll spend some time looking at more of the capabilities offered by QTKit.

---

Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him at monroe@mactech.com. The views expressed here are not necessarily shared by his employer.