Developing Applications with the QuickTime for Cocoa Kit
Volume Number: 21 (2005)
Issue Number: 7
Column Tag: Programming
QuickTime Toolkit
Back To The Future, Part III
by Tim Monroe
Developing Applications with the QuickTime for Cocoa Kit
In the previous two QuickTime Toolkit articles (in MacTech, May and June 2005), we've taken a look at using
QTKit, Apple's new Cocoa framework for displaying and modifying QuickTime movies. We saw how to open and
manipulate movies using command-line tools; then we stepped through the process of building a fairly complete
document-based Cocoa application using QTKit. This application, which we called KitEez, supports the standard
document-related behaviors, including saving an edited movie into its original file or into some other file,
and reverting to the last saved version of a movie file. Since KitEez was built using the Cocoa class
NSDocument, we didn't need to write any code to track the edited state of the document window and its
associated movie or to display the standard dialog boxes that warn the user about unsaved changes and
overwriting existing files.
Introduction
In this article, we'll take a look at a few more of the capabilities offered in by QTKit. In particular,
we'll see how to add images to an existing QuickTime movie and work with notifications and delegates. We'll
also see how to call QuickTime functions that have no corresponding QTKit method and we'll take a second look
at opening files and URLs.
Adding Images to a Movie
Adding Images to an Existing Movie
Imagine that we've opened a QuickTime movie from a file and we'd like to tack a few images onto the end of
the movie. The QTMovie class provides a method that makes this extremely easy,
addImage:forDuration:withAttributes, whose signature is this:
-(void)addImage:(NSImage *)image
forDuration:(QTTime )duration
withAttributes:(NSDictionary *)attributes;
The image parameter of course specifies the image to add to the end of the movie, and the duration
parameter is a QTTime structure that specifies the desired duration of the new frame. The attributes parameter
is a dictionary that specifies the codec to be used to compress the image and possibly also the quality of the
compressed image. For instance, Listing 1 shows a simple method that reads an image from the application's nib
file and adds it to the end of the movie, with a duration of one second.
Listing 1: Appending an image to a movie
- (void)addImageFromNibFile:(NSString *)name
{
NSImage *image = [NSImage imageNamed:name];
QTTime duration = QTMakeTime(600, 600);
NSDictionary *dict = [NSDictionary dictionaryWithObject:
QTStringForOSType('pxlt') forKey:QTAddImageCodecType]
[_movie addImage:image forDuration:duration withAttributes:dict];
}
As you can see, the dict dictionary contains one key-value pair, where the key is QTAddImageCodecType. This
key specifies the codec to be used to compress the image before it's added to the movie. Notice that we have
used the function QTStringForOSType to convert a codec OSType (here, for the Pixlet compressor) to an NSString
object. QTKit also provides a utility function for going in the opposite direction: QTOSTypeForString will
return an OSType corresponding to a given NSString.
The dictionary of attributes can also contain a key-value pair in which the key is QTAddImageCodecQuality.
If this pair is present, then the specified value, which should be an NSNumber, is used as the quality setting
for the compressor. This key-value pair is optional; if it is not included in the dictionary, the quality
setting codecNormalQuality is used. The acceptable values are listed in this enumeration in the header file
ImageCompression.h in the framework QuickTime.framework:
enum {
codecLosslessQuality = 0x00000400,
codecMaxQuality = 0x000003FF,
codecMinQuality = 0x00000000,
codecLowQuality = 0x00000100,
codecNormalQuality = 0x00000200,
codecHighQuality = 0x00000300
};
Adding Images to a New Movie
Suppose that instead of adding an image to the end of an existing movie, we want to create a slideshow
movie from some images, starting this time from an empty movie. You might think that we could first create a
new, empty QTMovie object and then tack on the images, like this:
QTMovie *movie = [[QTMovie alloc] init];
[movie addImageFromNibFile:@"A.jpg"];
[movie addImageFromNibFile:@"B.jpg"];
// and so forth...
In fact, however, this would fail. The underlying reason is that a new QTMovie object created in this way
has no writable data reference associated with it. In layman's terms, this means that the associated QuickTime
Movie has not yet been assigned a place to store any new media data that is added to the movie. Since there is
no place to write the media data, the call to addImage:forDuration:withAttributes fails.
You might think that this is a bug in QTMovie's init method, which could easily assign a block of data in
memory as the writable data reference for a new QTMovie object. But that revised behavior would lead to
problems of its own. For instance, adding a number of large images could soon fill up the available physical
memory or at least put the virtual memory system into severe thrashing.
Future versions of QTKit will almost certainly provide one or more new methods in QTMovie to create a new
empty QTMovie object that is associated with a writable file or block of memory. In the meantime, it is
possible to work around this limitation by creating a new empty file and associating it with a new QTMovie
object using the movieWithQuickTimeMovie:disposeWhenDone:error: method, as shown in Listing 2. As you can see,
we use the movie storage APIs (discussed in "Modern Times" in MacTech, May, 2004) to create a new empty movie
file.
Listing 2: Creating a new empty movie that is writable
- (QTMovie)movieWithWritableFile:(NSString *)filename handler:(DataHandler *)dataHandler
{
OSErr err = noErr;
Handle dataRef = nil;
OSType dataRefType;
Movie qtMovie = NULL;
QTMovie movie = nil;
// make sure we are passed a location to return the data handler identifier
if (!dataHandler) return nil;
// create a file data reference for the specified filename
err = QTNewDataReferenceFromFullPathCFString(
(CFStringRef)fileName,
kQTNativeDefaultPathStyle,
0, &dataRef, &dataRefType);
if (err != noErr) return nil;
// create a QuickTime movie from the file data reference
err = CreateMovieStorage(dataRef, dataRefType, 'TVOD',
smSystemScript, newMovieActive,
dataHandler, &qtMovie);
if (err != noErr) return nil;
// instantiate a QTMovie from the QuickTime movie
movie = [QTMovie movieWithQuickTimeMovie:qtMovie
disposeWhenDone:YES error:nil];
// mark the movie as editable
[movie setAttribute:[NSNumber numberWithBool:YES]
forKey:QTMovieEditableAttribute];
return movie;
}
We need to return to the caller both the new QTMovie object and the identifier of the data handler
associated with the open movie file. That's because, when we are finished adding frames to the movie and have
called updateMovieFile, we need to call CloseMovieStorage to close the movie:
CloseMovieStorage(dataHandler);
Creating a Movie from a Single Image
It's worth mentioning one further twist on this issue, namely using QTKit to create a movie from a single
image. Suppose we want to create a 10-second movie from a single image. We could use the strategy employed in
the previous section, first creating a new empty movie file and then adding the specified image to its
associated QTMovie object. But there is in fact a much simpler way to do this, using the
dataReferenceWithReferenceToData:name:MIMEType: method in the QTDataReference class. In a nutshell, this
method creates a data reference to some block of memory addressed using an NSData object and optionally
attaches a filenaming extension or a data reference extension of type 'mime' to the data reference. As we have
seen in earlier articles (particularly in "Somewhere I'll Find You" in MacTech, October, 2000), these
extensions help QuickTime find the appropriate movie or graphics importer when a data reference is to a block
of memory.
In the present case, we'll create an NSData object that contains a TIFF representation of the image and
then create a QTDataReference object that has, as its filenaming extension, an arbitrary name with the
extension .tiff. Listing 3 shows the code we can use to do this.
Listing 3: Creating a movie from a single image
- (void)createMovieFileFromImage:(NSString *)filename
(NSImage *)image
{
if (!filename || !image)
return;
NSData *data = [image TIFFRepresentation];
QTDataReference *dataRef = [QTDataReference
dataReferenceWithReferenceToData:data
name:@"some_image.tiff" MIMEType:nil];
QTMovie *movie = [QTMovie movieWithDataReference:dataRef
error:nil];
// make the movie editable
[movie setAttribute:[NSNumber numberWithBool:YES]
forKey:QTMovieEditableAttribute];
// set the duration to 10 seconds
QTTimeRange range = QTMakeTimeRange(QTZeroTime,
[movie duration]);
[movie scaleSegment:range newDuration:QTMakeTime(10, 1)];
// export as a 3gpp file
NSDictionary *dict= [NSDictionary
dictionaryWithObject:[NSNumber numberWithBool:YES]
forKey:QTMovieFlatten];
[movie writeToFile: filename withAttributes:dict];
}
This works because we are not actually editing the movie's media data. Instead, we are creating a QTMovie
object that gets its media data from the NSData block. We are editing the movie when we call the
scaleSegment:newDuration: method, but that's okay even when the movie does not have a writable data reference.
QuickTime Functions
Even though QTKit exposes a fairly large number of classes and methods, and even though it does quite a bit
of movie-related processing automatically, it's fairly likely that we will want to use QuickTime capabilities
that it does not currently support. For instance, we've already seen that QTKit does not yet provide a method
for creating a new, empty movie that has a writable data reference associated with it. So we used the
underlying QuickTime APIs to create one, which we then used to initialize a QTMovie object. It can also happen
that we've already got a QTMovie object and we want to perform some operation not yet supported by QTKit. In
this case, the QTMovie class provides two useful methods that allow us to retrieve the Movie and
MoveController identifiers associated with a QTMovie object:
-(Movie)quickTimeMovie;
-(MovieController)quickTimeMovieController;
Suppose, for example, that we want to set the magnification (or "zoom") level of a Flash movie. We can do
so by using the quickTimeMovie method to retrieve the QuickTime Movie associated with a QTMovie object and
then using Movie Toolbox and Flash media handler functions, as shown in Listing 4.
Listing 4: Setting the zoom level of a Flash movie
- (void)setZoom:(float)zoomPct
{
Track flashTrack = NULL;
Media flashMedia = NULL;
MediaHandler flashHandler = NULL;
flashTrack = GetMovieIndTrackType([self quickTimeMovie],
1, FlashMediaType, movieTrackMediaType |
movieTrackEnabledOnly);
if (flashTrack) {
flashMedia = GetTrackMedia(flashTrack);
flashHandler = GetMediaHandler(flashMedia);
FlashMediaSetZoom(flashHandler, zoomPct);
}
}
In general, it should be safe to call virtually any QuickTime functions on the Movie or the MoveController
associated with a QTMovie object. Some operations, however, might not be safe and should probably be avoided.
In particular, QTKit generally assumes that it will be disposing of the Movie and MovieController associated
with a QTMovie object, so you should not call DisposeMovie or DisposeMovieController. (The exception to this
rule is when you pass the value YES as the disposeWhenDone parameter in QTMovie's
movieWithQuickTimeMovie:disposeWhenDone:error: method, which tells QTKit that you want to dispose of the movie
yourself.) Also, deleting tracks from a movie using the Movie Toolbox function DisposeMovieTrack is likely to
confuse QTKit and may even lead to crashes. It's likely that QTKit will in future versions add methods that
provide a way to delete tracks, since this is a reasonably common operation for some kinds of applications.
Notifications and Delegate Methods
As you no doubt already know, a notification is a way for one Cocoa object to inform other objects about
changes in its state or its properties. The QTMovie class defines a large number of notifications that other
objects can listen for; here are the notifications that are currently defined:
NSString *QTMovieEditabilityDidChangeNotification;
NSString *QTMovieEditedNotification;
NSString *QTMovieLoadStateDidChangeNotification;
NSString *QTMovieLoopModeDidChangeNotification;
NSString *QTMovieMessageStringPostedNotification;
NSString *QTMovieRateDidChangeNotification;
NSString *QTMovieSelectionDidChangeNotification;
NSString *QTMovieSizeDidChangeNotification;
NSString *QTMovieStatusStringPostedNotification;
NSString *QTMovieTimeDidChangeNotification;
NSString *QTMovieVolumeDidChangeNotification;
NSString *QTMovieDidEndNotification;
NSString *QTMovieChapterDidChangeNotification;
NSString *QTMovieChapterListDidChangeNotification;
NSString *QTMovieEnterFullScreenRequestNotification;
NSString *QTMovieExitFullScreenRequestNotification;
NSString *QTMovieCloseWindowRequestNotification;
Most of these are fairly obvious. For instance, QTMovieDidEndNotification is posted when a QTMovie object
reaches its end. And QTMovieEditedNotification is posted when a QTMovie object has been edited or changed in
some way. A few of these are however somewhat less obvious. The QTMovieTimeDidChangeNotification notification
is not, for instance, posted whenever the movie time changes; rather, it's posted whenever the movie time
changes to a value different from what it would be during normal movie playback. These sorts of time changes
include operations like the user clicking in the movie controller bar to change the movie time or a wired
action setting the movie time to some specific time.
In most cases, these notifications are posted using the NSNotificationCenter method
postNotificationName:object:, where no accompanying userInfo dictionary is passed. The expectation is that the
notification listener will be able to retrieve whatever information about the QTMovie object is needed at the
time the notification is received. So, for instance, a listener receiving the
QTMovieVolumeDidChangeNotification notification could call the QTMovie method volume to retrieve the current
volume of the movie. In three cases, however, a dictionary of information is passed along with the
notification, because there is no easy way for the receiver to ask for that information; these are:
QTMovieMessageStringPostedNotification
QTMovieRateDidChangeNotification
QTMovieStatusStringPostedNotification
We've already seen an example of registering for notifications in the previous article. When we initialize
a movie view to hold a movie, we want the associated document to be informed of any size changes that occur
programmatically or via wired actions; we did that by registering for the QTMovieSizeDidChangeNotification
notification, like this:
[[NSNotificationCenter defaultCenter] addObserver:self
selector:@selector(boundsDidChange:)
name:QTMovieSizeDidChangeNotification object:movie];
Listing 5 shows our implementation of the boundsDidChange: method, which is invoked when we receive this notification.
Listing 5: Handling size-changed notifications
- (void)boundsDidChange:(NSNotification *)notification
{
// set the size of the movie window to exactly enclose the movie at its current size
NSSize size = [[[movieView movie] attributeForKey:
QTMovieCurrentSizeAttribute] sizeValue];
[[movieView window] setContentSize:
[self windowContentSizeForMovieSize:size]];
}
Notifications provide a very loose sort of coupling between objects in Cocoa. One object posts a
notification and one or more other objects can listen for that notification and respond appropriately. By
contrast, a much tighter association of two objects can be formed by designating one of them as the delegate
of the other. Currently QTKit defines only five public delegate methods, all in the QTMovie class, of which
only three are likely to be of use to developers:
- (BOOL)movie:(QTMovie *)movie linkToURL:(NSURL *)url;
- (QTMovie *)externalMovie:(NSDictionary *)dictionary;
- (BOOL)movie:(QTMovie *)movie
shouldContinueOperation:(NSString *)op
withPhase:(QTMovieOperationPhase)phase
atPercent:(NSNumber *)percent
withAttributes:(NSDictionary *)attributes;
A delegate's movie:linkToURL: method is called when the movie controller is about to open a movie specified
by a URL. (This corresponds to the movie controller processing an mcActionLinkToURL action in the Carbon
world.) For most applications, the QTKit's normal processing of URLs is fine for most applications; you would
define this delegate method if you wanted to reroute URLs to some other location or cancel URL opening
altogether (by returning NO).
A delegate's externalMovie: method is called when the movie controller needs to find a movie external to
some given movie, most often to send that other movie a wired action. Once again, QTKit contains code to find
external movies and most applications can rely on that automatic processing.
The only QTMovie delegate method likely to be defined by applications is
movie:shouldContinueOperation:withPhase:atPercent: withAttributes:, which provides a Cocoa wrapper for a movie
progress procedure. Currently this is useful only when calling the writeToFile:withAttributes method, though
it's possible that more operations in QTMovie will support this delegate method in the future.
Opening Movies
In our first QTKit article (mentioned earlier), we opened a movie specified by a filename using the
initWithFile:error: method, like this:
QTMovie *movie = [QTMovie initWithFile:filename error:nil];
We can also open a movie specified by a URL using the initWithURL:error: method, like this:
QTMovie *movie = [QTMovie initWithURL:url error:nil];
It's useful to know that there is a more general method for opening movies which offers several advantages
over these methods, namely initWithAttributes:error:. The following lines of code do exactly the same thing as
the line of code that uses initWithFile:error: above.
NSDictionary *dict = [NSDictionary dictionaryWithObject:
filename forKey:QTMovieFileNameAttribute]
QTMovie *movie = [QTMovie initWithAttributes:dict
error:nil];
The idea is that we pass into initWithAttributes:error: a dictionary containing one or more key-value pairs
that specify the attributes we want the new QTMovie object to have. One of those attributes must specify the
location of the movie data, either via a filename or a URL or a QTDataReference object or a pasteboard or an
NSData block. But there can be an indefinite number of other attributes in that dictionary, which are applied
to the movie object before it's returned to the caller. For instance, we can ensure that all movies we open
are editable by creating the dictionary like this:
dict = [NSDictionary dictionaryWithObjectsAndKeys:
filename, QTMovieFileNameAttribute,
[NSNumber numberWithBool:YES], QTMovieEditableAttribute,
nil];
The initWithAttributes:error: method is not merely a convenience that saves us from setting attributes on the QTMovie object returned by calls to initWithFile:error: or initWithURL:error:. There are at least two important reasons we might need to call it rather than those other methods. First, we can pass in attributes that override some of the default behaviors of QTKit when opening movies. For example, we learned in an earlier article that QTKit opens all movies asynchronously. (That is, a call to initWithURL:error: returns almost immediately, so that the application can continue with its processing while the movie data loads.) But it's possible that an application would want to load movies synchronously. In that case, if could construct an attributes dictionary like this:
dict = [NSDictionary dictionaryWithObjectsAndKeys:
url, QTMovieURLAttribute,
[NSNumber numberWithBool:NO], QTMovieOpenAsyncOKAttribute,
nil];
Setting NO as the value of the QTMovieOpenAsyncOKAttribute attribute indicates that we want the movie to be
opened synchronously. Obviously, this attribute needs to be set before we start opening the remote movie, and
initWithAttributes:error: is the only way to do that using QTKit methods.
The second case in which this method is sometimes necessary is to set a delegate for the QTMovie object
being created so that that delegate is operational during the movie loading. In particular, it's quite
possible that an mcActionLinkToURL action is encountered by the movie controller before the initWithURL:error:
method returns to the caller. So this sequence of calls might not work if the delegate needs to handle all
link-to-URL actions:
QTMovie *movie = [QTMovie initWithURL:url error:nil];
[movie setDelegate:self];
Instead, we can call initWithAttributes:error:, adding the QTMovieDelegateAttribute and its associated
object to the dictionary of attributes:
dict = [NSDictionary dictionaryWithObjectsAndKeys:
url, QTMovieURLAttribute,
self, QTMovieDelegateAttribute,
nil];
QTMovie *movie = [QTMovie initWithAttributes:dict
error:nil];
By setting the delegate in this way, we can guarantee that all link-to-URL actions will be seen by the
movie:linkToURL: delegate method.
If your application does not need to use any of the delegate methods defined in the QTMovie class and if it
does not need to override any of the default behaviors provided by QTMovie, then the initWithFile:error: and
initWithURL:error: methods are just fine.
Conclusion
This article and the previous two articles should help convince you that QTKit is a powerful framework that
allows you to develop robust QuickTime applications with a minimum of code. QTKit provides the most
comprehensive set of built-in capabilities and the most complete automatic processing of any QuickTime
framework that I have yet encountered. It's easy to use, it dovetails nicely with existing Cocoa frameworks,
and it's already serving as the QuickTime underpinnings of powerful in-house and commercial multimedia
applications.
In the next article, we'll take one more look at QTKit, to investigate how to execute QTKit methods on a
secondary thread. That will give us the machinery we need to perform computationally intensive operations
(like exporting large movies or creating slideshow movies from a large number of images) without negatively
impacting the responsiveness of the application's user interface.
References
The movieWithWritableFile:handler: method shown earlier is based on sample code that is available at
http://developer.apple.com/samplecode/QTKitCreateMovie/QTKitCreateMovie.html.
Tim Monroe is a member of the QuickTime engineering team at Apple. You can contact him at
monroe@mactech.com. The views expressed here are not necessarily shared by his employer.