Honey, I Shrunk the Kids

By Tim Monroe

Using QuickTime's Standard Image Compression Dialog Component

Introduction

In a previous QuickTime Toolkit article ("Making Movies" in MacTech, June 2000), when we built our very first QuickTime movie, we used a couple of Image Compression Manager functions to compress each video frame so that it (and hence the entire movie) took up less space on disk. The size reduction was significant: simply adding 100 uncompressed frames to the movie would have resulted in a movie file that was about 12 megabytes in size. Using JPEG compression, we were able to reduce the final movie file size to about 470 kilobytes.

In that article, however, we cut some corners by hard-coding the compression type when we called GetMaxCompressionSize and CompressImage. It would have been nice to provide the user with a choice of compression algorithms, and indeed perhaps even an indication of what any particular compression algorithm would do to the penguin images. Happily, QuickTime makes this very easy to do, by supplying the standard image compression dialog component. We can use this component to perform two main tasks. First, as the name suggests, we can have it display a dialog box in which the user can adjust compression settings for a single image. Figure 1 shows the standard image compression dialog box.

The standard image compression dialog box contains a pop-up menu that lists the available image compressors. It also contains a pop-up menu that lists the available pixel depths supported by the selected compressor. Finally, the dialog box contains a slider control for adjusting the image quality. As the user varies the compressor, pixel depth, or image quality, the standard image compression dialog component adjusts the thumbnail picture to show what the image would look like if compressed using the selected settings.

The second main task that the standard image compression dialog component can perform is to compress the image. That is to say, it can not only retrieve the desired compression settings from the user, but it can also do the actual compression for us (thereby saving us from having to call GetMaxCompressionSize and CompressImage). For this reason, the component is sometimes also called the standard compression component. Figure 2 shows the result of using the standard image compression dialog component to compress our penguin picture using the PNG image compressor at 16 levels of grayscale and the highest available quality.

In this article, we'll see how to use the standard image compression dialog component to elicit compression settings from the user and to compress images using the settings selected by the user. We'll also see how to use the standard image compression dialog component to compress a sequence of images (for example, the sequence of images that make up our penguin movie). We'll begin by taking a more focused look at compression itself. While the basic idea is straightforward, there are a handful of concepts we'll need to understand before we can start using the standard image compression dialog component. Then we'll spend the rest of this article investigating the compression-related parts of this month's sample application, QTCompress. The Test menu for QTCompress is shown in Figure 3; as you can see, it contains only one menu item, which compresses the image or sequence of images in the frontmost window.

Compression

Compression is the process of reducing the size of some discrete collection of data, presumably without unduly compromising the integrity of that data. The basic goal, of course, is to be able to store the data in less space and to use less bandwidth when transferring the data over a network. Particularly for multimedia content like large color images, movies, and sounds, uncompressed data (also known as raw data) simply takes up too much space on disk or too much time to transfer over a network. It's almost always better to store and transfer compressed data, which is then decompressed during playback.

In QuickTime, compression and decompression are handled by components called codecs (which is short for compressor/decompressor). The available codecs effectively define the kinds of compressed data that QuickTime can handle. Apple has written a large number of codecs itself and also licensed some other codecs from third-party developers. Ideally, it would be nice if QuickTime supplied both a compressor and decompressor for every kind of data that it can handle, but sadly that isn't the case. For instance, QuickTime can decompress and play MP3 files, but it does not include a component that can compress sound data into the MP3 format.

For the present, we'll be concerned primarily with compression and decompression of images and sequences of images. In this case, there are two basic kinds of compression: spatial compression and temporal compression. Spatial compression is a means of compressing a single image by reducing redundant data in the image. For instance, our penguin picture has large areas of pure white; a good spatial compressor would encode the image so as to avoid having to store a 32-bit RGB value for every one of those white pixels. Exactly how the encoding is accomplished varies from compressor to compressor.

Temporal compression is a means of compressing a sequence of images by comparing two adjacent frames and storing only the differences between the two frames. It turns out that many common sorts of video change very little from frame to frame, so a significant size reduction can be achieved by storing a full frame of the video and then the subsequent differences to be applied to that frame in order to reconstruct the original image sequence. In QuickTime, the full frame of video is called a key frame, and the subsequent frames that contain only the differences from previous frames are called difference frames or delta frames. (Other media technologies use other nomenclature. Key frames are also called intraframes, and difference frames are also called interframes. MPEG calls key frames I-frames and has two sorts of difference frames, B-frames and P-frames.)

Figure 4 shows a key frame (on the left) and the immediately following difference frame (on the right). The difference frame is pretty worthless when viewed by itself; but when the decompressor, using its special algorithms, applies the difference frame to the key frame, we get the actual image displayed in the movie, shown in Figure 5.

In this particular case, the space savings are significant. In the movie file, the key frame occupies 4076 bytes, while the difference frame occupies only 3120 bytes. The next difference frame occupies only 1184 bytes. The kinds of space savings you'll see depend of course on the actual content of the movie. For instance, a music video with lots of fast cuts and motion will not compress as well temporally as a movie of paint slowly drying.

In theory, a temporally compressed movie could consist of a single key frame followed by a large number of difference frames. But in practice, key frames are interspersed throughout the movie at predetermined intervals. This is because, to be able to draw any particular frame, the preceding key frame and all difference frames following that key frame (up to the frame to be drawn) must be processed. It would be prohibitively slow to jump to a random spot in a movie, or play a movie backwards, if it consisted of a single key frame and a bunch of difference frames. The maximum number of frames that can occur before a key frame is inserted is the key frame rate. A compressor may insert key frames more often than the specified key frame rate, however (for instance, at a scene change, where there is very little similarity between one frame and the following frame).

Note that spatial and temporal compression are not competing forms of compression. Indeed, most QuickTime movies employ both spatial and temporal compression, since the key frames of a movie are typically spatially compressed images. Note also that the use of temporal compression forces us to revise our understanding of the data stored in a QuickTime movie file. Up to now, we've tended to think of the movie data as a sequence of images. Now we see that it's more accurate to think of the movie data as a sequence of images (key frames) and changes to those images (difference frames). Only at playback time (that is, after the movie data is decompressed) do we get an actual series of images.

Compressing Images

The sample applications that we've developed so far in this series of articles can open both QuickTime movie files and image files and display them in windows on the screen. When the user selects the "Compress..." menu item in the Test menu, QTCompress executes this code:

case IDM_COMPRESS:
   if (QTFrame_IsImageWindow(myWindow))
      QTCmpr_CompressImage(myWindowObject);
   else
      QTCmpr_CompressSequence(myWindowObject);
   myIsHandled = true;
   break;

As you can see, if the frontmost window contains an image, then QTCompress calls the function QTCmpr_CompressImage (which we'll consider in this section); otherwise, it calls QTCmpr_CompressSequence (which we'll consider in the next section).

The QTCmpr_CompressImage function is built mainly around two routines provided by the standard image compression dialog component, SCRequestImageSettings and SCCompressImage. SCRequestImageSettings displays and manages the standard image compression dialog box (Figure 1), and SCCompressImage performs the actual compression of the image data into a new buffer. (As you've probably guessed, all functions provided by the standard image compression dialog component begin with the letters "SC".)

Getting the Image Pixel Map

The first thing we need to do when compressing an image is to draw it into an offscreen graphics world. We'll use the pixel map associated with that offscreen graphics world in two ways. First, we'll pass it to the SCSetTestImagePixMap function to set the thumbnail image in the image compression dialog box. Then, later on, we'll pass it to SCCompressImage as the source image to be compressed.

As we've seen in earlier articles, we can use graphics importers to open and draw image files. In fact, we already have an instance of a graphics importer component associated with the image file; it's the one we use to draw the image into the window on the screen (namely, (**theWindowObject).fGraphicsImporter). But here we're going to create a new graphics importer instance to draw the image into the offscreen graphics world. This is because we'll want to use the existing graphics importer to redraw the image in the on-screen window inside of the modal-dialog event filter procedure QTCmpr_FilterProc (defined later). It might in fact be possible to cleverly juggle the graphics importer's graphics world (using GraphicsImportSetGWorld), but I never managed to get that strategy to work properly. So let's create a new graphics importer instance for the image to be compressed:

myErr = GetGraphicsImporterForFile(
            &(**theWindowObject).fFileFSSpec, &myImporter);
if (myErr != noErr)
   goto bail;
   
myErr = GraphicsImportGetNaturalBounds(myImporter, &myRect);
if (myErr != noErr)
   goto bail;

Now that we know the size of the image, we can use this code to create the requisite offscreen graphics world:

myErr = QTNewGWorld(&myImageWorld, 0, &myRect, NULL, NULL, 
                                    kICMTempThenAppMemory);
if (myErr != noErr)
   goto bail;
   
// get the pixmap of the GWorld; we'll lock the pixmap, just to be safe
myPixMap = GetGWorldPixMap(myImageWorld);
if (!LockPixels(myPixMap))
   goto bail;

Finally, we need to draw the image into myImageWorld.

GraphicsImportSetGWorld(myImporter, (CGrafPtr)myImageWorld, 
               NULL);
GraphicsImportDraw(myImporter);

At this point, the offscreen graphics world myImageWorld contains the image to be compressed.

Now we want to display the standard image compression dialog box, to get the user's desired compression settings. To do this, we need to open an instance of the standard image compression dialog component, like so:

myComponent = OpenDefaultComponent(StandardCompressionType, 
                                       StandardCompressionSubType);

Before we call the SCRequestImageSettings function to display the dialog box on the screen, we need to set the thumbnail picture (called the test image) that is displayed in the top-right part of the dialog box, by calling the SCSetTestImagePixMap function:

SCSetTestImagePixMap(myComponent, myPixMap, NULL, 
                                                scPreferScaling);

Here, the first two parameters are the instance of the standard image compression dialog component and the pixel map that contains the image. The third parameter is a pointer to a Rect structure that specifies the area of interest in the pixel map that is to be used as the test image. Passing the value NULL means to use the entire pixel map as the test image, suitably reduced into the 80 pixel by 80 pixel area in the dialog box. The fourth parameter indicates how the image reduction is to occur; it can be any of these constants:

enum {
   scPreferCropping                              = 1 << 0,
   scPreferScaling                                 = 1 << 1,
   scPreferScalingAndCropping                  = scPreferScaling | 
                                                         scPreferCropping,
   scDontDetermineSettingsFromTestImage   = 1 << 2
};

You can also pass the value 0 in the fourth parameter to request the component's default method of displaying the test image, which is currently a combination of scaling and cropping. Personally, I prefer just scaling the image to fit into the available space, so I've passed the value scPreferScaling.

Installing Extended Procedures

We're almost ready to call SCRequestImageSettings to display the dialog box. We need to do just one more thing to configure the dialog box, namely install one or more callback procedures that extend the basic functionality of the standard image compression dialog component. (These are therefore called extended procedures.) We install these callback procedures by calling the SCSetInfo function with the scExtendedProcsType selector:

SCSetInfo(theComponent, scExtendedProcsType, &gProcStruct);

Here, gProcStruct is an extended procedures structure, of type SCExtendedProcs, which is defined like this:

struct SCExtendedProcs {
   SCModalFilterUPP             filterProc;
   SCModalHookUPP                hookProc;
   long                            refcon;
   Str31                            customName;
};

The filterProc field specifies a modal-dialog event filter function, to handle events that are not handled by the standard image compression dialog component itself. As we've done in the past, we'll provide a filter function that looks for update events for our application's windows and redraws those windows accordingly. Listing 1 shows our custom filter function.

Listing 1: Filtering events in the image compression dialog box

QTCmpr_FilterProc
static PASCAL_RTN Boolean QTCmpr_FilterProc 
      (DialogPtr theDialog, EventRecord *theEvent, 
                                    short *theItemHit, long theRefCon)
{
#pragma unused(theItemHit, theRefCon)
   Boolean         myEventHandled = false;
   WindowRef      myEventWindow = NULL;
   WindowRef      myDialogWindow = NULL;

#if TARGET_API_MAC_CARBON
   myDialogWindow = GetDialogWindow(theDialog);
#else
   myDialogWindow = theDialog;
#endif
   
   switch (theEvent->what) {
      case updateEvt:
         // update the specified window, if it's behind the modal dialog box
         myEventWindow = (WindowRef)theEvent->message;
         if ((myEventWindow != NULL) && 
                     (myEventWindow != myDialogWindow)) {
#if TARGET_OS_MAC
            QTFrame_HandleEvent(theEvent);
#endif
            myEventHandled = false;
         }
         break;
   }
   
   return(myEventHandled);
}

This is a fairly typical event filter function. It looks for update events that are not destined for the dialog box and sends them to the framework's event-handling function. Notice that this step isn't necessary on Windows; on that platform, redraw messages are sent directly to the window procedure of the affected window.

We can also install a hook function, which is called whenever the user selects (or "hits") an item in the dialog box. We can then intercept those hits and handle them in any way we like. A typical way to use the hook function is in connection with a custom button in the standard image compression dialog box. Figure 6 shows the dialog box with a new button labeled "Defaults".

We install this custom button by specifying a name for the button in the customName field of the extended functions structure. We can do that with these two lines of code:

StringPtr       myButtonTitle = 
                     QTUtils_ConvertCToPascalString("Defaults");

BlockMove(myButtonTitle, gProcStruct.customName, 
                                          myButtonTitle[0] + 1);

Then we can handle user clicks on this custom button inside our hook function, QTCmpr_ButtonProc, defined in Listing 2.

Listing 2: Intercepting events in a hook function

QTCmpr_ButtonProc
static PASCAL_RTN short QTCmpr_ButtonProc 
         (DialogPtr theDialog, short theItemHit, 
                                    void *theParams, long theRefCon)
{
#pragma unused(theDialog)
   // in this sample code, we'll have the settings revert to their default values
   // when the user clicks on the custom button
   if (theItemHit == scCustomItem)
      SCDefaultPixMapSettings(theParams, 
                                 (PixMapHandle)theRefCon, false);

   // always return the item passed in
   return(theItemHit);
}

This hook function is extremely simple; it just looks for hits on the custom button (signaled by the constant scCustomItem) and then calls the SCDefaultPixMapSettings function to reset the dialog box to its default values. Notice that the reference constant passed to our hook function (in the theRefCon parameter) is expected to be a handle to the pixel map we created earlier. We install this reference constant by setting the refcon field of the extended procedures structure. Listing 3 shows our definition of the QTCmpr_InstallExtendedProcs function, which we use to set up our extended procedures.

Listing 3: Installing the extended procedures

QTCmpr_InstallExtendedProcs
static void QTCmpr_InstallExtendedProcs 
                  (ComponentInstance theComponent, long theRefCon)
{
   StringPtr       myButtonTitle = 
                     QTUtils_ConvertCToPascalString(kButtonTitle);

   // the modal-dialog filter function can be used to handle any events that
   // the standard image compression dialog handler doesn't know about, such
   // as any update events for windows owned by the application
   gProcStruct.filterProc = 
                              NewSCModalFilterUPP(QTCmpr_FilterProc);

#if USE_CUSTOM_BUTTON   
   // the hook function can be used to handle clicks on the custom button
   gProcStruct.hookProc = 
                     NewSCModalHookUPP(QTCmpr_ButtonProc);
   
   // copy the string for our custom button into the extended procs structure
   BlockMove(myButtonTitle, gProcStruct.customName, 
                     myButtonTitle[0] + 1);
#else
   gProcStruct.hookProc = NULL;
   gProcStruct.customName[0] = 0;
#endif

   // in this example, we pass the pixel map handle as a refcon
   gProcStruct.refcon = theRefCon;
   
   // set the current extended procs
   SCSetInfo(theComponent, scExtendedProcsType, &gProcStruct);
   
   free(myButtonTitle);
}

You'll notice that we've used the compiler flag USE_CUSTOM_BUTTON to indicate whether we want to install a custom button in the standard image compression dialog box. Some image compressors want to install an Options button in that dialog box, and our custom button would prevent them from doing so. (See Figure 7, later, for a dialog box that contains an Options button.) For this reason, we usually won't install a custom button. But you should at least know how to do so.

Finally we can call QTCmpr_InstallExtendedProcs to install our extended procedures and then SCRequestImageSettings to display the dialog box.

if (gUseExtendedProcs)
   QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);
   
myErr = SCRequestImageSettings(myComponent);

If the user selects the Cancel button in the standard image compression dialog box, then SCRequestImageSettings returns the value scUserCancelled. Otherwise, if SCRequestImageSettings returns noErr, we want to go ahead and compress the image. Thankfully, we can do this with a single call to the function SCCompressImage.

myErr = SCCompressImage(myComponent, myPixMap, NULL, 
                                       &myDesc, &myHandle);

SCCompressImage compresses the specified pixel map using the current settings of the specified standard image compression dialog component. It allocates storage for the compressed image and returns a handle to that storage in the fifth parameter (myHandle). It also returns an image description in the fourth parameter (myDesc). We can write the compressed data into a new file by calling the application function QTCmpr_PromptUserForDiskFileAndSaveCompressed, passing in the compressed data and the image description. (See the file QTCompress.c in this month's source code for the definition of this function.) Listing 4 shows our complete definition of QTCmpr_CompressImage.

Listing 4: Compressing an image

QTCmpr_CompressImage
void QTCmpr_CompressImage (WindowObject theWindowObject)
{
   Rect                                 myRect;
   GraphicsImportComponent      myImporter = NULL;
   ComponentInstance               myComponent = NULL;
   GWorldPtr                           myImageWorld = NULL;
   PixMapHandle                     myPixMap = NULL;
   ImageDescriptionHandle         myDesc = NULL;
   Handle                              myHandle = NULL;
   OSErr                              myErr = noErr;

   if (theWindowObject == NULL)
      return;
      
   myErr = GetGraphicsImporterForFile(
                  &(**theWindowObject).fFileFSSpec, &myImporter);
   if (myErr != noErr)
      goto bail;
   
   myErr = GraphicsImportGetNaturalBounds(myImporter,
                &myRect);
   if (myErr != noErr)
      goto bail;

   // create an offscreen graphics world and draw the image into it
   myErr = QTNewGWorld(&myImageWorld, 0, &myRect, NULL, NULL, 
                                       kICMTempThenAppMemory);
   if (myErr != noErr)
      goto bail;
   
   // get the pixmap of the GWorld; we'll lock the pixmap, just to be safe
   myPixMap = GetGWorldPixMap(myImageWorld);
   if (!LockPixels(myPixMap))
      goto bail;
   
   // set the current port and draw the image
   GraphicsImportSetGWorld(myImporter, (CGrafPtr)myImageWorld, 
                                             NULL);
   GraphicsImportDraw(myImporter);
   
   // open the standard compression dialog component
   myComponent = OpenDefaultComponent(StandardCompressionType, 
                                          StandardCompressionSubType);
   if (myComponent == NULL)
      goto bail;
      
   // set the picture to be displayed in the dialog box
   SCSetTestImagePixMap(myComponent, myPixMap, 
                                             NULL, scPreferScaling);

   // install the custom procs, if requested
   if (gUseExtendedProcs)
      QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);
   
   // request image compression settings from the user
   myErr = SCRequestImageSettings(myComponent);
   if (myErr == scUserCancelled)
      goto bail;

   // compress the image
   myErr = SCCompressImage(myComponent, myPixMap, NULL, 
                                                   &myDesc, &myHandle);
   if (myErr != noErr)
      goto bail;

   // save the compressed image in a new file
   QTCmpr_PromptUserForDiskFileAndSaveCompressed(myHandle, 
                                             myDesc);
   
bail:
   if (gUseExtendedProcs)
      QTCmpr_RemoveExtendedProcs();

   if (myPixMap != NULL)
      if (GetPixelsState(myPixMap) & pixelsLocked)
         UnlockPixels(myPixMap);
   
   if (myImporter != NULL)
      CloseComponent(myImporter);

   if (myComponent != NULL)
      CloseComponent(myComponent);

   if (myDesc != NULL)
      DisposeHandle((Handle)myDesc);

   if (myHandle != NULL)
      DisposeHandle(myHandle);

   if (myImageWorld != NULL)
      DisposeGWorld(myImageWorld);
}

Compressing Image Sequences

Compressing a sequence of images is not fundamentally different from compressing a single image. We'll need to display the standard image compression dialog box, as before, to get the user's desired compression settings. Then, however, instead of compressing a single image, we'll need to loop through all the images in the sequence and compress each one individually. We'll obtain our sequence of images by extracting the individual frames from an existing QuickTime movie, and then we'll write the compressed images into a new QuickTime movie. (In effect, we'll be converting a QuickTime movie from one compression scheme to another; this operation is often called transcoding.) So we'll have the added overhead of creating a new QuickTime movie, track, and media, and of adding samples to the media by calling AddMediaSample. We've done this kind of thing numerous times before, so that part of the code shouldn't slow us down too much.

When we want to compress a sequence of images, we need to call SCRequestSequenceSettings instead of SCRequestSettings. The dialog box that it displays is shown in Figure 7; as you can see, it contains an additional pane of controls for specifying the number of frames per second, the key frame rate, and the maximum data rate (the number of bytes per second that can be processed).

When configuring the dialog box and when responding to its dismissal, we'll need to add code to handle this additional information. So let's get started.

Getting the Image Sequence

As mentioned above, we're going to obtain our sequence of images by reading the individual frames from the video track of a QuickTime movie. (Let's call this the source movie.) We can get the source movie by reading the fMovie field of the window object record, and we can get the source movie's video track by calling the GetMovieIndTrackType function:

mySrcMovie = (**theWindowObject).fMovie;
if (mySrcMovie == NULL)
   goto bail;

mySrcTrack = GetMovieIndTrackType(mySrcMovie, 1, 
                                 VideoMediaType, movieTrackMediaType);
if (mySrcTrack == NULL)
   goto bail;

To make things look a bit cleaner, we don't want the movie to be playing while we are compressing its frames into a new movie, so we call SetMovieRate to stop the movie. We also need to keep track of the current movie time, since we'll be changing it as we move from frame to frame through the movie. Later, when we're done recompressing the frames of the movie, we'll reset the movie time to this saved value.

SetMovieRate(mySrcMovie, (Fixed)0L);
myOrigMovieTime = GetMovieTime(mySrcMovie, NULL);

Finally, we need to know how many video frames are in the source movie, so that we know (for instance) how many iterations our loop should make. QTCmpr_CompressSequence includes this line of code for counting the frames of the source movie:

myNumFrames = QTUtils_GetFrameCount(mySrcTrack);

There are several methods we could use to determine how many frames the source movie contains. Probably the best method is just to step though the interesting times in the movie using the GetTrackNextInterestingTime function, as shown in Listing 5.

Listing 5: Counting the frames in a movie

QTUtils_GetFrameCount
long QTUtils_GetFrameCount (Track theTrack)
{   
   long            myCount = -1;
   short         myFlags;
   TimeValue   myTime = 0;
   
   if (theTrack == NULL)
      goto bail;
      
   // we want to begin with the first frame (sample) in the track
   myFlags = nextTimeMediaSample + nextTimeEdgeOK;

   while (myTime >= 0) {
      myCount++;
      
      // look for the next frame in the track; when there are no more frames,
      // myTime is set to -1, so we'll exit the while loop
   GetTrackNextInterestingTime(theTrack, myFlags, myTime, 
                                                   fixed1, &myTime, NULL);
      
      // after the first interesting time, don't include the time we're currently at
      myFlags = nextTimeStep;
   }

bail:
   return(myCount);
}

For more discussion of GetTrackNextInterestingTime, see "Word Is Out" in MacTech, November 2000.

Configuring the Standard Image Compression Dialog Component

As before, we need to open an instance of the standard image compression dialog component and configure the initial settings of the dialog box. Opening an instance of the component uses the same code we used in the case of a single image:

myComponent = OpenDefaultComponent(StandardCompressionType, 
                                             StandardCompressionSubType);

To configure the settings in the dialog box, we first want to turn off the "Best Depth" menu option in the pixel depth pop-up menu. This is because we are going to draw the movie frames into a 32-bit offscreen graphics world, regardless of the pixel depth of the original source images. A better approach might be to determine the maximum bit depth used in the source images (by looping through the video sample descriptions of the video frames) and then create an offscreen graphics world of that depth. (This refinement, of course, is left as an exercise for the reader.) We can disable the "Best Depth" option using this code:

SCGetInfo(myComponent, scPreferenceFlagsType, &myFlags);
myFlags &= ~scShowBestDepth;
SCSetInfo(myComponent, scPreferenceFlagsType, &myFlags);

Next, we want to allow the user to leave the frame rate field blank (in which case the compression component will preserve the original frame durations of the source movie). To do this, we need to specify that 0 is an acceptable value in that field. We do that by executing these lines of code:

SCGetInfo(myComponent, scPreferenceFlagsType, &myFlags);
myFlags |= scAllowZeroFrameRate;
SCSetInfo(myComponent, scPreferenceFlagsType, &myFlags);

If the user enters a number in the frame rate field, we'll use that number as the new sample rate for the destination movie.

Setting the Test Image

Before we display the compression settings dialog box to the user, we want to set the test image. In the present case, however, we have an entire sequence of images to handle, not just a single image. Which of those images shall we select as the test image? Let's select the movie poster image, on the assumption that that image is representative of the content of the entire sequence of images (that is, of the source movie itself). So we can call GetMoviePosterPict to get a PicHandle to the test image:

myPicture = GetMoviePosterPict(mySrcMovie);

Then we can get the size of the poster image and create an offscreen graphics world large enough to hold that image:

GetMovieBox(mySrcMovie, &myRect);
myErr = NewGWorld(&myImageWorld, 32, &myRect, NULL, NULL, 
                                                      0L);

And, as before, we'll lock the pixel map of that graphics world:

myPixMap = GetGWorldPixMap(myImageWorld);
if (!LockPixels(myPixMap))
   goto bail;

Next we want to draw the poster image into the offscreen graphics world. Since we've got a handle to a QuickDraw picture, we can use the DrawPicture function to draw the picture. First, however, we need to make sure to set the current graphics world to our new offscreen graphics world and to erase the destination graphics world.

GetGWorld(&mySavedPort, &mySavedDevice);
SetGWorld(myImageWorld, NULL);
EraseRect(&myRect);
DrawPicture(myPicture, &myRect);
KillPicture(myPicture);
SetGWorld(mySavedPort, mySavedDevice);

Finally we are ready to call SCSetTestImagePixMap to set the test image:

SCSetTestImagePixMap(myComponent, myPixMap, NULL, 
                                    scPreferScaling);

Once again, we have a couple of things still to do before we can display the standard image compression dialog box. For one thing, we need to install the extended procedures; here we can use exactly the same application function as in the single-image case:

if (gUseExtendedProcs)
   QTCmpr_InstallExtendedProcs(myComponent, (long)myPixMap);

Next, we want to set some default settings for the dialog box. The standard image compression dialog component can examine the pixel map that we just created and derive some sensible default settings based on the characteristics of that image. So let's take advantage of that capability:

SCDefaultPixMapSettings(myComponent, myPixMap, true);

Also, we want to clear out whatever default frame rate was selected by the standard image compression dialog component. As we discussed above, we'd like to use the frame rate 0, indicating that the frame rate of the source movie should be used. (The user is free to change this rate, but at least we want the default value to be 0.) We can first retrieve and then reset the current temporal settings of the component:

myErr = SCGetInfo(myComponent, scTemporalSettingsType, 
                                 &myTimeSettings);
if (myErr != noErr)
   goto bail;

myTimeSettings.frameRate = 0;
SCSetInfo(myComponent, scTemporalSettingsType, 
                                 &myTimeSettings);

SCGetInfo and SCSetInfo expect temporal settings to be stored in a structure of type SCTemporalSettings, defined like this:

struct SCTemporalSettings {
   CodecQ                      temporalQuality;
   Fixed                         frameRate;
   long                         keyFrameRate;
};

Finally, we're ready to call SCRequestSequenceSettings to display the standard image compression dialog box:

myErr = SCRequestSequenceSettings(myComponent);
if (myErr == scUserCancelled)
   goto bail;

If we've made it this far in the QTCmpr_CompressSequence function, we have successfully displayed the standard image compression dialog box and the user has selected his or her desired compression settings. In the single-image case, we could finish up rather quickly, by immediately calling SCCompressImage and then saving the compressed data into a new file. In the current case, however, we still have a good bit of work left to do. We need to retrieve the temporal settings - which may indicate a new frame rate for the destination movie - and configure the destination movie accordingly. Then we need to step though the frames of the source movie and compress each frame in the movie. We'll begin by first retrieving the temporal settings selected by the user:

myErr = SCGetInfo(myComponent, scTemporalSettingsType, 
                                 &myTimeSettings);

If the user wants to change the frame rate of the movie (as indicated by a non-zero value in the frameRate field of the temporal setting structure myTimeSettings), then we need to calculate the number of frames in the destination movie and the duration of the destination movie. We can do that like this:

if (myTimeSettings.frameRate != 0) {
   long   myDuration = GetMovieDuration(mySrcMovie);
   long   myTimeScale = GetMovieTimeScale(mySrcMovie);
   float   myFloat = (float)myDuration * 
                                       myTimeSettings.frameRate;
      
   myNumFrames = myFloat / myTimeScale / 65536;
   if (myNumFrames == 0)
      myNumFrames = 1;
}

Suppose now that myFile is a file system specification for the destination movie file (perhaps we called our framework function QTFrame_PutFile to elicit that file specification from the user). At this point, we need to create the destination movie file and movie, as shown in Listing 6.

Listing 6: Creating a new movie file and movie

QTCmpr_CompressSequence
myErr = CreateMovieFile(&myFile, sigMoviePlayer, 
                                    smSystemScript, 
                                    createMovieFileDeleteCurFile | 
                                    createMovieFileDontCreateResFile, 
                                    &myRefNum, &myDstMovie);
if (myErr != noErr)
   goto bail;
   
// create a new video movie track with the same dimensions as the entire source movie
myDstTrack = NewMovieTrack(myDstMovie,
                        (long)(myRect.right - myRect.left) << 16,
                        (long)(myRect.bottom - myRect.top) << 16, 
                        kNoVolume);
if (myDstTrack == NULL)
   goto bail;
   
// create a media for the new track with the same time scale as the source movie;
// because the time scales are the same, we don't have to do any time scale conversions
myDstMedia = NewTrackMedia(myDstTrack, VIDEO_TYPE, 
                        GetMovieTimeScale(mySrcMovie), 0, 0);
if (myDstMedia == NULL)
   goto bail;

We of course want to copy the movie user data and other settings from the source movie to the destination movie:

CopyMovieSettings(mySrcMovie, myDstMovie);

Also, we want to set the movie matrix of the destination movie to the identity matrix and clear out the movie clip region. This is because the process of recompressing the movie transforms and composites all the video tracks into a single, untransformed video track.

SetIdentityMatrix(&myMatrix);
SetMovieMatrix(myDstMovie, &myMatrix);
SetMovieClipRgn(myDstMovie, NULL);

Finally, since we're about to start adding compressed video samples to the destination movie, we need to call BeginMediaEdits:

myErr = BeginMediaEdits(myDstMedia);
if (myErr != noErr)
   goto bail;

We are now ready to start compressing the frames of the source movie into the destination movie. We are going to step through the source movie, draw the current frame into our offscreen graphics world, compress that data, and then add the compressed data as a video sample to the destination movie. To start things off, we call SCCompressSequenceBegin:

myErr = SCCompressSequenceBegin(myComponent, myPixMap, NULL, 
                                       &myImageDesc);

SCCompressSequenceBegin initiates a compression sequence, using the current settings of the specified component instance and the characteristics of the specified pixel map. The third parameter is a pointer to a Rect structure that indicates what portion of the pixel map we're going to be compressing; the value NULL indicates that we want to compress the entire pixel map. The fourth parameter is a pointer to an image description. The standard image compression dialog component will allocate storage for that description and fill in its fields when it compresses some data for us. In turn, we shall use that image description when we call AddMediaSample.

Next we need to erase the offscreen graphics world that we're going to be drawing movie frames into and set the source movie to draw its frames into that graphics world:

SetGWorld(myImageWorld, NULL);
EraseRect(&myRect);
SetMovieGWorld(mySrcMovie, myImageWorld, 
                        GetGWorldDevice(myImageWorld));

From now on, until we reset the movie graphics world, calling MoviesTask on the source movie will cause the current movie frame to be drawn into myImageWorld.

So let's start looping through the frames of the source movie. We begin by setting a variable that holds the current movie time to the beginning of the movie and by retrieving the duration of the source movie:

myCurMovieTime = 0;
mySrcMovieDuration = GetMovieDuration(mySrcMovie);

The remainder of our code in this section will occur inside of a for loop that iterates through all the frames of the source movie:

for (myFrameNum = 0; myFrameNum < myNumFrames; myFrameNum++) {
   // get a frame, compress it, and add it to the new movie
}

The first thing we want to do inside this loop is get the next frame of the source movie. If we are not resampling the movie (that is, changing the frame rate), we can use GetMovieNextInterestingTime to step us forward in the movie; otherwise, we need to step forward by the appropriate amount, based on the desired new frame rate. Listing 7 shows our code for this step.

Listing 7: Getting the next frame of the source movie

QTCmpr_CompressSequence
if (myTimeSettings.frameRate) {
   myCurMovieTime = myFrameNum * mySrcMovieDuration / 
                                             (myNumFrames - 1);
   myDuration = mySrcMovieDuration / myNumFrames;
} else {
   OSType      myMediaType = VIDEO_TYPE;
         
   myFlags = nextTimeMediaSample;

   // if this is the first frame, include the frame we are currently on      
   if (myFrameNum == 0)
      myFlags |= nextTimeEdgeOK;
         
   // if we are maintaining the frame durations of the source movie,
   // skip to the next interesting time and get the duration for that frame
   GetMovieNextInterestingTime(mySrcMovie, myFlags, 1, 
         &myMediaType, myCurMovieTime, 0, &myCurMovieTime, 
         &myDuration);
}

Then we need to set the current movie time of the source movie to the time just calculated and draw the movie into the offscreen graphics world:

SetMovieTimeValue(mySrcMovie, myCurMovieTime);
MoviesTask(mySrcMovie, 0);
MoviesTask(mySrcMovie, 0);
MoviesTask(mySrcMovie, 0);

(Why three calls to MoviesTask here? Partly it's to make sure that the current movie frame is completely drawn before continuing with our work, and partly it's just because that's the way the sample code on which QTCmpr_CompressSequence is based was written. Consider this your first taste of QuickTime black magic. No harm results from a few extra calls here.) If the user has requested that the data rate of the destination movie be constrained, then we need to tell the standard image compression dialog component the duration of the current frame, in milliseconds. We can do that using this code:

if (!SCGetInfo(myComponent, scDataRateSettingsType,
                        &myRateSettings)) {
   myRateSettings.frameDuration = myDuration * 1000 / 
                        GetMovieTimeScale(mySrcMovie);
   SCSetInfo(myComponent, scDataRateSettingsType, 
                        &myRateSettings);
}

Finally we can actually compress the pixel map data, by calling SCCompressSequenceFrame:

myErr = SCCompressSequenceFrame(myComponent, myPixMap, 
         &myRect, &myCompressedData, &myDataSize, &mySyncFlag);

If SCCompressSequenceFrame completes successfully, then myCompressedData will hold a handle to the compressed data and myDataSize will be the size of the compressed data. In addition, the mySyncFlag parameter will hold a value that indicates whether the compressed frame is a key frame (0) or a difference frame (mediaSampleNotSync). We will pass this value directly to the AddMediaSample function, like this:

myErr = AddMediaSample(myDstMedia, myCompressedData, 0, 
                                 myDataSize, myDuration, 
                                 (SampleDescriptionHandle)myImageDesc, 
                                 1, mySyncFlag, NULL);

And so we've completely handled a frame of the source movie.

Finishing Up

Once we've exited the for loop that iterates through all the frames of the source movie, we need to do a little cleaning up. First, we need to close the compression sequence that we opened by calling SCCompressSequenceBegin:

SCCompressSequenceEnd(myComponent);

SCCompressSequenceEnd disposes of the image description and compressed data handles allocated by SCCompressSequenceBegin.

Next we need to close our media-editing session by calling EndMediaEdits:

myErr = EndMediaEdits(myDstMedia);

Then we proceed as normal, inserting the edited media into the track and adding the movie resource to the destination movie file.

InsertMediaIntoTrack(myDstTrack, 0, 0, 
               GetMediaDuration(myDstMedia), fixed1);
myErr = AddMovieResource(myDstMovie, myRefNum, NULL, NULL);

And at this point, we can safely close the destination movie file:

CloseMovieFile(myRefNum);

We also need to close down the instance of the standard image compression dialog component that's been doing all this work for us:

if (myComponent != NULL)
   CloseComponent(myComponent);

Finally, let's restore the source movie to its original graphics world and movie time, and dispose of the offscreen graphics world:

if (mySrcMovie != NULL) {
   // restore the source movie's original graphics port and device
   SetMovieGWorld(mySrcMovie, mySavedPort, mySavedDevice);
   // restore the source movie's original movie time
   SetMovieTimeValue(mySrcMovie, myOrigMovieTime);
}
// restore the original graphics port and device
SetGWorld(mySavedPort, mySavedDevice);
// delete the GWorld we were drawing frames into
if (myImageWorld != NULL)
   DisposeGWorld(myImageWorld);

We have now successfully compressed our image sequence. (Whew!) I won't bother to give the entire listing of the QTCmpr_CompressSequence function, as it would run for several pages. You can find the complete story in the source file QTCompress.c.

Asynchronous Compression

If you run the QTCompress sample application and have it recompress a sizable movie file, you'll notice that, after you dismiss the compression settings dialog box and the file-saving dialog box, no other events are processed until the entire recompression sequence is completed. The mouse continues tracking, but you cannot access the application's menus; worse yet, the application windows are not updated. This is all perfectly understandable, as we've jumped into a for loop and just keep processing movie frames until we are finished.

One way to address this problem would be to display a progress dialog box, which we could update each time though the for loop. For good measure, we could also see if any of our windows need updating and redraw them if necessary. This would cost us some time, but it would significantly improve the user's experience.

Another way to address this problem would be to compress the movie frames asynchronously. That is to say, we could launch a compression operation and then process other application events while we are waiting for the operation's completion procedure to be executed. That would allow us to redraw our windows and even grant the operating system some time to handle its own periodic tasks.

QuickTime has supported asynchronous compression for many years, but until recently we had to use the lower-level services of the Image Compression Manager (namely, CompressSequenceFrame) to do so. As we've seen, there is no parameter to the SCCompressSequenceFrame function that allows us to specify a completion routine, which we'd need in order to be able to use the standard image compression dialog component asynchronously. QuickTime 5, however, introduces the new function SCCompressSequenceFrameAsync, which we can use for asynchronous compression. In this section, we'll see how to modify QTCmpr_CompressSequence to support asynchronous compression.

Setting Up for Asynchronous Compression

SCCompressSequenceFrameAsync is exactly like SCCompressSequenceFrame, except that it takes one additional parameter, a pointer to an ICM completion procedure record. The ICM completion procedure record, of type ICMCompletionProcRecord, is declared like this:

struct ICMCompletionProcRecord {
   ICMCompletionUPP            completionProc;
   long                            completionRefCon;
};

The completionProc field is a universal procedure pointer to our completion routine, and the completionRefCon field is an application-specific reference constant. When we first enter QTCmpr_CompressSequence, we'll declare and initialize some additional local variables, like this:

#if USE_ASYNC_COMPRESSION
   ICMCompletionProcRecord         myICMComplProcRec;
   ICMCompletionProcRecordPtr   myICMComplProcPtr = NULL;
   OSErr                                 myICMComplProcErr = noErr;

   myICMComplProcRec.completionProc = NULL;
   myICMComplProcRec.completionRefCon = 0L;
#endif

Then, just before we enter the for loop, we need to set some codec flags:

#if USE_ASYNC_COMPRESSION
   myFlags = codecFlagUpdatePrevious + 
                  codecFlagUpdatePreviousComp + codecFlagLiveGrab;
   SCSetInfo(myComponent, scCodecFlagsType, &myFlags);
#endif

The codecFlagUpdatePrevious and codecFlagUpdatePreviousComp flags are used to optimize temporal compression. The codecFlagLiveGrab flag is required to get SCCompressSequenceFrameAsync to work correctly. (More QuickTime black magic.)

Once we're inside the for loop, we'll fill in the ICM completion procedure record:

if (myICMComplProcPtr == NULL) {
   myICMComplProcRec.completionProc = 
                        NewICMCompletionProc(QTCmpr_CompletionProc);
   myICMComplProcRec.completionRefCon = 
                        (long)&myICMComplProcErr;
   myICMComplProcPtr = &myICMComplProcRec;
}

For our reference constant, we're passing the address of a local variable of type OSErr. The idea is that we can undertake other processing inside the for loop as long as the value of that variable is set to a known value; the completion routine, when it is triggered, will be responsible for changing that value, as we'll see shortly. Before we initiate the asynchronous compression operation, we'll set myICMComplProcErr to a default value:

myICMComplProcErr = kAsyncDefaultValue;

Performing Asynchronous Compression

As we've seen, we can begin an asynchronous compression operation by calling SCCompressSequenceFrameAsync, like this:

myErr = SCCompressSequenceFrameAsync(myComponent, myPixMap, 
         &myRect, &myCompressedData, &myDataSize, &mySyncFlag, 
         myICMComplProcPtr);

The parameters here are exactly those that we earlier passed to SCCompressSequenceFrame, with the additional final parameter myICMComplProcPtr. As with all asynchronous functions, SCCompressSequenceFrameAsync returns immediately. When the compressor is finished with the compression operation, our completion routine will be executed. Listing 8 shows our application's completion routine.

Listing 8: Signaling the end of a compression operation

QTCmpr_CompletionProc
static PASCAL_RTN void QTCmpr_CompletionProc 
               (OSErr theResult, short theFlags, long theRefCon)
{
   OSErr      *myErrPtr = NULL;
   if (theFlags & codecCompletionDest) {
      myErrPtr = (OSErr *)theRefCon;
      if (myErrPtr != NULL)
         *myErrPtr = theResult;
   }
}

When the codecCompletionDest flag is set in the theFlags parameter, we know that the compression operation has finished successfully. At that point, we can copy the result code passed to our completion routine into the location specified by theRefCon. This effectively changes the value of the myICMComplProcErr variable in our for loop, where we are executing this while loop until that value changes:

while (myICMComplProcErr == kAsyncDefaultValue) {
   EventRecord         myEvent;
   WaitNextEvent(0, &myEvent, 60, NULL);
   SCAsyncIdle(myComponent);
}

As you can see, we're repeatedly calling WaitNextEvent to yield time to other processes and SCAsyncIdle to yield time to the compressor component. I'll leave it as another exercise to add the code necessary to handle application events (for example, update events).

Weighing the Benefits

While adding support for asynchronous compression is not too terribly complicated, you might be wondering whether it's worth the trouble. In other words, are we going to see appreciable performance gains, or at least significant user experience improvements? Sadly, the answer is "no", at least at the moment. First of all, not all compression components support asynchronous operation. (Indeed, at this point, there is only one Apple-supplied codec capable of performing asynchronous compression, the H.263 codec.) You can still invoke compression components using SCCompressSequenceFrameAsync, but SCCompressSequenceFrameAsync will not return to the caller until the compression is complete and the completion routine has been executed. In other words, it will run synchronously. Second, and perhaps more important, even codecs that are able to compress asynchronously will not show any real improvement when running on a single-processor machine.

The standard image compression dialog component is a prime example of the kind of QuickTime APIs that I like best: it provides a high-level toolbox for handling a wide variety of typical image compression tasks, allowing us to compress both individual images and sequences of images. Moreover, it gives us a standard user interface for eliciting compression settings from the user (rather in the same way that the Standard File Package or the Navigation Services provide a standard user interface for eliciting files from the user).

Credits

Much of the code in the function QTCmpr_CompressSequence is based on code in the earlier sample code package called "ConvertToMovie Jr." (which was itself based on an even earlier sample called "ConvertToMovie").

Tim Monroe works in the QuickTime Engineering team at Apple. You can contact him at monroe@apple.com.